When should you use a robots.txt file?

Question: When should you use a robots.txt file?

• When you have multiple versions of a page to indicate the preferred version
• When your website receives a penalty from Google
• When you have pages that you don’t want search engines to crawl and index
• Whenever you feel like it

The right answer was: When you have pages that you don’t want search engines to crawl and index