Frequently Asked Questions


Please Scrape This Site

This site is made for the purpose of teaching people the art of web scraping. Please feel free to write or use software in order to automatically pull, harvest, scrape, download or otherwise access any information on this site.

Please be Well-Behaved

Just like any site you'd scrape out in the wild wild west (www), please be mindful of other users trying to access the site. From a technical standpoint, you must observe the following rules:

  • Clients may only make a maximum of one request per second
  • Clients must send an identifying user agent
  • Clients must respect this site's robots.txt file

Any client that violates the rules above or otherwise tries to interfere with the site's operation will be subject to a temporary or permanent ban.

Be a good web scraping citizen. 👋