A program used by search engines to gather and interpret data from pages on a website before the search engine indexes the site.