Typically, a search engine works by sending out a spider to fetch as many documents as possible. Another program, called an indexer, then reads these documents and creates an index based on the words contained in each document. Each search engine uses a proprietary algorithm to create its indices such that, ideally, only meaningful results are returned for each query.
On the Internet, a search engine is a coordinated set of programs that includes:
A spider (also called a "crawler" or a "bot") that goes to every page or representative pages on every Web site that wants to be searchable and reads it, using hypertext links on each page to discover and read a site's other pages
A program that creates a huge index (sometimes called a "catalog") from the pages that have been read
A program that receives your search request, compares it to the entries in the index, and returns results to you