What are Search Engine Crawlers?

A search engine crawler is also sometimes referred to as a web crawler. This is a program which is used to scour the entirety of the World Wide Web or the Internet. It also has rather apt bug-like names...