How Search Engines Work

Search engines are searchable databases of web content. They’re made up of two main parts:

Every search engine aims to provide the best, most relevant results for users. That’s partly how they gain market share.

Search engines have two types of search results:

Each time someone clicks a paid search result, the advertiser pays the search engine. This is known as pay-per-click (PPC) advertising, and it’s why market share matters. More users mean more ad clicks and more revenue.

Everything begins with a known list of URLs. Google discovers these in many ways, but the three most common are:

• From backlinks. Google has an index of hundreds of billions of webpages.² If someone links to a new page from a known page, Google can find it from there.
• From sitemaps. Sitemaps tell Google which pages and files you think are important on your site.³
• From URL submissions. Google lets site owners request crawling of individual URLs in Google Search Console.

Crawling is where a computer bot called a spider visits and downloads known URLs. Google’s crawler is Googlebot.⁴

Processing is where Google works to understand and extract key information from crawled pages. To do this, it has to render the page, which is where it runs the page’s code to understand how it looks for users.

Nobody outside of Google knows every detail about this process. But it doesn’t matter. All we really need to know is that it involves extracting links and storing content for indexing.

Search algorithms are formulas that match and rank relevant results from the index. Google uses many factors in its algorithms.

Nobody knows every Google ranking factor because Google hasn’t disclosed them. But we do know some key ones. Let’s look at a few of them.

Backlinks are links from a page on one website to another. They’re one of Google’s strongest ranking factors.⁶ This is probably why we saw a strong correlation between linking domains and organic traffic in our study of over a billion pages.⁷

It’s not all about quantity, though. Quality matters too. Pages with a few high-quality backlinks often outrank those with many low-quality backlinks.

Relevance is the usefulness of a given result for the searcher. Google has many ways of determining this. At the most basic level, it looks for pages containing the same keywords as the search query. It also looks at interaction data to see if others found the result useful.⁹

Freshness is a query-dependent ranking factor. It’s stronger for searches that call for fresh results.⁹ That’s why you see a recently published top result for “new netflix series” but not “how to solve a rubik’s cube.”

Page speed is a ranking factor on desktop and mobile.^{10 11} But it’s more of a negative ranking factor than a positive one. This is because it negatively affects the slowest pages rather than positively affect lightning-fast pages.

Mobile-friendliness has been a ranking factor on mobile and desktop since Google’s switch to mobile-first indexing in 2019.¹²

Google uses your location to personalize the results for searches with local intent. That’s why all results for “italian restaurant” are from or about local restaurants. Google knows you’re unlikely to fly halfway around the world for lunch.

Google knows there’s no point in showing English results to Spanish users. That’s why it ranks localized versions of content (if available) to users who speak different languages.

Google saves the things you do and places you go to give you a more personalized search experience.¹³ You can opt out of this, but most people probably don’t.