-: Are we talking about a search engine for the entire web,
like Google, or just some internet tool?
Kerri: I'm talking Google.
So billions and billions of web pages
and billions and billions of people searching.
-: Okay, we designed a web crawler earlier,
so I assume we can have a distributed data store
of page content already, right?
Kerri: Sure, we can start with a repository
of web pages you designed earlier.
-: All right.
Well, I'm not gonna recreate all of Google in 20 minutes
so can we just focus on the problem
of generating reasonable search results at massive scale?
Kerri: Sure. But how would you define reasonable though?
-: Well, we need to take some measures to prevent people
from gaming the system, at least.
So a webpage that consists of nothing
but a keyword repeated over and over and over again
shouldn't rank as number one for that keyword.
So we should talk about other signals
that might predict relevance,
but I'm not gonna try to recreate decades of work
to combat people trying to play SEO tricks.
To answer your question though,
a reasonable search result is a page that gives the searcher
the information they were seeking.
Kerri: How do you measure that?
-: Well, short of explicit ratings
that most people wouldn't bother to give,
we could look at things like whether the user returned
to the search results after viewing a page
that was presented to them.
That might mean that they didn't get
what they were looking for the first time.
If they don't click any search results at all,
that might mean that they were particularly unhelpful.
These sorts of implicit signals can be inaccurate though.
I mean, someone may have just gotten distracted
and left for all we know,
but over enough people, it should provide some data.
Kerri: Mmhmm, okay.
Let's talk about some algorithms you might employ
and an architecture to apply them at scale.