Hi,
I'm trying to get much better at using search engines for technical research.
If you just want the question, jump straight to QUESTION below. The rest is context for why I'm asking.
---
LLMs solve part of the problem today as it helps better information foraging quickly, for instance when what I need is a clear definition. But I have the feeling that there is still much more I could get out of web search if I knew how to use it properly. Serendipity is one obvious example. But I keep running into a problem that makes the whole process very inefficient: context explosion.
Take Apache Hadoop.
Let's say I encounter the term somewhere and I don't know what it is.
I search for it, land on Wikipedia, and get something like:
"Apache Hadoop is a collection of open-source software utilities for reliable, scalable, distributed computing. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model."
Now the problem is that if I don’t already know the field, this definition creates several new questions and so on.
What exactly is MapReduce? What does "programming model" mean in this particular context? "Programming model" is the kind of expression where I understand both words, but that doesn't necessarily tell me precisely what the author means by putting them together here. Then I end up with a lot of tabs open.
I'm deliberately using a slightly crude example. Obviously I can keep reading the Wikipedia page. There is an Overview section, HDFS and YARN get introduced, the architecture becomes more concrete, and after a while I can probably build a decent picture of what Hadoop is.
So my point isn't that Wikipedia gives a bad definition, or that the information isn't there. It's more that there is a difference between eventually reconstructing the answer from all of that and getting something like: "Hadoop is basically a system for turning a cluster of machines into distributed storage plus distributed compute for large datasets."
When I asked ChatGPT about Hadoop, it could give me something at roughly that level immediately. Maybe the answer here is simply that an LLM is better for that particular job. I don't know and don't care. I'm more interested in is whether I'm missing something on the web-search side.
---
However I'm not completely new to web search either. I know that different kinds of sources are useful for different things. I know about forums, aggregators, documentation, Stack Overflow, GitHub issues, Hacker News and so on. I also know enough of the technical web ecosystem that a SERP isn't completely flat to me anymore: I recognize names like USENIX, CNCF, LWN, good company engineering blogs, vendor documentation, independent technical blogs, and so on, and I often already have some sense of which results are worth opening and which ones raise a red flag. I also know that useful searches are often indirect, and that one good result can lead to a good author, another site, another term, another discussion, etc.
And yet I keep running into the same problem: context explosion. So even though I feel like I already have quite a few of the pieces, I don't feel that I have a precise method for this.
I've also had the opposite experience, where web search was much better than asking ChatGPT. Recently I was looking into some FastAPI questions around how to organize things — when something should be a query, when it should be an operation, how people actually structure that boundary. I ended up finding Stack Overflow discussions that made the issue much clearer to me than the answers I was getting from ChatGPT.
So there is obviously a larger question here about how to combine both efficiently, but that isn’t really the question I’m trying to ask in this post, its much narrower. Here it is:
Suppose you’re an experienced engineer, but you genuinely don’t know the subject you’re researching, you encounter Hadoop, you want to answer a fairly basic question: "What is this thing, concretely?"
What do you actually do? Not what would you recommend that I do in general. I’d be much more interested if you actually did the experiment yourself and described what happened. That is to say :
What was your first query, exactly?
What did you open?
What part did you read?
What made you keep reading that source rather than going somewhere else?
When you encountered a term you didn’t understand, which ones did you look up and which ones did you decide you could leave unresolved?
What did you consider noise? Why?
Did you reformulate the query? Search indirectly? Look for a discussion instead of documentation? Search a particular site?
And where did you stop? Why?
If you can, it would also be interesting to give the short, concrete definition you ended up with after doing the search.
---
One thing I’d ask you to be careful about during the experiment: try to notice which words or concepts feel completely obvious to you but probably wouldn’t be obvious to someone new to the field. I’m interested in that too, because otherwise it’s hard to separate search skill from knowledge you already had.
And if you already know Hadoop well, that can still be useful. Just try to separate what you already knew, what you only had to look up because you remembered roughly where to find it, and what you actually had to figure out through the search.
The idea is basically a small experiment: do the search, tell me your background, and describe as precisely as you can what you actually did (questions above).
---
"REAL-WORLD" EXAMPLES
And if you have a concrete example you remember where you had to be ridiculously good at web search to find something, please tell me about it too. I framed the Hadoop example narrowly mainly so the discussion wouldn’t go in every direction, but Hadoop isn’t a hard requirement. Another example is completely fine as long as you’re precise about what you actually did - roughly along the lines of the questions above.
---
LLMs solve part of the problem today as it helps better information foraging quickly, for instance when what I need is a clear definition. But I have the feeling that there is still much more I could get out of web search if I knew how to use it properly. Serendipity is one obvious example. But I keep running into a problem that makes the whole process very inefficient: context explosion.
Take Apache Hadoop.
Let's say I encounter the term somewhere and I don't know what it is.
I search for it, land on Wikipedia, and get something like:
"Apache Hadoop is a collection of open-source software utilities for reliable, scalable, distributed computing. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model."
Now the problem is that if I don’t already know the field, this definition creates several new questions and so on.
What exactly is MapReduce? What does "programming model" mean in this particular context? "Programming model" is the kind of expression where I understand both words, but that doesn't necessarily tell me precisely what the author means by putting them together here. Then I end up with a lot of tabs open.
I'm deliberately using a slightly crude example. Obviously I can keep reading the Wikipedia page. There is an Overview section, HDFS and YARN get introduced, the architecture becomes more concrete, and after a while I can probably build a decent picture of what Hadoop is.
So my point isn't that Wikipedia gives a bad definition, or that the information isn't there. It's more that there is a difference between eventually reconstructing the answer from all of that and getting something like: "Hadoop is basically a system for turning a cluster of machines into distributed storage plus distributed compute for large datasets."
When I asked ChatGPT about Hadoop, it could give me something at roughly that level immediately. Maybe the answer here is simply that an LLM is better for that particular job. I don't know and don't care. I'm more interested in is whether I'm missing something on the web-search side.
---
However I'm not completely new to web search either. I know that different kinds of sources are useful for different things. I know about forums, aggregators, documentation, Stack Overflow, GitHub issues, Hacker News and so on. I also know enough of the technical web ecosystem that a SERP isn't completely flat to me anymore: I recognize names like USENIX, CNCF, LWN, good company engineering blogs, vendor documentation, independent technical blogs, and so on, and I often already have some sense of which results are worth opening and which ones raise a red flag. I also know that useful searches are often indirect, and that one good result can lead to a good author, another site, another term, another discussion, etc.
And yet I keep running into the same problem: context explosion. So even though I feel like I already have quite a few of the pieces, I don't feel that I have a precise method for this.
I've also had the opposite experience, where web search was much better than asking ChatGPT. Recently I was looking into some FastAPI questions around how to organize things — when something should be a query, when it should be an operation, how people actually structure that boundary. I ended up finding Stack Overflow discussions that made the issue much clearer to me than the answers I was getting from ChatGPT.
So there is obviously a larger question here about how to combine both efficiently, but that isn’t really the question I’m trying to ask in this post, its much narrower. Here it is:
---
QUESTION
Suppose you’re an experienced engineer, but you genuinely don’t know the subject you’re researching, you encounter Hadoop, you want to answer a fairly basic question: "What is this thing, concretely?"
What do you actually do? Not what would you recommend that I do in general. I’d be much more interested if you actually did the experiment yourself and described what happened. That is to say : What was your first query, exactly? What did you open? What part did you read? What made you keep reading that source rather than going somewhere else? When you encountered a term you didn’t understand, which ones did you look up and which ones did you decide you could leave unresolved? What did you consider noise? Why? Did you reformulate the query? Search indirectly? Look for a discussion instead of documentation? Search a particular site? And where did you stop? Why?
If you can, it would also be interesting to give the short, concrete definition you ended up with after doing the search.
---
One thing I’d ask you to be careful about during the experiment: try to notice which words or concepts feel completely obvious to you but probably wouldn’t be obvious to someone new to the field. I’m interested in that too, because otherwise it’s hard to separate search skill from knowledge you already had.
And if you already know Hadoop well, that can still be useful. Just try to separate what you already knew, what you only had to look up because you remembered roughly where to find it, and what you actually had to figure out through the search.
The idea is basically a small experiment: do the search, tell me your background, and describe as precisely as you can what you actually did (questions above).
---
"REAL-WORLD" EXAMPLES
And if you have a concrete example you remember where you had to be ridiculously good at web search to find something, please tell me about it too. I framed the Hadoop example narrowly mainly so the discussion wouldn’t go in every direction, but Hadoop isn’t a hard requirement. Another example is completely fine as long as you’re precise about what you actually did - roughly along the lines of the questions above.
Thanks!