Wednesday, June 3, 2009

Bing and Google: Searching for Technical Terms

I wondered how Microsoft's recently released and highly advertised Bing search engine would stack up against the renowned Google search engine for searching specifically for technical terms. I planned to try it out on some favorite terms and report here, but then I ran across a link from DZone to a blog posting that has already started down that road. In Bing Survives the Search Engine Achilles Heel: Programmers, Brandon Cannaday ("The Reddest") has beaten me to the punch.

The intention of this post is not to steal The Reddest's Thunder, but is to instead build upon what he has started and show a few of my representative Java-related queries in both the Bing search engine and the Google search engine. Like the Reddest, my past experience has been that no other search engine remotely matches Google for searching for technical terms and concepts.

The only time I have really used a search engine other than Google's is when 15-30 minutes of Googling has not found me the answer and I want a fresh (search engine) perspective. In fact, I like the Google search engine so much for searching for technical and personal things that the Google search engine main page is my home page on my Firefox browser and is the default browser in my Firefox search engine toolbar. Further evidence of my satisfaction in using the Google search engine as a development tool can be found in my blog post Day-to-Day Java Development Tools and Techniques.

Round #1: A Java Exception

What do the respective search engines provides when I query for "StackOverflowError"? The results with top returned queries from both browsers are shown next.

There are a few (I think interesting) observations here. First, Bing's first returned link is to the Sun-hosted Java SE 6 version of the Javadoc-generated API documentation for this Java error whereas Google's first-returned link is to the Sun-hosted J2SE 5 version of the Javadoc-generated API documentation. Because I work mostly with Java SE 6, I like Bing's giving me that version first. It's not difficult to get there myself, but it is a little more convenient.

The second interesting observation is that although I did not include "Java" in the search query, both favored Java. However, Bing seemed to favor Flex more by returning a link to Flex API documentation on the same-named error object. A third and related observation is that Google returned a Microsoft link related to stack overflow errors on Windows in the top several provided links and the Microsoft search engine did not.

Round One goes to Bing.

Round #2: Specialized Java Concept

For my next search results comparison, I try searching for "jmx connector versus adaptor" because this is a common beginning JMX user question (what is the difference between the two and because the spelling issue of adaptor versus adapter can make this one interesting). The screen snapshots of top results from both browsers for this query is shown next.

I try to be unbiased, but it is difficult for me not to give Google significant points and perhaps award them the entire contest for returning one of my blog posts as the first result for this query. However, if I'm forced to be honest, I have to admit that the page the Bing search returns is a very useful one. It is written by one of the members of the Sun JMX team (Daniel Fuchs), provides a nice introduction to Java Management Extensions, and definitely spells out the difference between a connector and an adapter in JMX. Still, I must admit to being a little hurt that my post specifically on the differences between a JMX connector and adaptor does not come up at all in Bing's first several pages worth of query results. That's going to cost them in my assessment. Round 2 goes to Google.

I don't show it here, but changing the "adaptor" to "adapter" is interesting. In Google, the first returned result is the JavaWorld-syndicated version of my same post returned first above. That one letter difference dramatically affects the top results of both search engines. This is evidence of something I've found with search engines; the key to a good search is often knowing the appropriate terms to search by, but we often don't know what those appropriate terms are when we're new to a subject.

Round #3: Not Quite Right Search Terms

For some reason, it seems tricky for many Java developers to remember if the operator known for its question mark and colon syntax is the ternary operator (which it is) or the tertiary operator (which it is apparently not). What happens in these two search engines if I search for "java tertiary operator"?

Bing seems to more closely literally match "tertiary" while Google seems to know that I really mean "ternary." In fact, Google's first returned result is explicitly on "ternary" with the second result using "tertiary." Sometimes I don't want the search engine trying to guess what I really want, but in this case it was beneficial. Round 3 goes to Google.

Round #4: Not Quite Right Search Terms Part 2

REST is very trendy in software development circles these days. What if I wanted to find out more about REST using these two search engines?

The first interesting observation here is that both Google and Bing return identical results for the first two: the Wikipedia REST entry and Building Web Services the REST Way. I was curious where Roy Fielding's PhD dissertation that started the whole REST phenomenon was in the respective search results. It was the 5th result on Google and 8th on Bing. The concept of REST the architectural style is obviously easily confused with the concept of rest. If I was looking for the latter, I'd actually prefer Bing's results because Google's initial page results are exclusively related to REST the architectural style. However, because I am comparing them in terms of technical terms searching, Google's emphasis on this is an advantage.

Two Key Areas Still to Explore

There are two things I use Google heavily for that tend to be unique to or at least more commonly associated with software development. First, I often use Google to search by an error message to find out how that particular problem was solved. Second, I often find myself needing to include some code syntax or syntax-like query term.

For the first item (querying by error message), my minimal comparisons of Google and Bing have so far shown them to return similar items (forums, online e-mail threads, etc.). I suspect I need to wait until my next real-life situation when I need to find an answer based on an unintuitive error message and try both browsers out then.

For the second item (needing to use syntax in the query), I have found that including operators generally is not helpful (operators are ignored or impact the search) with Google. For example including a minus before a word actually instructs Google to search for matches that explicitly do not include the term following the minus sign. Similarly, trying to search for discussions on Groovy's ?. operator with the phrase "?." are unhelpful. The following three snapshots of Google search engine results for querying on "?.", "?. groovy", and "?. groovy null checking" show that the operator itself doesn't really help and that its the use of "extra" knowledge that the operator has to do with Groovy and with null checking that makes the search successful. When I'm searching something entirely new to me, this extra knowledge may be exactly what I don't have.

I don't show the Bing results for the same three queries here because those results are largely the same as Google's.


From my admittedly small sample of tests, I still think Google comes out as the top search engine for software development searches (at least in the Java world and probably outside the .NET world in general). However, Bing has done itself proud in terms of jumping to second favorite search engine on my list ahead of Yahoo!'s search engine. When I need a different search engine's perspective for a particularly difficult to find concept, I will use Bing. While Google does a great job as a search engine, I still look forward to a day when developers can use their programming language's operators and get matches based on the actual use of those operators in source code. I dream big.

Google will remain the main page that comes up when my browser comes up, but Bing has certainly earned its spot on the search engine toolbar selector.

Related Resources

Bing Survives the Search Engines Achilles Heel: Programmers

FAQ: How to Add Some Bing to Your Browser

Bing Isn't All That

Bing: How Useful is Microsoft's New Search Offering?

Why Bing Sucks. Top 5 Reasons

Microsoft, You Got Binged

No comments: