Software development

Search - The best friend and worst enemy of 21st century websites

Search is by far one of the most powerful navigational paradigms of the modern day internet. Yet the internet is littered with sites that have completely useless internal searches. We might want to face the fact that with current tools and understanding, searching is still an enormous challenge that’ll easily devour any project budget thrown at it.

“Search”

The word seems too sweet and innocent. So harmless and uninteresting, even.

Yet every time I hear it nowadays, I flinch.

I spent quite a bit of my time zigzagging between business needs, concept planning and technical implementation, trying to make sure all the three dimensions are joining hands and aiming at a common goal.

For the most part, this is one of the best things about my job: Most successful projects have quite clearly succeeded exactly due to success in this area: business, concept and implementation have come together to create unique and functional solutions for real world problems.

But then there’s the concept of “search“.

Google

We all google stuff, right? Actually that was a pretty rhetoric question as everybody knows Google pretty much owns the internet. Their minimalistic search UI and great relevance engine cleared the room of rivals in the late 90′s and are still very much the elephant in the search engine living room.

But the thing is, Google did a lot of things right. Even if they’re now trying hard to beat Apple in who gets the “Microsoft of the 2010s” throne (ie. most hated tech bullies in the world) throne, for 10+ years they’ve provided us with a search service that’s fast, simple and damn good at guessing what we really meant.

But this didn’t happen overnight. Over the past 10 years they’ve spend more money on product development for their indexing and searching engine than I care to even try to imagine.

Why am I bringing this up?

This leads me to the beef I have with the world – or at least its digital business side – that made me write this post: Treating search as a trivial feature.

When it comes to search, we’re all spoiled brats. When we want something, Daddy Google just provides it to us – no fuss, no muss. We’ve also learned to love searching as an amazingly good navigation paradigm! You can check Google Trends and what you’ll notice is that very often popular search terms are just a TLD away from being URLs. The meaning of the address bar – and all other navigational features for that matter – is diminishing fast and being replaced by the navigation idiom of “type what you want in this box and we’ll give it to ya”.

The other side of the coin is that what seems dead simple on the outside is actually a huge cognitive mess on the inside! There is a very good reason why some of the world’s leading engineers, linguists, sociologists and… actually, any scientific disclipline in the world could more of less be listed here – are working at 1600 Amphitheatre Parkway, Mountain View, CA. Letting a user type text freely into that one simple text box and trying to understand her as she types away is nothing short of a awe inspiring problem. Seriously, when I was a kid, I used to look at the stars and wonder at the infinity that opens above me. Now I just open my browser and stare blankly at the marvel that is the Google search box.

Which FINALLY brings me to my problem at hand:

Websites and their search.

Pretty much by now every single entity running a web based service have caught wind of the “search as a leading navigational paradigm” and have come up with the “we want some of that action” -mantra. Fair enough, from a conceptual point of view they’re exactly on the right track.

The big pitfall comes when it’s time to start whipping up some wireframes and throwing around feature definitions. It’s still far too common to see “search” defined more or less as “the best damn search in the business” (obviously while laughing at their competitors’ feeble attempts at providing a good search for their site) and search result sorting with “sorting order: most relevant result first”.

I might exaggerate a bit, but I ain’t kidding.

What this blog post is in essence is a plea. It’s a small man’s silent prayer in hope of getting all the relevant parties from business owners to concept designers to digital strategy planners… heck, even engineers to own up to how important this whole search-thingy is. Sure, there’s a lot of great tools to use, indexing engines, relevancy algorithms and even Google’s own Site Search and Search Appliance, but a 3rd party tool can never fully understand the context you operate in. Not to mention the linquistic difficulties our language – or any locale for that matter (maybe apart from Esperanto) – poses.

If this was a purely technical problem, it would have been solved ages ago. What it all comes down to is that advanced searching is actually closer to artificial intelligence / machine learning than just your run-of-the-mill database querying problems.

So, now what?

If your budget for a project is 150k – a pretty decent if not overwhelming wad of dough for a web-based service – and you expect to include a world class search with full capabilities of predicting every last hidden intention of the user plus correcting their faults while providing only the BEST and MOST RELEVANT results from your huge catalog of content / products, maybe it’s time for a tiny reality check. At least make sure you’re not basing your entire business model on search as it could be reality that will be checking you afterwards.

A great way to approach this is to look at search with a strong sense of realism. A bad internal search is useless at best, but more often just obnoxious and a hindrance.

Also, please don’t forget that Google – scary as it might be – is still your friend in the end. You save yourself a world of headaches by putting in more effort in ranking high in Google than struggling to get that “best of class” search on your own site.

As a worldwide community of professionals we need to aim to understand the search domain better and better each year. Even up to the point that a user doesn’t have to do more than think about a given subject and we show him the BEST and MOST RELEVANT content.

But until then, let’s just be realistic. A good search that behaves as expected (or even an integration to Google Site Search!) combined with a strong strategy of using external search sources (-> good SEO) and a strong navigational structure is far better than a superhuman concept of a search that swallows the entire budget, but fails to live up to its expectations thus making your site a navigation nightmare.

Let’s make sure the search functionality we’re planning to make our service doesn’t end up breaking it.