There is a discussion of Google going on with Tim O’Reilly and Read Write Web regarding the well being of Google. Read Write Web proposed the idea that Google was flailing and spreading itself too thin, but O’Reilly dismissed the notion suggesting Google is healthy.
Funny enough, an idea hit pretty hard the other day when I was looking at my friendfeed sidebar I have here on my blog (to the right). Check out all the conversation that is happening around these posts. Yes, I call them posts, for they are not just aggregated feed items from my other accounts around the web. Friendfeed has become a favorite of mine and it’s lot like my blog, its just easier to post over there and there is more community for me there.
But the problem is, for me, none of that data is showing up in Google search. Its not being indexed because FF has opted to disallow search spiders. As a result, all of this potentially useful data will never be discovered outside of FF. Even the sidebar on my blog is not indexed.
So why would FF block the spiders you ask? For the same reason Twitter did I assume: The new search engine that Twitter bought, Summize, is a major property value. An enormous amount of the value of Twitter is the ability to search news updates every .01 seconds. (e.g. I wanted to know if there was still a line at the Apple store in Soho when the iPhone came out, so I went to twitter search, typed in “apple soho” and found a tweet from someone published 20 seconds prior that said the line was around the block to Houston street, info Google could not offer). Not sure if anyone is articulating this, but a no brainer way for Twitter to generate a lot of revenue would be to sell the rights for search engines to index them. To my point, Twitter could end up selling spider rights for even more by going exclusive, dealing with only one search giant, like Yahoo for instance, leaving Google out in the cold.
Facebook, Myspace, iTunes, Twitter, Friendfeed, the list goes on and on – all not indexable by Google spiders.
Are you seeing a trend here? I know I am. For all of the time, value and umph I put into publishing on these networks, I get no Google juice in return – the traffic gain and value goes to others and thus I’m decentralized and I’m also unable to monetize until they are ready to monetize my work on their terms.
While there is no denying that there is still a lot of value for me personally nonetheless, and while I am okay with continuing on in this direction (i.e. its better than not being there at all), it seems like the internet is trending away from a centralized search.
I used to think of Google as my entry point into the web but recently, have found that Google search is where I go just to get old news and well established findings. More recently, I find myself going other places to explore and research.
In my opinion this trend is a potential threat to Google search. Its not any one company that is coming in to rival the superpower by doing it better, and its not about Google spreading itself too thin, its just that Google is being denied access to important, growing pools of data. Perhaps the trend is a slow breakdown, more like the melting ice caps. Slow, but quick enough to be concerned.
*update: apparently FF and Twitter DO allow alot of indexing, though I maintain my argument re: Facebook, Myspace, etc as major pools of inaccessible data.
