Krishwords

Today Scott Karp made a rather naive argument saying that user generated content is a myth. I do agree that there is way too much noise in the user generated content but it doesn’t make UGC (User Generated Content) a myth. I suggest that Mr. Karp spends some time reading history before making a bold statement like this. There are many historical precedents to show that user generated contents can add value. Two of the most prominent cases are

1. Open Source Software
2. Wikipedia

We all know that open source software are developed with the contributions of thousands and thousands of users worldwide. In fact, many contributions to open source software are a result of user’s scratching their own personal itch. The contributions to open source software doesn’t come just from star programmers. Kiddie hackers and noobs trying to learn programming, equivalent of people in the streets, have contributed immensely to open source software. Before he became what he is, Linus Torvalds was just yet another programmer frustrated with the established system in the software world. Scott tries to get rid of the argument saying that the contributors are not people in the streets but programmers with immense potential. Unfortunately, Scott isn’t making the distinction that the bar is set to people with programming skills in the case of open source software because we are actually talking about software, a specialized form of content. In the case of content, as in user generated content, the barrier is automatically believed but the process remains the same. Anyone with experience with open source development process will agree that open source software is indeed a user generated software, contributed by thousands and thousands of actual users from around the world. In fact, many thought leaders of Web 2.0 will concede that open source software is the precursor to Web 2.0 approach.

Another example of user generated content is Wikipedia. Unlike in the case of open source software, the barrier to participate in Wikipedia is almost none. Literally, a person walking in the street can contribute to Wikipedia. It is a well known fact that the quality of Wikipedia is almost on par with Encyclopedia Brittanica. The breadth and depth of wikipedia articles talk about the value generated by thousands and thousands of people on the streets.

Clearly, there are historical precedent to the concept of user generated content. The above two examples are clear evidences for the value that can be created by user collaboration. Either Mr. Karp has failed to understand the role played by open source and wikipedia to the development of Web 2.0 concept or he is just trying to increase his position in the techmeme leadership board

Tags: Web2.0, Techmeme, UGC

Robert Scoble kicked off a discussion today with his prediction that Mahalo, Techmeme and Facebook will beat Google. In the videos, he seems to be pretty convinced that Mahalo can beat Google. I disagree with Robert’s take on this. I had a few back to back tweets with Robert on the topic. In one of the tweets he asked whether several Mahalos can get together to beat Google. In this post, I am going to disagree with the thesis that “Mahalo can beat Google”. I am using Mahalo here but it applies to any human powered search engine with Mahalo’s approach to search.

Before I offer my views on the topic, I want to make the following clear.

• I completely agree with Robert that SEMs (Search Engine Marketers) are a pain and we need better search results.
• If, by any chance, Robert had meant that Google should be beaten in the search game, I am in full agreement with it too. We need to stop Google’s possible search monopoly and there should be heavy competition to force Google to continue innovating.

Having said that, I want to categorically state that Mahalo, in its current form, can’t beat Google. The reasons are many. Some of them include

• Human powered search cannot scale like machine based search
• Human powered search has built in bias in the system
• Human powered search will definitely lack in freshness
• Human powered search doesn’t mean much in non text search world like Image Search, Video Search, Book Search, etc.
• Human powered search is a ridiculous thing when it comes to desktop search.

There are many other reasons where Mahalo and other similar search engines will fail. However, these are the few reasons that just popped out immediately after watching Scoble’s video. Let us dig a little deeper on these reasons.

Human powered search engine in the Mahalo mould cannot scale like an algorithm based search like Google. Even if we assume that Mahalo grows like how Robert expects in the next four years, with 100K “volunteers”, it still cannot cover all topics in the world, it still cannot cover all languages in the world, it still cannot cover all the specifics in a particular topic, it cannot cover all the variations of a particular query, etc.. It might cover a search query “HDTV” much better than Google. But I searched Mahalo with queries “Dent on HDTV” and “Scratch on HDTV”, an hour back. It returned back saying there are no matching results. Even though it offers better results for HDTV, it fails miserably when you get into specifics on HDTV. Imagine the variations in queries on HDTV from all over the world, “HDTV with a spot”, “HDTV with a broken edge”, “HDTV and Bestbuy problem”, “HDTV with loose switch”, “HDTV’s remote not working”, “I just peed on HDTV”, etc. There is no way Mahalo can scale to take care of such specifics on even a single topic, along with everything else. Comparatively, HDTV is an easy problem to solve. There are much more difficult problems in this world which Mahalo cannot solve or may not bother to solve. Let us say I make a web page about a topic HDTV that contains a sentence like “not gone triple ribbon belt mother cook with mould acumen”. If someone searches for HDTV, my page may show up after the first 50 pages of Google results and not at all on Mahalo. But if someone searches for the sentence I mentioned above with double quotes, it will show up in the Google results and there is almost zero chance that it will show up in Mahalo (unless if my content is a killer content except for that particular sentence I quoted above). Now imagine endless new queries. Ask.com CEO claimed that 60% of their searches are unique. Even Google seems to have claimed that 20-25% of their total queries were not made before. Mahalo cannot even attempt to solve the search engine needs of all the internet users in this world, even if it scales like what Scoble hopes. It is like trying to build a 200 storey building without involving any machines and just with one brick at a time. It is just not humanly possible. Scaling problems will ensure that any attempts to create a completely human powered search engine is a failure.

To highlight my arguments, I am adding the picture of the result I got in Mahalo when I queried “mod_security rules for SQL injection attack”. You can see how badly it failed for this query. The scaling of Mahalo in the next 4 years, which Robert describes, may answer this particular query well. But, there are billions of other queries which may throw up similar results even after four years.

Human powered search like Mahalo has inbuilt bias which is difficult to manage. Jason feels that this bias can be managed with stuff like transparency, ethics codes, hiring people well, firing bad people quickly, early warning systems, etc. He claims that they take a Neutral Point of View (NPOV) on stuff like abortion, 9/11 and George Bush. I agree with him that with stuff like transparency, ethics code, etc., you can take a NPOV on few things like the queries he has quoted. Bring in diversity of queries and the different points of view attached with it. Now bring scale these number of queries. There is no way you can maintain NPOV consistently on all the topics and various queries associated with these topics. Jason even suggested the idea of Mahalo Ombudsman and included Jeff Jarvis as one possible candidate, which he rejected immediately. Even if he makes it ombudsmen instead of ombudsman, the scaling problem will ensure that the bias will stay in the system. Add to this the issues like embargo. For example, the embargo on Iran might prevent Jason to hire guides from Iran and this may result in inferior Mahalo results as far as topics related to Iran are concerned. But, an algorithmic engine like Google can just crawl the websites in Iran and have better results on the topics related to Iran.

Human powered search engine like Mahalo cannot keep the content fresh. Google crawls the web more frequently to keep its contents fresh. Now, I think, we can even inform Google about how often it should update our websites and this helps Google to update its results with fresh contents. There is no way Mahalo can keep the content fresh on all topics and related to all queries. The scalability problem described above, will make sure that the Mahalo content is stale except for a smaller percentage of terms.

I think it is pretty obvious that human powered engines like Mahalo cannot do much in the areas like Image search, Video search, Book search, etc. Add to this, the search in other languages, reverse phone number look up, etc. I don’t even have to talk about how ridiculous it will be to consider human powered search when it comes to desktop search.

Robert ignores the longtail concept when he talks about human powered search engines replacing the algorithm based ones. The very concept of longtail came into existence due to the phenomenon called internet. Under such a scenario, betting the future of internet on something that uses the hits and ignores the longtail (remember his discussion on social fabric? He talks about bumping the results up based on the number of people who trust a particular person (Michael Arringtons and Robert Scobles of the world)). I do agree that Google uses pagerank which can, in some ways, be likened to the hits in the longtail concept. However, a well structured query in Google will also get results from the longtail of web pages whereas it will not be fetched in the social fabric approach of Mahalo. This is a very important point and I just hope I have put it forward clearly.

So what is the solution to the search engine problem? Is Mahalo doomed for failure? What about 100s of Mahalos? Well, I am not an expert in search engines to offer a solution to the problems, in the current day search engines. But I strongly believe that human based search engines like Mahalo cannot replace (or beat) algorithm based search engines like Google. My solution is still algorithmic based. My hunch feeling is that the future search engine will be a collection of vertical search engines which may use Robert Scoble’s trust concept to optimize the search results. As I told Robert in our twitter conversation, Mahalo can, at the very best, be a Google optimizer but not a Google beater. Vertical search engines, with Mahalos in the front for optimization, may offer the much needed solution to our search engine problems. Humans cannot replace machines in the search engine world. Mahalo may serve well for a niche market but it is not a Google beater. At best, it can be termed as a Google optimizer.

What is your take on Scoble’s thesis that Mahalo and other trust based human search engines will beat Google? Do you agree with him? If not, why do you think human search engines cannot beat Google? What do you think will be the Google killer?

PS: It is 2:15 AM. I will correct any mistakes in the morning.

If you are not a Silicon Valley kinda entrepreneur, you may want to check out this article by Matt on Web 2.0 for entrepreneurs. Even if you are a Web 2.0 junkie, this is an interesting read. If you Web 2.0 is greek and latin for you, this is a must read. It will give you an idea about what Web 2.0 is and how you can build exciting and valuable propositions into your business.

Tags: Web2.0, Entrepreneurship

I have around 5 pownce invites. If you want one, leave a comment with your email address in the email field.

Remember the Killer Facebook App idea I was talking about? When I wrote the post, Zoho CEO, Sridhar, assured us that they will come out with the Facebook App soon. They have done it with the release of Zoho Facebook app. I strongly urge you to check it out. Now you can create a new document or presentation or spreadsheet inside Facebook. You can also see the list of your docs and other friends who use Zoho inside Facebook. Kudos to Sridhar and his team for this app. I am sure the implementation is still in the initial stages. By the time they complete, I would like to see the following features

• If it is not going to mess up the layout, I would like to see the apps inside the facebook like Piknik
• When you save and close a document or spreadsheet or presentation, it automatically updates the list of documents, spreadsheets and presentations inside the facebook
• A way to delete docs from inside the facebook
• A way to share documents with other facebook friends

Thanks Zoho for the app. I am sure this is going to be a success among facebook community. Thanks for listening to blogosphere and coming up with neat apps. Read more at Zoho Blogs.

Dear Zoho team,

I know you guys listen to blogosphere. Today I saw the release of izoho.com for iPhone. It is a neat idea. How about making it available for Windows mobile devices too. I am sure it shouldn’t be all that difficult for your company. I am not planning to buy iPhone till version 2. Unfortunately, I am struck with a Windows Mobile Device (T-mobile Dash). I just hope that you will create a similar mini site for Windows Mobile browser. All the poor souls who are struck with Windows Mobile device will thank your company for this. You might even end up getting a few converts from Google Docs. Hoping to see a Windows Mobile version soon.

Cheers!!
Krish…..

Dave Winer makes an astute observation about social networks. (Also see the discussion on Techmeme)

Eventually, soon I think, we’ll see an explosive unbundling of the services that make up social networks. What was centralized in the form of Facebook, Linked-in, even YouTube, is going to blow up and reconstitute itself. How exactly it will happen is something the historians can argue about 25 years from now. It hasn’t happened yet, but it will, unless the rules of technology evolution have been repealed (and they haven’t, trust me). Permalink to this paragraph

I have been thinking about this for sometime now. I am not happy with the centralized form of social networking sites like facebook. I am fascinated by Marc Canter’s idea of Digital Lifestyle Aggregator which doesn’t lock you into a single system. Thatz why I liked his PeopleAggregator networking system. My hunch feeling tells me that the future of social networking is an open standards based decentralized networks in the People Agrregator mould, with something like OpenID being the single point of entry needed to connect with people. Facebook is an amazing platform. There is no doubt about it. I am excited about their platform, which can be seen by the number of facebook related posts I have made. Still, there is something in me, the open source part of me, which tells me that facebook could be the next Microsoft or the hundreds of other companies, that lock you into their product. We need a decentralized social networks based on open standards. After all, what we have in Facebook or Myspace, are our friends. They are fruits of our time. We should be able to take them anywhere we go at any time we want. Marc Canter may be too early in his idea of open standards based social networking or the “old age” can be attributed, by others, to Dave Winer for making a hypothesis like the one above. But the social networking platform, in its current form, cannot serve the needs of our society. We needed a Microsoft to understand the importance of open source and open standards. Probably, Myspace and Facebook (to some extent), will provide us the “Microsoft” of the social networking world. I can show a simple analogy from the real life to drive home the ideas of Marc Canter and Dave Winer. People go to various community events to socialize but they could also stay at home and socialize with their friends, from the convenience of their home. Similarly, I might be heading over to Facebook to socialize but I should also be able to socialize, in the same way, from my blog or my website. This is where decentralized services and OpenID kinda authentication mechanisms play a pivotal role. I agree with Marc Canter when he says that we need to take our friends with us wherever we go. I also agree with Dave Winer when he says that the centralized system like Facebook or Linked-In will blow up and result in explosive unbundling of (decentralized) services that make up social networks.

If you just want to see what Facebook platform can do to your web business, just check out the following graph. You can see the growth of apps in the pre-facebook world and how it has changed in the facebook world. The viral nature of Facebook platform is amazing and it open up a great opportunity for all the web companies to tap into the vast user pool.

Source: iLike Team Blog

According to Comscore,

The growth of widgets is amazing. It is where the money is right now. In fact, most of the companies don’t have their widget strategy right. There is a good opportunity for the first mover advantage if the companies manage to get their strategies right.

Oliver Ryan, blogging for Fortune magazine, points out to the optimism expressed by LinkedIn CEO Dan Nye about the future of LinkedIn in the Facebook dominated world. They are confident that they will dominate the business networking scene. As Steve O’ Hear and Jeremiah Owyang clearly point out, it is going to be much difficult than the optimism expressed by the people at LinkedIn. LinkedIn in a Facebook world is akin to a vertical search engine in this Google world. It is possible to survive but it is extremely difficult. As both Steve and Jeremiah correctly point out, it will only take a few filters to be developed by Facebook to enter into the market of LinkedIn. Does this mean it is the end of LinkedIn? Not necessarily. They just have to readjust their strategy. As JP says, every enterprise now is a platform. It should be open and inclusive. LinkedIn should open up its API strategy and also integrate tightly into the Facebook platform. It is time for LinkedIn to get their API strategy correct. From what I read in media, they seem to be working on it.

The “social climate” inside Facebook is much warmer than LinkedIn. I do agree that LinkedIn is just a good networking tool for business but they could still make the process of business networking much more interesting than what it is right now. I do not agree with people who claim that there is no need for LinkedIn in this Facebook world. I am a kinda person who would like to separate my social life from my business life. LinkedIn proves to be a valuable tool for people like me. By using Facebook as an app for my social life and LinkedIn for my business networking, I could establish a clear demarcation between these two worlds. Using the Seinfeld quote, I can ensure that these two worlds do not collide. Any talk about Facebook consuming LinkedIn is premature. However, it is vital for LinkedIn to re-adjust their strategy in this Facebook dominated world, to ensure that a vertical player like them can also survive along with a monster. The fate of LinkedIn is now linked in directly to their approach in the current world where every enterprise is a platform. In short, it is a case of open up, collaborate and survive.