Should Information Have Expiration Date?

 This blog post was originally published on Linkedin.com

It will be an understatement to say that the decision from EU’s Court of Justice, virtually enshrining Right to be Forgotten, and its compliance by Google, has kicked off a firestorm, on both sides of Atlantic.

Americans have a trepidation about this newly minted "right” colliding heads-on with their First Amendment Rights, guaranteeing the Freedom of Speech. They are also apprehensive that this development will potentially impede free flow of information, the life blood of internet. The issue of online privacy is also held in a similar light. Not long ago Eric Schmidt reportedly remarked that online users would have to fight for privacy or they may loose it.

Europeans, on the other hand, have a deeper affinity to the idea of privacy, data or otherwise, and seek to restrict public availability of information about individuals. This right has its historical roots in French Law covering Right of Oblivion.

A professor at Yale summarized these differences as originating from differing perception of Dignity vs. Liberty

Google’s (somewhat hurried) implementation of EU's Court order sparked more reactions. Last week EU gave an extended questionnaire with 26 questions to Google, Microsoft and Yahoo, in order to delve deeper into the matter.

Balancing Public Interest vs. Privacy

Immediately after the Take-down Request form went live, thousands of requests poured in. A review of of these request showed that half of these stem from criminal conviction or past sins. This raises a question about how would Google strike a fair balance between the privacy and public right to know? How long the information about a person should be easily accessible?

Court Records are archived (after a mandated period). Newspapers have their own archives for the business or social needs, that can be accessed with some effort.

Search Engines have not defined any set period for accessing the information once it get into their indexes. The keyword based “relevancy” of their algorithm ensures retrieval of all information held in the repository, regardless of its recency or public needs. The indexed information does not have any attributes to cover “privacy” or “public interest.” The later is usually handled via ad-hoc requests from governments or courts.

Courts are competent to balance individual’s privacy rights against broader public interest. Search Engines should not be expected to walk that thin, fuzzy line.

A case for specifying a shelf life

This raises a question should some types of information have expiration dates to preclude those from appearing in search results?

Note that the EU Court while considering the case deemed that the act of 1990 was old and “no longer relevant.” [emphasis is mine]

Attaching an expiration date will give search algorithms an objective criteria for excluding the information from its results. The information would still be in the index, just not accessible through a casual search.

Search Engines cannot (and should not) adjudicate what that period should be. In the case of a conviction that determination should rest solely with the courts. For example courts could specify an expiration date, before which the information would be freely available (or searchable) in order to serve the public and social interests adequately.

Once that date is over the individual's right of privacy will gain primacy. Courts could do this as part of their sentencing guidelines or through judicial discretion. This will, of course, require amendment of the statues currently on the books.

…and signaling with META DATA

Once a period is set by the decision of a competent court, it should not be too difficult to attach a META DATA carrying that date to that piece of information. Search engine’s information retrieving algorithm could be tasked to read this META DATA, and to leave the information out from search results if it has gone over its expiration date.

Such a step should ease the technology related burden in removing numerous links and also address the issue of removed links in one domain still accessible in other domains.

Above all, it will free the search engines from the herculean task of dealing with “removal requests” from individuals.


Obviously such a step will not be a silver bullet for all types of “takedown requests” where other information about a user has been proliferated by others on internet. However, once a META DATA on Expiry Date has been created, the original owner of the content would have gained more control on the data. This will make it easy for media sites/SEs to ignore the time-expired result in casual search sessions.

Such a step will serve to ameliorate the Type 1 and Type 3 issues referred to by Peter Fleischer, Google’s Chief Privacy Counsel. The existing laws should be able to adequately handle type 2 problems.

Image courtesy of usamedeniz / FreeDigitalPhotos.net

Will Bing soon be looking for a new parent?

-->

Businessweek reported that the potential Microsoft CEO candidate, Stephan Elop,

“…would consider ending Microsoft’s costly effort to take on Google with its Bing search engine, and would also consider selling healthy businesses such as the Xbox game console if he determined they weren’t critical to the company’s strategy,..”

 

-->

Maybe just a Test Balloon?

At this point this could just be second guessing, based on the opinions of some people who are said to be familiar with Elop's thinking. Search experts, however, were quick to analyze such a move. +Greg Sterling  made some great arguments here.

One obvious speculation was to whom Microsoft could sell Bing to!

+Danny Sullivan, of Search Engine Land held the view that Bing is too strategic for Microsoft to “just walk away” from the search engine. Bing is deeply integrated with the Windows Phone and other Microsoft products.  In his view it would be too disruptive for Microsoft to disengage from Bing. 

Yahoo and Apple are the two names being bandied about. 

-->

Possibility of search engines overseas, like Alibaba, Baidu and Yandex  acquiring Bing is being mentioned. This does make some sense, since the next (and potentially explosive) round of growth in Internet will come from Asia (and Africa). Along with more people accessing Internet, demand will rise for search engines that are better customized to the regional needs (my earlier post).

-->

But Competition also spawns Innovation

Competition is another reason why Bing needs to exist. Healthy rivalry does not let players sit on their hunches. It spurs the urge to innovate and to do better. Blackberry learned that lesson too late.  Microsoft’s Web Based Office 365 (and recently announced real-time collaboration), are two current examples.

 Note the differences between how Bing and Google describe Bing

 The problem that is generally associated with Bing, is that it continued, for far too long, into “me too” mode, trying to grow by doing the similar things as Google. It did not create much deeper differentiation.

Bing found it difficult to shake off those colors!

 Where search engines have not gone before

Search is one of the fundamental processes of discovery. But it is, by no means, need to be uni-dimensional.   

Facebook is proving this by building their Graph Search. By thinking differently, Bing could grow into other, less explored frontiers in searching, area that will be as natural as Googling something today.   

It is not that Bing lacks the brainpower to step into the unchartered waters. It is their managers that lacked the will (or vision) to venture forth.

Google is 15 and still lonely!

-->

Google celebrated its 15^th birthday on September 27.  The search giant chose to celebrate the day by announcing one of its biggest algorithm update, Hummingbird. This algo is designed to answer complex queries , in a conversational style. As an apt companion of Google Now, Google hopes that voice search on smartphones will be the focus growth area. I wrote about the Hummingbird update here.

 (image courtesy Forbes.com)

A quick look, at the search leader's evolution over the fifteen years, reveals the road that Google has traveled.

When you chart the revenue and growth in the search volume, over the last six years, an amazing picture emerges. In the (cluttered) chart below, I have superimposed some new features/products 

The search volume data has been taken from StatisticBrain. They have compiled this from comScore. Revenue data has been taken from Google's Investor Reports  

[caveat: I have taken the search volume for United States only, while the revenue numbers are Google's total revenue].

As you see from the graph above, Google continued to innovate across a wide spectrum of  activities. Its competition made only half hearted attempts to catch up, that too only on one single front - the search.

An Efficient Machine

Google has been growing its revenue strongly, in line with the growth in the web search volume (CORR. COEFF = 0.97 between 2006-2012).  Clearly it has been able to maintain a very high level of efficiency in converting the increasing search volumes into revenue.

Of course, it helps if you have cornered the market, and continue to be the undisputed search leader. But it is also a testimony to Google's strategy, that it has helped it retained the crown for now 9-10 years in running, while the competition got scattered.

Thinking Big, and building an intellectually capable, team that does not shy away from solving the giant scale problems, have helped Google in outdistancing the competition.

Taking search deep within the pyramid will open a new frontier

Search behavior in South East Asia and some underdeveloped regions of the world has a strong potential to expand. More and more people are getting mobile phones and hopping onto the carrier networks.  However, search in these regions, may not follow the same trajectory as in the situation where access to internet and smartphones is widespread.

Internet penetration generally grows at a much slower pace than the voice circuits, given infrastructure and resource constraints. 

 

Mobile phones have the deepest penetration, and in countries like India, are the most pervasive of all devices.  While smartphones are spreading like wildfire, their presence is more confined to urban centers or rural elites.  Some statistics place over 920 million total mobile connections, 44 million (5%) of which are smartphones with some sort of data connectivity. This leaves a large number of mobile phones with no access to search engines.

The language bottleneck is also a barrier to search when a large mass of population is not very proficient in the major internet languages. 

Search Engines have not been able to crack the language issue very well. Google has done some work where the search terms being keyed-in are translated into the selected language, and the results are displayed partially in the language selected.  The problem the search engines face is also due to lack of web assets in the  language of the query in their index.  This limits the quality of the results.

A moment reflection would show that this is really a chicken-and-egg situation. With almost no one to query, there is little incentive for anyone to put up a web property in the language of its users.

SMS based search

Traditional web search requires a browser running on desktops (or laptops or smart mobile phones).  The search index is queried via keywords entered into the search engine's browser interface.  The results are returned by the search engines, and are displayed in the standard fashion in the browser.

Search on mobile devices and smartphones follow a similar paradigm. One variation of this is voice-activated search on mobile devices (the famed SIRI) where both the query and the results are “spoken.”  However, voice agents are more geared towards “finding” rather than covering a broad canvas of “searching.”

The SMS functionality that all basic phones have can help solve this problem.  A user sends in his query via a text message to the search engine. The search engine responds by sending, say top 2-3 results customized for display on the small mobile screen.  The call-to-action (CTA) could be the ones appropriate for the device like - send a text message or place a call.  

Wait! This is not a new concept!

Google did launch a text based search interface. The service sms.google.com went live in 2004, and was shut down in May 2013! This service involved live agents who received the query, fished the results out, and sent those back to the phone. 

Few years ago another search engine ChaCha.com attempted something similar.  They had “guides” who received and responded to the user’s query via text messages. Last I know is that they are still active in US but have closed their shop in UK.

Then again three years ago a brave soul in India started Askbysms.com – a similar service. This also shut down soon after.

Why attempt something that Google, ChaCha and others could not succed? I believe the answer lies in targeting.  The search providers thought that SMS based searching would be a cool alternative to their audience who have internet connectivity via desktop or smartphones.  They did not have in mind the large swath of population for whom this could very well be the only way of searching. 

Airtel, the largest mobile service prvoder in India has SMS Gyan, a text based search service that is still running. However, a user has to pay Rs 1 for every query. 

The most promising start in this field appears to be from a young enterpernure who lauched Innoz. It will be interesting to see how this evolves.

Note that the search use case here does not involve complex queries. The person is probably interested in finding quick information about some local service to consume in his area – repair/price/availability of food/some commodity and like. Unlike browser based local search which is accompanied by maps and reviews, the SMS based results can have limited address and say distance from a known landmark in the vicinity as a navigational aid.

Cost & Revenue?

There are two very apparent snags in this approach.  A user has to pay every time a text is sent. In contrast a desktop and mobile user pays nothing for searching (discounting the payment to ISP or mobile carrier for the data plan that is spread over a host of activities other than search....streaming videos, email...).  So paying for search via texting would appear to be an extra fee on the user.   

However this is not an insurmountable issue. The search engines could set the system up with carriers, that the they, the SEs, rather than the searcher, pay for the query by the user.  Alternatively using their (considerable) clout they can negotiate a very very small price to the user.

Is that absurd? Studies have shown that it costs carriers next to nothing  to transmit a text message. These 160 character messages do not taken any space in the valuable wireless spectrum.  Search Engines, given the weight they can throw, would be able to work out a far more favorable price for themselves, much lower than say $0.10/ Rs. 10 per text.

Still absurd? Hardly so! 

Ad Revenue?

Now let us examine the questions of monetizing search via sponsored ads. Search is free because the search engines make money  in advertising.  In SMS based query the search results can be formatted inventively, so that first 120 chars are reserved for displaying the top 2 search results, leaving some room (40 chars) for showing the one ad that wins in the real time auction. If the user chooses to scroll for more results (say the next 2), the second ad could show up. 

Will 40 chars be enough? Google AdWords desktop search ads take 25+35+35 = 95 characters plus display URL.  In our case display URL is not needed because there is no web browser (or internet) to navigate to. A user should be able to connect to the sponsoring party via text or a call.

How about language?

 

Mobile phone manufacturers in India have introduced local language keyboards that are quite intuitive. Users do easily send or receive texts in their own language. 

When user start searching in their language, the local businesses/organizations will start hosting web assets in their language. The higher search volume will bring in advertisers as well.  When people who are the lower rungs of income pyramid start searching, the business segment that serves them will follow suit.

Today there is little evidence that the small businesses are motivated enough to get into search engines index. There is a strong perception that advertising on Google is expensive and unaffordable for street corner shop.  Makes sense as small businesses go where their customers congregate. A SMS based search will draw in users in large numbers, so even with relatively lower per capita advertising revenue, the total yield will not be insignificant. 

Why go to this length?

  The alternative is to wait till Internet infrastructure expands and people shift over to smartphones to avail of the existing search process.  The latest report shows that between April to June 2013, smartphones took 51.8% share of the total phones sold. The flip side of this report is that 209 Million basic phones were still sold throughout the world. There is (and will be) a large mass of population that will continue to use feature phones for a long time to come.

People obtain information either by searching or by asking. The basic phone users usually resort to the latter, since they do not have access to the former.

Several studies have shown that search creates economic benefits for users. Taking search down to the most common denominator will be equitable and will create a win-win situation for everyone, besides narrowing the digital divide.

 

Search Engines still a Frontier?

 

Flurry of “activities” related to search engines and website optimization drew me  to the digital marketing world.  The current hot flavor (actually has always been) is digging out the factors that Google (supposedly) takes into account in order to rank websites higher in the SERP.  

Mining activities range from the task of compiling 200 factors based on what SEO pundits have been talking about  (more importantly what is coming from Google’s Spam Master +Matt Cutts) to running yearlong correlation experiments on a large scale (I wrote about deconstructing the ranking factors last week in my other blog post) . 

Then there is an elegant “Periodic Table” compiled by +Danny Sullivan (from Search Engine Land) that groups and lays out these factors. 

                                                   (See the bigger version at SearchEngine Land)

The idea of second guessing Google is not very attractive (and may not be very productive as well).  It is obvious from the constant series of algo updates from the search giant, that it  is  maintains the integrity of the Page Rank  jealously, and is thwarting all attempts to outfox its search results.

There is one aspect that deserves some thought though. Google has always been consistent in its advice that the website owners should stick to creating quality content that is useful to users, unique and engaging. The attempts cited above should be better aimed at finding factors (and parameters) that help the website content and usability measure up against that goal. 

It may be more useful to gauge the visitor's view of your website than to focus on outsmarting searchbot's clinical perspective.

Playing with just mechanics results in a constantly moving goalpost.