Google, Image Recognition & Search

The Midnight Epiphany That Changed From An Over-Hyped Failure To A $100 Million Acquisition (Fast Company)

On April 25th, 2006, just after midnight, Shah was looking at Riya’s stats when he noticed that for every person uploading personal pictures and tagging them, 20 others were using Riya for search. He immediately emailed one of his cofounders, Gokturk, who replied within five minutes. “I wonder what they are searching for?” Gokturk wrote back. “Why are they using our site instead of Google or Yahoo Images?”

Read the whole thing, then see: Biometrics, object recognition and searchinspired by an earlier Google acquisition.
Key paragraph:

With this acquisition, I suspect Google doesn’t so much have facial recognition for identity search as they have object recognition in mind. First, Google has been wary of face recognition in public search. Whether this is due to the technical challenges of an unbound face rec application or a respect for the privacy of their users, I’ll leave it to the reader to judge. It is also a much different challenge to return the result “This is a human face” than it is to say “This is a human face and that face belongs to Guy Herbert.” In most object search the first type of result will be the most desirable anyway. If you submit a photo of an insect to a search, you aren’t asking about the insect’s individual identity, you probably just want to know what type of insect it is.”

The current case, well documented by Fast Company, seems to bear that out.

What if? Online Real-Time Searchable Sensor Data

Each of these articles is extremely interesting and when they’re read together, they become even more so.

The first one was published in the Washington Post a week or so ago and concentrates on industrial control systems (probably because Stuxnet has been in the news a lot lately). The second article below talks about the development of a search engine that could combine social networking with data collected by sensors that are hooked up to the internet.

Cyber search engine Shodan exposes industrial control systems to new risks (Washington Post)

It began as a hobby for a ­teenage computer programmer named John Matherly, who wondered how much he could learn about devices linked to the Internet.

After tinkering with code for nearly a decade, Matherly eventually developed a way to map and capture the specifications of everything from desktop computers to network printers to Web servers.

He called his fledgling search engine Shodan, and in late 2009 he began asking friends to try it out. He had no inkling it was about to alter the balance of security in cyberspace.

“I just thought it was cool,” said Matherly, now 28.

Smart City Search Engine Uses Sensors (Tech Week Europe)

Researchers at the University of Glasgow are helping build a search engine that will combine data from social networks with real-time sensor information such as recognition of faces in crowds to help users locate individual people or events.

The European-funded project, called Search engine for MultimediA Environment geneRated contenT (SMART), takes advantage of the already widespread presence of sensors such as CCTV cameras and microphone arrays, according to Dr. Iadh Ounis of the University of Glasgow’s School of Computing Science.

The combination of what these two efforts envision — a crawler that finds online devices, an engine that makes them searchable, datamining of social media and real-time access to sensor data — would really be something.

But what would it take for someone sitting at a computer terminal to find and commandeer a surveillance camera, grab an image of my face, run it through a facial recognition search of social media platforms and find out something as simple as my name?

For now, it would be pretty difficult. Without significant help from disparate entities, the challenges associated with such a query are extremely daunting and that will probably be true for the foreseeable future.

A simple facial recognition search of all the photos on (for example) Facebook’s servers would be pretty close to useless. The ‘book simply has far too many faces. Based upon the (low) image quality from surveillance cameras and the (high) number of Facebook photos, there would be far too many false positives. I’ll make an educated guess that the reason Facebook gets the facial recognition results that it does is that it uses its (highly proprietary) knowledge of its users to limit the face rec search only to people that Facebook already believes have a significant likelihood of actually knowing each other. So, without Facebook’s help, that random someone sitting at their computer would have a pretty difficult challenge even if their target is a heavy user of social media.

Other challenges apply. Finding a device online is not the same as controlling it. Controlling some functions of a device such as a surveillance camera doesn’t necessarily imply that all functions are available to the user. Speed and storage are also issues.

Nevertheless, some challenges, such as the co-operation of service providers, can already be overcome by governments. Others will become easier to overcome as technological progress is made.

What is possible? That’s an interesting question. What does it cost? That may be far more important. Stuxnet proved that some amazing things are possible. As for pulling a mini-Stuxnet to see if I’m over-sharing on the social webs, theoretical possibility may not be the most important consideration. A private detective is still the way to go.

But if Moore’s law holds and techy things continue get cheaper, better and faster, who knows?