Pichai began the event by acknowledging the gradual shift in how users interact with technology. Moving from mice and multi-touch to voice and VR. Google Lens fits into the latter of those categories.
The Google Lens app uses a smartphone’s camera to identify plants, buildings, and other kinds of objects before showing users what they can do with that information. For example, by holding the camera in front of a router, Google Lens will scan the Wi-Fi username and password then automatically connect the phone to the network. Also, the camera can extract information from movie/theatre posters to add events to Google Calendar accordingly.
The Google Lens app is a natural progression from Android N’s Google Assistant which analysed the information from any app to show users more information and functions relating to it. Instead of the screen, Google Lens takes the information from the world around you. It doesn’t just rely on the camera though. It pulls information from location services, the gyroscope, and other useful sensors to show augmented information.
Google’s VP of Engineering, Scott Huffman, demonstrated how Lens could be used to translate foreign words in real-time augmented reality however this is a feature that Google Translate and other similar apps have had for years now.
In the future, Google says Lens will be able to remove objects from a photograph - even large ones. In their demonstration, Huffman removed the chain fencing from a picture of a girl playing baseball. It is unclear whether this was a real example or simply a mock-up though.
Google Lens also works for pictures that you have already taken. By browsing your Google Photos albums, Google will be able to identify buildings, cars, and paintings to offer context. Google is also making it possible to extract website addresses and phone numbers from screenshots. OCR technology is nothing new but convenient I suppose.
The Photos app will also encourage you to share pictures with the people in them. Using facial recognition, Photos will scan faces and prompt users to share them with those people with a click of a button. Others can then add their photos to the album. Facebook currently runs a similar service however it is unlikely that Facebook Moments has anything close to the 500 million active monthly users like what Google is boasting.
The second new major feature in Google Photos is shared libraries. Users can now automatically share pictures of a certain person or from a certain date with someone they are close to. For example, if you want all the pictures you take of your children to be automatically shared with your wife, Google Photos can now do that. It can even share your whole library.
Shared libraries is a feature that is all shits and giggles until you get too close to your secretary at work. She invites you out for ‘work drinks’ which you assume is platonic but she has other intentions. Before you know it, you’re staggering out of a bar, half a bottle of wine still in your hand, and hailing a taxi to go back to her place for a night you’ll never forget. She suggests you take a playful selfie in the back of the cab, you put your arm around her and strike a pose kissing her on the cheek. Then BAM! It’s automatically shared with your wife who has spent all evening looking after your two young children - one of which is really struggling in school, you arsehole. What I’m getting at is that shared libraries can only end badly.
Google has thought about this though and are now offering a printed photo album service. It is called Google Photo Books and it will select your best photos to get printed in a physical album. A perfect present for your partner who you’re desperately trying to win back.
By in large, the updates to Google Photos and Google Lens seem to only build on what other services already offer and the innovation is small. Regardless, it is helpful that Google is pulling all this technology together and it is reassuring that Google continues to update these services. Google Lens will be available on smartphones running Android O however to me Lens would be suited better for an augmented reality headset more than anything else – a product line that Google has not touched since Google Glass in 2011 (which is now dead).