Google made a subtle announcement today on Twitter: it’s in the process of rolling out new AI features for its Lens platform that will let you search your Google Photos library for text that appears within photos and screenshots. Then, you’ll then be able to easily copy and paste that text into a note, document, or form.
Both of the new features make use of a technique known as optical character recognition (OCR), with the copy/paste option building on Lens’ existing ability to understand and pull out the text found within photos, be it a screenshot or a photo of a physical sign or document. According to 9to5Google, that feature is available now on some Android devices, although it does not appear to be active quite yet on iOS. You may already be able to search your photos for text using Google Photos on the web.
Google is positioning the Lens feature as a way to easily snag a confounding Wi-Fi password, but it can also be used to grab any portion of text from any photo or screenshot in your Google Photos library on a case-by-case basis. The more impressive feat, though, is the ability to search for any snippet of text without having to pick a photo first. That suggests Google is performing OCR on your entire photo collection automatically.
Wow, @googlephotos has OCR to turn screenshots into copy/paste text!
A. Open google photos and select screenshot
B. Pick “Lens” feature [image 1]
C. Highlight text [image 2]
D. Pick copy/paste [image 2,3]Nicely done Google team! pic.twitter.com/Um49ika2yT
— ☕️ (@hunterwalk) August 21, 2019
“Starting this month, we’re rolling out the ability to search your photos by the text in them,” Google wrote in response to prominent venture capitalist Hunter Walk, who noticed earlier this week that the Lens feature had been turned on for his account. “Once you find the photo you’re looking for, click the Lens button to easily copy and paste text. Take that, impossible wifi password.”
You spotted it! Starting this month, we’re rolling out the ability to search your photos by the text in them.
Once you find the photo you’re looking for, click the Lens button to easily copy and paste text. Take that, impossible wifi passwords
— Google Photos (@googlephotos) August 22, 2019
The features seem related to one announced back during Google’s 2017 I/O developer conference. At the time, CEO Sundar Pichai celebrated a new, in-the-works feature that would let the AI-powered Lens camera understand the context of a photo. That way, you could point your smartphone camera at a router’s password sticker and get automatically connected to Wi-Fi by way of the software reading the password and understanding that you’d like to connect to the network using it.
Since then, Google Lens has added the ability to search the web for photos of menu items, add contact information from a business card to your address book, and all manner of other OCR-related tricks. But these new Photos features seem like a more large-scale deployment of the AI technique, done so on a user’s entire library to make it searchable and to give you the ability to pull that text out of photos.
