Google Search Will Use MUM AI to Combine Text and Images

Google Lens visual search will be updated to incorporate the company’s new AI technology, the Multitask Unified Model (MUM), which understands context and draws from various formats, including text, images and videos. With MUM, users will be able to incorporate text in order to specify queries on visual search. For instance, you could use your phone to snap a photo of a favorite shirt using the Google Lens feature — or find a shirt you like through Google Search — then tap the Lens icon on the open image and type in “socks with this pattern” to search with specificity.

“By combining the image and the words into one query, Google may have a better shot at delivering relevant search results,” writes TechCrunch, citing a search for repairing a specific bicycle part: “you don’t know what the piece is called. Instead of delving into repair manuals, you could point Google Lens at the broken part of your bike, then type in ‘how to fix.’ This could connect you directly with the exact moment in a video that could help.”

The Google Lens update, announced at Google’s Search On event last week, will be deployed in the coming months. It is part of the company’s commitment to using AI to make its products smarter. It is also a way to keep Google Search at the forefront in a world where search is migrating to in-app platforms through Amazon or Apple’s App Store.

TechCrunch reports Apple is “developing its own alternative to Google Search,” evidenced in the iOS 15 update to Spotlight search that “now directly connects users to the information they need without the need for a Google query.”

Also announced at Search On was that “details and context about topics and sources” will be added to Google Search results in an effort to combat misinformation. Alphabet “will add descriptions about listed websites in its own words, reviews of sites from other parties, and information about topics from third-party sources,” accessible through the “About This Result” panel, according to Bloomberg.

“Lots of people are concerned about mis- and disinformation,” said Harris Cohen, a product manager on Google’s consumer trust team. “Across lots of areas, this tool really helps them with that journey.”

No Comments Yet

You can be the first to comment!

Sorry, comments for this entry are closed at this time.