Meta Platforms has published a new AI technology, the Segment Anything Model (SAM) that the company claims can identify objects it hasn’t seen before. Acting on a text prompt, SAM will highlight items in a photo or video, picking out all the cats, for instance, or flowers. It can also execute other functions, such as generating a 3D construct using a single 2D image or extrapolating from things viewed in a mixed reality headset. Segment Anything can work in concert with other models, potentially minimizing the need for voluminous data sets for training.

The developers admit that in its current state, SAM “is flawed,” writes Engadget, explaining “it might miss finer details, and isn’t as accurate at detecting the boundaries as some models.” And while the model can handle real-time prompts, “it bogs down when demanding image processing is involved.” Meta concurs that specialized artificial intelligence tools will likely outperform Segment Anything when it comes to core functionalities.

“You aren’t about to see this AI in robots or other devices where fast, accurate object detection is (usually) vital,” Engadget writes, noting that “models like this may still help in situations where it’s impractical to rely exclusively on training data.”

For example, a social network could leverage this type of machine learning to keep pace with a large volume of rapidly changing content. Meta already uses SAM-like tech on Facebook and Instagram to do things like tag photos, moderate content and power its recommendation engine.

“The SAM model and dataset will be available for download under a non-commercial license,” writes Reuters, which says “users uploading their own images to an accompanying prototype likewise must agree to use it only for research purposes.”

In addition to fostering further research into foundation models for computer vision, “we aim to democratize segmentation by introducing the Segment Anything project: a new task, dataset, and model for image segmentation,” Meta explains in a blog post.

The company has simultaneously posted a research paper that includes a download link to both the general Segment Anything Model and Segment Anything 1-Billion mask dataset (SA-1B), which Meta calls “the largest ever segmentation dataset.”

“Meta is no stranger to sharing AI breakthroughs,” writes Engadget, citing “a translator for unwritten languages” that debuted in October. “That said, there’s pressure on the company to show that it’s as much of a powerhouse in the category as tech heavyweights like Google and Microsoft,” Engadget suggests, adding that “it’s already planning generative AI ‘personas’ for its social apps.”