Amazon Web Services unveiled Amazon Augmented Artificial Intelligence (A2I), a fully managed service that makes it easy for developers to build workflows that use human reviewers to validate machine learning predictions. Human reviewers can be added via Mechanical Turk, third-party vendors or the developer’s own employees. The developer can also use Amazon A2I to structure the review process and manage the human reviewers. Users do not need to commit to use Amazon A2I, but instead pay only for each review needed.

CXOtoday reports that users can visit Amazon’s A2I site here. Today, developers that use machine learning rely on its highly accurate predictions (known as inferences) and a confidence score, with a high number indicating trusted accuracy.

A social media site, for example, can rely on an 80 percent confidence score to match a user’s photos with celebrity faces. But other applications, such as public safety involving law enforcement, require a confidence score up to 99 percent and human judgement to “validate the prediction.” Human reviews, however, are “challenging and expensive to build and operate at scale,” and developers either spend more time managing the human review process than building the application — or abandon it, “which leads to less confidence in deploying applications that utilize machine learning.”

Amazon A2I offers “over 60 pre-built human review workflows for common machine learning tasks … that allow machine learning predictions from Amazon Rekognition and Amazon Textract to be human-reviewed more easily.” Developers who use Amazon SageMaker (or any other on-premises or cloud tools) to build custom machine learning models, “can set up human review for their specific use case in the Augmented AI console or via its Application Programming Interface (API).”

Developers next set a “confidence threshold for model predictions” and can then “choose to have predictions below that threshold reviewed by Amazon Mechanical Turk and its 500,000 global workforce of independent contractors, third-party organizations who specialize in business process outsourcing … or their own private, in-house reviewers.”  Developers can also “specify the number of workers per review and Amazon A2I then routes each review to the precise number of reviewers.”

At AWS, Amazon Machine Learning vice president Swami Sivasubramanian noted that, even with machine learning tools SageMaker, Rekognition and Textract, “our customers still say there are critical use cases where human judgment is required like in law enforcement investigations, or times when human review can be used to resolve the ambiguity in predictions.”

“Amazon A2I … makes it significantly easier and faster to incorporate human judgment into machine learning applications in order to ensure higher quality predictions over a sustained period of time,” he said.

At T-Mobile, machine learning engineer Heather Nollis agreed. “Using A2I, we will be able to ensure that our models continuously deliver top-quality insights by having humans validate random samples of model predictions,” she said. “Trust is the hardest thing to build when it comes to machine learning, and A2I will allow us to make sure that our models are making the fewest mistakes.”