August 8, 2023
Spotify is rolling out its new Confidence software development product in private beta. Based on its learnings from more than 10 years of experience experimenting at scale, the company says Confidence aims to make it simple for teams to set up, run and analyze their own user tests so they can quickly perfect and productize ideas. “Whether it’s automatically coordinating simultaneous A/B tests or orchestrating the rollout of an AI recommendation system across mobile, desktop, and web, the platform we built scales experimentation best practices and capabilities,” Spotify explains, adding that it plans to make the technology available soon to all companies.
Spotify based the Confidence system on its internally developed A/B testing platform, dubbed ABBA, which did feature flagging and analysis based on standard metrics. Spotify says ABBA’s efficiencies “unlocked a wave of experimentation across the company” that allowed it to increase testing “from running fewer than 20 priority experiments per year to running hundreds of experiments per year across multiple squads,” according to TechCrunch.
Confidence will be made available to customers in three configurations: 1) as a managed service accessed as a standalone web service managed by Spotify; 2) as a Backstage plugin; 3) or integrated via API into a user’s own infrastructure. A Spotify engineering blog post by Spotify’s VP of Technology and Platforms Tyson Singer provides further detail as to how each of the three may be suitable to a development use case.
“Throughout our journey, we’ve compared notes with other companies struggling to scale reliable experimentation practices within their organizations,” writes Singer. “Often these companies have outgrown their existing A/B testing tools (whether purchased off-the-shelf or built internally) and are now seeking greater customization for how they run experiments.”
“Confidence is designed to bring value whether you’ve outgrown your current testing platform or are looking for a quick, easy way to get started with A/B testing that will scale with you as your needs change,” he adds.
Software development teams can now “sign up for the waitlist to be eligible for an invite and to get updates on features,” Spotify says.
TechCrunch writes that “Confidence is Spotify’s second commercial product for monetizing developers,” following the December announcement of its plan to “monetize developers via its open-source Backstage project.”
Spotify Backstage allows companies “to build customized ‘developer portals,’ combining all their tooling, apps, data, services, APIs and documents in a single interface,” TechCrunch reports, adding that the Confidence announcement “indicates that although Spotify is mainly known for its consumer-facing music streaming service, it’s looking to branch out beyond that notion.”