Spotify dances to the beat of open source

We’re excited to bring back Transform 2022 in person on July 19 and virtually from July 20-28. Join leaders in AI and data for in-depth discussions and exciting networking opportunities. Register today!


Leave him OSS Enterprise Newsletter guide your open source journey! register here.

Just about every tech company under the sun wants to align with open source, whether it’s Facebook open-sourcing its own internal projects or Microsoft doling out north of $7 billion to acquire the one of the biggest platforms for open source developers — GitHub.

Spotify is no different. The music streaming giant has open source a number of his projects over the years, as In the wingswhat was recently accepted in incubation at the Cloud Native Computing Foundation (CNCF) after two years as an open source project. The company also recently joined the Open Source Security Foundation, opened a dedicated open source program office, and is now launching a fund to support independent open source projects.

In short, Spotify is doubling down on its open source efforts.

Open for business

There are many reasons why a company may choose to open up its internal technologies or contribute to those maintained by other companies or individuals. For starters, it can help engage the wider software development community and is a useful recruiting tool. A company can also contribute resources to community projects where it plays a central role in its critical infrastructure, to help strengthen security, for example.

Backstage, on the other hand, is about creating custom “developer portals,” unifying a company’s myriad of tools, services, applications, data, and documents into a single interface through which they can access at the console of their cloud providers, troubleshoot Kubernetes and find all the documentation they need for their daily work.

“The problem Backstage solves is complexity — the kind of day-to-day complexity that can really bog down engineers and their teams, which then slows down your entire organization,” said Tyson Singer, Spotify’s chief technology and platform officer. , to VentureBeat. “Backstage as a product and as a platform is really about creating a better experience for engineers – streamlining their workflows, making it easier to share knowledge, and eliminating messy parts of the infrastructure. makes it easier to focus on creating business value – innovative products and features.

Spotify Backstage: An Open Source Platform for Developer Portals

Today, Backstage is used by dozens of companies, covering retail, gaming, finance, transportation, and more, including Netflix, American Airlines, IKEA, Splunk, HP, Expedia, and Peleton. But in the end, what does Spotify get from the open source Backstage? Well, for starters, it gets a better version of Backstage for itself due to the community nature of the project.

“Let’s imagine the counterfactual, where two years ago we didn’t open Backstage, and instead we invested the same amount of internal resources into it that we got from the external community – and based on the ‘huge community engagement so far would have been a huge investment and difficult to fund – it still wouldn’t be as good a product as it is today,’ Singer explained. viewpoints and use cases, ranging from adoption by companies like the world’s largest airline or a fast-growing financial startup, individual contributors and third-party software vendors, improved the product, the making it more robust and allowing the platform to keep up with the pace of changes that occur both inside and outside of a particular business.

But on top of that, Backstage being embraced by some of the biggest companies in the world also indirectly benefits Spotify, as it ensures that its own product is among the de facto “developer portal” tools.

“If we hadn’t opened [Backstage], we would be the only ones using and depending on Backstage,” Singer continued. “If eventually another open source solution emerged, we should have migrated to that solution, as community-powered innovation eclipsed our ability to keep pace.”

To support its ongoing open source efforts, Spotify has joined a long legion of companies to launch a dedicated Open Source Program Office (OSPO), designed to bring formality and order to all of their open source efforts, aligning OSS project goals with key business objectives, manage licensing and compliance issues, and more.

Winding

Spotify actually had something of an OSPO for the better part of a decade already, but it was more of an informal group of employees who had other full-time roles in the business. Going forward, the company now has a full-time OSPO manager in By Ploug and is actively recruiting for other roles.

So, until now, Spotify’s open source work has been driven primarily by the “passion and commitment” of the company’s engineering teams, according to Singer.

“The enthusiasm was always there, and we just needed to channel it,” Singer said. “A dedicated OSPO brings more clarity to this process for everyone, including what expectations are and what kind of support should be expected. This ensures that our efforts are properly prioritized and integrated into the way we work. We want the treat [open source] with the same level of ownership and dedication as we do with our internal applications – creating a formal OSPO allows us to do this.

Spotify’s OSPO is positioned within the company’s “platform strategy” unit – however, it will end up straddling multiple teams and departments as open source software cuts across everyone from engineering and security to legal, HR and beyond.

“Engineering teams have their areas of expertise, but we want our OSPO to span multiple teams,” Singer said. “The best position to do this is within our ‘platform strategy’ organization, which is the connective tissue between the various R&D teams. It gives OSPO visibility and independent positioning within this framework. This very much represents how closely open source is to the ways of working not just in Spotify, but indeed in any modern tech company.

A core element of any OSPO is security – ensuring that any open source element of the company’s technology stack is secure, is updated to the latest version, and also complies with the terms of the open source license. It is therefore perhaps fitting that Spotify recently joined the Open Source Security Foundation (OpenSSF), an industry-wide initiative launched by the Linux Foundation nearly two years ago to strengthen the software supply chain.

With regular members including Google, Microsoft and JPMorgan Chase, Spotify is in good company, and its decision to join follows the critical Log4j security bug that was exposed late last year. The OpenSSF also highlights how open source has become the de facto model for cross-enterprise collaboration – everyone benefits from more secure software, so it makes sense for everyone to step up.

“Open source security is a topic that affects all technology companies — or, really, all companies that rely on software,” Singer said. “We all depend on the open source ecosystem, so as a technical community we all have a responsibility to improve security wherever possible. Like when we joined others in creation of the Mobile Native Foundation, we see the problem as a problem of scale – how do we create solutions that can affect not just local problems, but an entire landscape? We believe that participating in foundations – in collaboration with other large companies that reflect on the problems and opportunities for scale within their businesses every day – makes a lot of sense for finding scalable solutions.

show me the money

To further align with the open source realm, Spotify today lifted the lid on a new fund for maintainers of “independent” (i.e., not Kubernetes) open source projects. The Spotify FOSS fund will start at €100,000 ($109,000), with the company’s engineers selecting the projects they believe are most deserving of the funds, and a separate committee making the final decision. The first tranche of selected projects will be announced during the month of May.

“The idea for Spotify’s FOSS fund was born out of asking ourselves, what could we do to help support the quality of the open source code we all depend on?” Singer said. “It’s only natural that big tech players play a role in supporting the open source ecosystem. We use it, we contribute to it, we build projects that others can contribute to and depend on – we believe it is important and necessary for us to contribute to the success of this community.

However, €100,000 is not a huge sum in the grand scheme of things. Over the past year, we’ve seen Google pledge $100 million to support foundations such as OpenSSF and commit $1 million to a Linux Foundation open source security program. Recently, Google also partnered with Microsoft to fund another security program called Project Alpha-Omega for an initial $5 million.

But it is perhaps unfair to compare support foundations and larger projects with smaller-scale “independent” projects that receive no financial support. Plus, it’s still early days for the Spotify FOSS Fund, and it’s likely to evolve, which could mean a bigger pot.

“The fund will start with €100,000 – the keyword being ‘start’,” Singer explained. “We are ready and eager to grow the fund, but we are using this initial amount to help us gauge what kind of impact we can have. Funds will be distributed to ensure maintainers have the financial means to continue to maintain their projects, fix security vulnerabilities, and continue to improve the codebase. We will target independent projects that are actively maintained and relevant to our work here at Spotify.

VentureBeat’s mission is to be a digital public square for technical decision makers to learn about transformative enterprise technology and conduct transactions. Learn more about membership.

Comments are closed.