Launching Peloton has certainly been a tremendous ride – one that’s included an exhilarating acceleration resulting in two-fold year-over-year growth and a tripled subscriber base in under a year. Among the steepest hills climbed along this journey, though, has been keeping up with our database requirements and pushing through the pains experienced throughout intensely-paced user growth.
Our challenge managing an exponentially growing user base has been kicked into a higher gear by the fact that our consumer product utilizes so much user data. The idea for Peloton was born out of the founders’ desire to bring studio-style, group fitness classes home that still included the feeling – and benefits – of actual in-person classes. In practice, this has resulted in storing vast volumes of personal fitness data. As customers exercise on their home bikes and experience live or on-demand cycling classes, we need to continuously collect and retain a swath of telemetric data that includes pedaling speed, heart rate, the selected resistance setting, and many other metrics germane to the experience. This data is important to both our product and our customers, providing feedback that helps us shape the user experience and enabling us to deliver performance information to each customer via statistics, graphs, achievements, lists of top rides and routines, etc.
To handle the data load, our small engineering team decided on open source NoSQL Apache Cassandra for its virtues as a massively scalable distributed database able to keep pace with current and expected growth stages. However, our engineering team did not possess deep in-house Cassandra expertise and were intrigued by the option of turning to a managed service strategy. We reasoned that doing so could free internal resources to focus on iterating and improving our product rather than on learning to run Cassandra at the future scale we would require.
Our pressing need for a bit more expert Cassandra management came to a head as our production systems began to suffer from performance issues. Our architecture made use of dense Cassandra nodes rather than a greater quantity of smaller nodes – the latter we later learned to be the superior practice. This caused an incident where a reconciliation issue between Chef and AWS required a boost in read pressures on existing clusters.
The performance snag served as the catalyst for a strategy change and a turn toward managed services, both to address our most immediate database performance issues and to support further scaling. For our own needs, and after vetting, we selected Instaclustr largely due to the service provider’s extensive Cassandra expertise. Strategically, we also liked the deeper open-source-as-a-service platform, which aligns with our preference – wherever possible – for open source solutions (for superior reliability and community support, to greater flexibility and cost effectiveness). Additionally, we felt it was important here to be able to freely use different public cloud vendors if needed (i.e. avoiding lock-in).
The managed provider route helped us diagnose and fix our production issues on day one; the overly dense production nodes and environment were adjusted for better (and noticeable) optimization. With our database back to running smoothly, we faced a choice as to how we’d acquire the Cassandra expertise we clearly needed in the long term: either investing to build that expertise in-house or relying on managed services going forward. Given the higher expenditures in both time and money, and the focus building an in-house team would take away from our product, it was an easy decision to stick with the managed provider route for our Cassandra production environment.
Our data growth shows no signs of slowing – data volumes continue to increase 8% each month, and our node count should double within a year to encompass more than 500TB of data. At the same time, we’re adding new data centers to keep up with our extensive active monthly users. But as we plan for the next stages of this ride, we’re now confident that with a managed Cassandra strategy, we can tap into the open source database in a way that maintains performance while enabling our engineering staff to more acutely focused on product development.
About the Author
Ben Wu is a Platform Engineering Manager at Peloton, a company revolutionizing the fitness industry by merging high design with modern technology to provide access to live and on-demand fitness group classes led by elite NYC instructors.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind