Over the course of the last few years there has been a steady increase in the number of data breaches resulting from compromised API keys. Bad actors seem to be able to quite easily get ahold of API keys, certificates and tokens in order to access sensitive data. These are often the product of a lateral attack where adversaries have already gained access to the network and move across hybrid environments to uncover sensitive data.
In this blog article, we will walk through some notable examples of breaches that have been in the headlines over the last few years. We’ll then discuss the theme that is driving each one of these breaches – and no, it isn’t the just API secret that was used to get access to data in each case. The repeating theme you will see as we walk through each of these examples is the ease with which the API secret was compromised. It’s almost as though these API secrets were wrapped up in a pretty bow and gifted to the bad actor. API driven eco-systems behind many cloud-native solutions today provide an inherent risk.
On one hand, APIs offer organizations a terrific way to send, receive and manage data across a large landscape of applications and services, serving essentially as a communication contract into an service or application. They give developers the ability to move fast and develop focused services and application building blocks in a way that wasn’t possible before. To securely make API calls, essentially authenticate to an API, traditional options today include API keys, API tokens, and TLS certificates for client-side verification. The challenge here is that these secrets are growing in number as the number of API channels explode making workflows like secret provisioning and sharing, secret management, monitoring and control. These secrets often move across code repositories, continuous integration and continuous deliver (CICD) systems, test infrastructure, deployment applications and data lakes, logging tools or even team collaboration apps like Slack. The ability to access these secrets is becoming easier and easier for adversaries as they are long-lasting, sprayed and shared across environments, across applications, across pipelines. Once these secrets are accessed, they provide keys to the castle.
API Breaches from Compromised Secrets
One of the most notable API attacks, and the first of many, dates back to October of 2019 when Imperva suffered a high profile breach as a result of a stolen AWS API Key. On their post recapping the events that unfolded, Imperva wrote:
“Some key decisions made during the AWS evaluation process, taken together, allowed information to be exfiltrated from a database snapshot. These were: (1) we created a database snapshot for testing; (2) an internal compute instance that we created was accessible from the outside world and it contained an AWS API key; (3) this compute instance was compromised and the AWS API key was stolen; and (4) the AWS API key was used to access the snapshot,” they explained in their post.
The first step leading to the breach was the fact that an internal compute instance was left open to the outside world. While sometimes this may be done unintentionally, the deeper vulnerability point for Imperva was exposing a long-lived, static AWS API key in that instance.
Just a few months later, MGM suffered a breach that disclosed personal details and contact information of 10.6M guests in the summer of 2019. The breach was a result of poor API security practices, namely a developer who left credentials exposed in an API usage document, according to researchers.
In December of 2020, Ledger suffered a breach that resulted in personal information of over 270,000 customers, including phone numbers and physical addresses being disclosed. Their CEO was able to attribute a stolen API key as the cause of the breach
“It’s a wrong API key that got coded on the map client to import the database from the store that got coded in the wrong placements and so, therefore, was coded where it should not have been coded and exposed the database to a simple attack,” explained Gauthier.
In May of 2021, security researchers found a significant vulnerability in Peloton’s API whereby someone could make an unauthenticated request for user account data, without checking to make sure the person making that request was allowed to do so. This exposed API let anyone access a Peloton users’ personally identifiable information (PII) including birthday, gender, age, weight, city and workout statistics.
Shortly after the incident, Peloton received some backlash for not being transparent about whether or not user credentials had been compromised and also for the way they approached fixing it. The way they remediated the problem was to only allow API access to authenticated users. This meant that no longer could just anyone could access this user data, but rather only those with Peloton accounts. This still meant that any Peloton subscribers could still access all of the data of the other users and the larger API access issue remained.
Travis CI was at the center of two security incidents in less than a year. In September of 2021, the continuous integration and delivery provider disclosed a vulnerability that could have leaked secure environment variables that included access credentials, API tokens and signing keys. There was backlash in the development community because of the way of handling the incident (“Rotate your keys”) without acknowledging that it was Travis’ infrastructure that was causing these issues.
Earlier this year, Travis was again at the center of news headlines when a security flaw in their API left tens of thousands of developers’ users tokens and other sensitive information exposed. Bad actors could easily leverage these credentials to launch attacks in cloud environments including GitHub, AWS and Docker Hub. The issue dated back several years but the vulnerability could still be exploited in order to launch lateral attacks. Researchers were able to access more than 770 million cleartext logs full of sensitive data.
So what do all of these have in common (Other than the compromised API secret?) The theme in each of these cases is a tale as old as cloud — Certainly the speed, agility and flexibility that cloud technologies offer engineering teams increases the ability to ship products faster Unfortunately security considerations often become an afterthought. But it’s much more than ‘developers want to move fast and security wants to slow them down.’ It’s important for security and development teams to recognize that risk is predominantly shifting from human to machine and the consider what needs to be done account for that.
Before Infrastructure as Code (IaC) has become a common way to manage and provision infrastructure, the security posture of your cloud infrastructure was most at risk when someone logged into a console and spun up compute instances or left an S3 bucket open. While automation has provided enterprises with the ability to optimize efficiency and scale for many development and deployment processes, it has also done nothing more but shift much of that risk from a human logging into a console to manually spin out infrastructure, to the machines, and the APIs that infrastructure as code systems leverage to provision those workloads.
We’re past the days where a developer can write code, push it to a VM and call it production. CI/CD pipelines have helped development teams securely shephard code through code repositories, CI and test systems and orchestration tools – and they are ALL driven by APIs to automate that pipeline. With automation and APIs driving the complex pipeline, it’s no wonder that these keys are winding up in open S3 buckets, hard coded and pushed out to artifact repositories or leaked in code repositories.
Engineering teams are still trying to and should move fast to the cloud.he benefits of microservices architecture, serverless, as well as multi and hybrid cloud offerings have only exponentially increased the number of APIs that are required to power these environments and our call to action to you today is to stay vigilant about your API authentication and access and security hygiene about API secrets..
