Shattering the Rotation Illusion: Part 2 - Package Managers

The Hidden Playground: Secrets at Risk in Package Managers

Forty seconds. That’s how quickly an exposed AWS key on PyPI was exploited.

In less time than it takes to make your morning espresso, attackers had already acted—leveraging automation and relentless scanning tools to turn a minor mistake into a major security risk. Docker Hub painted a similarly alarming picture. Yet on npm, the silence was almost unsettling. Were attackers skipping it? Or are they simply focusing on the easiest, most accessible targets?

In our prelude, we unveiled the illusion of secret rotation: a practice that feels secure but crumbles under the speed and sophistication of modern attackers. Then, in the previous blog in the series, we exposed how secrets on GitHub and GitLab were compromised in mere minutes.

Today, we take you deeper into the ecosystem of package managers — platforms like Docker Hub, PyPI, npm, and Crate. They are essential to developers but equally enticing to attackers when secrets accidentally slip into published packages.

Our experiments show just how quickly attackers seize these opportunities, while the quiet around npm suggests something different: Attackers either aren’t scanning it yet, or they prioritize platforms like PyPI and Docker Hub which are tied to backend systems, or perhaps they don’t have mature pipelines for analyzing and exploiting secrets stored in npm. This is a plausible explanation, but we can’t know for sure. Lucky for now, but this doesn’t mean npm is safe. Security practices should stay ahead of attackers—not wait for them to evolve.

This blog unpacks the scenarios, timing, and lessons from these exploits, revealing why traditional defenses like rotation fail in the face of relentless automation. Let’s dive in.

Faster Than You Think: Attackers in the Package Managers Ecosystem

Scenarios

In the table below, we have listed all the scenarios we executed across popular package managers. We wanted to see if any specific package manager is more popular among attackers, and if any are not favored at all. We also wanted to take this opportunity to verify Cybenari’s recent research¹ on npm's popularity among attackers for finding secrets.

Platform	Scenario Name	Description
	Environmental Hazards	Exposed an AWS access key in a Docker image's environment variables and uploaded it to Docker Hub with relevant keywords and tags.
	Layer Leak	Exposed an AWS access key in between the layers of a Docker image and uploaded it to Docker Hub with relevant keywords and tags.
	Secret npm Nightmare	Published an npm package containing an AWS access key within the code.
	npm Deja Vu	Repeated the previous scenario in order to validate our findings.
	Replicating Cybenari’s Scenario	Replicated a Cybenari’s research, by publishing an npm package with a Canary token simulating an AWS access key.
	Python Oopsie	Published a PyPI package containing an AWS access key within the code.
	Python Oopsie 2	Repeated the previous scenario in order to reaffirm attackers' detection speed.
	Rusty Blunder	Published a Crate (Rust package) containing an AWS access key within the code.
	Rusty Blunder 2	Repeated the Crate package publication with a new AWS access key to see if it would be detected over time.

Results

The biggest takeaway is that there was no exploitation whatsoever on npm. This finding contradicts Cybenari's research and establishes with high confidence that attackers are not actively scanning npm packages for AWS access keys.

In contrast, our experiments on other package managers like Docker Hub, PyPI, and Crate showed varying degrees of exploitation. Docker Hub images were scanned and exploited rapidly, with attackers accessing the secrets within minutes. PyPI packages saw moderate scanning activity, and Crate packages had delayed exploitation, indicating that attackers might scan these repositories less frequently.

Platform	Scenario	Time to Exploit	Time to Alert ⚠️
	Environmental Hazards	1 minute	No alert sent
	Layer Leak	3 minute	No alert sent
	Secret npm Nightmare	No exploitation	1 minute
	npm Deja Vu	No exploitation	1 minute
	Replicating Cybenari’s Scenario	No exploitation	8 minute
	Python Oopsie	2 hours	46 minute
	Python Oopsie 2	40 seconds	43 minute
	Rusty Blunder	No exploitation	No alert sent
	Rusty Blunder 2	4 days	No alert sent

The Scenarios We Tested: How AWS Keys End Up in the Wrong Hands

Docker Hub Scenarios

Environmental Hazards

We exposed an AWS access key in a Docker image's environment variables and uploaded it to Docker Hub with relevant keywords and tags. Note: This specific scenario was tested using Canary tokens.

Not only was the secret picked within 1 minute of the leak, but an alert was also never sent by AWS.

Layer Leak

We exposed an AWS access key in between the layers of a Docker image and uploaded it to Docker Hub with relevant keywords and tags. Note: This specific scenario was tested using Canary tokens.

Not only was the secret picked up within 3 minutes of the leak, but an alert was also never sent by AWS.

npm Scenarios

Secret npm Nightmare

We decided to try and expose an AWS access key in an npm package, especially due to the notoriety it received after Cybenari’s findings in his own research.

We received an email from AWS notifying us that the key was quarantined within 40 seconds of publishing the package. However, we haven’t seen any evidence of attackers exploiting this secret, which raised two questions for us:

• Could this indicate that attackers are not scanning npm packages? And
• Are we missing something? Cybenari’s research yielded very different (and rather alarming) results.

We decided to explore this question further. But we weren’t ready for that yet—we needed to be certain, so we ran it one more time.

npm Deja Vu

Given the results of Secret npm Nightmare, we decided to repeat the scenario, as we saw zero hits and couldn’t understand why. We thought we must be missing something.

We received an email from AWS notifying us that the key was quarantined within a minute of publishing the package, though no hits were detected. Interestingly, when we leaked a GitHub PAT in one of the npm packages during our research (spoiler alert!), it did receive hits. Later, we attempted to replicate the scenario with another set of GitHub and AWS access keys but, once again, saw no hits for our AWS access key in npm.

Additionally, we noticed that whenever we leaked an AWS access key on npm, AWS alerted us within minutes, and the access key had the AWSCompromisedKeyQuarantineV2 policy attached to it.

This led us to conclude that, similar to GitHub, npm is also scanning its packages, and when it detects an AWS access key, it alerts AWS. Based on this, we conclude that attackers are not scanning npm packages.

However, Cybenari’s research yielded very different results from ours, so it was finally time to put his findings to the test.

Replicating Cybenari’s Scenario

Summarizing the previous two scenarios, we were surprised by the consistent pattern: our keys were quarantined almost immediately without being exploited, whereas in Cybenari’s research¹, his Canary token received hits in under a minute. To see if we were missing something, we decided to replicate his exact setup. We began by generating a Canary token, just as he did, and the content we placed in index.js was identical to his. He also had an aws folder, but since it was empty and contained no information, we omitted it and proceeded with index.js, including our Canary Token credentials. Within a minute of publishing, we received a single hit from an AWS internal IP. However, despite 62 downloads of our package over a week, we saw no further hits beyond the initial AWS internal IP.

While Cybenari observed a hit in under a minute, our conclusion is that he may have mistaken this initial hit for a scanner, rather than recognizing it as an internal AWS process that quarantines the key. The hit we observed on our Canary token was from an internal AWS IP address, and no further hits followed, despite a significant number of downloads over the course of a week. This aligns with our findings from the previous two scenarios, in which we used actual AWS access keys rather than Canary tokens, as he did in his research.

While we can’t verify the exact timing of an AWS alert, we observed activity after 8 minutes, originating from an 'AWS internal' IP with the event name 'AttachUserPolicy.' This likely represents the attachment of the AWSCompromisedKeyQuarantineV2 policy to the IAM User. Based on this information, we estimate that the alert was sent around the 8-minute mark.

PyPI Scenarios

Python Oopsie

We exposed an AWS access key in a Python package

We received an email from AWS 46 minutes after publishing the package on PyPI, and the first attempt to use it was 2 hours from publishing. What we observed taking place, was a random scanner picking up our package and pushing it to a GitHub repository.

We received an email from AWS 46 minutes after publishing the package on PyPI, and the first attempt to use it occurred 2 hours from the time of publishing. What we observed was a random scanner picking up our PyPI package and pushing it to a public GitHub repository. This exposure on GitHub triggered the AWS alert, also allowing subsequent scanners to easily detect and exploit the secret.

Python Oopsie 2

We repeated the above scenario to see if the 2 hour interval will stay somewhat consistent or if there’s a possibility an additional key will be picked up much faster than the previous one.

The first attempt to use it occurred 40 seconds after publishing, and we received an email from AWS 43 minutes after the package was published on PyPI. What we observed mirrored the previous scenario: a random scanner picked up our package and pushed it to a GitHub repository.

Crate Scenarios

Rusty Blunder

We exposed an AWS access key in a Crate package, and were curious to see if Rust is of any interest to scanners.

Interestingly enough, there were zero hits on our secret, as well as no email alerts from AWS. We had thought initially that this means that Crate packages are not being scanned continuously by attackers, an entire day has gone by, and nothing. We thought, what’s next? But then came the sequel.

Rusty Blunder II

We weren’t satisfied with the results of the previous scenario. Are attackers not scanning Crate packages? We needed to be sure, so we re-tested the scenario, making only two changes: a different AWS access key (to maintain the integrity of the test) and an updated package version number.

Lo and behold, after 4 days, we got a hit on our key, indicating that Crate packages are not safe from attackers either. Yikes. The package ended up with more than 400 downloads.

What’s Next

In this blog, we explored how attackers exploit package managers with alarming efficiency. Docker Hub and PyPI proved to be hot zones, while npm stood apart—for now. But this doesn’t mean it’s safe. We don’t recommend testing how soon their capabilities evolve.

From GitHub to Docker Hub, our findings in the series so far unveil an unavoidable truth: attackers operate faster than traditional defenses. Secrets leaked in these environments were exploited in seconds, making secret rotation as effective as locking your door after an intruder has already entered.

The lesson remains clear: proactive measures, not reactive fixes, are the only way forward.

In our next post, we’ll delve into another risky frontier: code-sharing platforms like Pastebin and GitHub Gist, where secrets often make unintentional appearances, ripe for exploitation. If you can’t wait for the next blog post, we invite you to download the full report, with all scenarios we ran, a deeper dive into our methodologies, platform-specific insights, attacker behavior patterns, and the tool we built to neutralize exposed secrets instantly. It’s an essential read for any organization looking to secure its systems against the escalating risks of exposed secrets.

[1]https://cybenari.com/2024/08/whats-the-worst-place-to-leave-your-secrets/