upvote
Years ago I worked for a company where customers started seeing other customers' data.

The cause was a bad hire decided to do a live debugging session in the production environment. (I stress bad hire because after I interviewed them, my feedback was that we shouldn't hire them.)

It was kind of a mess to track down and clean up, too.

reply
Maybe dynamodb was inconsistent for a period and as that backs IAM credentials were scrambled? Do you have references to this, because if it is true that is really really bad.
reply
AWS IAM doesn't use or depend on DynamoDB
reply
Got references? This is crazy.
reply
I saw a link to https://old.reddit.com/r/webdev/comments/1obtbmg/aws_site_re... at one point but then it was deleted
reply
This is not about the AWS Console. It is talking about the customer's site hosted on CloudFront. It is possible to cross wires with user sessions when using CloudFront if you haven't set caching granular enough to be specific to an end user. This scenario is customer error, not AWS.
reply
I'd argue it's a classic footgun and a flaw of CloudFront (they should at least warn about it much more).
reply
electricity_is_life's comment on reddit seems to explain it:

> Not sure if this is what happened to you, but one thing I ran into a while back is that even if you return Cache-Control: no-store it's still possible for a response to be reused by CloudFront. This is because of something called a "collapse hit" where two requests that occur at the same time and are identical (according to your cache key) get merged together into a single origin request. CloudFront isn't "storing" anything, but the effect is still that a user gets a copy of a response that was already returned to a different user.

> https://stackoverflow.com/a/69455222

> If your app authenticates based on cookies or some other header, and that header isn't part of the cache key, it's possible for one user to get a response intended for a different user. To fix it you have to make sure any headers that affect the server response are in the cache key, even if the server always returns no-store.

---

Though the AWS docs seem to imply that no-store is effective:

> If you want to prevent request collapsing for specific objects, you can set the minimum TTL for the cache behavior to 0 and configure the origin to send Cache-Control: private, Cache-Control: no-store, Cache-Control: no-cache, Cache-Control: max-age=0, or Cache-Control: s-maxage=0.

https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...

reply
Collapse-hits... hadn't thought about those in years. Brought back some trauma.
reply
This isn't about an aws account, this is about the auth inside the project that user is running.
reply
deleted
reply
> couple folks on reddit said while they were refreshing during the outage, they were briefly logged in as a whole different user

Didn't ChatGPT have a similar issue recently? Would sound awfully similar.

reply
Steam also had this, classic caching issue.
reply
This happened to me on Twitter maybe like, 9 years ago? What's the mechanism of action that causes this to happen?
reply
The easiest way to do this is to misconfigure your CDN so that it caches set-cookie headers.
reply
A security incident like this would dwarf in comparision to partial unavailability of services.
reply
A friend of a friend knows a friend who logged in to Netflix root account. Source: trust me bro
reply