Thank you author Ian Mckay! This is one of those good hygiene conventions that save time by not having to think/worry each time buckets are named. As pointed out in the article, AWS seems to have made this part of their official naming conventions [1].
I'm excited for IaC code libraries like Terraform to incorporate this as their default behavior soon! The default behavior of Terraform and co is already to add a random hash suffix to the end of the bucket name to prevent such errors. This becoming standard practice in itself has saved me days in not having to convince others to use such strategies prior to automation.
[1] https://aws.amazon.com/blogs/aws/introducing-account-regiona...
[dead]
[dead]
That took a decade to resolve? Surprising, but hindsight is 20/20 I guess.
> If you wish to protect your existing buckets, you’ll need to create new buckets with the namespace pattern and migrate your data to those buckets.
My pet conspiracy theory: this article was written by bucket squatters who want to claim old bucket names after AI agents read this and blindly follow.
> For Azure Blob Storage, storage accounts are scoped with an account name and container name, so this is far less of a concern.
The author probably misunderstood what "account name" is in Azure Storage's context, as it's pretty much the equivalent of S3's bucket name, and is definitely still a large concern.
A single pool of unique names for storage accounts across all customers has been a very large source of frustration, especially with the really short name limit of only 24 characters.
I hope Microsoft follows suit and introduces a unique namespace per customer as well.
I recall being shocked the first time I used Azure and realizing so many resources aren’t namespaced to account level. Bizarre to me this wasn’t a v1 concern.
Author here. Thanks for the call out! I've updated the article with attribution.
I take it advertising your account id isn't a security risk?
It is not hygienic, but with only the account-id you are fine. In the IAM rules the attacker can always just use a * on their end, so it does not make a difference. You have to be conscious to set proper rules for your (owner) end tho.
I started treating long random bucketnames as secrets years ago. Ever since I noticed hackers were discovering buckets online with secrets and healthcare info.
This is where IaC shines.
I just started using hashes for names. The deployment tooling knows the "real" name. The actual deployment hash registers a salt+hash of that name to produce a pseudo-random string name.
This is all good and we'll on the IaC side,yes. But at the end of the day, buckets are also user facing resources, and nobody likes random directory / bucket names.
Why all that stuff with namespaces when they could just not allow name reuse?
Potential reasons I can think of for why they don't disallow name reuse:
a) AWS will need to maintain a database of all historical bucket names to know what to disallow. This is hard per region and even harder globally. Its easier to know what is currently in use rather know what has been used historically.
b) Even if they maintained a database of all historically used bucket names, then the latency to query if something exists in it may be large enough to be annoying during bucket creation process. Knowing AWS, they'll charge you for every 1000 requests for "checking if bucket name exists" :p
c) AWS builds many of its own services on S3 (as indicated in the article) and I can imagine there may be many of their internal services that just rely on existing behaviour i.e. allowing for re-creating the same bucket name.
I'd allow re-use, but only by the original account. Not being able to re-create a bucket after deleting it would be annoying.
I think that's an important defense that AWS should implement for existing buckets, to complement account scoped bucket.
Huh? Hash your bucket names
if your bucket name is ever exposed and you later delete it, then this doesn't help you.
I don't think that'd prevent this attack vector.
Ok; salt, and then hash your bucket names
I saw “bucketsquatting,” and an entirely different image came to mind…
It sounds like a sensitive subject, very delicate, and of no concern to law enforcement, for private videos of an artistic nature.
I'd ask politely to refrain from such comments :)
This is not me criticising you. I totally understand the urge to say it. We're all thinking the thing you're thinking of. It takes effort not to give into it ;)
The reason I personally would refrain from making such comments is that they have the potential to end up as highest ranked comment. That would be a shame. Topic of S3 bucketsquatting is rather important and very interesting.
He is just comment squatting :)
You did not really give a reason to refrain from making a joke. Don't take yourself too serious
>We're all thinking the thing you're thinking of.
I wasn't but I sure am now.