Using HeadBucket for backups is dangerous in multi-tenant S3 buckets

[DimitriosLisenko]

Proposal

This project uses HeadBucket to determine the signature version to use, as well as whether the bucket exists before creating backups. The problem with this is that in IAM, there is literally no way to let a user call HeadBucket while also preventing them from listing objects inside that bucket. This is because HeadBucket requires the s3:ListBucket permission.

There are zero air-tight IAM workarounds that would prevent this permission from also allowing users to list objects inside the bucket. Here are some examples:

1. Grant s3:ListBucket conditionally on the "" prefix - does not allow HeadBucket because HeadBucket does not pass a prefix parameter, and so it is denied.
2. Grant s3:ListBucket conditionally on a specific directory - same as above.
3. Grant s3:ListBucket conditionally on "Null": {"s3:prefix": "true"} - this actually still allows listing all objects in the bucket, just not specifying a prefix.
4. Grant s3:ListBucket conditionally on everything except */* - can still specify a prefix without a slash and list objects.

There has been extensive discussion around this issue as it relates to the CNPG plugin:
EnterpriseDB/barman#929
EnterpriseDB/barman#1101
hashicorp/terraform-provider-aws#17195

The initial suggestion was to use ListObjectsV2 with a prefix rather than HeadBucket - this works in terms of security, but it is more expensive in terms of AWS API calls, and so was rejected.

The final solution as it relates to CNPG was to completely remove cloud connectivity + bucket existence checks before running backups. I propose the same here, which would require to remove the signature checks and require that it is provider; or instead to use HeadObject, which can be scoped to a directory.

Use-Case

The use case is for multi-tenant S3 buckets. Using HeadBucket in this scenario is a non-starter because it would allow users to see backups of other users.

Anything else?

This is a requirement to resolve percona/percona-xtradb-cluster-operator#2251, because that operator calls HeadBucket in itself, as well as by virtue of calling xbcloud.

[it-percona-cla]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}