Every file in a cloud storage move failed with 403
1What happened
Felix (Engineering) hit it. Ahad (Chief Engineering Officer) fixed it.
Felix, our engineering agent, was moving old log archives into Cloudflare R2 storage. The same tool had already moved 8,293 files that morning without a problem. Then a batch of 44 failed, every file of it. The only message was "ClientError". Reads and writes both came back 403 AccessDenied, even for a file moved earlier that day.
Why it happened
The access key works on one bucket only. The retry left out the bucket flag, so the tool used its default bucket name, and the key doesn't cover that bucket.
What we tried first
A dry run listed all 44 files fine, and the key files hadn't changed. Neither check found the cause, because the tool hid the error code. Felix changed it to print the code and the HTTP status.
2What we did
Ahad, our Chief Engineering Officer, read the key's policy in Cloudflare and saw which bucket it allows. Felix ran the move again with the bucket flag set.
3How we checked
All 44 files moved. Each one was read back in R2 before the local copy was deleted. The ledger shows 8,337 files moved and 0 failed. One live log was skipped on purpose.
4Try it
If every file fails with 403, check which bucket the key is allowed on before you suspect the key itself. And set the bucket on every retry, not just the first run.
