s3.PutObject silently fails when key starts with a slash

[leonid-shevtsov]

Describe the bug

An s3.PutObject will return no error - and a seemingly valid PutObjectOutput - when the PutObjectInput.Key contains a leading slash. No object appears in the bucket.

The same input in aws-sdk-go v1 worked fine.

Expected Behavior

Either the operation succeeding, or failing with an error, either way is fine by me.

Current Behavior

Error is nil. PutObjectOutput is valid and indistinguishable from a successful upload.

{
"BucketKeyEnabled": false,
"ChecksumCRC32": null,
"ChecksumCRC32C": null,
"ChecksumSHA1": null,
"ChecksumSHA256": null,
"ETag": "\"d3ac5112fe464b81184352ccba743001\"",
"Expiration": null,
"RequestCharged": "",
"SSECustomerAlgorithm": null,
"SSECustomerKeyMD5": null,
"SSEKMSEncryptionContext": null,
"SSEKMSKeyId": null,
"ServerSideEncryption": "",
"VersionId": null,
"ResultMetadata": {}
}

Reproduction Steps

s3Client.PutObject(context.Background(), &s3.PutObjectInput{
		Bucket:      "mybucket",
		Key:         "/test.txt",
		Body:        strings.NewReader("contents"),
		ContentType: aws.String("text/plain"),
	})

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

	github.com/aws/aws-sdk-go-v2 v1.16.3
	github.com/aws/aws-sdk-go-v2/config v1.15.4
	github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.18.2
	github.com/aws/aws-sdk-go-v2/service/ecs v1.18.6
	github.com/aws/aws-sdk-go-v2/service/firehose v1.14.4
	github.com/aws/aws-sdk-go-v2/service/s3 v1.26.7

Compiler and Version used

go version go1.18.1 darwin/arm64

Operating System and version

macOS 12.3.1

[skmcgrail]

Hey @leonid-shevtsov I am not able to reproduce this behavior you have described.

Can you please enable request and response logging for requests following the Logging developer guide?

As show in my testing I was able to put this key in the bucket, and that key was visible within the aws console. In this example I used the key /slashtest.

PutObject

SDK 2022/05/12 17:05:26 DEBUG Request
PUT //slashtest?x-id=PutObject HTTP/1.1
Host: mcgrails-test-data.s3.us-west-2.amazonaws.com
User-Agent: aws-sdk-go-v2/1.16.3 os/macos lang/go/1.18 md/GOOS/darwin md/GOARCH/amd64 api/s3/1.26.9
Content-Length: 12
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: 8bc7a09e-2f8c-44e2-be55-6d7072c40a45
Amz-Sdk-Request: attempt=1; max=3
Authorization: AWS4-HMAC-SHA256 Credential=ASIAWIZ2BD2C6R3LGKE4/20220513/us-west-2/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=dffa75023a0d8e46d318eb42f900bef68a79fe108f922314f0af0fa543c59439
Content-Type: application/octet-stream
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20220513T000526Z
X-Amz-Security-Token: IQoJb3JpZ2luX2VjEHgaCXVzLWVhc3QtMSJGMEQCIBdPso4UlwGIh8YOwEBs7MpenHE9dafoYdqVfB7xkEEaAiBY1ATbnRNeJ9kgnGwGEav0CWBb9DgbiZdRI4EjqpmDbCqfAghREAMaDDQzMTIyOTA1MDUwMSIMaEuMCkd6sLCBa/UwKvwBWdlQo/kTrhqIlOx69tz7y+1NNsMTkQjjJl8UDDTjrfE3+XRhzsekutN7Tytnbutr/en7Mz6/CUV8trhG7IIQFA0JaB4klM0XPZTbRKQQLFlKUbgIz33TlmC5Ldq/kmVGLZU/Gdi1/QyE6E5L/knl8upIb/9rVVwhqaHeR+fWtlF7XOOgfegBSmlDZvvN6oirsaoNzzZojB2v0PyYBWZSGGpxai1S1zGz77jnwMmTIR8esbAl1UMU0Ir/B2udA71Ebk3JJ6zo1PzSDGHdBhEWYKkrn5AepG6+lHBc+E4KwQSOe8JYFbx/4TCDEWIFhmxmz+ipKD1MopTXGCETMP6y9pMGOp4BPIdNB0TWrbnYpkdv+eXL/w4KOsT0aILgx+tFozbe9bGLQVfbOZWj/R+rjiIj3hYuLbp2zkX7BkbNxiihQeeZOoZ+mWp4TYIIHakzWAk2IakpRWHeXc3FFQ0nqYbcHwhlVW2qwqAcAzKhMmih3ayY7nJLHbiBSHdm5C75SrQYcK7b3nQ6WHCBES3qpPxw89U1bgh+c3EkO0EmM9Gfn5E=

SDK 2022/05/12 17:05:27 DEBUG Response
HTTP/1.1 200 OK
Content-Length: 0
Date: Fri, 13 May 2022 00:05:28 GMT
Etag: "9893532233caff98cd083a116b013c0b"
Server: AmazonS3
X-Amz-Id-2: H8r3KqiFgSawIfa5fr3i2QvRHjhOH8QfeaKkmYhJs/26YgbxqVL5eBs7AefxloCF5GVKg1pAD/A=
X-Amz-Request-Id: Y28G48ZP8WA2SQ9Q
X-Amz-Version-Id: td27Vp4_x8ZLAbiGgHTPkU_qnuylMv_9

GetObject

SDK 2022/05/12 17:05:27 DEBUG Request
GET //slashtest?x-id=GetObject HTTP/1.1
Host: mcgrails-test-data.s3.us-west-2.amazonaws.com
User-Agent: aws-sdk-go-v2/1.16.3 os/macos lang/go/1.18 md/GOOS/darwin md/GOARCH/amd64 api/s3/1.26.9
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: 62a746e3-d9d8-484b-906b-df23abadbc7d
Amz-Sdk-Request: attempt=1; max=3
Authorization: AWS4-HMAC-SHA256 Credential=ASIAWIZ2BD2C6R3LGKE4/20220513/us-west-2/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=37c832f97044d7337223880dce359dda278df442768ccd9acec4fe41d86391f0
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220513T000527Z
X-Amz-Security-Token: IQoJb3JpZ2luX2VjEHgaCXVzLWVhc3QtMSJGMEQCIBdPso4UlwGIh8YOwEBs7MpenHE9dafoYdqVfB7xkEEaAiBY1ATbnRNeJ9kgnGwGEav0CWBb9DgbiZdRI4EjqpmDbCqfAghREAMaDDQzMTIyOTA1MDUwMSIMaEuMCkd6sLCBa/UwKvwBWdlQo/kTrhqIlOx69tz7y+1NNsMTkQjjJl8UDDTjrfE3+XRhzsekutN7Tytnbutr/en7Mz6/CUV8trhG7IIQFA0JaB4klM0XPZTbRKQQLFlKUbgIz33TlmC5Ldq/kmVGLZU/Gdi1/QyE6E5L/knl8upIb/9rVVwhqaHeR+fWtlF7XOOgfegBSmlDZvvN6oirsaoNzzZojB2v0PyYBWZSGGpxai1S1zGz77jnwMmTIR8esbAl1UMU0Ir/B2udA71Ebk3JJ6zo1PzSDGHdBhEWYKkrn5AepG6+lHBc+E4KwQSOe8JYFbx/4TCDEWIFhmxmz+ipKD1MopTXGCETMP6y9pMGOp4BPIdNB0TWrbnYpkdv+eXL/w4KOsT0aILgx+tFozbe9bGLQVfbOZWj/R+rjiIj3hYuLbp2zkX7BkbNxiihQeeZOoZ+mWp4TYIIHakzWAk2IakpRWHeXc3FFQ0nqYbcHwhlVW2qwqAcAzKhMmih3ayY7nJLHbiBSHdm5C75SrQYcK7b3nQ6WHCBES3qpPxw89U1bgh+c3EkO0EmM9Gfn5E=

SDK 2022/05/12 17:05:27 DEBUG Response
HTTP/1.1 200 OK
Content-Length: 12
Accept-Ranges: bytes
Content-Type: application/octet-stream
Date: Fri, 13 May 2022 00:05:28 GMT
Etag: "9893532233caff98cd083a116b013c0b"
Last-Modified: Fri, 13 May 2022 00:05:28 GMT
Server: AmazonS3
X-Amz-Id-2: Aqin1OYkuTPqkZfauMmWga7tSrYsF3pw5S5N7BIwVoQlbO6nHNImFfILZWSvEx3WepbD8B4O0do=
X-Amz-Replication-Status: PENDING
X-Amz-Request-Id: Y28Z2DTKKB4H3014
X-Amz-Version-Id: td27Vp4_x8ZLAbiGgHTPkU_qnuylMv_9

Console View

[leonid-shevtsov]

Thanks! After inspecting responses - and the bucket - I found the files.

So, the change from API v1 is that the leading slash is not trimmed from the key. And then in the quasi-nesting S3 filesystem, it appears in a nameless "folder", and not in the root.

This seems logical (setting the difference with v1 aside). It's also consistent with the ruby sdk (that we use to read the files - turns out, we even had to the leading slash manually).

So, now I see nothing wrong with the handling of keys.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.