Files
wiki/存储/Ceph/知识库/10-RGW-S3与Swift接口篇.md
2026-09-18 13:49:11 +08:00

162 KiB
Raw Permalink Blame History

基于 Ceph 官方文档整理 | 文档版本:Tentacle v20.2.4 | 抓取日期:2026-09-17 | 源文:ceph/ceph@v20.2.4 doc/radosgw/s3、doc/radosgw/swift 官网入口:https://docs.ceph.com/en/tentacle/radosgw/s3/

第 10 册 · RGW 对象存储篇(二):S3 / Swift 接口与 SDK


本册导读

本册是《Ceph 中文学习手册》RGW 四册中的第二册。

分册 主题 位置
第 9 册 RGW 对象存储篇(一):部署、配置与多站点 前序
第 10 册 RGW 对象存储篇(二):S3 / Swift 接口与 SDK 本册
第 11 册 RGW 对象存储篇(三):管理与运维 后续
第 12 册 RGW 对象存储篇(四):高级特性 后续

本册定位

本册是接口参考(API Reference),不是从零讲起的教程。因此写作上遵循三条原则:

  1. 逐个操作列出:每一个 S3 / Swift 操作单独成节,给出「操作名 | HTTP 方法与路径 | 关键参数/请求头 | 响应要点 | 示例」五要素;禁止「支持若干 bucket 操作」这类概括。
  2. 可查阅(速查型):主体由大表格 + 示例块构成,请求/响应实体按官方原表逐行搬运,便于按操作名或 URL 路径检索。
  3. 讲透:本册范围内的内容在这里讲完,不以「详见官方文档」收尾。

术语与书写约定

  • 中文叙述;操作名、HTTP 方法、请求头/响应头名、查询参数名、XML 元素名、错误码、语言标识一律保留英文原文(如 PUT Bucket、x-amz-copy-source、NoSuchUpload、BucketAlreadyExists、python)。
  • 代码块按内容标注语言:http / bash / python / java / cpp / csharp / php / perl / ruby / xml / text。
  • 官方没写、由编者补充的内容,统一以 > 💡 补充:… 标注。
  • 官方文档确实没有说明的,直接写「文档未明确」。

本册输入的官方页面(共 22 个 rst 源文件)

# 源文件 本册章节
1 radosgw/s3.rst 一、S3 接口总览
2 radosgw/s3/authentication.rst 二、S3 认证
3 radosgw/s3/commons.rst 三、S3 公共请求头与错误响应
4 radosgw/s3/serviceops.rst 四、Service 操作
5 radosgw/s3/bucketops.rst 五、Bucket 操作
6 radosgw/s3/objectops.rst 六、Object 操作
7 radosgw/s3/cpp.rst 七(C++)
8 radosgw/s3/csharp.rst 七(C#)
9 radosgw/s3/java.rst 七(Java)
10 radosgw/s3/perl.rst 七(Perl)
11 radosgw/s3/php.rst 七(PHP)
12 radosgw/s3/python.rst 七(Python)
13 radosgw/s3/ruby.rst 七(Ruby)
14 radosgw/swift.rst 八、Swift 接口总览
15 radosgw/swift/auth.rst 九、Swift 认证
16 radosgw/swift/serviceops.rst 十(Service)
17 radosgw/swift/containerops.rst 十(Container)
18 radosgw/swift/objectops.rst 十(Object)
19 radosgw/swift/tempurl.rst 十(Temp URL)
20 radosgw/swift/tutorial.rst 十(Tutorial 流程)
21 radosgw/swift/java.rst 十一(Java)
22 radosgw/swift/python.rst 十一(Python)
23 radosgw/swift/ruby.rst 十一(Ruby)

引自:Ceph Object Gateway S3 API、Ceph Object Gateway Swift API


一、S3 接口总览

Ceph Object Gateway(RGW)提供一套 RESTful API,兼容 Amazon S3 API 的基本数据访问模型。也就是说:能跑在 AWS S3 上的客户端,只要改 endpoint 与密钥,原则上也能对接 Ceph RGW;但不是 100% 等价,差异集中在下表的 Remarks 列。

1.1 功能支持矩阵(Features Support)

下表逐行搬运官方 s3.rst 的 Features Support 表。Status 列:Supported = 支持;Partial = 部分支持。

Feature(能力) 是否支持 限制说明(Remarks,原文)
List Buckets Supported —
Delete Bucket Supported —
Create Bucket Supported Different set of canned ACLs(canned ACL 集合与 AWS 不同)
Bucket Lifecycle Supported —
Bucket Replication Partial Permitted only across zones(仅允许跨 zone 复制)
Policy (Buckets, Objects) Supported ACLs & bucket policies are supported
Bucket Website Supported —
Bucket ACLs (Get, Put) Supported Different set of canned ACLs
Bucket Location Supported —
Bucket Notification Supported See S3 Notification Compatibility
Bucket Object Versions Supported —
Get Bucket Info (HEAD) Supported —
Bucket Request Payment Supported —
Put Object Supported —
Delete Object Supported —
Get Object Supported —
Object ACLs (Get, Put) Supported —
Get Object Info (HEAD) Supported —
POST Object Supported —
Copy Object Supported —
Multipart Uploads Supported —
Object Tagging Supported See tag_policy for Policy verbs
Bucket Tagging Supported —
Storage Class Supported See storage_classes
Bucket Logging Supported —

💡 补充:上表中若干能力(Bucket Lifecycle、Bucket Replication、Bucket Website、Bucket Request Payment、POST Object、Object/Bucket Tagging、Storage Class)在 s3.rst 标记为 Supported,但本册输入范围内的 s3/bucketops.rst 与 s3/objectops.rst 并未提供这些操作的请求/响应规格(没有对应小节)。因此本册第五章、第六章只讲源文实际写了规格的操作;其余操作的具体语法属于「文档未明确」,见文末。

1.2 不支持的公共头字段(Unsupported Header Fields)

Name Type
x-amz-id-2 Response

即:RGW 的 S3 响应不返回 x-amz-id-2 这个 AWS S3 常见的响应头。依赖该头做排障的客户端需要注意。

1.3 S3 API 页面结构

官方 s3.rst 的 toctree 顺序即本册 S3 部分的组织顺序:

Common            -> s3/commons
Authentication    -> s3/authentication
Service Ops       -> s3/serviceops
Bucket Ops        -> s3/bucketops
Object Ops        -> s3/objectops
C++ / C# / Java / Perl / PHP / Python / Ruby

引自:Ceph Object Gateway S3 API


二、S3 认证(Authentication and ACLs)

2.1 认证与匿名请求

  • 发往 RGW 的请求可以是**已认证(authenticated)或未认证(unauthenticated)**两种。
  • RGW 假定未认证请求来自一个 anonymous user(匿名用户)。
  • RGW 支持 canned ACLs。
  • 请求使用 AWS Signatures 认证,签名由用户的凭证(S3 access key 与 secret key)派生。
  • 绝大多数 S3 客户端与 AWS SDK 会替你生成签名;只有直接发裸 HTTP 请求时,才需要手工构造签名。

2.2 AWS Signature v4

  • 规范参见 AWS 官方 Authenticating Requests (AWS Signature Version 4)。
  • x-amz-content-sha256 请求头支持以下取值:
x-amz-content-sha256 取值 说明
Actual payload checksum value 真实 payload 校验和
UNSIGNED-PAYLOAD 未签名 payload
STREAMING-UNSIGNED-PAYLOAD-TRAILER 流式未签名 payload(带 trailer)
STREAMING-AWS4-HMAC-SHA256-PAYLOAD 流式 AWS4-HMAC-SHA256 签名 payload
STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER 上式带 trailer

2.3 AWS Signature v2

  • 规范参见 AWS 官方 Authenticating Requests (AWS Signature Version 2)。
  • 注意:虽然 v2 签名在 AWS 已废弃,RGW 仍然继续支持 v2。

2.4 认证头的形式

源文中 Service 操作示例给出的认证头形式为:

Authorization: AWS {access-key}:{hash-of-header-and-secret}

💡 补充:这是 AWS Signature v2 风格的 Authorization 头写法,源文在多个操作的 Syntax 中都以此示意;Signature v4 的 Authorization 头格式(含 AWS4-HMAC-SHA256 Credential=...)源文未在本页展开,指向 AWS 官方文档。

2.5 对接 OpenStack Keystone 做 S3 认证

若 radosgw 实例配置了 Keystone 认证,可把 Keystone 作为 S3 API 认证的权威源。需要设置:

  • ../keystone 页中说明的 rgw keystone 系列配置项;
  • rgw s3 auth use keystone = true。

此外,使用者必须持有 AWS 风格的 access key / secret key,可用如下命令获取:

$ openstack --os-interface public ec2 credentials create
+------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                       |
+------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| access     | c921676aaabbccdeadbeef7e8b0eeb2c                                                                                                            |
| links      | {u'self': u'https://auth.example.com:5000/v3/users/7ecbebaffeabbddeadbeefa23267ccbb24/credentials/OS-EC2/c921676aaabbccdeadbeef7e8b0eeb2c'} |
| project_id | 5ed51981aab4679851adeadbeef6ebf7                                                                                                            |
| secret     | ********************************                                                                                                            |
| trust_id   | None                                                                                                                                        |
| user_id    | 7ecbebaffeabbddeadbeefa23267cc24                                                                                                            |
+------------+---------------------------------------------------------------------------------------------------------------------------------------------+

由此生成的 access / secret key 即可用于 S3 API 访问 radosgw。

注意:多数生产环境在 Keystone 认证的同时也会配置 multitenancy,此时 S3 signed URLs 与 public read ACLs 有特殊注意事项。

2.6 Access Control Lists(ACL)

ACL 是一组 access grants,指明某个用户能对 bucket 或 object 执行什么操作。同一个 Permission 作用在 bucket 与 object 上含义不同:

Permission Bucket 上的含义 Object 上的含义
READ Grantee 可以列出 bucket 中的对象 Grantee 可以读该对象
WRITE Grantee 可以在 bucket 中写或删对象 N/A
READ_ACP Grantee 可以读 bucket ACL Grantee 可以读对象 ACL
WRITE_ACP Grantee 可以写 bucket ACL Grantee 可以写对象 ACL
FULL_CONTROL Grantee 对 bucket 中的对象拥有全部权限 Grantee 可以读或写对象 ACL

2.7 S3 操作到 ACL 权限的映射(完整表)

RGW 内部把 S3 操作映射为 ACL 权限,映射表如下(逐行搬运官方原表):

Operation Permission
s3:GetObject READ
s3:GetObjectTorrent READ
s3:GetObjectVersion READ
s3:GetObjectVersionTorrent READ
s3:GetObjectTagging READ
s3:GetObjectVersionTagging READ
s3:ListAllMyBuckets READ
s3:ListBucket READ
s3:ListBucketMultipartUploads READ
s3:ListBucketVersions READ
s3:ListMultipartUploadParts READ
s3:AbortMultipartUpload WRITE
s3:CreateBucket WRITE
s3:DeleteBucket WRITE
s3:DeleteObject WRITE
s3:s3DeleteObjectVersion WRITE
s3:PutObject WRITE
s3:PutObjectTagging WRITE
s3:PutObjectVersionTagging WRITE
s3:DeleteObjectTagging WRITE
s3:DeleteObjectVersionTagging WRITE
s3:RestoreObject WRITE
s3:GetAccelerateConfiguration READ_ACP
s3:GetBucketAcl READ_ACP
s3:GetBucketCORS READ_ACP
s3:GetBucketLocation READ_ACP
s3:GetBucketLogging READ_ACP
s3:GetBucketNotification READ_ACP
s3:GetBucketPolicy READ_ACP
s3:GetBucketRequestPayment READ_ACP
s3:GetBucketTagging READ_ACP
s3:GetBucketVersioning READ_ACP
s3:GetBucketWebsite READ_ACP
s3:GetLifecycleConfiguration READ_ACP
s3:GetObjectAcl READ_ACP
s3:GetObjectVersionAcl READ_ACP
s3:GetReplicationConfiguration READ_ACP
s3:GetBucketEncryption READ_ACP
s3:DeleteBucketPolicy WRITE_ACP
s3:DeleteBucketWebsite WRITE_ACP
s3:DeleteReplicationConfiguration WRITE_ACP
s3:PutAccelerateConfiguration WRITE_ACP
s3:PutBucketAcl WRITE_ACP
s3:PutBucketCORS WRITE_ACP
s3:PutBucketLogging WRITE_ACP
s3:PutBucketNotification WRITE_ACP
s3:PutBucketPolicy WRITE_ACP
s3:PutBucketRequestPayment WRITE_ACP
s3:PutBucketTagging WRITE_ACP
s3:PutPutBucketVersioning WRITE_ACP
s3:PutBucketWebsite WRITE_ACP
s3:PutLifecycleConfiguration WRITE_ACP
s3:PutObjectAcl WRITE_ACP
s3:PutObjectVersionAcl WRITE_ACP
s3:PutReplicationConfiguration WRITE_ACP
s3:PutBucketEncryption WRITE_ACP

原文说明:某些映射(例如 s3:CreateBucket → WRITE)对 S3 操作本身并不适用,之所以存在,是为了在 Swift user ACL 生效时,让 Swift 与 S3 能访问同一份资源。这也是「能用 S3 bucket policy 就尽量别用 S3 ACL」的原因之一。

💡 补充:官方原表中 s3:PutPutBucketVersioning 多了一个 Put,s3:s3DeleteObjectVersion 多了一个 s3: 前缀,均为官方文档原文的笔误,本册按原文保留未改。

2.8 常见认证相关错误码

authentication.rst 本身没有给出错误码清单;认证/授权失败时的错误码在公共错误表中给出(见第三章 3.3),其中最相关的是:

场景 HTTP 状态 错误码
权限不足 / 签名不通过导致拒绝 403 AccessDenied
用户被停用 403 UserSuspended
请求时间与服务器时间偏差过大(签名里的 Date 太旧/太新) 403 RequestTimeTooSkewed
访问不存在的 key 404 NoSuchKey
访问不存在的 bucket 404 NoSuchBucket

💡 补充:上面「场景」一列是对错误码语义的解释性归纳,非官方原文逐字描述;错误码与状态码本身来自 commons.rst 原表。官方源文未单独列出预签名 URL(presigned URL)的专用错误码;各 SDK 示例中展示了如何生成签名 URL(见第七章)。

引自:Authentication and ACLs


三、S3 公共请求头与错误响应(Common Entities)

3.1 Bucket 与 Host Name:两种寻址模式

访问 bucket 有两种模式:path-style 与 virtual-hosted-style。

Path-style:bucket 作为请求路径的顶层目录。

GET /mybucket HTTP/1.1
Host: cname.domain.com

Virtual-hosted-style:bucket 作为完全限定域名的一部分(大多数 S3 客户端默认使用此模式)。

GET / HTTP/1.1
Host: mybucket.cname.domain.com

相关配置与注意点:

  • Path-style 已被 AWS 废弃(参见 Amazon S3 Path Deprecation Plan)。

  • 配置 virtual hosted bucket:在 ceph.conf 中设置 rgw_dns_name = cname.domain.com,或把 cname.domain.com 加入 zonegroup 配置的 hostnames 列表。

  • 设置示例:

    $ ceph config set client.rgw.<ceph authx client for rgw> rgw_dns_name cname.domain.dom
    
  • 可以用 rgw_dns_name 参数直接定义多个 hostname。

  • 启用 SSL 时,证书必须使用通配域名,才能匹配各 bucket 子域。

  • RGW 位于代理之后时,应使用代理的 DNS 名;可用 ceph config set client.rgw 一次性为所有实例设置。

  • s3website API 的静态网站视图必须在另一个域名下提供,用 rgw_dns_s3website_name 单独配置(与 rgw_dns_name 分开)。

3.2 Common Request Headers(公共请求头,逐个)

Request Header Description(含义)
CONTENT_LENGTH Length of the request body.(请求体长度)
DATE Request time and date (in UTC).(请求时间日期,UTC)
HOST The name of the host server.(主机名)
AUTHORIZATION Authorization token.(认证令牌)

文档未明确:commons.rst 只列出上述 4 个公共请求头,没有给出公共响应头(Common Response Headers)清单。唯一明确提到的响应头是不支持的 x-amz-id-2(见 1.2)。

3.3 Common Response Status(公共响应状态码与错误码,逐个)

HTTP Status Response Code 中文含义说明
100 Continue 继续
200 Success 成功
201 Created 已创建
202 Accepted 已接受
204 NoContent 无内容
206 Partial content 部分内容(Range 请求)
304 NotModified 未修改
400 InvalidArgument 参数非法
400 InvalidDigest 摘要非法
400 BadDigest 摘要错误
400 InvalidBucketName bucket 名非法
400 InvalidObjectName 对象名非法
400 UnresolvableGrantByEmailAddress 无法通过邮箱地址解析 grantee
400 InvalidPart 分片非法
400 InvalidPartOrder 分片顺序非法
400 RequestTimeout 请求超时
400 EntityTooLarge 实体过大
403 AccessDenied 拒绝访问
403 UserSuspended 用户被停用
403 RequestTimeTooSkewed 请求时间偏移过大
404 NoSuchKey key 不存在
404 NoSuchBucket bucket 不存在
404 NoSuchUpload upload 不存在
405 MethodNotAllowed 方法不允许
408 RequestTimeout 请求超时
409 BucketAlreadyExists bucket 已存在
409 BucketNotEmpty bucket 非空
411 MissingContentLength 缺少 Content-Length
412 PreconditionFailed 前置条件失败
416 InvalidRange Range 非法
422 UnprocessableEntity 无法处理的实体
500 InternalError 内部错误

注:上表前两列逐行来自官方原表;第三列中文含义为编者对应补充的说明(💡 补充),便于速查。

💡 补充:除上表外,第五、六章各操作还定义了各自特有的错误码,例如 MalformedXML(400)、InvalidBucketState(409)、NoSuchUpload(404)、PositionNotEqualToLength(409)、ObjectNotAppendable(409) 等,它们不在 commons.rst 的公共表中,而在具体操作页给出。

引自:Common Entities


四、Service 操作(Service Operations)

S3 层级的 Service 操作只有两个:List Buckets 与 Get Usage Stats。

4.1 List Buckets

要素 内容
操作名 List Buckets
HTTP 方法与路径 GET /
关键参数/请求头 无查询参数;需 Authorization: AWS {access-key}:{hash-of-header-and-secret}
响应要点 返回发起请求的已认证用户所创建的 bucket 列表;不能匿名请求

Syntax

GET / HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

Response Entities

Name Type Description
Buckets Container Container for list of buckets.
Bucket Container Container for bucket information.
Name String Bucket name.
CreationDate Date UTC time when the bucket was created.
ListAllMyBucketsResult Container A container for the result.
Owner Container A container for the bucket owner's ID and DisplayName.
ID String The bucket owner's ID.
DisplayName String The bucket owner's display name.

响应示例

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult>
  <Owner>
    <ID>johndoe</ID>
    <DisplayName>John Doe</DisplayName>
  </Owner>
  <Buckets>
    <Bucket>
      <Name>mahbuckat1</Name>
      <CreationDate>2011-04-21T18:05:39.000Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

💡 补充:上面这段 XML 是编者按 Response Entities 表构造的示意,官方 serviceops.rst 未给出完整响应报文样例。

4.2 Get Usage Stats

要素 内容
操作名 Get Usage Stats
HTTP 方法与路径 GET /?usage
关键参数/请求头 查询参数 usage;需 Authorization 头
响应要点 按用户返回用量统计,效果类似管理命令 rgw_user_usage_stats

Syntax

GET /?usage HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

Response Entities

Name Type Description
Summary Container Summary of total stats by user.
TotalBytes Integer Bytes used by user
TotalBytesRounded Integer Bytes rounded to the nearest 4k boundary
TotalEntries Integer Total object entries

响应示例

<Usage>
  <Summary>
    <TotalBytes>1024</TotalBytes>
    <TotalBytesRounded>4096</TotalBytesRounded>
    <TotalEntries>2</TotalEntries>
  </Summary>
</Usage>

💡 补充:官方 serviceops.rst 的 Response Entities 表只给出 Summary/TotalBytes/TotalBytesRounded/TotalEntries 四个实体,未给出根元素名与完整响应样例;上面 XML 为编者按实体表构造的示意。

引自:Service Operations


五、Bucket 操作(Bucket Operations)

bucketops.rst 共定义 17 个 bucket 级操作。本节逐个讲解,每节按「概览表 + Syntax + 实体表 + 示例」组织。

5.0 本章操作速查表

# 操作名 HTTP 方法与路径
1 PUT Bucket PUT /{bucket}
2 DELETE Bucket DELETE /{bucket}
3 GET Bucket(List Objects) GET /{bucket}?max-keys=25
4 Get Bucket Location GET /{bucket}?location
5 Get Bucket ACL GET /{bucket}?acl
6 PUT Bucket ACL PUT /{bucket}?acl
7 List Bucket Multipart Uploads GET /{bucket}?uploads
8 ENABLE/SUSPEND BUCKET VERSIONING PUT /{bucket}?versioning
9 PUT BUCKET OBJECT LOCK PUT /{bucket}?object-lock
10 GET BUCKET OBJECT LOCK GET /{bucket}?object-lock
11 Create Notification PUT /{bucket}?notification
12 Delete Notification DELETE /{bucket}?notification[=<notification-id>]
13 Get/List Notification GET /{bucket}?notification[=<notification-id>]
14 Enable Bucket Logging PUT /{bucket}?logging
15 Disable Bucket Logging PUT /{bucket}?logging(空 BucketLoggingStatus)
16 Get Bucket Logging GET /{bucket}?logging
17 Flush Bucket Logging POST /{bucket}?logging

5.1 PUT Bucket

要素 内容
操作名 PUT Bucket
HTTP 方法与路径 PUT /{bucket} HTTP/1.1
关键参数/请求头 头:x-amz-acl(canned ACL)、x-amz-bucket-object-lock-enabled(true/false)
前置条件 必须有 user ID 与有效的 AWS Access Key ID;不能以匿名用户创建 bucket
响应要点 bucket 名唯一、符合约束且未被占用 → 成功;同名 bucket 存在且你是其 owner → 成功;名字已被他人占用 → 失败 409 BucketAlreadyExists

Syntax

PUT /{bucket} HTTP/1.1
Host: cname.domain.com
x-amz-acl: public-read-write

Authorization: AWS {access-key}:{hash-of-header-and-secret}

Parameters(请求头参数)

Name Description Valid Values Required
x-amz-acl Canned ACLs. private, public-read, public-read-write, authenticated-read No
x-amz-bucket-object-lock-enabled Enable object lock on bucket. true, false No

Constraints(bucket 命名约束)

总体上 bucket 名应遵循域名约束:

  • Bucket names must be unique.(必须唯一)
  • Bucket names cannot be formatted as IP address.(不能是 IP 地址格式)
  • Bucket names can be between 3 and 63 characters long.(长度 3–63 字符)
  • Bucket names must not contain uppercase characters or underscores.(不能含大写字母或下划线)
  • Bucket names must start with a lowercase letter or number.(必须以小写字母或数字开头)
  • Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.). Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number.(由一个或多个 label 组成,label 间以单个 . 分隔;可含小写字母、数字、连字符;每个 label 必须以小写字母或数字开头和结尾)

注意:若 rgw_relaxed_s3_bucket_names 设为 true,上述约束会放宽——但 bucket 名仍须唯一、仍不能是 IP 地址格式,且可由字母、数字、点、连字符、下划线组成,最长 255 字符。

Request Entities

Name Type Description
CreateBucketConfiguration Container A container for the bucket configuration.
LocationConstraint String A zonegroup api name, with optional s3_bucket_placement

HTTP Response

HTTP Status Status Code Description
409 BucketAlreadyExists Bucket already exists under different user's ownership.

示例:指定 LocationConstraint 创建 bucket

PUT /my-new-bucket HTTP/1.1
Host: cname.domain.com
x-amz-acl: private

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<CreateBucketConfiguration>
  <LocationConstraint>default</LocationConstraint>
</CreateBucketConfiguration>

💡 补充:官方 bucketops.rst 的 Syntax 只给了 x-amz-acl 一行,未给出带 CreateBucketConfiguration 请求体的完整样例;上面请求体为编者按 Request Entities 表构造示意。


5.2 DELETE Bucket

要素 内容
操作名 DELETE Bucket
HTTP 方法与路径 DELETE /{bucket} HTTP/1.1
关键参数/请求头 无查询参数;需 Authorization 头
响应要点 成功返回 204 No Content;成功删除后 bucket 名可复用

Syntax

DELETE /{bucket} HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

HTTP Response

HTTP Status Status Code Description
204 No Content Bucket removed.

示例

DELETE /my-old-bucket HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

5.3 GET Bucket(List Objects)

要素 内容
操作名 GET Bucket
HTTP 方法与路径 GET /{bucket}?max-keys=25 HTTP/1.1
关键参数/请求头 prefix、delimiter、marker、max-keys、allow-unordered
响应要点 200 OK;ListBucketResult 容器;IsTruncated 为 true 表示只返回了子集

Syntax

GET /{bucket}?max-keys=25 HTTP/1.1
Host: cname.domain.com

Parameters

Name Type Description
prefix String Only returns objects that contain the specified prefix.
delimiter String The delimiter between the prefix and the rest of the object name.
marker String A beginning index for the list of objects returned.
max-keys Integer The maximum number of keys to return. Default is 1000.
allow-unordered Boolean Non-standard extension. Allows results to be returned unordered. Cannot be used with delimiter.

HTTP Response

HTTP Status Status Code Description
200 OK Buckets retrieved

Bucket Response Entities(GET /{bucket} 返回的 bucket 容器字段)

Name Type Description
ListBucketResult Entity The container for the list of objects.
Name String The name of the bucket whose contents will be returned.
Prefix String A prefix for the object keys.
Marker String A beginning index for the list of objects returned.
MaxKeys Integer The maximum number of keys returned.
Delimiter String If set, objects with the same prefix will appear in the CommonPrefixes list.
IsTruncated Boolean If true, only a subset of the bucket's contents were returned.
CommonPrefixes Container If multiple objects contain the same prefix, they will appear in this list.

Object Response Entities(ListBucketResult 中每个对象位于 Contents 容器内)

Name Type Description
Contents Object A container for the object.
Key String The object's key.
LastModified Date The object's last-modified date/time.
ETag String An MD-5 hash of the object. (entity tag)
Size Integer The object's size.
StorageClass String Should always return STANDARD.
Type String Appendable or Normal.

响应示例

<ListBucketResult>
  <Name>my-new-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>25</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>myphoto1.jpg</Key>
    <LastModified>2011-08-08T21:35:48.000Z</LastModified>
    <ETag>&quot;5e5a2c1f7f0e1d2c3b4a5968778695a4&quot;</ETag>
    <Size>251262</Size>
    <StorageClass>STANDARD</StorageClass>
    <Type>Normal</Type>
  </Contents>
</ListBucketResult>

💡 补充:官方未给出完整响应样例,上面按 Response Entities 表构造示意。


5.4 Get Bucket Location

要素 内容
操作名 Get Bucket Location
HTTP 方法与路径 GET /{bucket}?location HTTP/1.1
关键参数/请求头 子资源 location;需 Authorization
权限 调用者必须是 bucket owner
响应要点 返回 LocationConstraint(bucket 所在 region;默认 region 为空字符串)

bucket 可在 PUT 时通过 LocationConstraint 约束到某个 region。

Syntax

GET /{bucket}?location HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

Response Entities

Name Type Description
LocationConstraint String The region where bucket resides, empty string for default region

响应示例

<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></LocationConstraint>

💡 补充:响应体示例为编者构造示意,官方未给出。


5.5 Get Bucket ACL

要素 内容
操作名 Get Bucket ACL
HTTP 方法与路径 GET /{bucket}?acl HTTP/1.1
关键参数/请求头 子资源 acl;需 Authorization
权限 必须是 bucket owner,或被授予该 bucket 的 READ_ACP 权限
响应要点 返回 AccessControlPolicy 容器

Syntax

GET /{bucket}?acl HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

Response Entities

Name Type Description
AccessControlPolicy Container A container for the response.
AccessControlList Container A container for the ACL information.
Owner Container A container for the bucket owner's ID and DisplayName.
ID String The bucket owner's ID.
DisplayName String The bucket owner's display name.
Grant Container A container for Grantee and Permission.
Grantee Container A container for the DisplayName and ID of the user receiving a grant of permission.
Permission String The permission given to the Grantee bucket.

响应示例

<AccessControlPolicy>
  <Owner>
    <ID>johndoe</ID>
    <DisplayName>John Doe</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>johndoe</ID>
        <DisplayName>John Doe</DisplayName>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

💡 补充:响应体示例为编者构造示意。


5.6 PUT Bucket ACL

要素 内容
操作名 PUT Bucket ACL
HTTP 方法与路径 PUT /{bucket}?acl HTTP/1.1
关键参数/请求头 子资源 acl;请求体为 AccessControlPolicy XML
权限 必须是 bucket owner,或被授予该 bucket 的 WRITE_ACP 权限
响应要点 为已存在的 bucket 设置访问控制

Syntax

PUT /{bucket}?acl HTTP/1.1

Request Entities

Name Type Description
AccessControlPolicy Container A container for the request.
AccessControlList Container A container for the ACL information.
Owner Container A container for the bucket owner's ID and DisplayName.
ID String The bucket owner's ID.
DisplayName String The bucket owner's display name.
Grant Container A container for Grantee and Permission.
Grantee Container A container for the DisplayName and ID of the user receiving a grant of permission.
Permission String The permission given to the Grantee bucket.

示例

PUT /my-new-bucket?acl HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<AccessControlPolicy>
  <Owner>
    <ID>johndoe</ID>
    <DisplayName>John Doe</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>mary</ID>
        <DisplayName>Mary</DisplayName>
      </Grantee>
      <Permission>READ</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

💡 补充:请求体示例为编者构造示意。


5.7 List Bucket Multipart Uploads

要素 内容
操作名 List Bucket Multipart Uploads
HTTP 方法与路径 GET /{bucket}?uploads HTTP/1.1
关键参数/请求头 prefix、delimiter、key-marker、max-keys、max-uploads、upload-id-marker(全部可选)
响应要点 返回当前**进行中(in-progress)**的分片上传列表,即已 initiate 但尚未 complete 的上传

Syntax

GET /{bucket}?uploads HTTP/1.1

Parameters(可指定,但都不是必需的)

Name Type Description
prefix String Returns in-progress uploads whose keys contains the specified prefix.
delimiter String The delimiter between the prefix and the rest of the object name.
key-marker String The beginning marker for the list of uploads.
max-keys Integer The maximum number of in-progress uploads. The default is 1000.
max-uploads Integer The maximum number of multipart uploads. The range from 1-1000. The default is 1000.
upload-id-marker String Ignored if key-marker is not specified. Specifies the ID of first upload to list in lexicographical order at or following the ID.

Response Entities

Name Type Description
ListMultipartUploadsResult Container A container for the results.
ListMultipartUploadsResult.Prefix String The prefix specified by the prefix request parameter (if any).
Bucket String The bucket that will receive the bucket contents.
KeyMarker String The key marker specified by the key-marker request parameter (if any).
UploadIdMarker String The marker specified by the upload-id-marker request parameter (if any).
NextKeyMarker String The key marker to use in a subsequent request if IsTruncated is true.
NextUploadIdMarker String The upload ID marker to use in a subsequent request if IsTruncated is true.
MaxUploads Integer The max uploads specified by the max-uploads request parameter.
Delimiter String If set, objects with the same prefix will appear in the CommonPrefixes list.
IsTruncated Boolean If true, only a subset of the bucket's upload contents were returned.
Upload Container A container for Key, UploadId, InitiatorOwner, StorageClass, and Initiated elements.
Key String The key of the object once the multipart upload is complete.
UploadId String The ID that identifies the multipart upload.
Initiator Container Contains the ID and DisplayName of the user who initiated the upload.
DisplayName String The initiator's display name.
ID String The initiator's ID.
Owner Container A container for the ID and DisplayName of the user who owns the uploaded object.
StorageClass String The method used to store the resulting object. STANDARD or REDUCED_REDUNDANCY
Initiated Date The date and time the user initiated the upload.
CommonPrefixes Container If multiple objects contain the same prefix, they will appear in this list.
CommonPrefixes.Prefix String The substring of the key after the prefix as defined by the prefix request parameter.

响应示例

<ListMultipartUploadsResult>
  <Bucket>my-new-bucket</Bucket>
  <KeyMarker></KeyMarker>
  <UploadIdMarker></UploadIdMarker>
  <MaxUploads>1000</MaxUploads>
  <IsTruncated>false</IsTruncated>
  <Upload>
    <Key>largeobject</Key>
    <UploadId>2~abc123def456</UploadId>
    <Initiator>
      <ID>johndoe</ID>
      <DisplayName>John Doe</DisplayName>
    </Initiator>
    <StorageClass>STANDARD</StorageClass>
    <Initiated>2021-09-20T17:47:07.317Z</Initiated>
  </Upload>
</ListMultipartUploadsResult>

💡 补充:响应体示例为编者构造示意。


5.8 ENABLE/SUSPEND BUCKET VERSIONING

要素 内容
操作名 ENABLE/SUSPEND BUCKET VERSIONING
HTTP 方法与路径 PUT /{bucket}?versioning HTTP/1.1
关键参数/请求头 子资源 versioning;请求体 VersioningConfiguration / Status
权限 必须是 bucket owner
响应要点 设置已存在 bucket 的版本ing 状态

两种取值

Status 含义
Enabled Enables versioning for the objects in the bucket. All objects added to the bucket receive a unique version ID.
Suspended Disables versioning for the objects in the bucket. All objects added to the bucket receive the version ID null.

关键行为:如果某个 bucket 从未设置过 versioning 状态,则它没有 versioning state,此时 GET versioning 请求不会返回 versioning state 值。

Syntax

PUT  /{bucket}?versioning  HTTP/1.1

Request Entities

Name Type Description
VersioningConfiguration Container A container for the request.
Status String Sets the versioning state of the bucket. Valid Values: Suspended/Enabled

示例

PUT /my-new-bucket?versioning HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<VersioningConfiguration>
  <Status>Enabled</Status>
</VersioningConfiguration>

5.9 PUT BUCKET OBJECT LOCK

要素 内容
操作名 PUT BUCKET OBJECT LOCK
HTTP 方法与路径 PUT /{bucket}?object-lock HTTP/1.1
关键参数/请求头 子资源 object-lock;请求体 ObjectLockConfiguration
语义 在指定 bucket 上放置 Object Lock 配置;其中的 rule 会默认应用于该 bucket 中每一个新建对象
响应要点 若创建 bucket 时未启用 object lock,本操作会失败:409 InvalidBucketState;XML 不合规:400 MalformedXML

Syntax

PUT /{bucket}?object-lock HTTP/1.1

Request Entities

Name Type Description Required
ObjectLockConfiguration Container A container for the request. Yes
ObjectLockEnabled String Indicates whether this bucket has an Object Lock configuration enabled. Yes
Rule Container The Object Lock rule in place for the specified bucket. No
DefaultRetention Container The default retention period applied to new objects placed in the specified bucket. No
Mode String The default Object Lock retention mode. Valid Values: GOVERNANCE/COMPLIANCE Yes
Days Integer The number of days specified for the default retention period. No
Years Integer The number of years specified for the default retention period. No

HTTP Response

HTTP Status Status Code Description
400 MalformedXML The XML is not well-formed
409 InvalidBucketState The bucket object lock is not enabled

示例

PUT /my-new-bucket?object-lock HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<ObjectLockConfiguration>
  <ObjectLockEnabled>Enabled</ObjectLockEnabled>
  <Rule>
    <DefaultRetention>
      <Mode>COMPLIANCE</Mode>
      <Days>30</Days>
    </DefaultRetention>
  </Rule>
</ObjectLockConfiguration>

💡 补充:请求体示例为编者按实体表构造示意;注意先决条件——创建 bucket 时要带 x-amz-bucket-object-lock-enabled: true。


5.10 GET BUCKET OBJECT LOCK

要素 内容
操作名 GET BUCKET OBJECT LOCK
HTTP 方法与路径 GET /{bucket}?object-lock HTTP/1.1
关键参数/请求头 子资源 object-lock
响应要点 返回该 bucket 的 Object Lock 配置;配置中的 rule 默认应用于每个新建对象

Syntax

GET /{bucket}?object-lock HTTP/1.1

Response Entities

Name Type Description Required
ObjectLockConfiguration Container A container for the request. Yes
ObjectLockEnabled String Indicates whether this bucket has an Object Lock configuration enabled. Yes
Rule Container The Object Lock rule in place for the specified bucket. No
DefaultRetention Container The default retention period applied to new objects placed in the specified bucket. No
Mode String The default Object Lock retention mode. Valid Values: GOVERNANCE/COMPLIANCE Yes
Days Integer The number of days specified for the default retention period. No
Years Integer The number of years specified for the default retention period. No

响应示例

<ObjectLockConfiguration>
  <ObjectLockEnabled>Enabled</ObjectLockEnabled>
  <Rule>
    <DefaultRetention>
      <Mode>GOVERNANCE</Mode>
      <Years>1</Years>
    </DefaultRetention>
  </Rule>
</ObjectLockConfiguration>

5.11 Create Notification

要素 内容
操作名 Create Notification
HTTP 方法与路径 PUT /{bucket}?notification HTTP/1.1
关键参数/请求头 子资源 notification;请求体为 XML 编码的 NotificationConfiguration
语义 为指定 bucket 创建一个指向某个 topic 的 publisher
响应要点 400 MalformedXML / 400 InvalidArgument(缺 Id、Topic ARN 缺失或非法、Event 非法)/ 404 NoSuchBucket / 404 NoSuchKey(topic 不存在)

Syntax

PUT /{bucket}?notification HTTP/1.1

请求体格式(官方原文)

<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <TopicConfiguration>
        <Id></Id>
        <Topic></Topic>
        <Event></Event>
        <Filter>
            <S3Key>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Key>
            <S3Metadata>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Metadata>
            <S3Tags>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Tags>
        </Filter>
    </TopicConfiguration>
</NotificationConfiguration>

Request Entities

Name Type Description Required
NotificationConfiguration Container Holding list of TopicConfiguration entities Yes
TopicConfiguration Container Holding Id, Topic and list of Event entities Yes
Id String Name of the notification Yes
Topic String Topic ARN. Topic must be created beforehand Yes
Event String List of supported events see: S3 Notification Compatibility. Multiple Event entities can be used. If omitted, all "Created" and "Removed" events are handled. "Lifecycle" and "Synced" event types must be specified explicitly. No
Filter Container Holding S3Key, S3Metadata and S3Tags entities No
S3Key Container Holding a list of FilterRule entities, for filtering based on object key. At most, 3 entities may be in the list, with Name be prefix, suffix or regex. All filter rules in the list must match for the filter to match. No
S3Metadata Container Holding a list of FilterRule entities, for filtering based on object metadata. All filter rules in the list must match the metadata defined on the object. However, the object still match if it has other metadata entries not listed in the filter. No
S3Tags Container Holding a list of FilterRule entities, for filtering based on object tags. All filter rules in the list must match the tags defined on the object. However, the object still match it it has other tags not listed in the filter. No
S3Key.FilterRule Container Holding Name and Value entities. Name would be: prefix, suffix or regex. The Value would hold the key prefix, key suffix or a regular expression for matching the key, accordingly. Yes
S3Metadata.FilterRule Container Holding Name and Value entities. Name would be the name of the metadata attribute (e.g. x-amz-meta-xxx). The Value would be the expected value for this attribute. Yes
S3Tags.FilterRule Container Holding Name and Value entities. Name would be the tag key, and Value would be the tag value. Yes

HTTP Response

HTTP Status Status Code Description
400 MalformedXML The XML is not well-formed
400 InvalidArgument Missing Id; Missing/Invalid Topic ARN; Invalid Event
404 NoSuchBucket The bucket does not exist
404 NoSuchKey The topic does not exist

示例

PUT /my-new-bucket?notification HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <TopicConfiguration>
        <Id>image-uploads</Id>
        <Topic>arn:aws:sns:default::mytopic</Topic>
        <Event>s3:ObjectCreated:Put</Event>
        <Filter>
            <S3Key>
                <FilterRule>
                    <Name>prefix</Name>
                    <Value>images/</Value>
                </FilterRule>
            </S3Key>
        </Filter>
    </TopicConfiguration>
</NotificationConfiguration>

5.12 Delete Notification

要素 内容
操作名 Delete Notification
HTTP 方法与路径 DELETE /{bucket}?notification[=<notification-id>] HTTP/1.1
关键参数/请求头 参数 notification-id(String):要删除的通知名;不提供则删除该 bucket 上的所有通知
响应要点 404 NoSuchBucket(bucket 不存在)

官方注意点(原文三条):

  • Notification deletion is an extension to the S3 notification API(删除通知是 S3 notification API 的扩展)
  • When the bucket is deleted, any notification defined on it is also deleted(bucket 被删除时,其上的通知一并删除)
  • Deleting an unknown notification (e.g. double delete) is not considered an error(删除未知通知,如重复删除,不算错误)

Syntax

DELETE /{bucket}?notification[=<notification-id>] HTTP/1.1

Parameters

Name Type Description
notification-id String Name of the notification. If not provided, all notifications on the bucket are deleted

HTTP Response

HTTP Status Status Code Description
404 NoSuchBucket The bucket does not exist

示例

DELETE /my-new-bucket?notification=image-uploads HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

DELETE /my-new-bucket?notification HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

💡 补充:上面把「删除单个通知」与「删除全部通知」两个请求并列展示,均按官方 Syntax 构造。


5.13 Get/List Notification

要素 内容
操作名 Get/List Notification
HTTP 方法与路径 GET /{bucket}?notification[=<notification-id>] HTTP/1.1
关键参数/请求头 参数 notification-id(String):不提供则列出该 bucket 上所有通知
响应要点 响应体为 XML NotificationConfiguration;404 NoSuchKey(给定的 notification 不存在)/ 404 NoSuchBucket

Syntax

GET /{bucket}?notification[=<notification-id>] HTTP/1.1

Parameters

Name Type Description
notification-id String Name of the notification. If not provided, all notifications on the bucket are listed

响应体格式(官方原文)

<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <TopicConfiguration>
        <Id></Id>
        <Topic></Topic>
        <Event></Event>
        <Filter>
            <S3Key>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Key>
            <S3Metadata>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Metadata>
            <S3Tags>
                <FilterRule>
                    <Name></Name>
                    <Value></Value>
                </FilterRule>
            </S3Tags>
        </Filter>
    </TopicConfiguration>
</NotificationConfiguration>

Response Entities

Name Type Description Required
NotificationConfiguration Container Holding list of TopicConfiguration entities Yes
TopicConfiguration Container Holding Id, Topic and list of Event entities Yes
Id String Name of the notification Yes
Topic String Topic ARN Yes
Event String Handled event. Multiple Event entities may exist Yes
Filter Container Holding the filters configured for this notification No

HTTP Response

HTTP Status Status Code Description
404 NoSuchBucket The bucket does not exist
404 NoSuchKey The notification does not exist (if provided)

示例

GET /my-new-bucket?notification=image-uploads HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

5.14 Enable Bucket Logging

要素 内容
操作名 Enable Bucket Logging
HTTP 方法与路径 PUT /{bucket}?logging HTTP/1.1
关键参数/请求头 子资源 logging;请求体 XML BucketLoggingStatus
响应要点 400 MalformedXML / 400 InvalidArgument(缺必填值或值非法)/ 404 NoSuchBucket;若本次配置变更触发了当前 logging object 的 flush,响应体返回被 flush 的对象名

Syntax

PUT /{bucket}?logging HTTP/1.1

请求体格式(官方原文)

<BucketLoggingStatus xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <LoggingEnabled>
    <TargetBucket>string</TargetBucket>
    <TargetGrants>
      <Grant>
        <Grantee>
          <DisplayName>string</DisplayName>
          <EmailAddress>string</EmailAddress>
          <ID>string</ID>
          <xsi:type>string</xsi:type>
          <URI>string</URI>
        </Grantee>
        <Permission>string</Permission>
      </Grant>
    </TargetGrants>
    <TargetObjectKeyFormat>
      <PartitionedPrefix>
        <PartitionDateSource>DeliveryTime|EventTime</PartitionDateSource>
      </PartitionedPrefix>
      <SimplePrefix>
      </SimplePrefix>
    </TargetObjectKeyFormat>
    <TargetPrefix>string</TargetPrefix>
    <LoggingType>Standard|Journal</LoggingType>
    <ObjectRollTime>integer</ObjectRollTime>
    <Filter>
      <S3Key>
        <FilterRule>
          <Name>suffix/prefix/regex</Name>
          <Value></Value>
        </FilterRule>
      </S3Key>
    </Filter>
  </LoggingEnabled>
</BucketLoggingStatus>

Request Entities

Name Type Description Required
BucketLoggingStatus Container Enabling/Disabling logging configuration for the bucket. Yes
LoggingEnabled Container Holding the logging configuration for the bucket. Yes
TargetBucket String The bucket where the logs are stored. The log bucket cannot have bucket logging enabled. Yes
TargetGrants Container Not supported. The owner of the log bucket is the owner of the log objects. No
TargetObjectKeyFormat Container The format of the log object key. Contains either PartitionedPrefix or SimplePrefix entities. No
PartitionedPrefix Container Indicates a partitioned log object key format. Note that PartitionDateSource is ignored and hardcoded as DeliveryTime No
SimplePrefix Container Indicates a simple log object key format (default format) No
TargetPrefix String The prefix for the log objects. Used in both formats. May be used to distinguish between different source buckets writing log records to the same log bucket. No
LoggingType String The type of logging. Valid values are: Standard (default) all bucket operations are logged after being performed. The log record will contain all fields. Journal only operations that modify and object are logged. Will record the minimum subset of fields in the log record that is needed for journaling. No
ObjectRollTime Integer The time in seconds after which a new log object is created, and the previous log object added to the log bucket. Default is 3600 seconds (1 hour). No

注:请求体 XML 中出现的 Filter 元素在本页的 Request Entities 表中没有对应行——官方表未列出 Filter 的字段说明。

Response Entities

只有当配置变更触发了当前 logging object 的 flush 时,响应体才会带 XML,返回被 flush 的 logging object 名:

<PostBucketLoggingOutput xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <FlushedLoggingObject>string</FlushedLoggingObject>
</PostBucketLoggingOutput>

HTTP Response

HTTP Status Status Code Description
400 MalformedXML The XML is not well-formed
400 InvalidArgument Missing mandatory value or invalid value
404 NoSuchBucket The bucket does not exist

示例

PUT /my-new-bucket?logging HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<BucketLoggingStatus xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <LoggingEnabled>
    <TargetBucket>my-log-bucket</TargetBucket>
    <TargetPrefix>my-new-bucket/</TargetPrefix>
    <LoggingType>Standard</LoggingType>
    <ObjectRollTime>3600</ObjectRollTime>
  </LoggingEnabled>
</BucketLoggingStatus>

5.15 Disable Bucket Logging

要素 内容
操作名 Disable Bucket Logging
HTTP 方法与路径 PUT /{bucket}?logging HTTP/1.1(与 Enable 同路径,靠空的 BucketLoggingStatus 区分)
关键参数/请求头 子资源 logging;请求体为空 BucketLoggingStatus
响应要点 404 NoSuchBucket

Syntax

PUT /{bucket}?logging HTTP/1.1

请求体(官方原文)

<BucketLoggingStatus xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
</BucketLoggingStatus>

HTTP Response

HTTP Status Status Code Description
404 NoSuchBucket The bucket does not exist

示例

PUT /my-new-bucket?logging HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<BucketLoggingStatus xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
</BucketLoggingStatus>

5.16 Get Bucket Logging

要素 内容
操作名 Get Bucket Logging
HTTP 方法与路径 GET /{bucket}?logging HTTP/1.1
关键参数/请求头 子资源 logging
响应要点 响应头含 Last-Modified(logging 配置的日期时间);响应体为 XML BucketLoggingStatus;404 NoSuchBucket

Syntax

GET /{bucket}?logging HTTP/1.1

响应体格式(官方原文)

<BucketLoggingStatus xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <LoggingEnabled>
    <TargetBucket>string</TargetBucket>
    <TargetGrants>
      <Grant>
        <Grantee>
          <DisplayName>string</DisplayName>
          <EmailAddress>string</EmailAddress>
          <ID>string</ID>
          <xsi:type>string</xsi:type>
          <URI>string</URI>
        </Grantee>
        <Permission>string</Permission>
      </Grant>
    </TargetGrants>
    <TargetObjectKeyFormat>
      <PartitionedPrefix>
        <PartitionDateSource>DeliveryTime|EventTime</PartitionDateSource>
      </PartitionedPrefix>
      <SimplePrefix>
      </SimplePrefix>
    </TargetObjectKeyFormat>
    <TargetPrefix>string</TargetPrefix>
    <LoggingType>Standard|Journal</LoggingType>
    <ObjectRollTime>integer</ObjectRollTime>
    <Filter>
      <S3Key>
        <FilterRule>
          <Name>suffix/prefix/regex</Name>
          <Value></Value>
        </FilterRule>
      </S3Key>
    </Filter>
  </LoggingEnabled>
</BucketLoggingStatus>

HTTP Response

HTTP Status Status Code Description
404 NoSuchBucket The bucket does not exist

示例

GET /my-new-bucket?logging HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

5.17 Flush Bucket Logging

要素 内容
操作名 Flush Bucket Logging
HTTP 方法与路径 POST /{bucket}?logging HTTP/1.1
关键参数/请求头 子资源 logging
语义 把给定源 bucket 的 logging object 刷出(若未 flush,日志对象是惰性写入 log bucket 的);返回被 flush 的对象名;即使日志对象为空也会 flush
响应要点 201 Created(成功刷出待写日志对象)/ 404 NoSuchBucket

Syntax

POST /{bucket}?logging HTTP/1.1

Response Entities

<PostBucketLoggingOutput xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <FlushedLoggingObject>string</FlushedLoggingObject>
</PostBucketLoggingOutput>

HTTP Response

HTTP Status Status Code Description
201 Created Flushed pending logging object successfully
404 NoSuchBucket The bucket does not exist

示例

POST /my-new-bucket?logging HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

5.18 官方文档提到但 bucketops.rst 未给出规格的 bucket 操作

以下能力在 s3.rst 的 Features Support 表中标记为 Supported,但 bucketops.rst 没有对应小节,因此请求/响应规格文档未明确:

能力 s3.rst 状态 本册可提供的规格
Bucket Lifecycle Supported 文档未明确
Bucket Replication Partial(Permitted only across zones) 文档未明确
Bucket Website Supported 文档未明确
Bucket Request Payment Supported 文档未明确
Bucket Tagging Supported 文档未明确
Bucket Encryption 见 ACL 映射表 s3:GetBucketEncryption / s3:PutBucketEncryption 文档未明确(bucketops.rst 无对应小节)
Bucket Policy Supported(见 Features Support 表 Policy 行) 文档未明确(bucketops.rst 无对应小节)
Bucket CORS 见 ACL 映射表 s3:GetBucketCORS / s3:PutBucketCORS 文档未明确
Bucket Ownership Controls — 文档未明确(本册输入范围内未出现)
Get Bucket Info (HEAD) Supported 文档未明确(bucketops.rst 无 HEAD /{bucket} 小节)
Get Bucket Versioning 见 ACL 映射表 s3:GetBucketVersioning 文档未明确(只有 PUT versioning,无 GET versioning 小节)

引自:Bucket Operations


六、Object 操作(Object Operations)

objectops.rst 共定义 17 个 object 级操作,其中 5 个属于 multipart upload 全流程。

6.0 本章操作速查表

# 操作名 HTTP 方法与路径
1 Put Object PUT /{bucket}/{object}
2 Copy Object PUT /{dest-bucket}/{dest-object} + x-amz-copy-source
3 Remove Object DELETE /{bucket}/{object}
4 Get Object GET /{bucket}/{object}
5 Get Object Info HEAD /{bucket}/{object}
6 Get Object ACL GET /{bucket}/{object}?acl
7 Set Object ACL PUT /{bucket}/{object}?acl
8 Initiate Multi-part Upload POST /{bucket}/{object}?uploads
9 Multipart Upload Part PUT /{bucket}/{object}?partNumber=&uploadId=
10 List Multipart Upload Parts GET /{bucket}/{object}?uploadId=123
11 Complete Multipart Upload POST /{bucket}/{object}?uploadId=
12 Abort Multipart Upload DELETE /{bucket}/{object}?uploadId=
13 Append Object PUT /{bucket}/{object}?append&position=
14 Put Object Retention PUT /{bucket}/{object}?retention&versionId=
15 Get Object Retention GET /{bucket}/{object}?retention&versionId=
16 Put Object Legal Hold PUT /{bucket}/{object}?legal-hold&versionId=
17 Get Object Legal Hold GET /{bucket}/{object}?legal-hold&versionId=

6.1 Put Object

要素 内容
操作名 Put Object
HTTP 方法与路径 PUT /{bucket}/{object} HTTP/1.1
关键参数/请求头 content-md5、content-type、x-amz-meta-<...>、x-amz-acl
权限 需要对该 bucket 有 write 权限
响应要点 把对象加入 bucket

Syntax

PUT /{bucket}/{object} HTTP/1.1

Request Headers

Name Description Valid Values Required
content-md5 A base64 encoded MD-5 hash of the message. A string. No defaults or constraints. No
content-type A standard MIME type. Any MIME type. Default: binary/octet-stream No
x-amz-meta-<...> User metadata. Stored with the object. A string up to 8kb. No defaults. No
x-amz-acl A canned ACL. private, public-read, public-read-write, authenticated-read No

示例

PUT /my-new-bucket/hello.txt HTTP/1.1
Host: cname.domain.com
content-type: text/plain
x-amz-meta-author: johndoe
x-amz-acl: public-read

Hello World!

6.2 Copy Object

要素 内容
操作名 Copy Object
HTTP 方法与路径 PUT /{dest-bucket}/{dest-object} HTTP/1.1,请求头 x-amz-copy-source: {source-bucket}/{source-object}
关键参数/请求头 x-amz-copy-source(必填)、x-amz-acl、x-amz-copy-if-modified-since、x-amz-copy-if-unmodified-since、x-amz-copy-if-match、x-amz-copy-if-none-match
响应要点 返回 CopyObjectResult 容器,含 LastModified 与 Etag

Syntax

PUT /{dest-bucket}/{dest-object} HTTP/1.1
x-amz-copy-source: {source-bucket}/{source-object}

Request Headers

Name Description Valid Values Required
x-amz-copy-source The source bucket name + object name. {bucket}/{obj} Yes
x-amz-acl A canned ACL. private, public-read, public-read-write, authenticated-read No
x-amz-copy-if-modified-since Copies only if modified since the timestamp. Timestamp No
x-amz-copy-if-unmodified-since Copies only if unmodified since the timestamp. Timestamp No
x-amz-copy-if-match Copies only if object ETag matches ETag. Entity Tag No
x-amz-copy-if-none-match Copies only if object ETag doesn't match. Entity Tag No

Response Entities

Name Type Description
CopyObjectResult Container A container for the response elements.
LastModified Date The last modified date of the source object.
Etag String The ETag of the new object.

示例

PUT /my-new-bucket/hello-copy.txt HTTP/1.1
Host: cname.domain.com
x-amz-copy-source: my-new-bucket/hello.txt
x-amz-acl: private

Authorization: AWS {access-key}:{hash-of-header-and-secret}
<CopyObjectResult>
  <LastModified>2021-09-20T17:47:07.317Z</LastModified>
  <Etag>&quot;5e5a2c1f7f0e1d2c3b4a5968778695a4&quot;</Etag>
</CopyObjectResult>

6.3 Remove Object

要素 内容
操作名 Remove Object
HTTP 方法与路径 DELETE /{bucket}/{object} HTTP/1.1
关键参数/请求头 x-amz-delete-if-unmodified-since(可选)
权限 需要对所在 bucket 有 WRITE 权限
响应要点 删除对象

Syntax

DELETE /{bucket}/{object} HTTP/1.1

Request Headers

Name Description Valid Values Required
x-amz-delete-if-unmodified-since Deletes only if unmodified since the timestamp a datetime string No

示例

DELETE /my-new-bucket/goodbye.txt HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.4 Get Object

要素 内容
操作名 Get Object
HTTP 方法与路径 GET /{bucket}/{object} HTTP/1.1
关键参数/请求头 range、if-modified-since、if-unmodified-since、if-match、if-none-match
响应要点 响应头 Content-Range:仅当请求中指定了 range 头时才返回

Syntax

GET /{bucket}/{object} HTTP/1.1

Request Headers

Name Description Valid Values Required
range The range of the object to retrieve. Range: bytes=beginbyte-endbyte No
if-modified-since Gets only if modified since the timestamp. Timestamp No
if-unmodified-since Gets only if not modified since the timestamp. Timestamp No
if-match Gets only if object ETag matches ETag. Entity Tag No
if-none-match Gets only if object ETag doesn't match. Entity Tag No

Response Headers

Name Description
Content-Range Data range, will only be returned if the range header field was specified in the request

示例:读取前 1024 字节

GET /my-new-bucket/perl_poetry.pdf HTTP/1.1
Host: cname.domain.com
range: bytes=0-1023

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.5 Get Object Info

要素 内容
操作名 Get Object Info
HTTP 方法与路径 HEAD /{bucket}/{object} HTTP/1.1
关键参数/请求头 与 Get Object 相同:range、if-modified-since、if-unmodified-since、if-match、if-none-match
响应要点 返回与 Get Object 相同的头信息,但只含 metadata,不含对象数据 payload

Syntax

HEAD /{bucket}/{object} HTTP/1.1

Request Headers

Name Description Valid Values Required
range The range of the object to retrieve. Range: bytes=beginbyte-endbyte No
if-modified-since Gets only if modified since the timestamp. Timestamp No
if-unmodified-since Gets only if not modified since the timestamp. Timestamp No
if-match Gets only if object ETag matches ETag. Entity Tag No
if-none-match Gets only if object ETag doesn't match Entity Tag No

示例

HEAD /my-new-bucket/hello.txt HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.6 Get Object ACL

要素 内容
操作名 Get Object ACL
HTTP 方法与路径 GET /{bucket}/{object}?acl HTTP/1.1
关键参数/请求头 子资源 acl
响应要点 返回 AccessControlPolicy 容器

Syntax

GET /{bucket}/{object}?acl HTTP/1.1

Response Entities

Name Type Description
AccessControlPolicy Container A container for the response.
AccessControlList Container A container for the ACL information.
Owner Container A container for the object owner's ID and DisplayName.
ID String The object owner's ID.
DisplayName String The object owner's display name.
Grant Container A container for Grantee and Permission.
Grantee Container A container for the DisplayName and ID of the user receiving a grant of permission.
Permission String The permission given to the Grantee object.

响应示例

<AccessControlPolicy>
  <Owner>
    <ID>johndoe</ID>
    <DisplayName>John Doe</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>johndoe</ID>
        <DisplayName>John Doe</DisplayName>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

6.7 Set Object ACL

要素 内容
操作名 Set Object ACL
HTTP 方法与路径 PUT /{bucket}/{object}?acl
关键参数/请求头 子资源 acl;请求体 AccessControlPolicy
响应要点 设置对象 ACL

Syntax

PUT /{bucket}/{object}?acl

Request Entities

Name Type Description
AccessControlPolicy Container A container for the response.
AccessControlList Container A container for the ACL information.
Owner Container A container for the object owner's ID and DisplayName.
ID String The object owner's ID.
DisplayName String The object owner's display name.
Grant Container A container for Grantee and Permission.
Grantee Container A container for the DisplayName and ID of the user receiving a grant of permission.
Permission String The permission given to the Grantee object.

示例

PUT /my-new-bucket/hello.txt?acl HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

<AccessControlPolicy>
  <Owner>
    <ID>johndoe</ID>
    <DisplayName>John Doe</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group">
        <URI>http://acs.amazonaws.com/groups/global/AllUsers</URI>
      </Grantee>
      <Permission>READ</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

6.8 Initiate Multi-part Upload

要素 内容
操作名 Initiate Multi-part Upload
HTTP 方法与路径 POST /{bucket}/{object}?uploads
关键参数/请求头 content-md5、content-type、x-amz-meta-<...>、x-amz-acl
响应要点 返回 InitiatedMultipartUploadsResult 容器,核心是 UploadId

Syntax

POST /{bucket}/{object}?uploads

Request Headers

Name Description Valid Values Required
content-md5 A base64 encoded MD-5 hash of the message. A string. No defaults or constraints. No
content-type A standard MIME type. Any MIME type. Default: binary/octet-stream No
x-amz-meta-<...> User metadata. Stored with the object. A string up to 8kb. No defaults. No
x-amz-acl A canned ACL. private, public-read, public-read-write, authenticated-read No

Response Entities

Name Type Description
InitiatedMultipartUploadsResult Container A container for the results.
Bucket String The bucket that will receive the object contents.
Key String The key specified by the key request parameter (if any).
UploadId String The ID specified by the upload-id request parameter identifying the multipart upload (if any).

响应示例

<InitiatedMultipartUploadsResult>
  <Bucket>my-new-bucket</Bucket>
  <Key>largeobject</Key>
  <UploadId>2~abc123def456</UploadId>
</InitiatedMultipartUploadsResult>

6.9 Multipart Upload Part

要素 内容
操作名 Multipart Upload Part
HTTP 方法与路径 PUT /{bucket}/{object}?partNumber=&uploadId= HTTP/1.1
关键参数/请求头 查询参数 partNumber、uploadId
响应要点 404 NoSuchUpload:指定的 upload-id 与该对象上任何已 initiate 的上传都不匹配

Syntax

PUT /{bucket}/{object}?partNumber=&uploadId= HTTP/1.1

HTTP Response

HTTP Status Status Code Description
404 NoSuchUpload Specified upload-id does not match any initiated upload on this object

示例

PUT /my-new-bucket/largeobject?partNumber=1&uploadId=2~abc123def456 HTTP/1.1
Host: cname.domain.com
Content-Length: 5242880

<5MB 分片数据>

6.10 List Multipart Upload Parts

要素 内容
操作名 List Multipart Upload Parts
HTTP 方法与路径 GET /{bucket}/{object}?uploadId=123 HTTP/1.1
关键参数/请求头 查询参数 uploadId
响应要点 返回 ListPartsResult 容器;IsTruncated 为 true 时可用 NextPartNumberMarker 翻页

Syntax

GET /{bucket}/{object}?uploadId=123 HTTP/1.1

Response Entities

Name Type Description
ListPartsResult Container A container for the results.
Bucket String The bucket that will receive the object contents.
Key String The key specified by the key request parameter (if any).
UploadId String The ID specified by the upload-id request parameter identifying the multipart upload (if any).
Initiator Container Contains the ID and DisplayName of the user who initiated the upload.
ID String The initiator's ID.
DisplayName String The initiator's display name.
Owner Container A container for the ID and DisplayName of the user who owns the uploaded object.
StorageClass String The method used to store the resulting object. STANDARD or REDUCED_REDUNDANCY
PartNumberMarker String The part marker to use in a subsequent request if IsTruncated is true. Precedes the list.
NextPartNumberMarker String The next part marker to use in a subsequent request if IsTruncated is true. The end of the list.
MaxParts Integer The max parts allowed in the response as specified by the max-parts request parameter.
IsTruncated Boolean If true, only a subset of the object's upload contents were returned.
Part Container A container for LastModified, PartNumber, ETag and Size elements.
LastModified Date Date and time at which the part was uploaded.
PartNumber Integer The identification number of the part.
ETag String The part's entity tag.
Size Integer The size of the uploaded part.

响应示例

<ListPartsResult>
  <Bucket>my-new-bucket</Bucket>
  <Key>largeobject</Key>
  <UploadId>2~abc123def456</UploadId>
  <PartNumberMarker>0</PartNumberMarker>
  <NextPartNumberMarker>1</NextPartNumberMarker>
  <MaxParts>1000</MaxParts>
  <IsTruncated>false</IsTruncated>
  <Part>
    <LastModified>2021-09-20T17:47:07.317Z</LastModified>
    <PartNumber>1</PartNumber>
    <ETag>&quot;a1b2c3d4e5f6&quot;</ETag>
    <Size>5242880</Size>
  </Part>
</ListPartsResult>

6.11 Complete Multipart Upload

要素 内容
操作名 Complete Multipart Upload
HTTP 方法与路径 POST /{bucket}/{object}?uploadId= HTTP/1.1
关键参数/请求头 查询参数 uploadId;请求体 CompleteMultipartUpload(必填)
语义 组装已上传的分片并创建新对象,从而完成分片上传
响应要点 返回 CompleteMultipartUploadResult,含 Location、Bucket、Key、ETag

Syntax

POST /{bucket}/{object}?uploadId= HTTP/1.1

Request Entities

Name Type Description Required
CompleteMultipartUpload Container A container consisting of one or more parts. Yes
Part Container A container for the PartNumber and ETag. Yes
PartNumber Integer The identifier of the part. Yes
ETag String The part's entity tag. Yes

Response Entities

Name Type Description
CompleteMultipartUploadResult Container A container for the response.
Location URI The resource identifier (path) of the new object.
Bucket String The name of the bucket that contains the new object.
Key String The object's key.
ETag String The entity tag of the new object.

示例

POST /my-new-bucket/largeobject?uploadId=2~abc123def456 HTTP/1.1
Host: cname.domain.com

<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>&quot;a1b2c3d4e5f6&quot;</ETag>
  </Part>
  <Part>
    <PartNumber>2</PartNumber>
    <ETag>&quot;b2c3d4e5f6a1&quot;</ETag>
  </Part>
</CompleteMultipartUpload>

6.12 Abort Multipart Upload

要素 内容
操作名 Abort Multipart Upload
HTTP 方法与路径 DELETE /{bucket}/{object}?uploadId= HTTP/1.1
关键参数/请求头 查询参数 uploadId
响应要点 取消该分片上传(官方未在本节给出额外响应实体/错误码)

Syntax

DELETE /{bucket}/{object}?uploadId= HTTP/1.1

示例

DELETE /my-new-bucket/largeobject?uploadId=2~abc123def456 HTTP/1.1
Host: cname.domain.com

Authorization: AWS {access-key}:{hash-of-header-and-secret}

6.13 Append Object

要素 内容
操作名 Append Object
HTTP 方法与路径 PUT /{bucket}/{object}?append&position= HTTP/1.1
关键参数/请求头 查询参数 append、position;头 content-md5、content-type、x-amz-meta-<...>、x-amz-acl
权限 需要对该 bucket 有 write 权限
响应要点 响应头 x-rgw-next-append-position(下一次追加的位置)

重要限制(官方原文三条,均为加粗强调)

  • Append Object can't be used if bucket versioning is enabled or suspended.(bucket 开启或挂起 versioning 时不能使用 Append Object)
  • Synced object will become normal in multisite, but you can still append to the original object.(在 multisite 中同步过去的对象会变成 normal 类型,但仍可对原始对象继续追加)
  • Compression and encryption features are disabled for Appendable objects.(Appendable 对象禁用压缩与加密)

对象类型区分:Append Object 创建的对象类型是 Appendable Object;Put Object 上传的对象类型是 Normal Object。

Syntax

PUT /{bucket}/{object}?append&position= HTTP/1.1

Request Headers

Name Description Valid Values Required
content-md5 A base64 encoded MD-5 hash of the message. A string. No defaults or constraints. No
content-type A standard MIME type. Any MIME type. Default: binary/octet-stream No
x-amz-meta-<...> User metadata. Stored with the object. A string up to 8kb. No defaults. No
x-amz-acl A canned ACL. private, public-read, public-read-write, authenticated-read No

Response Headers

Name Description
x-rgw-next-append-position Next position to append object

HTTP Response

HTTP Status Status Code Description
409 PositionNotEqualToLength Specified position does not match object length
409 ObjectNotAppendable Specified object can not be appended
409 InvalidBucketstate Bucket versioning is enabled or suspended

示例

PUT /my-new-bucket/mylog.txt?append&position=0 HTTP/1.1
Host: cname.domain.com
content-type: text/plain

first line
HTTP/1.1 200 OK
x-rgw-next-append-position: 11

💡 补充:响应头示例为编者按 Response Headers 表构造示意。


6.14 Put Object Retention

要素 内容
操作名 Put Object Retention
HTTP 方法与路径 PUT /{bucket}/{object}?retention&versionId= HTTP/1.1
关键参数/请求头 查询参数 retention、versionId;请求体 Retention
响应要点 在对象上放置 Object Retention 配置

Syntax

PUT /{bucket}/{object}?retention&versionId= HTTP/1.1

Request Entities

Name Type Description Required
Retention Container A container for the request. Yes
Mode String Retention mode for the specified object. Valid Values: GOVERNANCE/COMPLIANCE Yes
RetainUntilDate Timestamp Retention date. Format: 2020-01-05T00:00:00.000Z Yes

示例

PUT /my-new-bucket/hello.txt?retention&versionId=abc123 HTTP/1.1
Host: cname.domain.com

<Retention>
  <Mode>GOVERNANCE</Mode>
  <RetainUntilDate>2026-01-05T00:00:00.000Z</RetainUntilDate>
</Retention>

6.15 Get Object Retention

要素 内容
操作名 Get Object Retention
HTTP 方法与路径 GET /{bucket}/{object}?retention&versionId= HTTP/1.1
关键参数/请求头 查询参数 retention、versionId
响应要点 返回该对象的 Retention 配置

Syntax

GET /{bucket}/{object}?retention&versionId= HTTP/1.1

Response Entities

Name Type Description Required
Retention Container A container for the request. Yes
Mode String Retention mode for the specified object. Valid Values: GOVERNANCE/COMPLIANCE Yes
RetainUntilDate Timestamp Retention date. Format: 2020-01-05T00:00:00.000Z Yes

响应示例

<Retention>
  <Mode>COMPLIANCE</Mode>
  <RetainUntilDate>2026-01-05T00:00:00.000Z</RetainUntilDate>
</Retention>

要素 内容
操作名 Put Object Legal Hold
HTTP 方法与路径 PUT /{bucket}/{object}?legal-hold&versionId= HTTP/1.1
关键参数/请求头 查询参数 legal-hold、versionId;请求体 LegalHold
响应要点 对指定对象应用 Legal Hold 配置

Syntax

PUT /{bucket}/{object}?legal-hold&versionId= HTTP/1.1

Request Entities

Name Type Description Required
LegalHold Container A container for the request. Yes
Status String Indicates whether the specified object has a Legal Hold in place. Valid Values: ON/OFF Yes

示例

PUT /my-new-bucket/hello.txt?legal-hold&versionId=abc123 HTTP/1.1
Host: cname.domain.com

<LegalHold>
  <Status>ON</Status>
</LegalHold>

要素 内容
操作名 Get Object Legal Hold
HTTP 方法与路径 GET /{bucket}/{object}?legal-hold&versionId= HTTP/1.1
关键参数/请求头 查询参数 legal-hold、versionId
响应要点 返回该对象当前的 Legal Hold 状态

Syntax

GET /{bucket}/{object}?legal-hold&versionId= HTTP/1.1

Response Entities

Name Type Description Required
LegalHold Container A container for the request. Yes
Status String Indicates whether the specified object has a Legal Hold in place. Valid Values: ON/OFF Yes

响应示例

<LegalHold>
  <Status>ON</Status>
</LegalHold>

6.18 Multipart Upload 全流程串联

官方 objectops.rst 把 multipart upload 拆成 5 个独立操作,未给出串联流程图。下面按操作顺序串联(步骤编号与请求/响应字段均来自上述各节):

1) Initiate  : POST   /{bucket}/{object}?uploads
              -> InitiatedMultipartUploadsResult.UploadId = "2~abc123def456"

2) Upload    : PUT    /{bucket}/{object}?partNumber=1&uploadId=2~abc123def456
              PUT    /{bucket}/{object}?partNumber=2&uploadId=2~abc123def456
              ...(失败可能返回 404 NoSuchUpload)

3) List      : GET    /{bucket}/{object}?uploadId=123
              -> ListPartsResult.Part[].PartNumber / ETag / Size
              (可选,用于断点续传时确认已上传分片)

4) Complete  : POST   /{bucket}/{object}?uploadId=2~abc123def456
              请求体 CompleteMultipartUpload.Part[] = {PartNumber, ETag}
              -> CompleteMultipartUploadResult.{Location, Bucket, Key, ETag}

   或 Abort  : DELETE /{bucket}/{object}?uploadId=2~abc123def456

💡 补充:上面流程图为编者按五个操作的 Syntax 与响应实体串联整理,官方未给出成文流程。

6.19 官方文档提到但 objectops.rst 未给出规格的 object 操作

能力 出处 本册可提供的规格
POST Object s3.rst Features Support:Supported 文档未明确(objectops.rst 无 POST Object 小节)
Object Tagging(Get/Put/Delete Object Tagging) s3.rst:Supported 文档未明确(objectops.rst 无 tagging 小节)
Object Versioning(Get/Put Object Version ACL 等) authentication.rst ACL 映射表 文档未明确(objectops.rst 无版本相关小节)
Get Object Torrent authentication.rst ACL 映射表 s3:GetObjectTorrent / s3:GetObjectVersionTorrent 文档未明确(objectops.rst 无 torrent 小节)
RestoreObject authentication.rst ACL 映射表 s3:RestoreObject 文档未明确
Storage Class s3.rst:Supported 文档未明确(仅 GET Bucket 响应实体中出现 StorageClass 字段)

引自:Object Operations


七、S3 SDK 示例

本章覆盖 7 种语言的官方示例页:C++、C#、Java、Perl、PHP、Python、Ruby。代码原文逐字保留。

7.1 C++(libs3)

Setup — includes 与全局变量(原文)

#include "libs3.h"
#include <stdlib.h>
#include <iostream>
#include <fstream>

const char access_key[] = "ACCESS_KEY";
const char secret_key[] = "SECRET_KEY";
const char host[] = "HOST";
const char sample_bucket[] = "sample_bucket";
const char sample_key[] = "hello.txt";
const char sample_file[] = "resource/hello.txt";
const char *security_token = NULL;
const char *auth_region = NULL;

S3BucketContext bucketContext =
{
	host,
	sample_bucket,
	S3ProtocolHTTP,
	S3UriStylePath,
	access_key,
	secret_key,
	security_token,
	auth_region
};

S3Status responsePropertiesCallback(
		const S3ResponseProperties *properties,
		void *callbackData)
{
	return S3StatusOK;
}

static void responseCompleteCallback(
		S3Status status,
		const S3ErrorDetails *error,
		void *callbackData)
{
	return;
}

S3ResponseHandler responseHandler =
{
	&responsePropertiesCallback,
	&responseCompleteCallback
};

创建(与关闭)连接

S3_initialize("s3", S3_INIT_ALL, host);
// Do stuff...
S3_deinitialize();

列出拥有的 bucket

static S3Status listServiceCallback(
		const char *ownerId,
		const char *ownerDisplayName,
		const char *bucketName,
		int64_t creationDate, void *callbackData)
{
	bool *header_printed = (bool*) callbackData;
	if (!*header_printed) {
		*header_printed = true;
		printf("%-22s", "       Bucket");
		printf("  %-20s  %-12s", "     Owner ID", "Display Name");
		printf("\n");
		printf("----------------------");
		printf("  --------------------" "  ------------");
		printf("\n");
	}

	printf("%-22s", bucketName);
	printf("  %-20s  %-12s", ownerId ? ownerId : "", ownerDisplayName ? ownerDisplayName : "");
	printf("\n");

	return S3StatusOK;
}

S3ListServiceHandler listServiceHandler =
{
	responseHandler,
	&listServiceCallback
};
bool header_printed = false;
S3_list_service(S3ProtocolHTTP, access_key, secret_key, security_token, host, 
                auth_region, NULL, 0, &listServiceHandler, &header_printed);

创建 bucket

S3_create_bucket(S3ProtocolHTTP, access_key, secret_key, NULL, host, sample_bucket, S3CannedAclPrivate, NULL, NULL, &responseHandler, NULL);

列出 bucket 内容

static S3Status listBucketCallback(
		int isTruncated,
		const char *nextMarker,
		int contentsCount,
		const S3ListBucketContent *contents,
		int commonPrefixesCount,
		const char **commonPrefixes,
		void *callbackData)
{
	printf("%-22s",	"      Object Name");
	printf("  %-5s  %-20s", "Size", "   Last Modified");
	printf("\n");
	printf("----------------------");
	printf("  -----" "  --------------------");
	printf("\n");

    for (int i = 0; i < contentsCount; i++) {
	char timebuf[256];
		char sizebuf[16];
	const S3ListBucketContent *content = &(contents[i]);
		time_t t = (time_t) content->lastModified;

		strftime(timebuf, sizeof(timebuf), "%Y-%m-%dT%H:%M:%SZ", gmtime(&t));
		sprintf(sizebuf, "%5llu", (unsigned long long) content->size);
		printf("%-22s  %s  %s\n", content->key, sizebuf, timebuf);
    }

    return S3StatusOK;
}

S3ListBucketHandler listBucketHandler =
{
	responseHandler,
	&listBucketCallback
};
S3_list_bucket(&bucketContext, NULL, NULL, NULL, 0, NULL, 0, &listBucketHandler, NULL);

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(bucket 必须为空,否则失败)

S3_delete_bucket(S3ProtocolHTTP, S3UriStylePath, access_key, secret_key, 0, host, sample_bucket, NULL, NULL, 0, &responseHandler, NULL);

从文件创建对象

#include <sys/stat.h>
typedef struct put_object_callback_data
{
    FILE *infile;
    uint64_t contentLength;
} put_object_callback_data;


static int putObjectDataCallback(int bufferSize, char *buffer, void *callbackData)
{
    put_object_callback_data *data = (put_object_callback_data *) callbackData;

    int ret = 0;

    if (data->contentLength) {
	int toRead = ((data->contentLength > (unsigned) bufferSize) ? (unsigned) bufferSize : data->contentLength);
		ret = fread(buffer, 1, toRead, data->infile);
    }
    data->contentLength -= ret;
    return ret;
}

put_object_callback_data data;
struct stat statbuf;
if (stat(sample_file, &statbuf) == -1) {
	fprintf(stderr, "\nERROR: Failed to stat file %s: ", sample_file);
	perror(0);
	exit(-1);
}

int contentLength = statbuf.st_size;
data.contentLength = contentLength;

if (!(data.infile = fopen(sample_file, "r"))) {
	fprintf(stderr, "\nERROR: Failed to open input file %s: ", sample_file);
	perror(0);
	exit(-1);
}

S3PutObjectHandler putObjectHandler =
{
	responseHandler,
	&putObjectDataCallback
};

S3_put_object(&bucketContext, sample_key, contentLength, NULL, NULL, 0, &putObjectHandler, &data);
fclose(data.infile);

下载对象到文件

static S3Status getObjectDataCallback(int bufferSize, const char *buffer, void *callbackData)
{
	FILE *outfile = (FILE *) callbackData;
	size_t wrote = fwrite(buffer, 1, bufferSize, outfile);
	return ((wrote < (size_t) bufferSize) ? S3StatusAbortedByCallback : S3StatusOK);
}

S3GetObjectHandler getObjectHandler =
{
	responseHandler,
	&getObjectDataCallback
};
FILE *outfile = stdout;
S3_get_object(&bucketContext, sample_key, NULL, 0, 0, NULL, 0, &getObjectHandler, outfile);

删除对象

S3ResponseHandler deleteResponseHandler =
{
	0,
	&responseCompleteCallback
};
S3_delete_object(&bucketContext, sample_key, 0, 0, &deleteResponseHandler, 0);

修改对象 ACL(授予另一用户 full control)

#include <string.h>
char ownerId[] = "owner";
char ownerDisplayName[] = "owner";
char granteeId[] = "grantee";
char granteeDisplayName[] = "grantee";

S3AclGrant grants[] = {
	{
		S3GranteeTypeCanonicalUser,
		{{}},
		S3PermissionFullControl
	},
	{
		S3GranteeTypeCanonicalUser,
		{{}},
		S3PermissionReadACP
	},
	{
		S3GranteeTypeAllUsers,
		{{}},
		S3PermissionRead
	}
};

strncpy(grants[0].grantee.canonicalUser.id, ownerId, S3_MAX_GRANTEE_USER_ID_SIZE);
strncpy(grants[0].grantee.canonicalUser.displayName, ownerDisplayName, S3_MAX_GRANTEE_DISPLAY_NAME_SIZE);

strncpy(grants[1].grantee.canonicalUser.id, granteeId, S3_MAX_GRANTEE_USER_ID_SIZE);
strncpy(grants[1].grantee.canonicalUser.displayName, granteeDisplayName, S3_MAX_GRANTEE_DISPLAY_NAME_SIZE);

S3_set_acl(&bucketContext, sample_key, ownerId, ownerDisplayName, 3, grants, 0, &responseHandler, 0);

生成签名下载 URL(有效期 5 分钟)

#include <time.h>
char buffer[S3_MAX_AUTHENTICATED_QUERY_STRING_SIZE];
int64_t expires = time(NULL) + 60 * 5; // Current time + 5 minutes

S3_generate_authenticated_query_string(buffer, &bucketContext, sample_key, expires, NULL, "GET");

引自:C++ S3 Examples


7.2 C#(AWS SDK for .NET)

创建连接

using System;
using Amazon;
using Amazon.S3;
using Amazon.S3.Model;

string accessKey = "put your access key here!";
string secretKey = "put your secret key here!";

AmazonS3Config config = new AmazonS3Config();
config.ServiceURL = "objects.dreamhost.com";

AmazonS3Client s3Client = new AmazonS3Client(
	accessKey,
	secretKey,
	config
	);

列出拥有的 bucket

ListBucketsResponse response = client.ListBuckets();
foreach (S3Bucket b in response.Buckets)
{
	Console.WriteLine("{0}\t{1}", b.BucketName, b.CreationDate);
}

输出类似:

mahbuckat1	2011-04-21T18:05:39.000Z
mahbuckat2	2011-04-21T18:05:48.000Z
mahbuckat3	2011-04-21T18:07:18.000Z

创建 bucket

PutBucketRequest request = new PutBucketRequest();
request.BucketName = "my-new-bucket";
client.PutBucket(request);

列出 bucket 内容

ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = "my-new-bucket";
ListObjectsResponse response = client.ListObjects(request);
foreach (S3Object o in response.S3Objects)
{
	Console.WriteLine("{0}\t{1}\t{2}", o.Key, o.Size, o.LastModified);
}

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(必须为空)

DeleteBucketRequest request = new DeleteBucketRequest();
request.BucketName = "my-new-bucket";
client.DeleteBucket(request);

Forced Delete for Non-empty Buckets:官方标注 not available(不可用)。

创建对象

PutObjectRequest request = new PutObjectRequest();
request.BucketName  = "my-new-bucket";
request.Key         = "hello.txt";
request.ContentType = "text/plain";
request.ContentBody = "Hello World!";
client.PutObject(request);

修改对象 ACL

PutACLRequest request = new PutACLRequest();
request.BucketName = "my-new-bucket";
request.Key        = "hello.txt";
request.CannedACL  = S3CannedACL.PublicRead;
client.PutACL(request);

PutACLRequest request2 = new PutACLRequest();
request2.BucketName = "my-new-bucket";
request2.Key        = "secret_plans.txt";
request2.CannedACL  = S3CannedACL.Private;
client.PutACL(request2);

下载对象到文件

GetObjectRequest request = new GetObjectRequest();
request.BucketName = "my-new-bucket";
request.Key        = "perl_poetry.pdf";
GetObjectResponse response = client.GetObject(request);
response.WriteResponseStreamToFile("C:\\Users\\larry\\Documents\\perl_poetry.pdf");

删除对象

DeleteObjectRequest request = new DeleteObjectRequest();
request.BucketName = "my-new-bucket";
request.Key        = "goodbye.txt";
client.DeleteObject(request);

生成签名下载 URL(有效期 1 小时)

官方注意:C# S3 Library 没有生成非签名 URL 的方法,因此示例只展示签名 URL。

GetPreSignedUrlRequest request = new GetPreSignedUrlRequest();
request.BucketName = "my-bucket-name";
request.Key        = "secret_plans.txt";
request.Expires    = DateTime.Now.AddHours(1);
request.Protocol   = Protocol.HTTP;
string url = client.GetPreSignedURL(request);
Console.WriteLine(url);

输出类似:

http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

引自:C# S3 Examples


7.3 Java(AWS Java SDK 2.17.42)

Pre-requisites:所有示例基于 AWS Java SDK 2.17.42 编写,换用其他客户端可能需要改代码。

Setup — imports(原文)

import java.net.URI;
import java.net.URISyntaxException;
import java.nio.ByteBuffer;
import java.nio.file.Paths;
import java.util.List;
import java.util.ListIterator;
import java.time.Duration;

import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.Bucket;
import software.amazon.awssdk.services.s3.model.ListBucketsResponse;
import software.amazon.awssdk.services.s3.model.ListObjectsResponse;
import software.amazon.awssdk.services.s3.model.ObjectCannedACL;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.model.S3Object;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;

若只是测试 Ceph Object Storage 服务,建议使用 HTTP 协议而非 HTTPS。

创建连接

AwsBasicCredentials credentials = AwsBasicCredentials.create(accessKey, secretKey);

S3Client client = S3Client.builder()
	.endpointOverride(new URI("https://endpoint.com"))
	.credentialsProvider(StaticCredentialsProvider.create(credentials))
	.serviceConfiguration(srvcConf -> {
		srvcConf.pathStyleAccessEnabled();
	})
	.region(Region.US_EAST_1) // this is not used, but the AWS SDK requires it
	.build();

列出拥有的 bucket

ListBucketsResponse lbResponse = client.listBuckets();
for (Bucket bucket : lbResponse.buckets()) {
	System.out.println(bucket.name() + "\t" + bucket.creationDate());
}

输出类似:

mahbuckat1    2021-09-20T14:12:57.231Z
mahbuckat2    2021-09-20T14:12:59.402Z
mahbuckat3    2021-09-20T14:13:02.288Z

创建 bucket

client.createBucket(req -> {
	req.bucket("my-new-bucket");
});

列出 bucket 内容

ListObjectsResponse loResponse = client.listObjects(req -> {
	req.bucket("my-bucket");
});

for (S3Object object : loResponse.contents()) {
	System.out.println(
		object.key() + "\t" +
		object.size() + "\t" +
		object.lastModified()
	);
}

输出类似:

myphoto1.jpg	251262	2021-09-20T17:47:07.317Z
myphoto2.jpg	262518	2021-09-20T17:49:46.872Z

删除 bucket(必须为空)

client.deleteBucket(req -> {
	req.bucket("my-new-bucket");
});

Forced Delete for Non-empty Buckets:官方标注 not available。

创建对象

ByteBuffer input = ByteBuffer.wrap("Hello World!".getBytes());
client.putObject(
	req -> {
		req.bucket("my-bucket").key("hello.txt");
	},
	RequestBody.fromByteBuffer(input)
);

修改对象 ACL

client.putObjectAcl(req -> {
	req.bucket("my-bucket").key("hello.txt").acl(ObjectCannedACL.PUBLIC_READ);
});
client.putObjectAcl(req -> {
	req.bucket("my-bucket").key("secret_plans.txt").acl(ObjectCannedACL.PRIVATE);
});

下载对象到文件

client.getObject(
	req -> {
		req.bucket("my-bucket").key("perl_poetry.pdf");
	},
	Paths.get("/home/larry/documents/perl_poetry.pdf")
);

删除对象

client.deleteObject(req -> {
	req.bucket("my-bucket").key("goodbye.txt");
});

生成签名下载 URL

官方注意:Java library 没有生成非签名 URL 的方法,下方示例只生成签名 URL。

S3Presigner presigner = S3Presigner.builder()
	.endpointOverride(new URI("https://endpoint.com"))
	.credentialsProvider(StaticCredentialsProvider.create(credentials))
	.region(Region.US_EAST_1) // this is not used, but the AWS SDK requires it
	.build();

PresignedGetObjectRequest presignedRequest = presigner.presignGetObject(preReq -> {
	preReq.getObjectRequest(req -> {
		req.bucket("my-bucket").key("secret_plans.txt");
	}).signatureDuration(
		Duration.ofMinutes(20)
	);
});
System.out.println(presignedRequest.url());

输出类似:

https://endpoint.com/my-bucket/secret_plans.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210921T151408Z&X-Amz-SignedHeaders=host&X-Amz-Expires=1200&X-Amz-Credential=XXXXXXXXXXXX%2F20210921%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=yyyyyyyyyyyyyyyyyyyyyy

引自:Java S3 Examples


7.4 Perl(Amazon::S3)

创建连接

use Amazon::S3;
my $access_key = 'put your access key here!';
my $secret_key = 'put your secret key here!';

my $conn = Amazon::S3->new({
	aws_access_key_id     => $access_key,
	aws_secret_access_key => $secret_key,
	host                  => 'objects.dreamhost.com',
	secure                => 1,
	retry                 => 1,
});

列出拥有的 bucket

my @buckets = @{$conn->buckets->{buckets} || []};
foreach my $bucket (@buckets) {
	print $bucket->bucket . "\t" . $bucket->creation_date . "\n";
}

输出类似:

mahbuckat1	2011-04-21T18:05:39.000Z
mahbuckat2	2011-04-21T18:05:48.000Z
mahbuckat3	2011-04-21T18:07:18.000Z

创建 bucket

my $bucket = $conn->add_bucket({ bucket => 'my-new-bucket' });

列出 bucket 内容

my @keys = @{$bucket->list_all->{keys} || []};
foreach my $key (@keys) {
	print "$key->{key}\t$key->{size}\t$key->{last_modified}\n";
}

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(必须为空)

$conn->delete_bucket($bucket);

Forced Delete for Non-empty Buckets:官方标注 not available in the Amazon::S3 perl module。

创建对象

$bucket->add_key(
	'hello.txt', 'Hello World!',
	{ content_type => 'text/plain' },
);

修改对象 ACL

$bucket->set_acl({
	key       => 'hello.txt',
	acl_short => 'public-read',
});
$bucket->set_acl({
	key       => 'secret_plans.txt',
	acl_short => 'private',
});

下载对象到文件

$bucket->get_key_filename('perl_poetry.pdf', undef,
	'/home/larry/documents/perl_poetry.pdf');

删除对象

$bucket->delete_key('goodbye.txt');

生成下载 URL(非签名 + 签名,有效期 1 小时)

官方注意:Amazon::S3 模块没有生成下载 URL 的能力,示例改用 Muck::FS::S3(Amazon 的示例 S3 perl 模块,不在 CPAN)。可自行从 CPAN 安装 Muck::FS::S3,或手动安装 Amazon 的示例模块;若手动安装,可把示例中的 Muck::FS:: 去掉。

use Muck::FS::S3::QueryStringAuthGenerator;
my $generator = Muck::FS::S3::QueryStringAuthGenerator->new(
	$access_key,
	$secret_key,
	0, # 0 means use 'http'. set this to 1 for 'https'
	'objects.dreamhost.com',
);

my $hello_url = $generator->make_bare_url($bucket->bucket, 'hello.txt');
print $hello_url . "\n";

$generator->expires_in(3600); # 1 hour = 3600 seconds
my $plans_url = $generator->get($bucket->bucket, 'secret_plans.txt');
print $plans_url . "\n";

输出类似:

http://objects.dreamhost.com:80/my-bucket-name/hello.txt
http://objects.dreamhost.com:80/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

引自:Perl S3 Examples


7.5 PHP(AWS SDK for PHP)

安装 AWS PHP SDK(composer)

$ composer install aws/aws-sdk-php

创建连接

官方注意:客户端初始化需要 region,这里使用 ''(空字符串)。

<?php

use Aws\S3\S3Client;

define('AWS_KEY', 'place access key here');
define('AWS_SECRET_KEY', 'place secret key here');
$ENDPOINT = 'http://objects.dreamhost.com';

// require the amazon sdk from your composer vendor dir
require __DIR__.'/vendor/autoload.php';

// Instantiate the S3 class and point it at the desired host
$client = new S3Client([
    'region' => '',
    'version' => '2006-03-01',
    'endpoint' => $ENDPOINT,
    'credentials' => [
                'key' => AWS_KEY,
	'secret' => AWS_SECRET_KEY
    ],
    // Set the S3 class to use objects.dreamhost.com/bucket
    // instead of bucket.objects.dreamhost.com
    'use_path_style_endpoint' => true
]);

列出拥有的 bucket

<?php
$listResponse = $client->listBuckets();
$buckets = $listResponse['Buckets'];
foreach ($buckets as $bucket) {
    	    echo $bucket['Name'] . "\t" . $bucket['CreationDate'] . "\n";
}

输出类似:

mahbuckat1	2011-04-21T18:05:39.000Z
mahbuckat2	2011-04-21T18:05:48.000Z
mahbuckat3	2011-04-21T18:07:18.000Z

创建 bucket

<?php
$client->createBucket(['Bucket' => 'my-new-bucket']);

列出 bucket 内容

<?php
$objectsListResponse = $client->listObjects(['Bucket' => $bucketname]);
$objects = $objectsListResponse['Contents'] ?? [];
foreach ($objects as $object) {
    	    echo $object['Key'] . "\t" . $object['Size'] . "\t" . $object['LastModified'] . "\n";
}

注意:若 bucket 中对象超过 1000 个,需要检查 $objectsListResponse['isTruncated'],并用最后一个列出的 key 再次请求,直到 isTruncated 不为 true。

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(必须为空)

<?php
$client->deleteBucket(['Bucket' => 'my-old-bucket']);

创建对象

<?php
$client->putObject([
    	    'Bucket' => 'my-bucket-name',
    	    'Key' => 'hello.txt',
    	    'Body' => "Hello World!"
]);

修改对象 ACL

<?php
$client->putObjectAcl([
    	    'Bucket' => 'my-bucket-name',
    	    'Key' => 'hello.txt',
    	    'ACL' => 'public-read'
]);
$client->putObjectAcl([
    	    'Bucket' => 'my-bucket-name',
    	    'Key' => 'secret_plans.txt',
    	    'ACL' => 'private'
]);

删除对象

<?php
$client->deleteObject(['Bucket' => 'my-bucket-name', 'Key' => 'goodbye.txt']);

下载对象到文件

<?php
$object = $client->getObject(['Bucket' => 'my-bucket-name', 'Key' => 'poetry.pdf']);
file_put_contents('/home/larry/documents/poetry.pdf', $object['Body']->getContents());

生成下载 URL(非签名 + 签名 1 小时)

<?php
$hello_url = $client->getObjectUrl('my-bucket-name', 'hello.txt');
echo $hello_url."\n";

$secret_plans_cmd = $client->getCommand('GetObject', ['Bucket' => 'my-bucket-name', 'Key' => 'secret_plans.txt']);
$request = $client->createPresignedRequest($secret_plans_cmd, '+1 hour');
echo $request->getUri()."\n";

输出类似:

http://objects.dreamhost.com/my-bucket-name/hello.txt
http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=sandboxAccessKey%2F20190116%2F%2Fs3%2Faws4_request&X-Amz-Date=20190116T125520Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Signature=61921f07c73d7695e47a2192cf55ae030f34c44c512b2160bb5a936b2b48d923

引自:PHP S3 Examples


7.6 Python(boto / boto3)

创建连接

import boto
import boto.s3.connection
access_key = 'put your access key here!'
secret_key = 'put your secret key here!'

conn = boto.connect_s3(
	aws_access_key_id = access_key,
	aws_secret_access_key = secret_key,
	host = 'objects.dreamhost.com',
                #is_secure=False,               # uncomment if you are not using ssl
	calling_format = boto.s3.connection.OrdinaryCallingFormat(),
	)

列出拥有的 bucket

for bucket in conn.get_all_buckets():
	print("{name}\t{created}".format(
		name = bucket.name,
		created = bucket.creation_date,
	))

输出类似:

mahbuckat1	2011-04-21T18:05:39.000Z
mahbuckat2	2011-04-21T18:05:48.000Z
mahbuckat3	2011-04-21T18:07:18.000Z

创建 bucket

bucket = conn.create_bucket('my-new-bucket')

列出 bucket 内容

for key in bucket.list():
	print("{name}\t{size}\t{modified}".format(
		name = key.name,
		size = key.size,
		modified = key.last_modified,
	))

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(必须为空)

conn.delete_bucket(bucket.name)

Forced Delete for Non-empty Buckets:官方标注 not available in python。

创建对象

key = bucket.new_key('hello.txt')
key.set_contents_from_string('Hello World!')

上传对象或文件

key = bucket.new_key('logo.png')
key.set_contents_from_filename('logo.png')

修改对象 ACL

hello_key = bucket.get_key('hello.txt')
hello_key.set_canned_acl('public-read')
plans_key = bucket.get_key('secret_plans.txt')
plans_key.set_canned_acl('private')

下载对象到文件

key = bucket.get_key('perl_poetry.pdf')
key.get_contents_to_filename('/home/larry/documents/perl_poetry.pdf')

删除对象

bucket.delete_key('goodbye.txt')

生成下载 URL(非签名 + 签名 3600 秒)

hello_key = bucket.get_key('hello.txt')
hello_url = hello_key.generate_url(0, query_auth=False, force_http=True)
print(hello_url)

plans_key = bucket.get_key('secret_plans.txt')
plans_url = plans_key.generate_url(3600, query_auth=True, force_http=True)
print(plans_url)

输出类似:

http://objects.dreamhost.com/my-bucket-name/hello.txt
http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

Using S3 API Extensions

要用 boto3 客户端测试 RadosGW 对 S3 API 的扩展,需要把 extensions file 放到 ~/.aws/models/s3/2006-03-01/ 目录下。例如获取无序对象列表:

print(conn.list_objects(Bucket='my-new-bucket', AllowUnordered=True))

没有 extensions file 时,boto3 会报 AllowUnordered 参数非法。 官方给出的 extensions file 地址:https://github.com/ceph/ceph/blob/main/examples/rgw/boto3/service-2.sdk-extras.json

💡 补充:AllowUnordered 对应 5.3 节 GET Bucket 的非标准扩展参数 allow-unordered;两者是同一能力在 HTTP 层与 SDK 层的不同拼写。

引自:Python S3 Examples


7.7 Ruby

官方 Ruby 页给出两代 SDK的示例:aws-sdk gem ~>2(AWS::SDK)与 aws-s3 gem(AWS::S3)。

7.7.1 AWS::SDK(aws-sdk gem ~>2)

全局配置 + 实例化客户端

Aws.config.update(
	endpoint: 'https://objects.dreamhost.com.',
	access_key_id: 'my-access-key',
	secret_access_key: 'my-secret-key',
	force_path_style: true, 
	region: 'us-east-1'
)

s3_client = Aws::S3::Client.new

列出拥有的 bucket

s3_client.list_buckets.buckets.each do |bucket|
	puts "#{bucket.name}\t#{bucket.creation_date}"
end

输出类似:

mahbuckat1	2011-04-21T18:05:39.000Z
mahbuckat2	2011-04-21T18:05:48.000Z
mahbuckat3	2011-04-21T18:07:18.000Z

创建 bucket

s3_client.create_bucket(bucket: 'my-new-bucket')

私有 bucket(acl 可选值:private, public-read, public-read-write, authenticated-read):

s3_client.create_bucket(bucket: 'my-new-bucket', acl: 'private')

列出 bucket 内容

s3_client.get_objects(bucket: 'my-new-bucket').contents.each do |object|
	puts "#{object.key}\t#{object.size}\t#{object.last-modified}"
end

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

删除 bucket(必须为空)

s3_client.delete_bucket(bucket: 'my-new-bucket')

Forced Delete for Non-empty Buckets:先清空,再删除:

Aws::S3::Bucket.new('my-new-bucket', client: s3_client).clear!

s3_client.delete_bucket(bucket: 'my-new-bucket')

创建对象

s3_client.put_object(
	key: 'hello.txt',
	body: 'Hello World!',
	bucket: 'my-new-bucket',
	content_type: 'text/plain'
)

修改对象 ACL

s3_client.put_object_acl(bucket: 'my-new-bucket', key: 'hello.txt', acl: 'public-read')

s3_client.put_object_acl(bucket: 'my-new-bucket', key: 'private.txt', acl: 'private')

下载对象到文件

s3_client.get_object(bucket: 'my-new-bucket', key: 'poetry.pdf', response_target: '/home/larry/documents/poetry.pdf')

删除对象

s3_client.delete_object(key: 'goodbye.txt', bucket: 'my-new-bucket')

生成下载 URL(非签名 + 签名 1 小时)

puts Aws::S3::Object.new(
	key: 'hello.txt',
	bucket_name: 'my-new-bucket',
	client: s3_client
).public_url

puts Aws::S3::Object.new(
	key: 'secret_plans.txt',
	bucket_name: 'hermes_ceph_gem',
	client: s3_client
).presigned_url(:get, expires_in: 60 * 60)

输出类似:

http://objects.dreamhost.com/my-bucket-name/hello.txt
http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

7.7.2 AWS::S3(aws-s3 gem)

创建连接

AWS::S3::Base.establish_connection!(
	:server            => 'objects.dreamhost.com',
	:use_ssl           => true,
	:access_key_id     => 'my-access-key',
	:secret_access_key => 'my-secret-key'
)

列出拥有的 bucket

AWS::S3::Service.buckets.each do |bucket|
	puts "#{bucket.name}\t#{bucket.creation_date}"
end

创建 bucket

AWS::S3::Bucket.create('my-new-bucket')

列出 bucket 内容

new_bucket = AWS::S3::Bucket.find('my-new-bucket')
new_bucket.each do |object|
	puts "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}"
end

删除 bucket(必须为空)

AWS::S3::Bucket.delete('my-new-bucket')

Forced Delete for Non-empty Buckets

AWS::S3::Bucket.delete('my-new-bucket', :force => true)

创建对象

AWS::S3::S3Object.store(
	'hello.txt',
	'Hello World!',
	'my-new-bucket',
	:content_type => 'text/plain'
)

修改对象 ACL

policy = AWS::S3::S3Object.acl('hello.txt', 'my-new-bucket')
policy.grants = [ AWS::S3::ACL::Grant.grant(:public_read) ]
AWS::S3::S3Object.acl('hello.txt', 'my-new-bucket', policy)

policy = AWS::S3::S3Object.acl('secret_plans.txt', 'my-new-bucket')
policy.grants = []
AWS::S3::S3Object.acl('secret_plans.txt', 'my-new-bucket', policy)

下载对象到文件

open('/home/larry/documents/poetry.pdf', 'w') do |file|
	AWS::S3::S3Object.stream('poetry.pdf', 'my-new-bucket') do |chunk|
		file.write(chunk)
	end
end

删除对象

AWS::S3::S3Object.delete('goodbye.txt', 'my-new-bucket')

生成下载 URL(非签名 + 签名 1 小时)

puts AWS::S3::S3Object.url_for(
	'hello.txt',
	'my-new-bucket',
	:authenticated => false
)

puts AWS::S3::S3Object.url_for(
	'secret_plans.txt',
	'my-new-bucket',
	:expires_in => 60 * 60
)

输出类似:

http://objects.dreamhost.com/my-bucket-name/hello.txt
http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

引自:Ruby AWS::SDK Examples (aws-sdk gem ~>2)


八、Swift 接口总览

Ceph 提供一套 RESTful API,兼容 Swift API 的基本数据访问模型。

8.1 功能支持矩阵(Features Support)

下表逐行搬运 swift.rst 官方原表。

Feature(能力) 是否支持 限制说明(Remarks,原文)
Authentication Supported —
Get Account Metadata Supported —
Swift ACLs Supported Supports a subset of Swift ACLs(只支持 Swift ACL 的子集)
List Containers Supported —
Delete Container Supported —
Create Container Supported —
Get Container Metadata Supported —
Update Container Metadata Supported —
Delete Container Metadata Supported —
List Objects Supported —
Static Website Supported —
Create Object Supported —
Create Large Object Supported —
Delete Object Supported —
Get Object Supported —
Copy Object Supported —
Get Object Metadata Supported —
Update Object Metadata Supported —
Expiring Objects Supported —
Temporary URLs Partial Support No support for container-level keys(不支持容器级密钥)
Object Versioning Partial Support No support for X-History-Location
CORS Not Supported —

8.2 Swift API 页面结构

官方 swift.rst 的 toctree 顺序:

Authentication    -> swift/auth
Service Ops       -> swift/serviceops
Container Ops     -> swift/containerops
Object Ops        -> swift/objectops
Temp URL Ops      -> swift/tempurl
Tutorial          -> swift/tutorial
Java / Python / Ruby

8.3 与 S3 的术语对照

官方原文提示:Amazon S3 API 用 bucket 描述数据容器;在 Swift API 的语境下听到 'bucket' 时,可以理解为 container(容器) 的等价说法。

Swift 术语 S3 等价术语
account (S3 无直接对应;Swift 的 account ≈ 用户/租户级命名空间)
container bucket
object object

💡 补充:上表「account」一行的 S3 侧说明为编者归纳,官方只明确了 container ↔ bucket 的对应关系(见 containerops.rst 的 note)。

引自:Ceph Object Gateway Swift API


九、Swift 认证(Authentication)

9.1 认证令牌

  • 需要认证的 Swift API 请求,必须在请求头中携带 X-Storage-Token 认证令牌。
  • 令牌可以从 RADOS Gateway 获取,也可以从其他 authenticator 获取。
  • 要从 RADOS Gateway 获取令牌,必须先创建一个用户:
sudo radosgw-admin user create --subuser="{username}:{subusername}" --uid="{username}" --display-name="{Display Name}" --key-type=swift --secret="{password}" --access=full

注意(官方原文):对熟悉 Swift API 的人而言,这里实现的是 Swift auth v1.0 API。上式中的 {username} 大体等价于 Swift 的 account,{subusername} 是该 account 下的一个 user。

9.2 Auth Get

要素 内容
操作名 Auth Get
HTTP 方法与路径 GET /auth HTTP/1.1
关键参数/请求头 X-Auth-User(必填)、X-Auth-Key(必填)
响应要点 响应中应包含 X-Auth-Token;也可能包含 X-Storage-Url,其值为 {api version}/{account} 前缀,后续 API 请求都要用到

Syntax

GET /auth HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-User: johndoe
X-Auth-Key: R7UUOLFDI2ZI9PRCQ53K

Request Headers

头名 Description Type Required
X-Auth-User The key RADOS GW username to authenticate. String Yes
X-Auth-Key The key associated to a RADOS GW username. String Yes

Response Headers

头名 Description Type
X-Storage-Token The authorization token for the X-Auth-User specified in the request. String
X-Storage-Url The URL and {api version}/{account} path for the user. String

典型响应(官方原文)

HTTP/1.1 204 No Content
Date: Mon, 16 Jul 2012 11:05:33 GMT
Server: swift
X-Storage-Url: https://swift.radosgwhost.com/v1/ACCT-12345
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
Content-Length: 0
Content-Type: text/plain; charset=UTF-8

💡 补充:官方描述说「响应中应包含 X-Auth-Token」,而上例响应头里给出的是 X-Auth-Token;但 9.1 节开头又说请求需携带 X-Storage-Token。这两处命名在源文中并存,本册按原文照录,未做统一。

引自:Authentication


十、Swift 操作

10.0 本章操作速查表

# 操作名 HTTP 方法与路径 所属页
1 List Containers GET /{api version}/{account} serviceops
2 Create a Container PUT /{api version}/{account}/{container} containerops
3 List a Container's Objects GET /{api version}/{container} containerops
4 Update a Container's ACLs POST /{api version}/{account}/{container} containerops
5 Add/Update Container Metadata POST /{api version}/{account}/{container} containerops
6 Enable Object Versioning for a Container POST /{api version}/{account}/{container}(X-Versions-Location) containerops
7 Delete a Container DELETE /{api version}/{account}/{container} containerops
8 Create/Update an Object PUT /{api version}/{account}/{container}/{object} objectops
9 Copy an Object PUT + X-Copy-From 或 COPY + Destination objectops
10 Delete an Object DELETE /{api version}/{account}/{container}/{object} objectops
11 Get an Object GET /{api version}/{account}/{container}/{object} objectops
12 Get Object Metadata HEAD /{api version}/{account}/{container}/{object} objectops
13 Add/Update Object Metadata POST /{api version}/{account}/{container}/{object} objectops
14 POST Temp-URL Keys POST /{api version}/{account} tempurl
15 GET Temp-URL Objects GET ...?temp_url_sig=&temp_url_expires= tempurl

10.1 List Containers(Service 操作)

要素 内容
操作名 List Containers
HTTP 方法与路径 GET /{api version}/{account} HTTP/1.1
关键参数/请求头 X-Auth-Token(必需);查询参数 limit、format、marker
权限 返回的是特定用户 account 的容器列表,因此需要认证令牌;不能匿名请求
响应要点 返回容器列表;若没有内容则返回 HTTP 204

Syntax

GET /{api version}/{account} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

Request Parameters

参数 Description Type Valid Values Required
limit Limits the number of results to the specified value. Integer — No
format Defines the format of the result. String json | xml No
marker Returns a list of results greater than the marker value. String — No

Response Entities

Name Description Type
account A list for account information. Container
container The list of containers. Container
name The name of a container. String
bytes The size of the container. Integer

示例

GET /v1/ACCT-12345 HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa

引自:Service Operations


10.2 Create a Container

要素 内容
操作名 Create a Container
HTTP 方法与路径 PUT /{api version}/{account}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Container-Read、X-Container-Write、X-Container-Meta-{key}(均可选)
命名约束 容器名必须唯一、不能含正斜杠 /、应小于 256 bytes
幂等性 幂等:请求创建已存在的容器会返回 HTTP 202,但不会创建第二个
响应要点 同名容器已存在且你是其 owner → 成功;否则失败 409 BucketAlreadyExists

前置说明(containerops 开篇):一个 account 可以有多个 container,但 container 名必须唯一。本 API 的请求全部需要认证,除非容器 ACL 被显式设为公开可访问(允许匿名请求)。

对象存储不支持层级路径或目录,只支持「一层容器 + 容器内对象」。RGW 的 Swift 兼容 API 支持 pseudo-hierarchical containers(伪层级容器):可以用 photos/buildings/empire-state.jpg 这样的对象名模拟目录层级,但容器名本身不能含 /。

Syntax

PUT /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}
X-Container-Read: {comma-separated-uids}
X-Container-Write: {comma-separated-uids}
X-Container-Meta-{key}: {value}

Headers

头名 Description Type Required
X-Container-Read The user IDs with read permissions for the container. Comma-separated string values of user IDs. No
X-Container-Write The user IDs with write permissions for the container. Comma-separated string values of user IDs. No
X-Container-Meta-{key} A user-defined meta data key that takes an arbitrary string value. String No

HTTP Response

HTTP Status Status Code Description
409 BucketAlreadyExists The container already exists under a different user's ownership.

示例

PUT /v1/ACCT-12345/my-new-container HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Container-Meta-Color: red

10.3 List a Container's Objects

要素 内容
操作名 List a Container's Objects
HTTP 方法与路径 GET /{api version}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token;查询参数 format、prefix、marker、limit、delimiter、path、allow_unordered
响应要点 不带参数时返回容器中前 10,000 个对象名

Syntax

GET /{api version}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

Parameters

参数 Description Type Valid Values / Range Required
format Defines the format of the result. String json | xml No
prefix Limits the result set to objects beginning with the specified prefix. String — No
marker Returns a list of results greater than the marker value. String — No
limit Limits the number of results to the specified value. Integer 0 - 10,000 No
delimiter The delimiter between the prefix and the rest of the object name. String — No
path The pseudo-hierarchical path of the objects. String — No
allow_unordered Allows the results to be returned unordered to reduce computation overhead. Cannot be used with delimiter. Boolean Non-Standard Extension: Yes No

Response Entities

Name Description Type
container The container. Container
object An object within the container. Container
name The name of an object within the container. String
hash A hash code of the object's contents. String
last_modified The last time the object's contents were modified. Date
content_type The type of content within the object. String

示例

GET /v1/my-new-container?format=json&limit=100 HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa

10.4 Update a Container's ACLs

要素 内容
操作名 Update a Container's ACLs
HTTP 方法与路径 POST /{api version}/{account}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Container-Read、X-Container-Write(均可选)
语义 创建容器的用户默认对该容器有读写权限;要让其他用户读写,必须显式授权。把值设为 * 表示所有用户(即公开/匿名可读写)

官方建议:若打算为 Swift API 暴露 public read ACL 功能,强烈建议在 endpoint 定义中包含 Swift account 名,以便最贴近原生 OpenStack Swift 的行为。做法是设置 ceph.conf 选项 rgw swift account in url = true,并把 Keystone endpoint 的 URL 后缀改为 /v1/AUTH_%(tenant_id)s(而不是只有 /v1)。

Syntax

POST /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}
X-Container-Read: *
X-Container-Write: {uid1}, {uid2}, {uid3}

Request Headers

头名 Description Type Required
X-Container-Read The user IDs with read permissions for the container. Comma-separated string values of user IDs. No
X-Container-Write The user IDs with write permissions for the container. Comma-separated string values of user IDs. No

示例

POST /v1/ACCT-12345/my-new-container HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Container-Read: *
X-Container-Write: mary, bob

10.5 Add/Update Container Metadata

要素 内容
操作名 Add/Update Container Metadata
HTTP 方法与路径 POST /{api version}/{account}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Container-Meta-{key}(可选,任意字符串值)
权限 必须对该容器有 write 权限

Syntax

POST /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}
X-Container-Meta-Color: red
X-Container-Meta-Taste: salty

Request Headers

头名 Description Type Required
X-Container-Meta-{key} A user-defined meta data key that takes an arbitrary string value. String No

示例

POST /v1/ACCT-12345/my-new-container HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Container-Meta-Color: red
X-Container-Meta-Taste: salty

10.6 Enable Object Versioning for a Container

要素 内容
操作名 Enable Object Versioning for a Container
HTTP 方法与路径 POST /{api version}/{account}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Versions-Location: {archive-container}
权限 必须对该容器有 write 权限
前置条件 radosgw 默认不开启 object versioning,必须在 ceph.conf 中设置 rgw swift versioning enabled = true

Syntax

POST /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}
X-Versions-Location: {archive-container}

Request Headers

头名 Description Type Required
X-Versions-Location The name of a container (the "archive container") that will be used to store versions of the objects in the container that the POST request is made on (the "current container"). The archive container need not exist at the time it is being referenced, but once X-Versions-Location is set on the current container, and object versioning is thus enabled, the archive container must exist before any further objects are updated or deleted in the current container. String No

官方明确:X-Versions-Location 是 radosgw 唯一解释的 versioning 相关头;原生 OpenStack Swift 支持的 X-History-Location 目前不被 radosgw 支持。

若该头以空值传入,则关闭当前容器的 object versioning,但 archive container 仍然存在。

示例

POST /v1/ACCT-12345/my-new-container HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Versions-Location: my-archive-container

10.7 Delete a Container

要素 内容
操作名 Delete a Container
HTTP 方法与路径 DELETE /{api version}/{account}/{container} HTTP/1.1
关键参数/请求头 X-Auth-Token
前置条件 容器必须为空;可先对容器发 HEAD 请求检查是否为空;删除成功后容器名可复用
响应要点 204 NoContent:容器已删除

Syntax

DELETE /{api version}/{account}/{container} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

HTTP Response

HTTP Status Status Code Description
204 NoContent The container was removed.

示例

DELETE /v1/ACCT-12345/my-new-container HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa

引自:Container Operations


10.8 Create/Update an Object

要素 内容
操作名 Create/Update an Object
HTTP 方法与路径 PUT /{api version}/{account}/{container}/{object} HTTP/1.1
关键参数/请求头 X-Auth-Token;ETag(推荐)、Content-Type、Transfer-Encoding
权限 必须对容器有 write 权限
幂等性 不幂等:对象名不唯一时该请求会更新已有对象;可用伪层级命名区分同名对象

Syntax

PUT /{api version}/{account}/{container}/{object} HTTP/1.1 
Host: {fqdn}
X-Auth-Token: {auth-token}

Request Headers

头名 Description Type Valid Values Required
ETag An MD5 hash of the object's contents. Recommended. String — No
Content-Type The type of content the object contains. String — No
Transfer-Encoding Indicates whether the object is part of a larger aggregate object. String chunked No

示例

PUT /v1/ACCT-12345/my-new-container/hello.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
Content-Type: text/plain

Hello World!

10.9 Copy an Object

要素 内容
操作名 Copy an Object
HTTP 方法与路径 方式一:PUT /{api version}/{account}/{dest-container}/{dest-object} + 头 X-Copy-From: {source-container}/{source-object};方式二:COPY /{api version}/{account}/{source-container}/{source-object} + 头 Destination: {dest-container}/{dest-object}
关键参数/请求头 X-Copy-From(用 PUT 时必填)/ Destination(用 COPY 时必填);条件头 If-Modified-Since、If-Unmodified-Since、Copy-If-Match、Copy-If-None-Match(均可选)
权限 必须对容器有 write 权限;目标对象名在容器内必须唯一(否则覆盖)
幂等性 不幂等

Syntax(PUT 形式)

PUT /{api version}/{account}/{dest-container}/{dest-object} HTTP/1.1
X-Copy-From: {source-container}/{source-object}
Host: {fqdn}
X-Auth-Token: {auth-token}

Syntax(COPY 形式)

COPY /{api version}/{account}/{source-container}/{source-object} HTTP/1.1
Destination: {dest-container}/{dest-object}

Request Headers

头名 Description Type Required
X-Copy-From Used with a PUT request to define the source container/object path. String Yes, if using PUT
Destination Used with a COPY request to define the destination container/object path. String Yes, if using COPY
If-Modified-Since Only copies if modified since the date/time of the source object's last_modified attribute. Date No
If-Unmodified-Since Only copies if not modified since the date/time of the source object's last_modified attribute. Date No
Copy-If-Match Copies only if the ETag in the request matches the source object's ETag. ETag. No
Copy-If-None-Match Copies only if the ETag in the request does not match the source object's ETag. ETag. No

示例

PUT /v1/ACCT-12345/my-new-container/hello-copy.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Copy-From: my-new-container/hello.txt

10.10 Delete an Object

要素 内容
操作名 Delete an Object
HTTP 方法与路径 DELETE /{api version}/{account}/{container}/{object} HTTP/1.1
关键参数/请求头 X-Auth-Token
权限 必须对容器有 write 权限;删除后对象名可复用

Syntax

DELETE /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

示例

DELETE /v1/ACCT-12345/my-new-container/hello.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa

10.11 Get an Object

要素 内容
操作名 Get an Object
HTTP 方法与路径 GET /{api version}/{account}/{container}/{object} HTTP/1.1
关键参数/请求头 X-Auth-Token;条件/范围头 range、If-Modified-Since、If-Unmodified-Since、Copy-If-Match、Copy-If-None-Match(均可选)
权限 必须对容器有 read 权限
响应要点 Content-Range:仅当请求中指定了 range 头字段时才返回

Syntax

GET /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

Request Headers

头名 Description Type Required
range To retrieve a subset of an object's contents, you may specify a byte range. Date No
If-Modified-Since Only copies if modified since the date/time of the source object's last_modified attribute. Date No
If-Unmodified-Since Only copies if not modified since the date/time of the source object's last_modified attribute. Date No
Copy-If-Match Copies only if the ETag in the request matches the source object's ETag. ETag. No
Copy-If-None-Match Copies only if the ETag in the request does not match the source object's ETag. ETag. No

Response Headers

头名 Description
Content-Range The range of the subset of object contents. Returned only if the range header field was specified in the request

示例

GET /v1/ACCT-12345/my-new-container/hello.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
range: bytes=0-1023

10.12 Get Object Metadata

要素 内容
操作名 Get Object Metadata
HTTP 方法与路径 HEAD /{api version}/{account}/{container}/{object} HTTP/1.1
关键参数/请求头 X-Auth-Token
权限 必须对容器有 read 权限
响应要点 返回与 GET 对象相同的头信息,但不返回对象数据

Syntax

HEAD /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

示例

HEAD /v1/ACCT-12345/my-new-container/hello.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa

10.13 Add/Update Object Metadata

要素 内容
操作名 Add/Update Object Metadata
HTTP 方法与路径 POST /{api version}/{account}/{container}/{object} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Object-Meta-{key}(可选,任意字符串值)
权限 必须对父容器有 write 权限

Syntax

POST /{api version}/{account}/{container}/{object} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

Request Headers

头名 Description Type Required
X-Object-Meta-{key} A user-defined meta data key that takes an arbitrary string value. String No

示例

POST /v1/ACCT-12345/my-new-container/hello.txt HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Object-Meta-Author: johndoe

引自:Object Operations


10.14 POST Temp-URL Keys

要素 内容
操作名 POST Temp-URL Keys
HTTP 方法与路径 POST /{api version}/{account} HTTP/1.1
关键参数/请求头 X-Auth-Token;X-Account-Meta-Temp-URL-Key(必填)、X-Account-Meta-Temp-URL-Key-2(可选)
语义 为 account 设置 secret temp URL key,用于签发临时 URL
关键行为 最多支持 2 个 key;签名会同时与两个 key 校验(若都存在),因此可以轮换 key 而不使已签发的临时 URL 失效

背景:为了在不共享凭证的情况下允许对对象的临时访问(例如 GET),radosgw 的 swift endpoint 支持 temp url 功能。使用前需要先设置 X-Account-Meta-Temp-URL-Key(以及可选的 X-Account-Meta-Temp-URL-Key-2)。Temp URL 功能依赖针对这些 secret key 的 HMAC-SHA1 签名。

官方限制:原生 OpenStack Swift 还支持在容器级设置临时 URL key(对容器发 POST 或 PUT 设置 X-Container-Meta-Temp-URL-Key / X-Container-Meta-Temp-URL-Key-2)。radosgw 不支持该功能——临时 URL key 只能在 account 级设置和使用。

官方建议:若打算暴露 Temp URL 功能,强烈建议在 endpoint 定义中包含 Swift account 名(设 rgw swift account in url = true,并把 Keystone endpoint 后缀改为 /v1/AUTH_%(tenant_id)s)。

Syntax

POST /{api version}/{account} HTTP/1.1
Host: {fqdn}
X-Auth-Token: {auth-token}

Request Headers

头名 Description Type Required
X-Account-Meta-Temp-URL-Key A user-defined key that takes an arbitrary string value. String Yes
X-Account-Meta-Temp-URL-Key-2 A user-defined key that takes an arbitrary string value. String No

示例

POST /v1/ACCT-12345 HTTP/1.1
Host: swift.radosgwhost.com
X-Auth-Token: UOlCCC8TahFKlWuv9DB09TWHF0nDjpPElha0kAa
X-Account-Meta-Temp-URL-Key: secret
X-Account-Meta-Temp-URL-Key-2: secret2

10.15 GET Temp-URL Objects

要素 内容
操作名 GET Temp-URL Objects
HTTP 方法与路径 GET {path}?temp_url_sig={sig}&temp_url_expires={expires}
签名算法 HMAC-SHA1
被签名的三要素 ① 请求方法(如 GET)② 过期时间(Unix 时间,秒)③ 从 v1 开始的请求路径
归一化方式 三要素之间以换行符 \n 连接

官方 Python 签名脚本(原文)

import hmac
from hashlib import sha1
from time import time

method = 'GET'
host = 'https://objectstore.example.com/swift'
duration_in_seconds = 300  # Duration for which the url is valid
expires = int(time() + duration_in_seconds)
path = '/v1/your-bucket/your-object'
key = 'secret'
hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest()
rest_uri = "{host}{path}?temp_url_sig={sig}&temp_url_expires={expires}".format(
	host=host, path=path, sig=sig, expires=expires)
print(rest_uri)

# Example Output
# https://objectstore.example.com/swift/v1/your-bucket/your-object?temp_url_sig=ff4657876227fc6025f04fcf1e82818266d022c6&temp_url_expires=1423200992

引自:Temp URL Operations


10.16 Tutorial:容器与对象生命周期

官方 tutorial.rst 描述的 Swift 兼容 API 教程遵循一个简单的基于容器的对象生命周期:第一步建立客户端与 RADOS Gateway 服务器的连接,随后按自然的容器/对象生命周期推进(包括添加与获取对象元数据)。

流程图(官方 ditaa 图,本册以文本重绘,节点名保留英文原文):

+----------------------------+        +-----------------------------+
|                            |        |                             |
|    Create a Connection     |------->|      Create a Container     |
|                            |        |                             |
+----------------------------+        +-----------------------------+
                                                     |
                  +----------------------------------+
                  |
                  v
+----------------------------+        +-----------------------------+
|                            |        |                             |
|     Create an Object       |------->| Add/Update Object Metadata  |
|                            |        |                             |
+----------------------------+        +-----------------------------+
                                                     |
                  +----------------------------------+
                  |
                  v
+----------------------------+        +-----------------------------+
|                            |        |                             |
|   List Owned Containers    |------->| List a Container's Contents |
|                            |        |                             |
+----------------------------+        +-----------------------------+
                                                     |
                  +----------------------------------+
                  |
                  v
+----------------------------+        +-----------------------------+
|                            |        |                             |
| Get an Object's Metadata   |------->|     Retrieve an Object      |
|                            |        |                             |
+----------------------------+        +-----------------------------+
                                                     |
                  +----------------------------------+
                  |
                  v
+----------------------------+        +-----------------------------+
|                            |        |                             |
|      Delete an Object      |------->|      Delete a Container     |
|                            |        |                             |
+----------------------------+        +-----------------------------+

对应的示例代码见第十一章(Java / Python / Ruby)。

引自:Tutorial


十一、Swift SDK 示例

本章覆盖 3 种语言的官方 Swift 示例页:Java(joss)、Python(swiftclient)、Ruby(cloudfiles)。代码原文逐字保留。

11.1 Java(org.javaswift.joss)

Setup — imports(原文)

       import org.javaswift.joss.client.factory.AccountConfig;
       import org.javaswift.joss.client.factory.AccountFactory;
       import org.javaswift.joss.client.factory.AuthenticationMethod;
       import org.javaswift.joss.model.Account;
       import org.javaswift.joss.model.Container;
       import org.javaswift.joss.model.StoredObject;
       import java.io.File;
       import java.io.IOException;
       import java.util.*;

创建连接

       String username = "USERNAME";
       String password = "PASSWORD";
       String authUrl  = "https://radosgw.endpoint/auth/1.0";

       AccountConfig config = new AccountConfig();
       config.setUsername(username);
       config.setPassword(password);
       config.setAuthUrl(authUrl);
       config.setAuthenticationMethod(AuthenticationMethod.BASIC);
       Account account = new AccountFactory(config).createAccount();

创建容器

       Container container = account.getContainer("my-new-container");
       container.create();

创建对象(从文件 foo.txt 创建 foo.txt)

       Container container = account.getContainer("my-new-container");
       StoredObject object = container.getObject("foo.txt");
       object.uploadObject(new File("foo.txt"));

添加/更新对象元数据

       Container container = account.getContainer("my-new-container");
       StoredObject object = container.getObject("foo.txt");
       Map<String, Object> metadata = new TreeMap<String, Object>();
       metadata.put("key", "value");
       object.setMetadata(metadata);

列出拥有的容器

       Collection<Container> containers = account.list();
       for (Container currentContainer : containers) {
           System.out.println(currentContainer.getName());
       }

输出类似:

mahbuckat1
mahbuckat2
mahbuckat3

列出容器内容

       Container container = account.getContainer("my-new-container");
       Collection<StoredObject> objects = container.list();
       for (StoredObject currentObject : objects) {
           System.out.println(currentObject.getName());
       }

输出类似:

myphoto1.jpg
myphoto2.jpg

获取对象元数据

       Container container = account.getContainer("my-new-container");
       StoredObject object = container.getObject("foo.txt");
       Map<String, Object> returnedMetadata = object.getMetadata();
       for (String name : returnedMetadata.keySet()) {
           System.out.println("META / "+name+": "+returnedMetadata.get(name));
       }

获取对象(下载到 ./outfile.txt)

       Container container = account.getContainer("my-new-container");
       StoredObject object = container.getObject("foo.txt");
       object.downloadObject(new File("outfile.txt"));

删除对象

       Container container = account.getContainer("my-new-container");
       StoredObject object = container.getObject("foo.txt");
       object.delete();

删除容器

       Container container = account.getContainer("my-new-container");
       container.delete();

注意:容器必须为空!否则无法删除!

引自:Java Swift Examples


11.2 Python(swiftclient)

创建连接

	import swiftclient
	user = 'account_name:username'
	key = 'your_api_key'

	conn = swiftclient.Connection(
		user=user,
		key=key,
		authurl='https://objects.dreamhost.com/auth',
	)

创建容器

	container_name = 'my-new-container'
	conn.put_container(container_name)

创建对象(从 hello.txt 创建 hello.txt)

	with open('hello.txt', 'r') as hello_file:
		conn.put_object(container_name, 'hello.txt',
						contents= hello_file.read(),
						content_type='text/plain')

列出拥有的容器

	for container in conn.get_account()[1]:
		print(container['name'])

输出类似:

mahbuckat1
mahbuckat2
mahbuckat3

列出容器内容

	for data in conn.get_container(container_name)[1]:
		print('{0}\t{1}\t{2}'.format(data['name'], data['bytes'], data['last_modified']))

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

获取对象(下载到 ./my_hello.txt)

	obj_tuple = conn.get_object(container_name, 'hello.txt')
	with open('my_hello.txt', 'w') as my_hello:
		my_hello.write(obj_tuple[1])

删除对象

	conn.delete_object(container_name, 'hello.txt')

删除容器

注意:容器必须为空!否则请求不会生效!

	conn.delete_container(container_name)

引自:Python Swift Examples


11.3 Ruby(cloudfiles)

创建连接

	require 'cloudfiles'
	username = 'account_name:user_name'
	api_key  = 'your_secret_key'

	conn = CloudFiles::Connection.new(
		:username => username,
		:api_key  => api_key,
		:auth_url => 'http://objects.dreamhost.com/auth'
	)

创建容器

	container = conn.create_container('my-new-container')

创建对象(从 ./my_hello.txt 创建 hello.txt)

	obj = container.create_object('hello.txt')
	obj.load_from_filename('./my_hello.txt')
	obj.content_type = 'text/plain'

列出拥有的容器

	conn.containers.each do |container|
		puts container
	end

输出类似:

mahbuckat1
mahbuckat2
mahbuckat3

列出容器内容

	require 'date'  # not necessary in the next version

	container.objects_detail.each do |name, data|
		puts "#{name}\t#{data[:bytes]}\t#{data[:last_modified]}"
	end

输出类似:

myphoto1.jpg	251262	2011-08-08T21:35:48.000Z
myphoto2.jpg	262518	2011-08-08T21:38:01.000Z

获取对象(下载到 ./my_hello.txt)

	obj = container.object('hello.txt')
	obj.save_to_filename('./my_hello.txt')

删除对象

	container.delete_object('goodbye.txt')

删除容器

注意:容器必须为空!否则请求不会生效!

	container.delete_container('my-new-container')

引自:Ruby Swift Examples


十二、S3 与 Swift 的命名空间互通说明

RGW 同时提供 S3 与 Swift 两套接口,它们落在同一套底层存储资源上。本册输入范围内,官方对此的直接说明有四处,整理如下。

12.1 容器与桶是同一份资源:container == bucket

官方 swift/containerops.rst 的 note 原文:

The Amazon S3 API uses the term 'bucket' to describe a data container. When you hear someone refer to a 'bucket' within the Swift API, the term 'bucket' may be construed as the equivalent of the term 'container.'

即:在 Swift API 语境下听到的 bucket,可以当作 container 的同义词。两者指向同一类数据容器。

旁证:Swift 的 Create a Container 在容器已被其他用户占用时,返回的错误码是 BucketAlreadyExists(409)——直接沿用了 S3 的错误码命名。

12.2 为什么部分 ACL 映射对 S3「不适用」

官方 s3/authentication.rst 原文:

Some mappings, (e.g. s3:CreateBucket to WRITE) are not applicable to S3 operation, but are required to allow Swift and S3 to access the same resources when things like Swift user ACLs are in play. This is one of the many reasons that you should use S3 bucket policies rather than S3 ACLs when possible.

要点拆解:

要点 说明
现象 某些 S3 操作 → ACL 权限的映射(如 s3:CreateBucket → WRITE)对 S3 操作本身并不适用
原因 之所以保留这些映射,是为了在 Swift user ACL 生效时,让 Swift 与 S3 能访问同一份资源
实践建议 这是「能用 S3 bucket policy 就尽量别用 S3 ACL」的众多理由之一

12.3 两套接口的权限模型对照

维度 S3 侧 Swift 侧
认证机制 AWS Signature v2 / v4(access key + secret key);可对接 Keystone X-Auth-User + X-Auth-Key 换取 token(Swift auth v1.0)
授权载体 canned ACL(Put/Get Object ACL、Put/Get Bucket ACL)+ bucket policy X-Container-Read / X-Container-Write(逗号分隔的 uid 列表,或 * 表示公开)
权限粒度 READ / WRITE / READ_ACP / WRITE_ACP / FULL_CONTROL read / write 两类(官方称只支持 Swift ACL 的子集)
ACL 支持度 S3-compatible ACL;canned ACL 集合与 AWS 不同 Supports a subset of Swift ACLs

12.4 互通时的实践注意点(官方明确给出的)

场景 官方建议 出处
暴露 Swift public read ACL 在 endpoint 定义中包含 Swift account 名:设 rgw swift account in url = true,Keystone endpoint 后缀改为 /v1/AUTH_%(tenant_id)s(而非 /v1) swift/containerops.rst
暴露 Temp URL 同上(rgw swift account in url = true + /v1/AUTH_%(tenant_id)s) swift/tempurl.rst
Keystone + S3 signed URL / public read ACL 生产环境开启 multitenancy 时,对 S3 signed URLs 与 public read ACLs 有特殊注意事项 s3/authentication.rst
在 Swift 与 S3 间共用资源 优先使用 S3 bucket policy 而非 S3 ACL s3/authentication.rst

12.5 文档未明确的互通细节

以下互通细节在本册输入范围内官方未明确说明,不做推测:

  • S3 的 bucket 与 Swift 的 container 之间名称是否完全一一对应(例如 S3 允许的点分命名在 Swift 侧是否可见/可操作)——文档未明确。
  • S3 的 object 与 Swift 的 object 是否共享同一 key 命名空间(例如 S3 写入的对象能否被 Swift 直接列出并读取,反之亦然)——文档未明确。
  • S3 ACL 与 Swift X-Container-Read/Write 之间的具体互转规则(例如 Swift 设的 uid 对应 S3 ACL 中的哪个 Grantee)——文档未明确。
  • S3 版本化对象(VersioningConfiguration)与 Swift 的 X-Versions-Location archive container 是否共用同一套版本数据——文档未明确(两者在各自接口页分别描述,未见交叉说明)。
  • Swift 的 X-Auth-User / X-Auth-Key 与 S3 的 access key / secret key 是否同一套凭证——文档未明确(auth.rst 只说明 subuser 的 --key-type=swift 创建方式;authentication.rst 的 Keystone 段落说明 S3 需要 AWS-style access/secret key)。

引自:Authentication and ACLs、Container Operations、Temp URL Operations


文档未明确项

本节汇总本册编写过程中,官方源文没有明确说明的内容。分两类:一是用户/实践常见但官方文档未覆盖的 API 规格,二是官方文档提到能力却未给规格的。

A. S3 Bucket 操作规格缺失

能力 s3.rst / authentication.rst 中的状态 缺失内容
Bucket Lifecycle(PUT/GET/DELETE ?lifecycle) Supported 请求体、参数、响应实体、错误码全部未给
Bucket Replication Partial(Permitted only across zones) 同上
Bucket Website Supported 同上
Bucket Request Payment Supported 同上
Bucket Tagging Supported 同上
Bucket Policy Supported(Features Support 表 Policy 行) 同上
Bucket Encryption(s3:GetBucketEncryption / s3:PutBucketEncryption) 出现在 ACL 映射表 同上
Bucket CORS(s3:GetBucketCORS / s3:PutBucketCORS) 出现在 ACL 映射表 同上
Bucket Ownership Controls 本册输入范围内未出现 全部未给
Get Bucket Info (HEAD)(HEAD /{bucket}) Supported bucketops.rst 无该小节
Get Bucket Versioning(GET ?versioning) 出现在 ACL 映射表 只有 PUT ?versioning,无 GET 小节
Storage Class 的设置方式 Supported(See storage_classes) 未给具体请求方式

B. S3 Object 操作规格缺失

能力 出处 缺失内容
POST Object(浏览器表单上传) Features Support:Supported 无对应小节
Object Tagging(Get/Put/Delete Object Tagging) Features Support:Supported 无对应小节
Object Version 相关操作 ACL 映射表 s3:GetObjectVersion 等 objectops.rst 无版本相关小节,仅 retention / legal-hold 带 versionId 参数
Get Object Torrent ACL 映射表 s3:GetObjectTorrent / s3:GetObjectVersionTorrent 无对应小节
s3:RestoreObject ACL 映射表 无对应小节

C. 认证/签名细节缺失

项 缺失内容
AWS Signature v4 的 Authorization 头具体格式 authentication.rst 只指向 AWS 官方文档,未在本页展开
预签名 URL(presigned URL)的生成规则与专用错误码 未在 S3 页面给出规格;仅在各 SDK 示例中演示生成方式
预签名 URL 的错误码(如过期、签名不匹配返回什么) 文档未明确
Keystone + multitenancy 下 S3 signed URL / public read ACL 的「特殊注意事项」具体内容 官方只说「special considerations apply」,未展开

D. 公共实体缺失

项 缺失内容
公共响应头(Common Response Headers)清单 commons.rst 只给了 4 个公共请求头,无响应头表
x-amz-id-2 之外的其他不支持头 官方只列了 x-amz-id-2 一项
各错误码对应的典型触发场景 commons.rst 只给了状态码 + 错误码对照,未给场景说明

E. Swift 操作规格缺失

能力 swift.rst 状态 缺失内容
Get Account Metadata Supported serviceops.rst 无该小节
Get Container Metadata Supported containerops.rst 无该小节(只在 Delete 段落提到可用 HEAD 检查容器是否为空)
Delete Container Metadata Supported containerops.rst 无该小节(只有 Add/Update)
Static Website Supported 无对应小节
Create Large Object Supported 无对应小节(分段/清单对象的具体语义未给)
Expiring Objects Supported 无对应小节(X-Delete-At / X-Delete-After 未提及)
Object Versioning 的 X-History-Location Partial Support 官方明确说明不支持,故无规格
CORS Not Supported 明确不支持,无规格
Swift Temp URL 的容器级 key Partial Support 官方明确说明不支持,只能在 account 级设置

F. Swift 认证细节

项 缺失内容
token 有效期 / 过期行为 文档未明确
X-Storage-Token 与 X-Auth-Token 的关系 源文两处命名并存(9.1 说请求需带 X-Storage-Token,9.2 响应给出 X-Auth-Token),官方未统一说明
Swift 认证失败的 HTTP 状态码与错误码 文档未明确(auth.rst 未列错误响应)

G. S3 / Swift 互通细节

见 十二、12.5 文档未明确的互通细节,共 5 项(命名空间一一对应性、object key 共享、ACL 互转规则、版本数据是否共用、凭证是否同一套)。

H. 其他

项 缺失内容
各 SDK 示例页中的 endpoint(如 objects.dreamhost.com、https://radosgw.endpoint/auth/1.0、https://objects.dreamhost.com/auth) 官方文档未说明这些是示意值还是真实可访问地址;本册按原文保留,未做改写
各 SDK 的版本兼容性矩阵 只有 Java S3 页明确了「AWS Java SDK 2.17.42」,其余语言未给版本
官方响应报文样例 bucketops.rst / objectops.rst / serviceops.rst 均未给出完整响应 XML 样例;本册中标注为「💡 补充」的响应示例均为编者按 Response Entities 表构造的示意
Append Object 的 position 参数取值规则与 x-rgw-next-append-position 的单位 官方只说明「Next position to append object」,未说明单位(字节/其他)与初始取值

本册小结

章节 覆盖的官方操作/示例数
四、Service 操作 2 个(List Buckets、Get Usage Stats)
五、Bucket 操作 17 个(逐个展开)+ 11 项规格缺失说明
六、Object 操作 17 个(逐个展开,含 5 步 multipart 全流程)+ 6 项规格缺失说明
七、S3 SDK 7 种语言(C++、C#、Java、Perl、PHP、Python、Ruby;Ruby 含两代 SDK)
十、Swift 操作 15 个(Service 1 + Container 6 + Object 6 + Temp URL 2)+ Tutorial 流程图
十一、Swift SDK 3 种语言(Java、Python、Ruby)

本册是接口速查手册:需要查某个操作的路径、参数、响应字段时,按第五章 / 第六章 / 第十章的速查表定位到小节即可;需要对接客户端时,直接取第七章 / 第十一章的代码原文改 endpoint 与密钥。遇到本册未覆盖的规格,请先查 ## 文档未明确项 确认是否属于官方文档缺口,再决定是否需要查阅其他分册(如第 12 册高级特性)或社区资源。