Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

To find the API url you should use the command "openstack catalog show swift", and select the public URL from that command. When using S3 you should remove the "/swift/v1/<project-id" from the url.

Using the amazon aws command-line client:

The amazon aws client can be useful to get the full S3 functionality. The client is available in the ubuntu-repositories under the name "awscli". To use the client it needs to be configured with credentials in the file ".aws/credentials". The file could for example look like this:

Code Block
titleConfigure the aws client credentials
$ cat .aws/credentials 
[default]
aws_access_key_id = <access-key>
aws_secret_access_key = <secret-key>

Storage resilliency

The object storage solutions should be a safe space to store data. A very important aspect of this is to ensure that information stored here should not be lost. There are two levels of resilliency available, which are further described in this section.

...

  • Upload (PUT): Objects can be uploaded to a certain container, and given a certain name. If there is an object with the given name in the current container, that object will be overwritten (DELETE, then PUT).
  • Download (GET): Existing objects can be downloaded. If the container is private (that is the default) a user needs to be authenticated first to be allowed to download the object.
  • Delete (DELETE): Objects can be deleted. This will make all the three copies of this object being removed from the permanent storage, and the object will quickly become unavailable.

When a container is created, it works with these default settings. One can change a default-container to a versioned-container.

Versioned containers

Some use-cases requires object-storage with versioning. If it is desired to keep a history of overwritten and deleted objects the S3 API can be used to enable "versioning" which will change the PUT/GET/DELETE operations a little:

  • Upload (PUT): When an object is uploaded it will be given a "version ID", and there will be set a flag called "latest" related to that object. Any objects with the same name will have the "latest" flag cleared.
  • Download (GET): The download operation would select the latest object with the supplied name. If the newest object is a "deletion marker" the download operation would result in an 404, which is the same response one would get when requesting a non-existant object.
    • It is possible to provide a specific version-id to a download-request; in which case that specific version of the object can be downloaded. It can be used to download previous versions of a given object.
  • Delete (DELETE): The delete operations would upload an empty object with a "deleted" flag set. In other words; nothing will be deleted; just marked as deleted. 
    • It is possible to provice a specific version-id to the delete command. In this case the specific version of the object is actually deleted from storage, and is not recoverable anymore.

It is worth noting that there will never be deleted anything from a versioned container, unless you explicitly deletes older versions of a certain object. The storage consumption would thus in general only increase in such containers, and they are thus good candidates for data that rarely change, and that requires an audit-trail for changes.

Even though the versioning is a S3-specific command, the versioning would still apply to objects uploaded to the same container through swift. So; the versioning happens regardless of which interface is used to access the objects. To retrieve older versions of the objects requires the S3 API however.

The versioning of a container can be disabled. This will stop marking new objects with a version ID; but existing objects will still keep their version ID.

Configure versioning

To enable versioning to a certain container, it is best done using the aws cli client:

Code Block
titleEnabling/Disabling versioning for a container
$ # To see if the container "FOO" in my project in skyhigh has versioning enabled, the following command can be used:
$ aws s3api get-bucket-versioning  --bucket FOO --endpoint https://swift.skyhigh.iik.ntnu.no 
{
    "Status": "Enabled"
}
$ # The command will return either "Enabled" or "Suspended"
$
$ # To enable versioning on the FOO container in my skyhigh project, the following command can be used:
$ aws s3api put-bucket-versioning --bucket FOO --versioning-configuration Status=Enabled --endpoint https://swift.skyhigh.iik.ntnu.no
$
$ # To disable (suspend) versioning of the container "FOO" in my skyhigh project, the following command can be used:
$ aws s3api put-bucket-versioning --bucket FOO --versioning-configuration Status=Suspended --endpoint https://swift.skyhigh.iik.ntnu.no