Connect to a storage account

What a client needs

• Endpoint — https:// and the hostname under Endpoint on the account's Overview; by default https://<account>.s3.example.com. An account set to HTTP only is served on http:// instead.
• Path-style addressing — requests go to <endpoint>/<bucket>/<key>. Addresses with the bucket in the hostname (<bucket>.<endpoint>) are not served, so turn path-style on in every client.
• Credentials — an access key, or temporary credentials.
• Region — any value, such as us-east-1. The server checks the signature with the region the client signed with.

Requests must be signed with AWS Signature Version 4, and the client's clock must be within 15 minutes of the server's.

Credentials

S3 clients read them from the usual places — for the AWS CLI and SDKs, the variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN, or a profile.

AWS CLI

export AWS_ACCESS_KEY_ID=<access-key-id>
export AWS_SECRET_ACCESS_KEY=<secret-access-key>
export AWS_DEFAULT_REGION=us-east-1
aws configure set default.s3.addressing_style path

ENDPOINT=https://my-storage.s3.example.com
aws --endpoint-url "$ENDPOINT" s3 ls
aws --endpoint-url "$ENDPOINT" s3 cp ./report.pdf s3://my-bucket/reports/report.pdf
aws --endpoint-url "$ENDPOINT" s3 sync ./site s3://my-bucket/site

Python

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://my-storage.s3.example.com",
    aws_access_key_id="<access-key-id>",
    aws_secret_access_key="<secret-access-key>",
    region_name="us-east-1",
    config=Config(s3={"addressing_style": "path"}),
)
s3.upload_file("report.pdf", "my-bucket", "reports/report.pdf")

JavaScript

With the AWS SDK for JavaScript v3:

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  endpoint: "https://my-storage.s3.example.com",
  region: "us-east-1",
  forcePathStyle: true,
  credentials: { accessKeyId: "<access-key-id>", secretAccessKey: "<secret-access-key>" },
});
await s3.send(new PutObjectCommand({ Bucket: "my-bucket", Key: "hello.txt", Body: "Hello" }));

Presigned URLs

A presigned URL lets someone without credentials read or upload one object until it expires. The server accepts them; create them with your client, for example:

aws --endpoint-url "$ENDPOINT" s3 presign s3://my-bucket/reports/report.pdf --expires-in 3600

A URL signed with temporary credentials stops working when those credentials expire. The server sets no upper limit on the expiry itself: a URL signed with an access key works until it expires or the key is disabled or deleted.

CORS

A web page that calls the storage account from the browser — to upload with a presigned URL, for example — needs its origin on the account's list. Add it under Additional CORS origins, in Configuration → Networking & Security, as scheme, host and an optional port, such as https://app.example.com. The portal's own origin is always allowed.

Allowed origins may use GET, HEAD, PUT, POST and DELETE. A browser request from any other origin is refused. CORS rules per bucket (PutBucketCors) are not supported.

Connect from a workload

Workloads reach the account over its HTTPS endpoint like any other client; the boundary's network rules allow outbound HTTPS. They allow no outbound traffic on port 80, so an account set to HTTP only cannot be reached from workloads. Keep the credentials out of configuration:

1. Put the access key ID and the secret access key in a vault as two secrets — see Add a secret.
2. Reference both from the workload, for example as S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY — see Secrets in apps and functions or Secrets in container instances.
3. Put the endpoint in an ordinary setting, for example the environment variable S3_ENDPOINT.

An app reads its secrets from /secrets/env.json; a function gets them as environment variables with the prefix STSH_. In a Python app:

import json
import os

import boto3
from botocore.config import Config

secrets = json.load(open("/secrets/env.json"))
s3 = boto3.client(
    "s3",
    endpoint_url=os.environ["S3_ENDPOINT"],
    aws_access_key_id=secrets["S3_ACCESS_KEY_ID"],
    aws_secret_access_key=secrets["S3_SECRET_ACCESS_KEY"],
    region_name="us-east-1",
    config=Config(s3={"addressing_style": "path"}),
)

When requests fail