s3.make_s3_upath()

Build an S3 UPath with the shared retry policy.

Usage

s3.make_s3_upath(
    path,
    *,
    anon=False,
    profile=None,
    key=None,
    secret=None,
    token=None,
    region=None,
    endpoint_url=None,
    request_checksum_calculation="when_required",
    response_checksum_validation="when_required",
    client_kwargs=None
)

Parameters

path: str: S3 URI or prefix, e.g. "s3://my-bucket/some/key.tif". The s3 protocol is forced, so a bare "my-bucket/key" also works.
anon: bool = False: Whether to use an anonymous connection (public buckets only). If False, uses profile, the key/secret given, or boto’s default credential resolver. Mirrors s3fs.S3FileSystem’s anon.
profile: str = None: Named AWS configuration profile to use for this path. The profile name is stored on the path and used independently by both s3fs and Rasterio. May be combined with endpoint_url for S3-compatible stores, but not with anon=True or explicit key, secret, or token. Setting a profile explicitly makes boto skip its environment-variable credential provider on both layers, so such a path can never be signed by ambient AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY values – which is what makes profiles the safer way to authenticate here.
key: str = None: If not anonymous, use this access key ID, if specified. Mutually exclusive with aws_access_key_id in client_kwargs.
secret: str = None: If not anonymous, use this secret access key, if specified. Mutually exclusive with aws_secret_access_key in client_kwargs.
token: str = None: If not anonymous, use this security token, if specified, for temporary/STS credentials.
region: str = None: AWS region name. Relevant for region-scoped AWS-native buckets.
endpoint_url: str = None: S3-compatible endpoint URL. Omit for AWS-native S3.
request_checksum_calculation: str = "when_required": boto3 >=1.36 checksum behavior. "when_required" avoids the x-amz-checksum-* trailers that some S3-compatible gateways (e.g. Ceph) reject; it’s a safe default even for gateways not known to need it.
response_checksum_validation: str = "when_required": boto3 >=1.36 checksum behavior. "when_required" avoids the x-amz-checksum-* trailers that some S3-compatible gateways (e.g. Ceph) reject; it’s a safe default even for gateways not known to need it.
client_kwargs: dict = None: Extra kwargs forwarded to the underlying botocore client. Defaults to {"endpoint_url": endpoint_url}. region is injected here as region_name. Credentials passed here (CLIENT_CREDENTIAL_KEYS) reach s3fs only, never s3_env(), so they are rejected alongside anon=True and profile rather than silently signing the two layers differently.

Returns

UPath: An S3 UPath configured with S3_MAX_ATTEMPTS botocore retries in S3_RETRY_MODE mode, on top of the s3fs-level retry handler registered by this module. Pair it with s3_env() for raster I/O.

Raises

ValueError: If anon=True is combined with a profile or credentials, if a profile is combined with explicit key, secret, token, or credentials in client_kwargs, or if profile is an empty string.

Examples

Anonymous access to a public bucket::

path = make_s3_upath(
    "s3://copernicus-dem-30m/x.tif", region="eu-central-1", anon=True
)

Signed access via boto’s credential resolver, with no explicit key pair::

path = make_s3_upath("s3://my-private-bucket/x.tif")

Per-path profile selection, including for an S3-compatible endpoint::

path = make_s3_upath(
    "s3://my-bucket/x.tif",
    profile="research",
    endpoint_url="https://objects.example.org",
)