s3.list_object_sizes()

List a prefix as {relative path: size} from one listing call.

Usage

Source

s3.list_object_sizes(
    directory,
    *,
    recursive=False,
)

One listing replaces one remote open per object, which is what makes checking tens of thousands of objects affordable. Only the size is taken: an ETag is the MD5 for a single-part upload and something else for a multipart one, so keying on it would constrain how every writer uploads.

Parameters

directory: str, Path, or UPath

Prefix to list. Any fsspec-backed path works, including a local one.

recursive: bool = False

Include everything below the prefix, keyed by its path relative to the prefix with / separators. The default lists the prefix’s direct entries by name.

Returns

dict of str to int

Relative path to size in bytes. Empty when the prefix does not exist.