tigris-python-sdk

---
name: tigris-python-sdk
description: Use when working with Tigris from Python — boto3 setup, Django uploads via django-storages, snapshots, bucket forking, in-place object rename, conditional writes (IfMatch/IfNoneMatch), and the Bundle API for batch ML data fetches. Covers the tigris-boto3-ext extension library (context managers, decorators, helpers) plus framework integration.
---

# Tigris Python SDK

Use Tigris from Python via boto3 plus the `tigris-boto3-ext` extension package. Standard S3 calls work unchanged; the extension layers on Tigris-specific features — snapshots, bucket forking, in-place object rename, and the Bundle API for batch fetches — by injecting headers on activated boto3 events.

For Django uploads via `django-storages`, see [`./resources/django.md`](./resources/django.md).

## Quick Start

```bash
# 1. Install CLI & authenticate
npm install -g @tigrisdata/cli
tigris login

# 2. Create bucket and access key
tigris buckets create my-bucket
tigris access-keys create "my-bucket-key"
# ⚠ Save the Secret Access Key — shown only once
tigris access-keys assign tid_xxx --bucket my-bucket --role Editor

# 3. Install Python deps
pip install boto3 tigris-boto3-ext
```

```bash
# .env
AWS_ENDPOINT_URL_S3=https://t3.storage.dev
AWS_ACCESS_KEY_ID=tid_xxx
AWS_SECRET_ACCESS_KEY=tsec_yyy
AWS_REGION=auto
```

```python
import boto3
from tigris_boto3_ext import (
    TigrisSnapshotEnabled, create_snapshot, get_snapshot_version,
)

s3 = boto3.client("s3")  # picks up env vars

# Create a snapshot-enabled bucket (header injection scoped to the block)
with TigrisSnapshotEnabled(s3):
    s3.create_bucket(Bucket="my-bucket")

# Take a snapshot
result = create_snapshot(s3, "my-bucket", snapshot_name="daily-backup")
version = get_snapshot_version(result)
```

See **Getting Started with CLI** for detailed setup steps.

---

## Getting Started with CLI

### Step 1: Install CLI

```bash
npm install -g @tigrisdata/cli
tigris --version
```

`t3` is an alias for `tigris` — all commands work with either.

### Step 2: Authenticate

```bash
tigris login    # opens browser for OAuth
tigris whoami   # verify
```

For CI/CD:

```bash
tigris configure --access-key <key> --access-secret <secret>
```

### Step 3: Create Bucket

```bash
tigris buckets create my-bucket
```

Buckets are private and global by default. Use `--public` for anonymous reads, `--locations` to pin regions.

### Step 4: Create Access Key

```bash
tigris access-keys create "my-bucket-key"
```

Outputs an Access Key ID (`tid_xxx`) and Secret Access Key (`tsec_yyy`). **The secret is shown only once.**

### Step 5: Assign Key to Bucket

```bash
tigris access-keys assign tid_xxx --bucket my-bucket --role Editor
```

| Role       | Permissions                   | Use when                              |
| ---------- | ----------------------------- | ------------------------------------- |
| `Editor`   | Read + write + delete         | App code that uploads/deletes objects |
| `ReadOnly` | Read only                     | Apps that only download/list          |

### Step 6: Configure Environment

```bash
# .env (add to .gitignore)
AWS_ENDPOINT_URL_S3=https://t3.storage.dev
AWS_ACCESS_KEY_ID=tid_xxx
AWS_SECRET_ACCESS_KEY=tsec_yyy
AWS_REGION=auto
```

boto3 reads `AWS_ENDPOINT_URL_S3` natively (boto3 ≥ 1.34) — no client-side overrides needed if you set it.

### Step 7: Install Python Dependencies

```bash
pip install boto3 tigris-boto3-ext
```

Requires Python 3.9+ and boto3 ≥ 1.26.0.

---

## boto3 Client Setup

```python
import boto3

# Reads AWS_ENDPOINT_URL_S3, AWS_ACCESS_KEY_ID, etc. from env
s3 = boto3.client("s3")

# Or configure explicitly
s3 = boto3.client(
    "s3",
    endpoint_url="https://t3.storage.dev",
    aws_access_key_id="tid_xxx",
    aws_secret_access_key="tsec_yyy",
    region_name="auto",
)
```

`tigris-boto3-ext` works with any boto3 S3 client — it registers handlers on `before-sign.s3.*` to inject Tigris headers when activated. Calls outside an activated scope pass through unchanged.

---

## Usage Patterns

`tigris-boto3-ext` exposes three ways to scope Tigris-specific behavior. Pick whichever fits the call site:

| Pattern           | Best for                                           |
| ----------------- | -------------------------------------------------- |
| Context managers  | One-off blocks with a clear scope                  |
| Decorators        | Functions that always need a Tigris feature        |
| Helper functions  | Direct calls without scoping ceremony              |

### Context Managers

```python
from tigris_boto3_ext import (
    TigrisSnapshotEnabled, TigrisSnapshot, TigrisFork, TigrisRename,
)

# Enable snapshots on bucket creation
with TigrisSnapshotEnabled(s3):
    s3.create_bucket(Bucket="my-bucket")

# List snapshots for a bucket
with TigrisSnapshot(s3, "my-bucket"):
    snapshots = s3.list_buckets()

# Read from a specific snapshot version
with TigrisSnapshot(s3, "my-bucket", snapshot_version="12345"):
    obj = s3.get_object(Bucket="my-bucket", Key="file.txt")

# Fork a bucket (current state or from a snapshot)
with TigrisFork(s3, "source-bucket", snapshot_version="12345"):
    s3.create_bucket(Bucket="forked-bucket")

# Rename in place — keep scope tight
with TigrisRename(s3):
    s3.copy_object(
        Bucket="my-bucket",
        CopySource="my-bucket/old.txt",
        Key="new.txt",
    )
```

### Decorators

```python
from tigris_boto3_ext import snapshot_enabled, with_snapshot, forked_from, with_rename

@snapshot_enabled
def make_snapshot_bucket(s3, name):
    return s3.create_bucket(Bucket=name)

@with_snapshot("my-bucket", snapshot_version="12345")
def read_old_config(s3):
    return s3.get_object(Bucket="my-bucket", Key="config.json")

@forked_from("source-bucket", snapshot_version="12345")
def make_fork(s3, name):
    return s3.create_bucket(Bucket=name)

@with_rename
def rename(s3, bucket, old, new):
    return s3.copy_object(Bucket=bucket, CopySource=f"{bucket}/{old}", Key=new)
```

### Helper Functions

```python
from tigris_boto3_ext import (
    create_snapshot_bucket,
    create_snapshot,
    list_snapshots,
    create_fork,
    get_object_from_snapshot,
    get_snapshot_version,
    list_objects_from_snapshot,
    head_object_from_snapshot,
    has_snapshot_enabled,
    get_bucket_info,
    rename_object,
)

create_snapshot_bucket(s3, "my-bucket")

result = create_snapshot(s3, "my-bucket", snapshot_name="backup-1")
version = get_snapshot_version(result)

snapshots = list_snapshots(s3, "my-bucket")
create_fork(s3, "fork-bucket", "my-bucket", snapshot_version=version)

obj = get_object_from_snapshot(s3, "my-bucket", "file.txt", version)
objects = list_objects_from_snapshot(s3, "my-bucket", version, Prefix="data/")
metadata = head_object_from_snapshot(s3, "my-bucket", "file.txt", version)

rename_object(s3, "my-bucket", "old.txt", "new.txt")

# Inspect bucket
if has_snapshot_enabled(s3, "my-bucket"):
    info = get_bucket_info(s3, "my-bucket")
    # info: { snapshot_enabled, fork_source_bucket, fork_source_snapshot }
```

---

## Snapshots

Point-in-time bucket copies. Tigris stores only deltas, so they're cheap.

```python
# 1. Enable snapshots when creating the bucket
create_snapshot_bucket(s3, "production-data")

# 2. Take a named snapshot
result = create_snapshot(s3, "production-data", snapshot_name="daily-2026-05-07")
version = get_snapshot_version(result)

# 3. Read from the snapshot — historical view
with TigrisSnapshot(s3, "production-data", snapshot_version=version):
    obj = s3.get_object(Bucket="production-data", Key="config.json")

# 4. List all snapshots
snaps = list_snapshots(s3, "production-data")
for b in snaps.get("Buckets", []):
    print(b["Name"])
```

Snapshots must be enabled at bucket creation — there is no retroactive enable.

## Bucket Forking

Forks are independent buckets created from the live state or a snapshot of a source bucket. Storage is shared until divergence, so forks are essentially free.

```python
# Fork from current state
with TigrisFork(s3, "source-bucket"):
    s3.create_bucket(Bucket="forked-bucket")

# Fork from a snapshot — common for testing against prod data
create_fork(s3, "test-bucket", "production-data", snapshot_version=version)

# Inspect lineage
info = get_bucket_info(s3, "test-bucket")
print(info["fork_source_bucket"], info["fork_source_snapshot"])
```

Common uses: test/staging environments off prod data, time-travel debugging from a pre-incident snapshot, per-experiment workspaces in ML pipelines.

## In-Place Object Rename

Tigris implements rename as a `copy_object` plus the `X-Tigris-Rename: true` header — only the key changes, no data rewrite.

```python
from tigris_boto3_ext import TigrisRename, rename_object

# Helper form (recommended)
rename_object(s3, "my-bucket", "old-name.txt", "new-name.txt")

# Context manager — every copy_object inside the block becomes a rename
with TigrisRename(s3):
    s3.copy_object(
        Bucket="my-bucket",
        CopySource="my-bucket/old.txt",
        Key="new.txt",
    )
```

Keep `TigrisRename` / `with_rename` scope tight. Any `copy_object` inside the block becomes a rename — unrelated copies will not behave as expected.

## Conditional Writes

Tigris supports the standard S3 precondition headers — `If-Match`, `If-None-Match`, `If-Modified-Since`, `If-Unmodified-Since` — for optimistic concurrency and create-if-not-exists semantics. These are pure boto3 parameters; `tigris-boto3-ext` adds nothing on top because boto3 already passes them through.

```python
from botocore.exceptions import ClientError

# Create-only (fail if the key already exists)
try:
    s3.put_object(
        Bucket="my-bucket",
        Key="config.json",
        Body=b'{"version": 1}',
        IfNoneMatch="*",
    )
except ClientError as e:
    if e.response["Error"]["Code"] == "PreconditionFailed":
        print("Object already exists")
    else:
        raise

# Optimistic concurrency — read, modify, write only if unchanged
head = s3.head_object(Bucket="my-bucket", Key="config.json")
etag = head["ETag"]

new_body = b'{"version": 2}'
try:
    s3.put_object(
        Bucket="my-bucket",
        Key="config.json",
        Body=new_body,
        IfMatch=etag,
    )
except ClientError as e:
    if e.response["Error"]["Code"] == "PreconditionFailed":
        # Someone else wrote the object since we read it — retry the read-modify-write
        ...
    else:
        raise
```

| boto3 parameter      | Behavior                                                            | Common use                              |
| -------------------- | ------------------------------------------------------------------- | --------------------------------------- |
| `IfMatch=<etag>`     | Write only if current ETag matches                                  | Optimistic concurrency / lost-update    |
| `IfNoneMatch="*"`    | Write only if the object does not exist                             | Create-once / idempotent writes         |
| `IfNoneMatch=<etag>` | Write only if current ETag differs                                  | Skip identical re-uploads               |
| `IfModifiedSince`    | Proceed only if modified after the given RFC 1123 timestamp         | Conditional reads / cache validation    |
| `IfUnmodifiedSince`  | Proceed only if not modified after the given RFC 1123 timestamp     | Time-bounded optimistic concurrency     |

Failed preconditions surface as a `ClientError` with `Error.Code == "PreconditionFailed"` (HTTP 412). Catch that specific code — don't swallow all `ClientError`s, since the same exception type covers `NoSuchKey`, throttling, etc.

`IfMatch` / `IfNoneMatch` work on `put_object`, `copy_object`, `get_object`, `head_object`, and `delete_object`.

## Bundle API

Fetch many objects in a single streaming tar archive. Designed for ML training where per-object roundtrips dominate latency.

```python
import tarfile
from tigris_boto3_ext import bundle_objects, BundleError, BUNDLE_ON_ERROR_FAIL

keys = [f"dataset/train/img_{i:05d}.jpg" for i in range(1000)]
response = bundle_objects(s3, "my-dataset-bucket", keys)

with tarfile.open(fileobj=response, mode="r|") as tar:
    for member in tar:
        if member.name == "__bundle_errors.json":
            continue  # error manifest — skip
        f = tar.extractfile(member)
        if f is not None:
            data = f.read()
            # feed to training pipeline
```

**Error modes:**

- Default: missing/failed objects are reported in `__bundle_errors.json` inside the tar; partial success is OK.
- `on_error=BUNDLE_ON_ERROR_FAIL`: any failure raises `BundleError` with `status_code` and `body`. Use for inference where every key must be present.

```python
try:
    response = bundle_objects(s3, "my-bucket", keys, on_error=BUNDLE_ON_ERROR_FAIL)
except BundleError as e:
    print(f"Bundle failed (HTTP {e.status_code}): {e.body}")
```

Always open the tar in streaming mode (`mode="r|"`). Don't materialize the whole archive in memory.

---

## Common Workflows

### Backup and Restore

```python
import boto3
from tigris_boto3_ext import (
    create_snapshot_bucket, create_snapshot, create_fork, get_snapshot_version,
)

s3 = boto3.client("s3")

create_snapshot_bucket(s3, "production-data")

# Daily snapshot
result = create_snapshot(s3, "production-data", snapshot_name="daily-backup")
version = get_snapshot_version(result)

# Restore: fork the snapshot into a recovered bucket
create_fork(s3, "restored-data", "production-data", snapshot_version=version)
```

### Testing Against Production Data

```python
result = create_snapshot(s3, "production-data", snapshot_name="for-tests")
version = get_snapshot_version(result)
create_fork(s3, "test-data", "production-data", snapshot_version=version)

try:
    # Writes to test-data don't touch production
    s3.put_object(Bucket="test-data", Key="experiment.txt", Body=b"...")
finally:
    s3.delete_bucket(Bucket="test-data")
```

### Time-Travel Queries

```python
from tigris_boto3_ext import get_object_from_snapshot, list_objects_from_snapshot

old = get_object_from_snapshot(s3, "my-bucket", "config.json", "12345")
config = old["Body"].read()

old_logs = list_objects_from_snapshot(
    s3, "my-bucket", "12345", Prefix="logs/2025/12/",
)
for obj in old_logs.get("Contents", []):
    print(obj["Key"])
```

---

## How It Works

`tigris-boto3-ext` registers handlers on boto3's `before-sign.s3.*` event to inject Tigris headers when a context/decorator is active:

| Header                                        | Purpose                                         |
| --------------------------------------------- | ----------------------------------------------- |
| `X-Tigris-Enable-Snapshot: true`              | Enable snapshots on bucket creation             |
| `X-Tigris-Snapshot: true; name=<name>`        | Create a named snapshot                         |
| `X-Tigris-Snapshot: <bucket>`                 | List snapshots for a bucket                    |
| `X-Tigris-Snapshot-Version: <version>`        | Read from a specific snapshot version           |
| `X-Tigris-Fork-Source-Bucket: <bucket>`       | Fork from this bucket                           |
| `X-Tigris-Fork-Source-Bucket-Snapshot: <ver>` | Fork from a specific snapshot                   |
| `X-Tigris-Rename: true`                       | Turn `CopyObject` into in-place rename          |

`HeadBucket` responses include `X-Tigris-Enable-Snapshot`, `X-Tigris-Fork-Source-Bucket`, and `X-Tigris-Fork-Source-Bucket-Snapshot` — surfaced via `get_bucket_info()` and `has_snapshot_enabled()`.

---

## Critical Rules

**Always:**

- Enable snapshots at bucket creation — there is no retroactive enable
- Capture `get_snapshot_version(result)` immediately after `create_snapshot` — you need it to read or fork
- Stream Bundle API responses (`mode="r|"`) — never load the whole tar in memory
- Skip `__bundle_errors.json` when iterating tar members
- Keep `TigrisRename` / `with_rename` scope tight — every `copy_object` inside becomes a rename

**Never:**

- Hard-code `region_name` to a real AWS region — use `auto`
- Use the production bucket directly for tests when a fork costs nothing
- Forget `AWS_ENDPOINT_URL_S3` — without it boto3 hits AWS S3, not Tigris

---

## Known Issues

| Problem                              | Cause & Fix                                                                  |
| ------------------------------------ | ---------------------------------------------------------------------------- |
| Snapshot calls fail with 404/409     | Snapshots not enabled on the bucket. Recreate with `create_snapshot_bucket`. |
| `copy_object` renamed unrelated file | `TigrisRename` block too broad. Wrap only the rename call.                   |
| Bundle hangs on iteration            | Tar opened in non-streaming mode. Use `mode="r\|"`, not `"r"`.               |
| `BundleError` raised unexpectedly    | `BUNDLE_ON_ERROR_FAIL` was passed. Default mode reports errors in the tar.   |
| Calls go to AWS, not Tigris          | `AWS_ENDPOINT_URL_S3` unset. Export it or pass `endpoint_url=` explicitly.   |
| Fork has no snapshots enabled        | Forks don't inherit snapshot capability — enable on the fork separately.     |

---

## CLI Quick Reference

`t3` is an alias for `tigris`.

```bash
# Auth
tigris login
tigris whoami

# Buckets
tigris buckets create <name> [--public] [--locations <region>]
tigris buckets list
tigris buckets delete <name>

# Access keys
tigris access-keys create "<name>"
tigris access-keys assign <tid_xxx> --bucket <name> --role Editor

# Snapshots & forks (also doable from Python via this skill)
tigris buckets snapshot <bucket> --name <snapshot-name>
tigris buckets snapshots <bucket>
tigris buckets fork <new> --source <bucket> [--snapshot <version>]
```

---

## Related Skills

- **file-storage** — General Tigris setup and the JS/TS `@tigrisdata/storage` SDK
- **tigris-snapshots-forking** — CLI-side snapshot/fork workflows
- **tigris-agent-kit** — Agent storage workflows (forks for sandboxes, checkpoints)
- **tigris-sdk-guide** — Picking SDKs across languages

## Official Documentation

- Package: https://pypi.org/project/tigris-boto3-ext/
- Repo: https://github.com/tigrisdata/tigris-boto3-ext
- Tigris docs: https://www.tigrisdata.com/docs/

• SKILL.md 18 KB
• resources/django.md 2 KB