Skip to content
Skills Library
Data & analytics · Tool integration

materialize-generic-create

/materialize-generic-create

Create a materialization for any Estuary destination connector. Use this generic skill when no dedicated skill exists for the target system. Use when user says "materialize to", "send data to", "destination connector", or "set up materialization".

Licence
Apache-2.0
Last repository change
24.09.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
1
Deprecation

No signals detected. Repository changed on 24.09.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Create a materialization for any Estuary destination connector. Use this generic skill when no dedicated skill exists for the target system. Use when user says "materialize to", "send data to", "destination connector", or "set up materialization".

Triggers from the description

materialize tosend data todestination connectorset up materialization
Original author description

Create a materialization for any Estuary destination connector. Use this generic skill when no dedicated skill exists for the target system. Use when user says "materialize to", "send data to", "destination connector", or "set up materialization".

How it works

  1. Check for a Dedicated Skill First
  2. Step 0: Find the Connector and Load Documentation
  3. Step 1: Gather Requirements
  4. Step 2: Find the Correct Connector Version
  5. Step 3: Help User Complete Prerequisites
  6. Step 4: Create the Spec File
  7. Step 5: Publish
  8. Step 6: Verify
  9. Troubleshooting
  10. Related Skills
Original SKILL.md headings · 185 lines, 837 words
SKILL.md preview
---
name: materialize-generic-create
description: Create a materialization for any Estuary destination connector. Use this generic skill when no dedicated skill exists for the target system. Use when user says "materialize to", "send data to", "destination connector", or "set up materialization".
---

# Create Materialization (Any Connector)

Create a materialization using flowctl for any supported Estuary destination connector. This generic skill handles connectors that don't have a dedicated skill.

## Check for a Dedicated Skill First

Before using this generic workflow, check if a dedicated `materialize-<connector>-create` skill exists for the user's target destination. Dedicated skills have connector-specific guidance and troubleshooting that this generic skill can't provide. Only use this skill if no dedicated one exists.

## Step 0: Find the Connector and Load Documentation

### Find the connector

Query the connector registry to find available materialization connectors:

```bash
# List all materialization connectors
flowctl raw get --table connector_tags \
  --query 'protocol=eq.materialization' \
  --query 'select=image_tag,documentation_url' \
  --output yaml
```

To search for a specific connector by name:

```bash
flowctl raw get --table connector_tags \
  --query 'documentation_url=ilike.*<connector-name>*' \
  --query 'protocol=eq.materialization' \
  --query 'select=image_tag,documentation_url' \
  --output yaml
```

### Load the documentation

Use WebFetch to load the `documentation_url` returned from the query. This is the canonical docs page with prerequisites, config reference, and setup instructions.

**Search Kapa for tribal knowledge** (if the Estuary MCP is configured):

```
Search kapa ai knowledge sources for "materialize <connector-name> common issues"
```

If Kapa MCP is not configured, the user can set it up: https://docs.estuary.dev/features/mcp-integration/

## Step 1: Gather Requirements

Before writing any YAML, ask the user:

1. **Target system details?** — Connection info (host, port, credentials, etc.)
2. **Network path?** — Direct connection, SSH tunnel, or other networking requirements
3. **Non-default data plane?** — Most users use the default. Ask if they need a non-default data plane.
4. **Source collections?** — Which Estuary collections to materialize
5. **Any connector-specific requirements?** — Check the docs loaded in Step 0
6. **Hard deletes?** — Off by default. Without it, deleted rows stay in the destination marked with `_meta/op: 'd'`. Enable to physically remove them.
7. **Delta updates?** — Off by default. Switches from one-row-per-key (standard merge) to append-only. Use for event logs or history tables.
8. **Sync schedule?** — Controls how often batches are written to the destination (default: 30 minutes, `0s` for real-time). Affects latency and destination compute cost.

## Step 2: Find the Correct Connector Version

Use the `image_tag` from the Step 0 query. If you need to re-query for a specific connector:

```bash
flowctl raw get --table connector_tags \
  --query 'documentation_url=eq.<DOCS-URL-FROM-STEP-0>' \
  --query 'select=image_tag,documentation_url' \
  --output yaml
```

Use the returned `image_tag` — never hardcode a version.

## Step 3: Help User Complete Prerequisites

Walk the user through prerequisites from the docs loaded in Step 0. Common prerequisites across materializations:

1. **Destination accessible** — Network connectivity from Estuary cloud
2. **Credentials configured** — User/service account with write permissions
3. **Target location exists** — Database, schema, bucket, etc.

Refer to the docs page for connector-specific prerequisite details.

## Step 4: Create the Spec File

Build `flow.yaml` using the config reference from the docs. General structure:

```yaml
materializations:
  <TENANT>/<PATH>/materialize-<connector>:
    endpoint:
      connector:
        image: <IMAGE>:<VERSION>
        config:
          # Properties from the docs config reference
          # Each connector has different required fields
    bindings:
      - source: <TENANT>/<collection-path>
        resource:
          table: "<TABLE_NAME>"
          # Some connectors use different resource fields
          # (e.g., "topic" for Kafka, "index" for Elasticsearch)
```

Fill in the `config` section using the property reference from the connector's docs page.

For SSH tunnel (if supported by the connector), add `networkTunnel.sshForwarding` block — see docs.

## Step 5: Publish

```bash
flowctl catalog publish --source flow.yaml --auto-approve
```

## Step 6: Verify

```bash
# Check status
flowctl catalog status <TENANT>/<PATH>/materialize-<connector>

# View logs
flowctl logs --task <TENANT>/<PATH>/materialize-<connector> --since 5m | jq -c '{ts, message}'
```

**Status progression:**
1. `PENDING` — Normal for ~30 seconds during shard assignment
2. `BACKFILLING` — Initial data sync from collections
3. `OK` — Running normally with real-time updates

## Troubleshooting

### Materialization stuck in PENDING

Wait 30-60 seconds — this is normal during shard assignment. If still stuck:

```bash
flowctl logs --task <TENANT>/<PATH>/materialize-<connector> --since 5m | jq 'select(.level == "error")'
```

### Config validation errors

**Cause**: Config structure doesn't match the connector's schema

**Fix**: Re-read the docs page from Step 0 and verify all required properties are set with correct types and formatting.

### "collection not found"

**Cause**: Source collection doesn't exist or wrong prefix

**Fix**:
```bash
flowctl catalog list --prefix <TENANT>/ | grep collection
```
Verify the collection name matches exactly.

### "no connector tag found for image"

**Cause**: Wrong image name or tag

**Fix**: Re-query `connector_tags` (Step 0) for the correct image and version.

### Network connectivity failures

**Cause**: Destination not reachable from Estuary cloud

**Fix**:
1. Check firewall rules / security groups
2. Verify Estuary IP addresses are allowlisted (see docs)
3. Consider SSH tunnel for private networks
4. For local testing: use ngrok or similar

### Connector-specific errors

For errors specific to the connector, search Kapa:

```
Search kapa ai knowledge sources for "materialize <connector-name> <error message>"
```

## Related Skills

- `schema-field-selection` — Control which fields are materialized
- `connector-disable-enable` — Pause/restart existing materializations
- `connector-delete-recreate` — Nuclear option for stuck materializations
- `estuary-logs` — Deep log analysis
- `estuary-catalog-status` — Status checking

Versions

The database has no other skills named materialize-generic-create.

Database snapshot: 13.11.2026 · package version add5c053a834 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100materialize-bigquery-create
    Create a BigQuery materialization to stream Estuary collections into BigQuery tables. Use when setting up BigQuery as a destination for captured data. Use when user says "send to BigQuery", "load into BigQuery", "materialize to BigQuery", or "BigQuery destination".
  • 100materialize-snowflake-create
    Create a Snowflake materialization to stream Estuary collections into Snowflake tables. Use when setting up Snowflake as a destination for captured data. Use when user says "send to Snowflake", "materialize to Snowflake", "Snowflake destination", or "load into Snowflake".
  • 100estuary-webhooks
    Receive and authenticate Estuary webhooks. Use when setting up an endpoint for Estuary materialization triggers or the HTTP Webhook materialization connector, checking the secret header you configure (Estuary does not sign requests), debugging 401s, or handling POST, PUT or PATCH deliveries.
  • 100add-connector
    Use when adding a Power Platform connector to an Expo/React Native Power Apps mobile app and no dedicated mobile connector skill exists.
  • 90mz-docs
    Materialize documentation for SQL syntax, data ingestion, concepts, and best practices. Use when users ask about Materialize queries, sources, sinks, views, or clusters.
  • 100write-pipeline
    MUST use when creating or modifying a data pipeline, a set of scripts marked `pipeline` and wired together by `on` / `materialize` annotations.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.