Skip to content
Skills Library
Data & analytics · Building & coding

role-database:streaming-databases

/role-database:streaming-databases

Deep operational guide for 14 streaming databases and platforms. Kafka (KRaft, Streams, Connect, Schema Registry), Pulsar (multi-tenant, geo-replication), Redpanda (Kafka-compatible, no JVM), NATS/JetStream, Flink (streaming SQL), Materialize, RisingWave, Kinesis, Event Hubs, Pub/Sub, EventStoreDB. Use when implementing event streaming, CDC, real-time analytics, or event sourcing.

Licence
MIT
Last repository change
24.07.2026

Assessment and security

80/ 100Class: good
Format15 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content25 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
7
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 24.07.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Deep operational guide for 14 streaming databases and platforms. Kafka (KRaft, Streams, Connect, Schema Registry), Pulsar (multi-tenant, geo-replication), Redpanda (Kafka-compatible, no JVM), NATS/JetStream, Flink (streaming SQL), Materialize, RisingWave, Kinesis, Event Hubs, Pub/Sub, EventStoreDB. Use when implementing event streaming, CDC, real-time analytics, or event sourcing.

Original author description

Deep operational guide for 14 streaming databases and platforms. Kafka (KRaft, Streams, Connect, Schema Registry), Pulsar (multi-tenant, geo-replication), Redpanda (Kafka-compatible, no JVM), NATS/JetStream, Flink (streaming SQL), Materialize, RisingWave, Kinesis, Event Hubs, Pub/Sub, EventStoreDB. Use when implementing event streaming, CDC, real-time analytics, or event sourcing.

How it works

  1. When to use this skill
  2. Core Principles
  3. Reference Pointers
Original SKILL.md headings · 27 lines, 249 words
SKILL.md preview
---
name: role-database:streaming-databases
description: |
  Deep operational guide for 14 streaming databases and platforms. Kafka (KRaft, Streams, Connect, Schema Registry), Pulsar (multi-tenant, geo-replication), Redpanda (Kafka-compatible, no JVM), NATS/JetStream, Flink (streaming SQL), Materialize, RisingWave, Kinesis, Event Hubs, Pub/Sub, EventStoreDB. Use when implementing event streaming, CDC, real-time analytics, or event sourcing.
allowed-tools: Read, Grep, Glob, Bash
---

You are a streaming databases specialist informed by the Software Engineer by RN competency matrix.

## When to use this skill

Load this skill for event streaming pipelines, CDC (change data capture), real-time analytics, event sourcing, or any task involving message brokers, stream processors, or exactly-once delivery guarantees.

## Core Principles

- Kafka is the default choice for high-throughput durable streams; Redpanda when JVM overhead is unacceptable
- Exactly-once requires both idempotent producers AND transactional consumers — half-measures give at-least-once
- Partition count is a one-way door in most platforms — over-provision, don't under-partition
- CDC (Debezium) is the safest sync strategy: no dual-write race conditions, captures deletes

## Reference Pointers

Load the relevant reference file for implementation details:

| File | When to load |
|------|-------------|
| `references/kafka.md` | Kafka topic design, KRaft, EOS transactions, consumer rebalancing, Streams DSL, Debezium Connect, Schema Registry, MirrorMaker 2 |
| `references/pulsar-redpanda-nats.md` | Pulsar multi-tenancy, geo-replication, Functions; Redpanda single-binary setup, tiered storage; NATS JetStream streams, KV store, object store |
| `references/flink-materialize-risingwave.md` | Flink DataStream API, checkpointing, SQL, CDC source; Materialize incremental views; RisingWave streaming SQL with PostgreSQL sink |
| `references/cloud-platforms.md` | Amazon Kinesis (KCL, Firehose), Azure Event Hubs (Kafka protocol, Capture), Google Pub/Sub (ordering keys), Spark Structured Streaming |
| `references/eventsourcedb-rabbitmq-memphis.md` | EventStoreDB event sourcing, projections, catch-up subscriptions; RabbitMQ Streams, super streams; Memphis with dead-letter stations |
| `references/patterns-operations.md` | Platform comparison table, event sourcing + CQRS, CDC pipeline, saga pattern, event mesh, exactly-once comparison, capacity planning, schema evolution |
Files in this package (7)
  • SKILL.md 2 KB
  • references/cloud-platforms.md 4 KB
  • references/eventsourcedb-rabbitmq-memphis.md 4 KB
  • references/flink-materialize-risingwave.md 4 KB
  • references/kafka.md 5 KB
  • references/patterns-operations.md 5 KB
  • references/pulsar-redpanda-nats.md 4 KB

Versions

The database has no other skills named role-database:streaming-databases.

Database snapshot: 13.11.2026 · package version 3e008840e6bd (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100cdc-streaming-ingestion
    Real-time RAG ingestion. CDC (Debezium, Postgres logical replication), Kafka/ Pulsar topics for doc events, stream processing (Flink, Kafka Streams) to embedding service, exactly-once semantics, late-arriving updates, tombstones (deletes), upsert to vector DB, schema evolution.…
  • 90data-streaming
    Use this skill when asked about streaming, Kafka, Flink, Kinesis, stream processing, event stream, real-time, CDC, change data capture, message queue, or stream architecture. This skill enforces: Kafka topic design with partitioning strategy, Flink/ksqlDB stream processing with…
  • 90streaming
    Cross-platform guidance for streaming and real-time data integration architecture: batch-vs-streaming decisions, event-driven patterns, and technology selection. WHEN: \"streaming architecture\", \"batch vs streaming\", \"event-driven data\", \"real-time pipeline design\",…
  • 90streaming-and-messaging-systems
    Guides agents through event streaming and real-time data pipeline design. Use when working with Kafka, Kinesis, Flink, stream processing, windowing, stateful consumers, or near-real-time publish flows.
  • 100nats-and-pulsar-lightweight-messaging-configuration
    Configures NATS core pub/sub and JetStream persistence, and Apache Pulsar topics/subscriptions, as lighter-weight alternatives to Kafka or RabbitMQ, and gives a decision framework for when each fits better. Use when the user asks to "set up NATS," "configure JetStream streams…
  • 100connect-cdc-sqlserver
    Streams change data capture from Microsoft SQL Server into Redpanda or Kafka using Redpanda Connect's microsoft_sql_server_cdc input, which reads native CDC change tables in LSN order. Use when enabling SQL Server CDC capture instances, configuring microsoft_sql_server_cdc,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.