Skip to content
Skills Library
Programming · Tool integration

walkeros-create-source

/walkeros-create-source

Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern.

Repository
walkerOS ↗
Licence
MIT
Last repository change
22.10.2026

Assessment and security

91/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content26 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
12
Scripts
10
HTML comments
0
URLs in SKILL.md
2
Deprecation

No signals detected. Repository changed on 22.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What to check before installing
  • 10 scripts in this package. Review them before installing, as Claude may execute them.

Scripts: examples/index.ts, examples/inputs.ts, examples/mapping.ts, examples/outputs.ts, examples/requests.ts, templates/server/dev.ts, templates/server/index.test.ts, templates/server/index.ts

What it does and when to use it

Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern.

Original author description

Use when creating a new walkerOS source to capture events (browser source, dataLayer interception, server/HTTP source, webhook receiver, event capture), web or server-side. Example-driven workflow: research the input format and define step examples before implementing the push interface, createTrigger, and env pattern.

How it works

  1. Prerequisites
  2. Source Types
  3. Augmenting SourceMap
  4. Source Categories
  5. Choose Your Template
  6. Process Overview
  7. Phase 1: Research
  8. Phase 2: Classify Source Type
  9. Phase 3: Create Input Examples (BEFORE Implementation)
  10. Phase 4: Define Mapping
Original SKILL.md headings · 921 lines, 4,586 words
SKILL.md preview
---
name: walkeros-create-source
description: >-
  Use when creating a new walkerOS source to capture events (browser source,
  dataLayer interception, server/HTTP source, webhook receiver, event capture),
  web or server-side. Example-driven workflow: research the input format and
  define step examples before implementing the push interface, createTrigger,
  and env pattern.
---

# Create a New Source

## Prerequisites

Before starting, read these skills:

- [understanding-flow](../walkeros-understanding-flow/SKILL.md) - How sources
  fit in architecture
- [understanding-sources](../walkeros-understanding-sources/SKILL.md) - Source
  interface
- [understanding-transformers](../walkeros-understanding-transformers/SKILL.md) -
  Transformer chaining from sources
- [understanding-events](../walkeros-understanding-events/SKILL.md) - Event
  structure sources emit
- [understanding-mapping](../walkeros-understanding-mapping/SKILL.md) -
  Transform raw input to events
- [testing-strategy](../walkeros-testing-strategy/SKILL.md) - How to test with
  env pattern and dev examples
- [using-step-examples](../walkeros-using-step-examples/SKILL.md) -
  Authoritative `Flow.StepExample` pattern, `createTrigger`, Three Type Zones
- [using-logger](../walkeros-using-logger/SKILL.md) - For an HTTP source, see
  "Choosing a level on public endpoints": rejected client input is debug or warn
  plus a `rejected` counter and a 4xx, never an error-level log
- [writing-documentation](../walkeros-writing-documentation/SKILL.md) -
  Documentation standards (for Phase 9)

## Source Types

A source's identity is split into two fields:

| Field             | Meaning                                                         | Examples                                       |
| ----------------- | --------------------------------------------------------------- | ---------------------------------------------- |
| `source.type`     | The kind of source (its role / mechanism)                       | `browser`, `dataLayer`, `cookiefirst`, `fetch` |
| `source.platform` | The runtime that hosts the source (`web`, `server`, `app`, ...) | `web`, `server`                                |

| Platform | Input                   | Example types                       |
| -------- | ----------------------- | ----------------------------------- |
| `web`    | DOM events, dataLayer   | `browser`, `dataLayer`              |
| `server` | HTTP requests, webhooks | `gcp`, `express`, `lambda`, `fetch` |

## Augmenting `SourceMap`

Every source registers its `type` literal and any source-specific `source.*`
fields by augmenting `SourceMap` from `@walkeros/core`. This is how the
collector and downstream consumers know about the new source kind without
loosening the union to `string`.

Add this to the source's `src/types.ts` (or `src/types/index.ts`):

```typescript
import type { Source, Elb } from '@walkeros/core';

declare module '@walkeros/core' {
  interface SourceMap {
    // Replace `mySource` with the source's package-level identifier.
    mySource: {
      type: 'mySource'; // matches the literal you return from Source.Init
      platform?: 'web'; // 'web' | 'server' | 'app' | ...
      // Add any extra fields the source surfaces in `event.source.*` here.
      // e.g. `version?: string;` is already on the base Source - only add
      // truly source-specific keys.
    };
  }
}
```

Reference implementations: `packages/web/sources/browser/src/types/index.ts`,
`packages/web/sources/demo/src/types.ts`. Conflicting declarations cause compile
errors on purpose, this surfaces naming collisions early.

## Source Categories

| Category           | Purpose                                   | Examples                | Key Concern          |
| ------------------ | ----------------------------------------- | ----------------------- | -------------------- |
| **Transformation** | Convert external format → walkerOS events | `dataLayer`, `fetch`    | Mapping accuracy     |
| **Transport**      | Receive events from specific platform     | `gcp`, `aws`, `express` | Platform integration |

## Choose Your Template

| Complexity            | Template       | When to Use                           |
| --------------------- | -------------- | ------------------------------------- |
| Simple transformation | `fetch/`       | Generic HTTP handler, data conversion |
| Platform transport    | `gcp/`, `aws/` | Cloud platform integration            |
| Browser interception  | `dataLayer/`   | DOM events, array interception        |

## Process Overview

```
1. Research     → Deeply understand external system, SDK, and data format
2. Classify     → Determine source type and integration approach
3. Examples     → Define in/out pairs FIRST (start with the end result)
4. Mapping      → Define input → walkerOS event transformation
5. Scaffold     → Copy template and configure
6. Convention   → Add walkerOS.json metadata and buildDev
7. Implement    → Build using examples as test fixtures
8. Test         → Verify against example variations
9. Document     → Write README
```

---

## Phase 1: Research

**Goal:** Deeply understand the external system before writing any code.
Research quality determines implementation quality.

### 1.1 Find and Install Official SDK

Always prefer the vendor's official SDK package over raw HTTP API calls. The SDK
handles transport, data formatting, and platform specifics - don't reinvent
these.

- [ ] **Install the SDK** - `npm install @vendor/sdk` and read the actual source
- [ ] **Read TypeScript types** - Import types from the SDK package directly.
      Never duplicate type definitions. This ensures IntelliSense completeness
      and consistency with SDK updates.
- [ ] **Understand the full API surface** - List every public method and type
      export. What data structures does the platform provide? What request/event
      formats exist?

```bash
# Search npm for official packages
npm search [vendor-name]
npm search @[vendor]

# Install and inspect actual types
npm install @vendor/sdk
ls node_modules/@vendor/sdk/lib/esm/
```

### 1.2 Understand SDK Architecture

- [ ] **Init options** - What does the SDK expose? How is the platform
      connection established?
- [ ] **Call ordering** - When does data arrive? Is it pushed (webhooks,
      callbacks) or pulled (polling, intercepting)? What are the timing
      implications?
- [ ] **Data format** - What does the raw event/request look like? Headers, body
      structure, query params, authentication tokens?
- [ ] **Identity signals** - Does the external system provide user IDs, session
      IDs, device IDs? How are they delivered (headers, cookies, body fields)?
- [ ] **Consent** - Does the platform have consent signals? How are they
      communicated?

### 1.3 Identify All Data Entry Points

Go beyond just the primary event payload. Most external systems provide multiple
data channels:

| Data Channel     | Examples                            | walkerOS Handling    |
| ---------------- | ----------------------------------- | -------------------- |
| Event payload    | Request body, DOM event data        | Default `push()`     |
| Headers/metadata | Auth tokens, content-type, origin   | `context` or `user`  |
| Query params     | UTM parameters, tracking IDs        | `data` or `context`  |
| Platform context | Cloud function metadata, Lambda ctx | `source` or `custom` |
| Identity         | User ID, session ID, device ID      | `user`               |
| Consent signals  | Opt-in/out flags, consent string    | `consent`            |

### 1.4 Check Existing Patterns

Review similar sources in the codebase:

```bash
# List existing sources
ls packages/web/sources/
ls packages/server/sources/

# Reference implementations
# - dataLayer: DOM-based, array interception
# - express: HTTP middleware
# - fetch: Generic HTTP handler (simplest server pattern)
# - gcp: Cloud Functions specific
```

### Gate: Research Complete

- [ ] Official SDK installed and types inspected (or HTTP API documented if no
      SDK exists)
- [ ] All data entry points listed with their format
- [ ] Init options and call ordering understood
- [ ] Identity and consent signal delivery documented
- [ ] Raw event/request structure mapped to walkerOS event fields

### Checkpoint: Research Review (Optional)

If working with human oversight, pause here to confirm:

- SDK integration approach correct?
- Data capture strategy makes sense for the use case?
- Any platform quirks or limitations to handle?

---

## Phase 2: Classify Source Type

**Goal:** Understand what the source captures and how it delivers data, which
determines implementation complexity.

### 2.1 Source Categories

| Category           | Description                                | Mapping Needed                  | Example Sources          |
| ------------------ | ------------------------------------------ | ------------------------------- | ------------------------ |
| **Transformation** | Converts external event format to walkerOS | Essential - must map fields     | `dataLayer`, `fetch`     |
| **Transport**      | Receives events from a specific platform   | Structural - platform unwrap    | `gcp`, `aws`, `express`  |
| **Interception**   | Intercepts existing data flows             | Varies - depends on data format | `dataLayer`, CMP sources |

### 2.2 Determine Integration Approach

| Approach                    | When to use                         | Pattern                                                   |
| --------------------------- | ----------------------------------- | --------------------------------------------------------- |
| **Platform SDK as host**    | SDK provides typed request/response | Use SDK types, wrap handler in walkerOS source            |
| **DOM interception**        | Capture browser-side events         | Listen to DOM events, intercept arrays/globals            |
| **HTTP handler**            | Generic webhook/API receiver        | Parse request, extract events, forward to collector       |
| **Callback/event listener** | Platform provides event emitter     | Register listener, transform events, forward to collector |

**Prefer the vendor SDK** - it provides typed request/response objects and
handles platform specifics. Raw HTTP parsing is a fallback when no SDK exists.

When using the vendor SDK:

- Import types from the SDK package directly
- Use SDK request/response types for handler signatures
- Let the SDK handle platform-specific parsing (body parsing, header extraction)

For an HTTP handler, forward the full `DeepPartialEvent` the body carries,
renaming only the wire-level event-name key. Never project the body through a
field allowlist: that strips `source` provenance (`release`, `trace`) and other
legitimate fields. `express` and `fetch` are the reference pass-through
implementations.

### Gate: Classification Complete

- [ ] Source category identified (transformation / transport / interception)
- [ ] Integration approach chosen (SDK / DOM / HTTP / callback)
- [ ] Know what the source captures and how it delivers data

---

## Phase 3: Create Input Examples (BEFORE Implementation)

**Mandatory.** Examples are the test fixtures for Phase 8. Define expected
`trigger` / `in` / `out` triples FIRST - start with the end result in mind.
Without examples, you cannot test. Even for simple sources, step examples are
the single source of truth for tests, simulations, and documentation.

> **Authoritative pattern:** See
> [using-step-examples](../walkeros-using-step-examples/SKILL.md) for the Three
> Type Zones, `createTrigger` contract, and CI integration. This skill reuses
> that contract - do not diverge.

### 3.1 Scaffold Directory Structure

```bash
mkdir -p packages/<platform>/sources/[name]/src/examples
mkdir -p packages/<platform>/sources/[name]/src/{schemas,types}
```

### 3.2 Required Files (3-4 files)

All reference sources in the monorepo use this exact layout in `src/examples/`.
Match it - no `inputs.ts`, `outputs.ts`, `requests.ts`, or standalone
`mapping.ts`.

| File                  | Required? | Purpose                                                             |
| --------------------- | --------- | ------------------------------------------------------------------- |
| `examples/step.ts`    | yes       | `Flow.StepExample` entries with `trigger` / `in` / `out` triples    |
| `examples/trigger.ts` | yes       | `createTrigger` implementation following `Trigger.CreateFn`         |
| `examples/index.ts`   | yes       | Barrel exports: `env` (if present), `step`, `createTrigger`         |
| `examples/env.ts`     | if needed | Mock env for platform deps (browser window/document, express, etc.) |

`env.ts` is included whenever the source touches platform globals or injected
deps - all web sources and every server source that wraps a platform SDK ship
one. Sources whose tests drive the collector entirely through `trigger.ts` (e.g.
`web/sources/session`) may omit it. When in doubt, **include it**.

The old `inputs.ts` / `outputs.ts` / `requests.ts` / `mapping.ts` files are gone

- their contents now live inline in each `Flow.StepExample` entry in `step.ts`.

### 3.3 Three Type Zones for Sources

Sources are the **inverse** of destinations in the Three Type Zones model:

| Zone      | Source semantics                                                                      |
| --------- | ------------------------------------------------------------------------------------- |
| `trigger` | How to simulate the invocation (HTTP method, DOM event type, cloud event)             |
| `in`      | External trigger content - HTTP request, DOM HTML, SDK payload (NOT a walkerOS event) |
| `out`     | The walkerOS event(s) the source should emit (`WalkerOS.Event`)                       |

Where a destination does `WalkerOS.Event → vendor output`, a source does
`external content → WalkerOS.Event`. Read
[using-step-examples](../walkeros-using-step-examples/SKILL.md) before authoring
entries.

### 3.4 Typing Rules (strict)

**No `any`.** Every example value must be explicitly typed.

- **`trigger`** uses the local source trigger type or a platform-native type
  (e.g. `'load' | 'click'` for DOM, HTTP method strings for server sources).
- **`in`** uses the **vendor / platform SDK types** imported from the official
  package whenever available (Express `Request`, Fetch `Request`, API Gateway
  `APIGatewayProxyEvent`, Lambda `Context`, GCP `CloudEvent`, etc.). Do not
  invent local request types when the platform publishes them.
- **`out`** uses `WalkerOS.Event` (or `DeepPartialEvent` for fragments).
- **Step entries** are typed `Flow.StepExample` from `@walkeros/core`.
- **Mock env** is typed against the source's local `Env` type from `../types`.
- `createTrigger` is typed as `Trigger.CreateFn<Content, Result>` - the
  `Content` and `Result` generics come from the source's own types module.

### 3.5 Code Template - `examples/step.ts`

```typescript
import type { Flow } from '@walkeros/core';

// One step example per captured trigger / input shape.
// `trigger` tells createTrigger how to simulate the invocation.
// `in` is the platform-specific content (HTTP request, DOM HTML, SDK payload) -
//      typed against the platform SDK's published types where available.
// `out` is the walkerOS event the source is expected to emit.

// Set `title` + `description` for public examples; mark test-only fixtures
// with `public: false`. See
// [walkeros-using-step-examples](../walkeros-using-step-examples/SKILL.md).

export const pageView: Flow.StepExample = {
  trigger: {
    type: 'load',
    options: {
      url: 'https://example.com/docs',
      title: 'Documentation',
    },
  },
  in: '', // no external content - DOM-driven trigger
  out: {
    name: 'page view',
    data: { domain: 'example.com', title: 'Documentation', id: '/docs' },
    entity: 'page',
    action: 'view',
    trigger: 'load',
    source: {
      type: 'browser',
      platform: 'web',
      url: 'https://example.com/docs',
    },
  },
};

// Server example: HTTP POST carrying a walker event payload.
export const orderComplete: Flow.StepExample = {
  trigger: { type: 'POST' },
  in: {
    method: 'POST',
    path: '/collect',
    body: { name: 'order complete', data: { id: 'ORD-123', value: 149.97 } },
  },
  out: {
    name: 'order complete',
    data: { id: 'ORD-123', value: 149.97 },
    entity: 'order',
    action: 'complete',
  },
};
```

### 3.6 `examples/index.ts` (barrel)

```typescript
export * as env from './env'; // omit if the source has no env.ts
export * as step from './step';
export { createTrigger, trigger } from './trigger';
```

### 3.7 `examples/trigger.ts` - `createTrigger`

Every source exports a `createTrigger` following the unified
`Trigger.CreateFn<Content, Result>` interface. It simulates real-world
invocations from the outside - no source instance access, full blackbox.

```typescript
import type { Trigger } from '@walkeros/core';
import { startFlow } from '@walkeros/collector';

export const createTrigger: Trigger.CreateFn<Content, Result> = async (
  config,
) => {
  let flow: Trigger.FlowHandle | undefined;

  const trigger: Trigger.Fn<Content, Result> =
    (type?: string) => async (content) => {
      if (!flow) {
        const result = await startFlow(config);
        flow = { collector: result.collector, elb: result.elb };
      }
      // Package-specific: make real HTTP request, inject DOM, dispatch SDK call.
      // Return the Result type declared by this source.
      return /* ... */;
    };

  return {
    get flow() {
      return flow;
    },
    trigger,
  };
};
```

Reference implementations:

- **Browser:** `packages/web/sources/browser/src/examples/trigger.ts` - DOM
  injection + native event dispatch
- **Session:** `packages/web/sources/session/src/examples/trigger.ts` - no
  env.ts, trigger drives collector directly
- **Express:** `packages/server/sources/express/src/examples/trigger.ts` - real
  HTTP `fetch()` to running server
- **CMP (Usercentrics):**
  `packages/web/sources/cmps/usercentrics/src/examples/trigger.ts` - dispatches
  CMP events, asserts on collector consent state
- **Fetch (function handler):**
  `packages/server/sources/fetch/src/examples/trigger.ts` - accesses source
  instance via `collector.sources`, calls `source.push()` with platform-native
  `Request`
- **AWS Lambda:** `packages/server/sources/aws/src/lambda/examples/trigger.ts` -
  constructs API Gateway event + Lambda context
- **GCP CloudFunction:**
  `packages/server/sources/gcp/src/cloudfunction/examples/trigger.ts` -
  synthesizes mock req/res (matching GCP Functions Framework)

### 3.8 Test Fixture Contract (hard rule)

The examples authored here **are** the Phase 8 test fixtures. No parallel
fixtures allowed.

- `src/index.test.ts` **MUST** iterate examples via
  `it.each(Object.entries(examples.step))`.
- Tests **must NOT** contain hardcoded trigger payloads, HTTP requests, DOM
  HTML, or expected events.
- If a test needs a value that is not in `examples.step`, **add it to `step.ts`
  first**, then consume it from the test.
- Tests invoke `examples.createTrigger(config)` and dispatch each example's
  `trigger.type` + `in` content, asserting the collector receives `out`.

See the canonical source tests under
`packages/web/sources/browser/src/index.test.ts` and
`packages/server/sources/express/src/index.test.ts`.

### 3.9 Export via `dev.ts`

```typescript
export * as schemas from './schemas';
export * as examples from './examples';
```

### Phase 3 Acceptance Checklist

- [ ] `src/examples/step.ts` - one `Flow.StepExample` per captured trigger /
      input shape, typed `trigger` / `in` / `out`
- [ ] `src/examples/trigger.ts` - exports `createTrigger` typed as
      `Trigger.CreateFn<Content, Result>`
- [ ] `src/examples/index.ts` - barrel exports `step`, `createTrigger`, and
      `env` (when present)
- [ ] `src/examples/env.ts` - included whenever the source touches platform
      globals or injected deps; typed against local `Env`; no real network
- [ ] No standalone `inputs.ts`, `outputs.ts`, `requests.ts`, or `mapping.ts`
      files
- [ ] All platform / SDK types imported from the official package - no `any`, no
      reinvented request / response shapes
- [ ] `src/index.test.ts` iterates `examples.step` via
      `it.each(Object.entries(...))`
- [ ] Tests contain zero hardcoded payloads, requests, or expected events -
      everything flows from `examples.step`
- [ ] Edge cases included (minimal input, invalid input)
- [ ] `npm run build` passes - examples compile against published types
- [ ] Each example traces: `trigger` + `in` → source push → matches `out`

---

## Phase 4: Define Mapping

**Goal:** Document transformation from input format to walkerOS events.

Mapping lives **inside** each `Flow.StepExample` entry in `step.ts` - no
separate `mapping.ts` file. Sources typically carry the mapping either in the
source's own `settings` (see `dataLayer` for an example) or inline via the
`trigger` → `in` → `out` relationship: the `in` content is the raw platform
payload; the `out` is the walkerOS event after the source's transformation.

### Verify Mapping Logic

For each entry in `step.ts`, trace:

```text
Input: examples.step.pageView.trigger + examples.step.pageView.in
  ↓ createTrigger dispatches the trigger
  ↓ Source receives platform content, runs its transformation
  ↓ Source calls env.push / collector.push
Output: Should match examples.step.pageView.out (a WalkerOS.Event)
```

### Gate: Mapping Verified

- [ ] Step examples cover the main input event types
- [ ] Each example name transforms to correct walkerOS event name
- [ ] Each example traces correctly from `(trigger, in)` to `out`

---

## Phase 5: Scaffold

**Template sources:**

- Web: `packages/web/sources/dataLayer/`
- Server: `packages/server/sources/fetch/` (simplest pattern)

```bash
cp -r packages/server/sources/fetch packages/server/sources/[name]
cd packages/server/sources/[name]

# Update package.json: name, description, repository.directory
```

**Directory structure:**

```
packages/server/sources/[name]/
├── src/
│   ├── index.ts           # Main export
│   ├── index.test.ts      # Tests against examples
│   ├── dev.ts             # Exports schemas and examples
│   ├── examples/
│   ├── schemas/
│   └── types/
├── package.json
├── tsconfig.json
├── tsup.config.ts
├── jest.config.mjs
└── README.md
```

### Transformer Chain Integration

Sources can wire to transformer chains via `next` in the init config:

```typescript
sources: {
  mySource: {
    code: sourceMySource,
    config: { settings: { /* ... */ } },
    next: 'validate'  // Events go through validator before collector
  }
}
```

---

## Phase 6: walkerOS.json Convention

Every walkerOS package ships a `walkerOS.json` file for CDN-based schema
discovery.

### Add `walkerOS` field to package.json

```json
{
  "walkerOS": { "type": "source", "platform": "web" },
  "keywords": ["walkerOS", "walkerOS-source", ...]
}
```

### Use `buildDev()` in tsup.config.ts

Replace `buildModules({ entry: ['src/dev.ts'] })` with `buildDev()`:

```typescript
import { buildDev } from '@walkeros/config/tsup';
// In defineConfig array:
buildDev(),
```

This auto-generates `dist/walkerOS.json` from your Zod schemas at build time.

### Hints (Optional)

If your source has capabilities, behaviors, or troubleshooting patterns not
obvious from schemas alone, add hints. See `walkeros-writing-documentation`
skill for full guidelines.

Create `src/hints.ts`:

```typescript
import type { Hint } from '@walkeros/core';

export const hints: Hint.Hints = {
  'capture-timing': {
    text: 'Describes when events are captured. See settings schema for options.',
    code: [{ lang: 'json', code: '{ "settings": { ... } }' }],
  },
};
```

Export from `src/dev.ts`:

```typescript
export * as schemas from './schemas';
export * as examples from './examples';
export { hints } from './hints';
```

Guidelines:

- Expand awareness - describe capabilities ("supports X, Y, Z"), don't prescribe
  one path
- Reference schemas and examples, don't duplicate them
- Verify every claim against actual implementation before publishing
- Key naming: kebab-case, group with prefixes (auth-\*, capture-\*,
  troubleshoot-\*)
- Most sources don't need hints - schemas and examples cover the common case

### Gate: Convention Met

- [ ] `walkerOS` field in package.json with type and platform
- [ ] `buildDev()` in tsup.config.ts
- [ ] Build generates `dist/walkerOS.json`
- [ ] Keywords include `walkerOS` and `walkerOS-source`

### Runtime-only npm dependencies

If your package wraps a third-party npm dep that **cannot be ESM-bundled** (uses
`__dirname`, ships a `.node` binary, etc.), declare it under
`walkerOS.bundle.external` in your `package.json`. See
[walkeros-using-cli → Bundle externals](../walkeros-using-cli/SKILL.md#bundle-externals-per-package-walkerosbundleexternal)
for the complete contract.

---

## Phase 7: Implement

**Now write code to produce the outputs defined in Phase 3.**

### Template Files

| File               | Purpose                | Template                                    |
| ------------------ | ---------------------- | ------------------------------------------- |
| `types/index.ts`   | Type definitions       | [types.ts](./templates/server/types.ts)     |
| `schemas/index.ts` | Zod validation schemas | [schemas.ts](./templates/server/schemas.ts) |
| `index.ts`         | Main source            | [index.ts](./templates/server/index.ts)     |

### Key Patterns

1. **Context destructuring**: Extract `config`, `env`, `logger`, `id` from
   context
2. **Schema validation**: Use Zod schemas to validate settings and provide
   defaults
3. **Forward to collector**: Call `env.push()` to send events to the collector
4. **Platform deps via env**: All platform dependencies (window, document,
   express, cors) must go through `env` with fallback to globals/imports:
   `env.express ?? express`. This enables testing without mocking globals.
5. **Failures are reported, not logged by the source**: a source that cannot
   start throws a `StepError` (`code: 'config'`) from its factory or `init`; it
   never returns early and never returns `false`. Input the collector rejects
   comes back as `invalid` in the push result and is counted as `rejected` by
   the collector. A failure on the source's own side that the push result does
   not carry (for example an `elbLayer` entry that fails) goes through
   `reportCause` from `@walkeros/core` (step `source.<id>`, named by its event,
   never its payload). Use `logger` for `debug` traces of routine operations.
6. **Tell the producer the truth**: an HTTP source builds its answer from the
   push result with `pushResultToOutcome` (or `acknowledge(result)`): `ack`
   answers 200 with today's body, `retry` (a retryable failure) answers 500, or
   207 for the failed events of a batch, `reject` (invalid input) answers 400. A
   failure a resend cannot fix is acknowledged, because redelivery would loop. A
   queue consumer acks on `ack` and `reject` and nacks only on `retry`. The body
   never contains a destination id or a step's error text.
7. **Return Source.Instance**: Return `{ type, config, push }` object
8. **Optional `destroy` method**: Implement if the source holds resources (HTTP
   servers, timers, connections) that need cleanup on shutdown

### Gate: Implementation Compiles

- [ ] `npm run build` passes
- [ ] `npm run verify:touched -- <source-name>` passes (L1: typecheck + lint +
      test)

---

## Adding setup (optional)

A source package can implement an optional `setup()` function to provision
external resources idempotently: Pub/Sub subscriptions, webhook registrations on
upstream platforms, queue declarations, SQS queues, polling cursors, inbound API
keys. Setup runs only when an operator explicitly types
`walkeros setup source.<name>`. The runtime never auto-invokes it from `init()`,
push, or `destroy()`.

The framework provides the slot, the CLI command, and a `resolveSetup` helper.
The package owns: what setup means, idempotency, error handling, return value.

For background on how setup fits the source lifecycle, see
[understanding-sources](../walkeros-understanding-sources/SKILL.md#setup-optional).

### Types

```typescript
// types/index.ts
import type { CoreSource } from '@walkeros/core';

export interface Settings {
  /* runtime push settings */
}
export interface InitSettings {
  /* one-time init settings */
}
export interface Mapping {
  /* event extraction mapping */
}
export interface Env {
  /* injected platform deps (SDK clients, request handlers, etc.) */
}

// The package's own setup options interface.
// Becomes the U slot of Types; surfaces as `config.setup: boolean | Setup` for users.
export interface Setup {
  // package-specific provisioning options
  // e.g. for Pub/Sub source: subscriptionName, ackDeadlineSeconds, filter
  subscriptionName?: string;
  ackDeadlineSeconds?: number;
}

export type Types = CoreSource.Types<
  Settings,
  Mapping,
  Env,
  InitSettings,
  Setup
>;
```

### Implementation

```typescript
// setup.ts
import type { CoreSource, SetupFn } from '@walkeros/core';
import { resolveSetup } from '@walkeros/core';
import type { Setup, Types } from './types';

const DEFAULT_SETUP: Setup = {
  ackDeadlineSeconds: 60,
};

export const setup: SetupFn<
  CoreSource.Config<Types>,
  CoreSource.Env<Types>
> = async ({ config, env, logger }) => {
  const options = resolveSetup(config.setup, DEFAULT_SETUP);
  if (!options) return; // config.setup is false or unset

  // Package-specific provisioning, idempotent.
  // Returning a structured object (e.g. { subscriptionCreated: true })
  // makes that data available to operators via `walkeros setup ... | jq`.
};
```

Wire it in your default export:

```typescript
// index.ts
import { setup } from './setup';

export default {
  type: 'my-source',
  init: /* ... */,
  setup,
};
```

### When to implement

Implement `setup()` when your source needs first-time provisioning of upstream
resources before events can be received: Pub/Sub subscriptions bound to a topic,
webhook registrations on upstream SaaS platforms (Stripe, GitHub, Shopify), SQS
queue declarations, message broker bindings, polling cursors. Skip it when your
source only listens on an HTTP port the runtime already owns or intercepts data
already present (DOM events, dataLayer pushes).

### Contract

- Triggered only by `walkeros setup <kind>.<name>`. Never by runtime push, init,
  or destroy.
- **Idempotency is your responsibility.** Re-running setup against a fully
  provisioned environment must be a safe no-op. Use try-create-catch-409 on REST
  APIs, `IF NOT EXISTS` on SQL, native idempotent operations where available.
  The framework does not retry, track state, or detect drift.
- Return structured data from `setup()` when useful for operator scripting. The
  CLI emits non-undefined return values as JSON to stdout.
- For packages where `setup: true` (boolean form) is meaningless because
  mandatory fields have no safe defaults (e.g., GitHub webhook `webhookUrl`,
  Pub/Sub source `topicName`), reject the boolean form with a clear runtime
  error listing required fields:

```typescript
if (config.setup === true) {
  throw new Error(
    'github-webhook source setup requires explicit options: ' +
      '{ webhookUrl, repo, events }. There is no safe default.',
  );
}
```

---

## Phase 8: Test Against Examples

> Tests verify implementation against the examples from Phase 3. If examples are
> incomplete, tests will be incomplete.
>
> See [testing-strategy](../walkeros-testing-strategy/SKILL.md) for the shared
> env / dev-examples conventions this phase depends on.

**Verify implementation produces expected outputs.**

### Test Template

Use the test template: [index.test.ts](./templates/server/index.test.ts).
Canonical references:

- `packages/web/sources/browser/src/index.test.ts`
- `packages/server/sources/express/src/index.test.ts`

### Key Test Patterns

1. **`it.each(Object.entries(examples.step))` is mandatory** - one iteration per
   step example. Do not write per-feature tests with hand-rolled payloads.
2. **Drive via `createTrigger`** - construct the trigger with `startFlow`
   config, then dispatch each example's `trigger.type` + `in` content.
3. **Use `createSourceContext()` helper** for any direct context construction.
4. **Zero hardcoded payloads** - every trigger type, request body, DOM HTML, and
   expected event comes from `examples.step` or `examples.env`. If you need
   something new, add it to examples first.
5. **Test error paths** - verify graceful error handling for invalid input (add
   an error example to `examples.step` if needed), and that a push that fails
   for a retryable reason is answered with a retry, a deterministic one with an
   ack.
6. **Conformance** - run the source with
   `conformance('source', source, options)` from `@walkeros/collector/dev`
   (profile `source`, `createTrigger` for how to feed it); it runs on a real
   collector. A web source may pass `page` from `@walkeros/web-core/dev`; the
   kit requires it only for the browser profiles `web-sdk` and `vendor-global`.

### Gate: Tests Pass

- [ ] `npm run verify:touched -- <source-name>` passes (L1)
- [ ] Tests iterate via `it.each(Object.entries(examples.step))`
- [ ] Tests contain no hardcoded payloads, requests, or expected events
- [ ] Every assertion reads from `examples.step[...].out`
- [ ] Invalid input handled gracefully (no crashes)

---

## Phase 9: Document

Follow the [writing-documentation](../walkeros-writing-documentation/SKILL.md)
skill for:

- README structure and templates
- Example validation against `apps/quickstart/`
- Quality checklist before publishing

Key requirements for source documentation:

- [ ] Input format table documenting expected fields
- [ ] Event name mapping table (source format → walkerOS format)
- [ ] Configuration options table
- [ ] Working code example with imports
- [ ] Installation instructions

---

## Validation Checklist

Beyond
[understanding-development](../walkeros-understanding-development/SKILL.md)
requirements (build, test, lint, no `any`):

- [ ] `dev.ts` exports `schemas` (with `settings`; `z.object({})` when the step
      has none) and `examples`
- [ ] Multi-export package (two or more `walkerOS.exports`): `exportExamples`
      and `exportSchemas` in `dev.ts` list exactly those exports, the default
      included, and every `exportSchemas` entry has `settings`; the build fails
      otherwise (see
      [using-step-examples](../walkeros-using-step-examples/SKILL.md))
- [ ] Examples include edge cases (minimal, invalid input)
- [ ] Invalid input returns gracefully (no crashes, clear error)
- [ ] Tests use examples for assertions (not hardcoded values)
- [ ] `walkerOS.json` generated at build time
- [ ] `walkerOS` field in package.json

---

## Reference Files

| What            | Where                               |
| --------------- | ----------------------------------- |
| Web template    | `packages/web/sources/dataLayer/`   |
| Server template | `packages/server/sources/fetch/`    |
| Source types    | `packages/core/src/types/source.ts` |
| Event creation  | `packages/core/src/lib/event.ts`    |

## Related Skills

- [understanding-sources](../walkeros-understanding-sources/SKILL.md) - Source
  interface and push pattern
- [understanding-events](../walkeros-understanding-events/SKILL.md) - Event
  structure
- [using-step-examples](../walkeros-using-step-examples/SKILL.md) -
  Authoritative `Flow.StepExample` + `createTrigger` pattern, Three Type Zones
- [testing-strategy](../walkeros-testing-strategy/SKILL.md) - Testing with env
  mocking and dev-examples-as-fixtures conventions
- [writing-documentation](../walkeros-writing-documentation/SKILL.md) -
  Documentation standards
Files in this package (12)
  • SKILL.md 35 KB
  • TESTING.md 2 KB
  • examples/index.ts 1 KB
  • examples/inputs.ts 1 KB
  • examples/mapping.ts 1 KB
  • examples/outputs.ts 1 KB
  • examples/requests.ts 1 KB
  • templates/server/dev.ts 1 KB
  • templates/server/index.test.ts 4 KB
  • templates/server/index.ts 4 KB
  • templates/server/schemas.ts 1 KB
  • templates/server/types.ts 1 KB

Versions

The database has no other skills named walkeros-create-source.

Database snapshot: 13.11.2026 · package version 6b6c17d2d007 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100walkeros-understanding-sources
    Use when working with walkerOS sources, understanding event capture, or learning about the push interface. Covers browser, dataLayer, and server source patterns.
  • 91walkeros-create-destination
    Use when creating a new walkerOS destination to send events to a vendor or API (GA4/gtag, Meta/Facebook Pixel, Mixpanel, Amplitude, a custom HTTP API, Measurement Protocol), web or server-side. Example-driven workflow: research the vendor SDK and define step examples before…
  • 100server-sent-events-tests
    Test Server-Sent Events (SSE) flows, one-way server-to-client push only (not bidirectional, use websocket-tests for client-to-server messaging): `EventSource` API on the browser side (`onmessage`, `onerror`, `readyState` 0/1/2), event stream format (`data:`, `event:`, `id:`,…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.