Przejdź do treści
Biblioteka skilli
AI i agenci · Automatyzacja

automations

/automations

Konfiguruje automatyzacje, alerty i raporty LangWatch dostarczane na Slack, e-mail, webhook, do zbioru danych lub kolejki adnotacji, na przykład alert przy skoku błędów lub mail przy negatywnej ocenie trace.

Repozytorium
langwatch ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
27.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
1
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
1
Przestarzałość

Brak sygnałów. Repozytorium zmienione 27.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co robi i kiedy

Konfiguruje automatyzacje, alerty i raporty LangWatch dostarczane na Slack, e-mail, webhook, do zbioru danych lub kolejki adnotacji, na przykład alert przy skoku błędów lub mail przy negatywnej ocenie trace.

Wyzwalacze z opisu

alert #support-alerts when errors spikeemail me when a trace gets a thumbs-down
Oryginalny opis (EN)

Set up LangWatch automations, alerts and reports that deliver to Slack, email, a webhook, a dataset or an annotation queue. Use when the user wants to be told when something happens ("alert #support-alerts when errors spike", "email me when a trace gets a thumbs-down"), wants matching traces collected automatically, wants a scheduled report, or wants to pause, resume, test or review an existing automation.

Jak działa

  1. Prerequisites
  2. Inspect First
  3. Secrets Never Travel Through the Conversation
  4. Deliver to Slack
  5. Deliver by Email, Webhook, Dataset or Annotation Queue
  6. Write the Condition
  7. Pause, Resume, Test and Review
  8. Verify
  9. Common Mistakes
Sekcje z nagłówków SKILL.md, w oryginale · 96 linii, 1 103 słów
Podgląd SKILL.md
---
name: automations
user-prompt: "Alert me on Slack when my agent's error rate goes up"
description: Set up LangWatch automations, alerts and reports that deliver to Slack, email, a webhook, a dataset or an annotation queue. Use when the user wants to be told when something happens ("alert #support-alerts when errors spike", "email me when a trace gets a thumbs-down"), wants matching traces collected automatically, wants a scheduled report, or wants to pause, resume, test or review an existing automation.
license: MIT
compatibility: Requires the LangWatch CLI. Works with Claude Code and similar AI assistants.
metadata:
  category: skill
---

# Set Up Automations, Alerts and Reports

An automation watches something in a LangWatch project and acts when it happens. There are three kinds, all created with `langwatch trigger create`:

- **Automation**: acts on each trace that matches a trace condition (`--filter-query`).
- **Alert**: watches one series of a custom graph and fires when it crosses a threshold (`--custom-graph-id`, `--graph-alert`, `--alert-type`).
- **Report**: renders a dashboard, a graph or a table of traces on a schedule (`--report`).

Each one delivers through one action: `SEND_SLACK_MESSAGE`, `SEND_EMAIL`, `SEND_WEBHOOK`, `ADD_TO_DATASET` or `ADD_TO_ANNOTATION_QUEUE`.

## Prerequisites

Read the reference once before the first create, and the command's own help. Only read the docs pages this skill names; never guess a docs path:

```bash
langwatch docs api-reference/triggers/overview
langwatch trigger create --help
```

## Inspect First

List what already exists before creating anything, so you extend an automation instead of duplicating it:

```bash
langwatch trigger list --format json
```

A Slack automation already in the list carries the id of the Slack connection it posts through in `actionParams.slackIntegrationId`: that is a connection this project can use.

## Secrets Never Travel Through the Conversation

Slack bot tokens, Slack webhook URLs, webhook signing secrets and webhook header values are credentials. Never ask for one, never accept one pasted into the conversation, never pass `--slack-webhook`, and never repeat a stored value back: reads return them as `[redacted]`, and that is all you ever show. A user who offers one is told, in one line, to add it as a Slack connection on the Automations page instead.

## Deliver to Slack

A Slack automation posts through a **Slack connection** the project can already use, named with `--slack-connection <id>`. A bot connection also needs `--slack-channel`, which takes a channel id (`C0123…`) or its name (`#support-alerts`). An incoming-webhook connection posts to its own channel and needs nothing else.

List the connections this project can use first, by name, kind (`bot` or `webhook`) and scope; the list never carries a secret:

```bash
langwatch slack-connection list --format json
```

1. When the user named a connection, take the `id` of the one whose `name` matches; when an existing Slack automation shows the one to reuse, take that. Create with it.
2. Otherwise, including when the list is empty, create the automation with `--action SEND_SLACK_MESSAGE` and the channel, and no connection. Do not ask a question first, and never offer to take a webhook URL or token as an option. The refusal is expected: the user's panel shows the Slack connections this project can use, each with a button that answers you with its name and id, and, when there is none, a button that opens the Slack connection form on the Automations page. Say in one line that they should pick or add a connection, and stop the turn.
3. When the answer names a connection id, run the same create again with `--slack-connection <id>`.

Never create, edit, widen or delete a Slack connection yourself: connections are added and scoped by the user on the Automations page, where a new one is scoped to this project unless they choose otherwise.

## Deliver by Email, Webhook, Dataset or Annotation Queue

- Email: `--action SEND_EMAIL --action-params '{"members":["me@example.com"]}'`. "Email me" means the user's own address: the turn names it ("You are talking to …"). Use that; ask with the `question` tool only when no address is given. Never run `langwatch whoami` for it: the worker is not device-logged-in.
- Webhook: `--action SEND_WEBHOOK --action-params '{"url":"https://…"}'`, only for an https address the user gave for this purpose. Never add headers or a signing secret they would have to paste.
- Dataset: `--action ADD_TO_DATASET` with `datasetId` and `datasetMapping`; read `langwatch docs datasets/automatically-from-traces` for the mapping.
- Annotation queue: `--action ADD_TO_ANNOTATION_QUEUE` with `annotators`, each `{"id":"user-<userId>","name":"…"}` or `{"id":"queue-<queueId>","name":"…"}`.

## Write the Condition

For a trace automation, write `--filter-query` in the same syntax the Traces view uses, for example `status:error`; `langwatch trace fields` lists every field. Check it selects what the user means before saving:

```bash
langwatch trace search --filter '<query>' --format json
```

An evaluation result is keyed by the MONITOR that ran it, not the evaluator: take the id from `langwatch monitor list --format json`.

A thumbs-down from an end user is a `thumbs_up_down` event, not an annotation: `--filters '{"events.metrics.value":{"thumbs_up_down":{"vote":["-1"]}}}'`. A reviewer's thumbs-down is an annotation; when the user could mean either, ask which. Never guess a filter key: a refused key means read the reference above, not try another spelling.

For an alert, find the graph that plots the metric with `langwatch graph list --format json`; when none does, create one with `langwatch graph create`. `--graph-alert` names the series as `<index>/<metric>/<aggregation>` (for example `0/performance.completion_time/p95`), a comparison `gt`, `gte`, `lt`, `lte` or `eq`, a threshold in the metric's own unit and a window in minutes (1, 5, 15, 30, 60 or 1440). A percentage written as "5%" is the threshold `5` on a percentage series and `0.05` on a rate series: read the graph to tell which.

Analytics has no error-rate metric: never pass `--metric error-rate`. For errors, plot a trace count (`metadata.trace_id` / `cardinality`) on a series filtered to errored traces (`"filters":{"traces.error":["true"]}`) and alert on the count per window; tell the user the threshold is a count, not a percentage.

## Pause, Resume, Test and Review

```bash
langwatch trigger disable <id> --format json   # pause
langwatch trigger enable <id> --format json    # resume
langwatch trigger test-fire <id> --format json # send a test to the saved destination
langwatch trigger fires <id> --format json     # what it has done, newest first
```

"Pause that alert" means the automation you created or showed earlier in the conversation: reuse its id. Prefer pausing to deleting. Deleting an automation is the user's to do on the Automations page.

## Verify

A write only succeeded if its result names what it wrote. The create or update result is the automation as stored: confirm from it that the kind, condition, destination and active state are the ones the user asked for. Run `langwatch trigger get <id> --format json` only when that result did not name them; the panel already shows the created automation, so a read-back just repeats the card.

## Common Mistakes

- Do not take or pass a Slack token, a Slack webhook URL or a signing secret, and do not use `--slack-webhook`.
- Do not create or scope a Slack connection: send the user to add one.
- Do not key an evaluation condition by the evaluator id; it is the monitor id.
- Do not create a duplicate of an automation the list already shows.
- Do not delete an automation to stop it: pause it.

Wersje

RepozytoriumWynikLicencja
Ta strona · langwatch/langwatch100Apache-2.0
builderio/agent-native100MIT
cardstack/boxel82MIT

Identyczna treść występuje jeszcze w 1 innych folderach na GitHubie. Tu pokazujemy jedną, najlepiej ocenioną kopię.

Migawka bazy z 13.11.2026 · wersja paczki 2bd6c4c2ace1 (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100sentry-create-alert
    Tworzy alerty Sentry przez API silnika workflow: powiadomienia, alerty o priorytecie issue i automatyzacje z akcjami e-mail, Slack, PagerDuty i Discord.
  • 95sentry-create-alert
    Tworzy alerty w Sentry przez API silnika workflow: powiadomienia e-mail, Slack, PagerDuty i Discord.
  • 95langwatch
    Odczytuje ślady LLM z LangWatch przez CLI: co agent lub model zrobił w produkcji, także dla konkretnej sesji lub identyfikatora.
  • 100lens-report
    Buduje potok raportowania: raporty cykliczne z zapytaniami SQL, dostarczanie przez Slack lub e-mail, alerty progowe i porównania historyczne.
  • 91openhands-automation
    Tworzy automatyzacje w OpenHands Cloud: zadania harmonogramowane, cron, integracje przez webhook i automatyzacje wyzwalane zdarzeniami.
  • 91openhands-automation
    Tworzy automatyzacje w OpenHands Cloud: zadania cron, integracje webhook i zdarzenia uruchamiające.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.