Skip to content
Skills Library
AI & agents · Analysis

autocontext

/autocontext

Iterative strategy generation and evaluation system. Use when the user wants to evaluate agent output quality, run improvement loops, queue tasks for background evaluation, check run status, inspect runtime artifacts and session branch lineage, or discover available scenarios. Provides LLM-based judging with rubric-driven scoring.

Repository
autocontext ↗
Licence
Apache-2.0
Last repository change
26.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 26.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Iterative strategy generation and evaluation system. Use when the user wants to evaluate agent output quality, run improvement loops, queue tasks for background evaluation, check run status, inspect runtime artifacts and session branch lineage, or discover available scenarios. Provides LLM-based judging with rubric-driven scoring.

Original author description

Iterative strategy generation and evaluation system. Use when the user wants to evaluate agent output quality, run improvement loops, queue tasks for background evaluation, check run status, inspect runtime artifacts and session branch lineage, or discover available scenarios. Provides LLM-based judging with rubric-driven scoring.

How it works

  1. Available Tools
  2. Quick Start
  3. Configuration
  4. CLI Companion
Original SKILL.md headings · 100 lines, 367 words
SKILL.md preview
---
name: autocontext
description: >
  Iterative strategy generation and evaluation system. Use when the user wants
  to evaluate agent output quality, run improvement loops, queue tasks for
  background evaluation, check run status, inspect runtime artifacts and
  session branch lineage, or discover available scenarios. Provides LLM-based
  judging with rubric-driven scoring.
allowed-tools: autocontext_judge autocontext_improve autocontext_status autocontext_scenarios autocontext_queue autocontext_runtime_snapshot
---

# autocontext

autocontext is an iterative strategy generation and evaluation system that uses
LLM-based judging to score and improve agent outputs.

## Available Tools

- **autocontext_judge** — Evaluate agent output against a rubric. Returns a 0–1
  score with reasoning and per-dimension breakdowns.
- **autocontext_improve** — Run a multi-round improvement loop. The agent output
  is judged, revised based on feedback, and re-evaluated until the quality
  threshold is met or max rounds are exhausted.
- **autocontext_queue** — Enqueue a task for background evaluation by the task
  runner daemon.
- **autocontext_status** — Check the status of runs and queued tasks.
- **autocontext_scenarios** — List available evaluation scenarios and their
  families.
- **autocontext_runtime_snapshot** — Inspect run artifacts, package provenance,
  branchable session lineage, and recent event-stream entries.

## Quick Start

### 1. Evaluate output quality

Use `autocontext_judge` with a task prompt, the agent's output, and a rubric:

```
autocontext_judge(
  task_prompt="Write a Python function to parse CSV files",
  agent_output="def parse_csv(path): ...",
  rubric="Correctness, error handling, edge cases, documentation"
)
```

### 2. Improve output iteratively

Use `autocontext_improve` to automatically revise output through
judge-guided feedback loops:

```
autocontext_improve(
  task_prompt="Write a Python function to parse CSV files",
  initial_output="def parse_csv(path): ...",
  rubric="Correctness, error handling, edge cases, documentation",
  max_rounds=5,
  quality_threshold=0.85
)
```

### 3. Queue background tasks

Use `autocontext_queue` with a scenario name to enqueue evaluation tasks
for asynchronous processing:

```
autocontext_queue(spec_name="my_scenario")
```

Check results later with `autocontext_status`.

For deeper context, use `autocontext_runtime_snapshot` with the run ID. Add
`session_id` when you need the active branch path before continuing work:

```
autocontext_runtime_snapshot(run_id="run_123", session_id="sess_123")
```

### 4. Discover scenarios

Use `autocontext_scenarios` to see what evaluation scenarios are available:

```
autocontext_scenarios()
autocontext_scenarios(family="agent_task")
```

## Configuration

The extension auto-detects configuration from these sources:

1. **Project config** — `.autoctx.json` in the working directory (created via `autoctx init`)
2. **Environment variables:**
   - `AUTOCONTEXT_AGENT_PROVIDER` or `AUTOCONTEXT_PROVIDER` — Provider type
   - `AUTOCONTEXT_AGENT_API_KEY` or `AUTOCONTEXT_API_KEY` — Provider API key
   - `AUTOCONTEXT_AGENT_DEFAULT_MODEL` or `AUTOCONTEXT_MODEL` — Model override
   - `AUTOCONTEXT_DB_PATH` — SQLite database path override
3. **Pi provider** — Falls back to Pi's configured LLM provider

## CLI Companion

For standalone usage outside Pi, install the `autoctx` CLI:

```bash
npm install -g autoctx
autoctx init
autoctx solve "your problem" --iterations 5
autoctx simulate --description "your simulation" --runs 3
```

Versions

RepositoryScoreLicence
This page · greyhaven-ai/autocontext100Apache-2.0
greyhaven-ai/autocontext100Apache-2.0

Database snapshot: 13.11.2026 · package version 49d43aa4d2b0 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100autocontext-creator
    Use when an agent needs to CREATE knowledge with Autocontext - run a scenario or plain-language task through the improvement loop, judge or improve a single output, and inspect what the run produced. Host-agnostic; requires only the autoctx CLI.
  • 100agentic-evaluation-framework
    This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs pairwise", or "measure agent quality".
  • 98skill-101
    Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when implementing self-critique and reflection loops, building evaluator-optimizer pipelines for quality-critical generation, creating test-driven code refinement workflows, designing…
  • 100autocontext-consumer
    Use when an agent needs to USE knowledge Autocontext already produced - find which scenarios have knowledge, read the playbook and lessons for one, understand the on-disk file and folder layout, and move knowledge between checkouts. Host-agnostic; requires only the autoctx CLI and the filesystem.
  • 100agent-evaluation
    This skill should be used when the user asks to "evaluate agent performance", "build test framework", "measure agent quality", "create evaluation rubrics", "implement LLM-as-judge", "compare model outputs", "mitigate evaluation bias", or mentions multi-dimensional evaluation,…
  • 100eval
    Evaluate everything the PA agent manages — tasks, skills, PA network health, billing, calendar connections, and memory quality. Use when: owner asks for an evaluation, wants to know what's working and what isn't, or requests a performance report. Combines supervisor status with quality scoring.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.