init
/init
Initialize a new agentic-usability benchmark pipeline project. Use when setting up a new SDK benchmark, creating a config.json, or starting a new evaluation project.
- Author
- pspdfkit-labs ↗
- Repository
- agentic-usability ↗
- Licence
- Apache-2.0
- Last repository change
- 11.06.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 6
No signals detected. Repository changed on 11.06.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Initialize a new agentic-usability benchmark pipeline project. Use when setting up a new SDK benchmark, creating a config.json, or starting a new evaluation project.
Arguments
/init [project-directory]
Original author description
Initialize a new agentic-usability benchmark pipeline project. Use when setting up a new SDK benchmark, creating a config.json, or starting a new evaluation project.
How it works
- Option 1: Interactive Wizard
- Option 2: Direct Config Creation
SKILL.md preview
---
name: init
description: Initialize a new agentic-usability benchmark pipeline project. Use when setting up a new SDK benchmark, creating a config.json, or starting a new evaluation project.
argument-hint: "[project-directory]"
disable-model-invocation: true
allowed-tools: Bash(agentic-usability *) Write Read Glob
---
# Initialize Pipeline Project
Set up a new agentic-usability benchmark pipeline in the given project directory.
```!
echo "Project directory: $ARGUMENTS"
```
You have two approaches:
## Option 1: Interactive Wizard
Run `agentic-usability init -p $ARGUMENTS` for a step-by-step interactive setup.
## Option 2: Direct Config Creation
If the user has described their SDK, create `config.json` directly. This is faster and allows you to tailor the config to their exact setup.
### Project Directory Structure
After init, the project should have:
```
<project>/
config.json # Configuration (you create this)
suite.json # Test suite (created by generate)
results/ # Run results (created by eval/execute)
cache/repos/ # Git repo cache (created automatically)
```
### config.json Schema
```json
{
"privateInfo": [],
"publicInfo": [],
"agents": {},
"targets": [],
"sandbox": {}
}
```
#### `privateInfo` (required, non-empty array)
SDK source code and internal docs. Visible to generator and judge, **never to executor**. Each entry is a SourceConfig with a `type` discriminator:
**Local source** — filesystem path:
```json
{ "type": "local", "path": "./src", "subpath": "packages/core", "additionalContext": "Focus on the Builder API" }
```
Fields: `path` (required), `subpath` (optional), `additionalContext` (optional)
**Git source** — clone a repository:
```json
{ "type": "git", "url": "https://github.com/org/sdk.git", "branch": "main", "subpath": "src", "sparse": ["src/api"], "additionalContext": "..." }
```
Fields: `url` (required), `branch`, `subpath`, `sparse` (sparse checkout paths), `additionalContext` (all optional)
**URL source** — fetch documentation:
```json
{ "type": "url", "url": "https://internal-docs.example.com/api-ref", "additionalContext": "..." }
```
Fields: `url` (required), `additionalContext` (optional)
**Package source** — metadata about the SDK package:
```json
{ "type": "package", "name": "@example/sdk", "installCommand": "npm install @example/sdk", "language": "typescript", "additionalContext": "..." }
```
Fields: `name` (required), `installCommand`, `language`, `additionalContext` (all optional)
#### `publicInfo` (optional array)
Public docs and package info visible to **both executor and judge**. Same SourceConfig types as above. Typically includes:
- A `package` source so executors know what to install
- A `url` source for public documentation
#### `agents` (optional object)
| Role | Type | Runs in sandbox? | Secret required? |
|------|------|-------------------|-----------------|
| `generator` | AgentConfig | No (host) | No |
| `executor` | SandboxAgentConfig | **Yes** | **Yes** |
| `judge` | SandboxAgentConfig | **Yes** | **Yes** |
| `insights` | AgentConfig | No (host) | No |
**AgentConfig fields** (generator, insights):
- `command` (required): `"claude"`, `"codex"`, `"gemini"`, or custom CLI name
- `systemPrompt` (optional): supports `{{packageName}}` and `{{docsUrl}}` placeholders
**SandboxAgentConfig** — extends AgentConfig with required `secret`:
```json
{
"command": "claude",
"secret": { "value": "$ANTHROPIC_API_KEY" }
}
```
**AgentSecretConfig fields**:
- `value` (required): raw API key or `"$ENV_VAR"` reference
- `envVar`: env var name for key inside sandbox — auto-detected for known agents
- `baseUrl`: API base URL — auto-detected for known agents
- `baseUrlEnvVar`: env var for base URL override — auto-detected for known agents
**Known agent defaults** (auto-filled, user only needs `value`):
| command | envVar | baseUrl | baseUrlEnvVar |
|---------|--------|---------|---------------|
| `claude` | `ANTHROPIC_API_KEY` | `https://api.anthropic.com` | `ANTHROPIC_BASE_URL` |
| `codex` | `CODEX_API_KEY` | `https://api.openai.com/v1` | `OPENAI_BASE_URL` |
| `gemini` | `GEMINI_API_KEY` | `https://generativelanguage.googleapis.com` | `GEMINI_API_BASE_URL` |
**Custom agents** must explicitly set `envVar` and `baseUrl` in the secret.
#### `targets` (required, non-empty array)
Docker images for sandboxed execution:
```json
{ "name": "node-20", "image": "node:20-slim", "timeout": 1200, "additionalContext": "Node.js 20 with npm" }
```
Fields: `name` (required), `image` (required), `timeout` (optional, seconds), `additionalContext` (optional, included in generator prompt)
#### `sandbox` (required object, can be `{}`)
```json
{
"concurrency": 3,
"defaultTimeout": 600,
"memoryMib": 2048,
"cpus": 2,
"secrets": {
"EXTRA_API_KEY": {
"value": "$EXTRA_KEY",
"allowHosts": ["api.extra-service.com"],
"allowHostPatterns": ["*.extra-service.com"]
}
},
"env": {
"LICENSE_KEY": "$MY_LICENSE_KEY"
}
}
```
- `secrets`: TLS-injected secrets that never enter the VM. Each needs `value` and non-empty `allowHosts`.
- `env`: Plain env vars passed directly into sandbox. Values can use `$VAR` to reference host env.
#### `workspace` (optional)
```json
{ "template": "./workspace-template", "setupScript": "./setup.sh" }
```
For the full schema with all validation rules, see [config-schema.md](../_reference/config-schema.md).
### Complete Example
```json
{
"privateInfo": [
{ "type": "local", "path": "./sdk-source", "additionalContext": "Main SDK source code" }
],
"publicInfo": [
{ "type": "package", "name": "my-sdk", "installCommand": "npm install my-sdk", "language": "typescript" },
{ "type": "url", "url": "https://docs.my-sdk.io/getting-started" }
],
"agents": {
"generator": { "command": "claude" },
"executor": { "command": "claude", "secret": { "value": "$ANTHROPIC_API_KEY" } },
"judge": { "command": "claude", "secret": { "value": "$ANTHROPIC_API_KEY" } }
},
"targets": [
{ "name": "node-20", "image": "node:20-slim", "timeout": 1200 }
],
"sandbox": {
"concurrency": 3,
"defaultTimeout": 600
}
}
```
After creating config.json, run `agentic-usability generate -p <project>` to create the test suite.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · pspdfkit-labs/agentic-usability | 100 | Apache-2.0 |
| CodaColaLight/nightmill | 100 | Apache-2.0 |
| userKael/rastro | 100 | MIT |
| elalfner/relevo | 100 | MIT |
| abatyuk/okf | 100 | MIT |
| duronext/jig | 100 | MIT |
| sh5623/e2e-rail | 100 | MIT |
| Calvin-LLC/skills | 100 | MIT |
| arxdsilva/ai-loops | 100 | MIT |
Database snapshot: 13.11.2026 · package version 5a285ad41881 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100eval
Run the full evaluation pipeline (execute, judge, report) for an SDK usability benchmark. Use when running a complete benchmark end-to-end, resuming an interrupted pipeline, or checking pipeline status. - 100scaffold-init
Initialize an AI-native development workspace with spec-driven workflow infrastructure. Use when starting a new project, setting up Claude Code on a codebase for the first time, or when the user says scaffold, initialize, set up AI workflow, onboard this repo, or similar. Do not… - 100scaffold-init
Initialize an AI-native development workspace with spec-driven workflow infrastructure. Use when starting a new project, setting up Codex on a codebase for the first time, or when the user says scaffold, initialize, set up AI workflow, onboard this repo, or similar. Do not use… - 100init-pipeline
This skill should be used when the user asks to set up or initialize the subagent-system pipeline for a project ('init pipeline', 'set up the pipeline', 'khoi tao pipeline'). Creates PIPELINE_CONFIG.md and checkpoint scaffolding. - 92init-project
Use when setting up a new or existing project for AI-assisted development. - 100pm-init
Initialize or refresh Project Memory for a repository. Use after amplifying a new repo, when the user says "init PM", "set up project memory", or when pm-index.md is missing or empty. Surveys the codebase, creates starter PM units, and populates the index.