Skip to content
Skills Library
Programming · Automation

nlweb-setup

/nlweb-setup

Bootstrap a local NLWeb development environment from scratch — clone the repo, configure .env, install Python deps via `nlweb init-python`, run `nlweb init` for interactive LLM/retrieval selection, load sample Schema.org data, and verify with `nlweb check`. Use when starting a new NLWeb deployment from zero.

Licence
MIT
Last repository change
14.09.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
8
Deprecation

No signals detected. Repository changed on 14.09.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Bootstrap a local NLWeb development environment from scratch — clone the repo, configure .env, install Python deps via `nlweb init-python`, run `nlweb init` for interactive LLM/retrieval selection, load sample Schema.org data, and verify with `nlweb check`. Use when starting a new NLWeb deployment from zero.

Original author description

Bootstrap a local NLWeb development environment from scratch — clone the repo, configure .env, install Python deps via `nlweb init-python`, run `nlweb init` for interactive LLM/retrieval selection, load sample Schema.org data, and verify with `nlweb check`. Use when starting a new NLWeb deployment from zero.

How it works

  1. Before writing code
  2. Conceptual Architecture
Original SKILL.md headings · 109 lines, 724 words
SKILL.md preview
---
name: nlweb-setup
description: Bootstrap a local NLWeb development environment from scratch — clone the repo, configure .env, install Python deps via `nlweb init-python`, run `nlweb init` for interactive LLM/retrieval selection, load sample Schema.org data, and verify with `nlweb check`. Use when starting a new NLWeb deployment from zero.
allowed-tools: Read, Write, Edit, Bash, Grep, Glob, WebSearch, WebFetch
---

# NLWeb Setup

> **Config layout changed upstream.** NLWeb replaced the single `site_types.xml` with two files in `config/`:
> **`sites.xml`** (site name → `itemType` list + description) and **`tools.xml`** (per-site / per-type tool
> definitions, prompts and examples, scoped by `<Site id="…">` / `<Item>` blocks). Older guidance — including any
> `site_type` / `extends` inheritance syntax — describes the retired file. **Fetch `config/sites.xml` and
> `config/tools.xml` from the live repo before editing anything.**

## Before writing code

**Fetch live docs first**:
1. Fetch https://github.com/nlweb-ai/NLWeb (README) for the current minimum Python version and required deps.
2. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/nlweb-hello-world.md for the canonical hello-world flow.
3. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/nlweb-cli.md for current `nlweb` CLI flags.
4. Web-search `site:github.com/nlweb-ai/NLWeb docs/release_notes` and read the **most recent** dated release note — config keys and required env vars change between releases.
5. Identify the default `write_endpoint` and verify which backends are enabled by default in `config/config_retrieval.yaml` on `main`.

## Conceptual Architecture

### What "setup" produces

A working NLWeb dev environment has four parts:

1. **Cloned repo** + Python virtualenv with requirements installed.
2. **`.env`** file with provider credentials (OpenAI/Azure OpenAI key + retrieval backend secrets).
3. **Sample data** ingested into the local vector store (Qdrant local by default).
4. **A running aiohttp server** on `:8000` with `/ask`, `/mcp`, `/sites` reachable.

### Three Default-Enabled Backends — Watch Out

NLWeb ships with **three retrieval backends enabled by default** in `config_retrieval.yaml`:
- `qdrant_local` (file-backed, fine for dev)
- `nlweb_west` (Azure AI Search — requires Azure credentials)
- `shopify_mcp` (queries Shopify's MCP endpoint, requires network)

For most local-dev cases, disable the latter two by setting `enabled: false` so you don't get connection errors at startup. The `write_endpoint` should point to `qdrant_local` for dev.

### Setup Decision Checklist

- **LLM provider** — OpenAI, Azure OpenAI (default), Anthropic, Gemini, Ollama (offline), Snowflake Cortex?
- **Embedding provider** — must match between ingest and query; default is `text-embedding-3-small` on Azure OpenAI.
- **Retrieval write endpoint** — Qdrant local for dev, Azure AI Search / Snowflake Cortex / pgvector for prod.
- **Data source** — Schema.org JSON-LD on the site, RSS/Atom feed, sitemap.xml, or CSV?
- **Mode** — `development` (allows query-string config overrides) or `production` in `config_webserver.yaml`?
- **OAuth** — anonymous-only, or login-gated (GitHub/Google/Microsoft/Facebook)?

### Project Layout (after setup)

```
NLWeb/                                 # cloned repo
├── AskAgent/python/
│   ├── app-aiohttp.py                 # main entry
│   ├── core/, methods/, webserver/    # core code
│   ├── llm_providers/, embedding_providers/, retrieval_providers/
│   └── data_loading/
├── config/
│   ├── config_llm.yaml
│   ├── config_embedding.yaml
│   ├── config_retrieval.yaml
│   ├── config_nlweb.yaml
│   ├── config_webserver.yaml
│   ├── config_oauth.yaml
│   ├── config_storage.yaml
│   ├── config_tools.yaml
│   ├── tools.xml
│   └── prompts.xml
├── data/db/                           # qdrant_local file store
├── .env                               # YOUR credentials (gitignored)
└── docs/, scripts/, demo/, tests/
```

### Setup Sequence

1. `git clone https://github.com/nlweb-ai/NLWeb && cd NLWeb`
2. `nlweb init-python` (or manual `python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt`)
3. `nlweb init` — interactive prompts walk through LLM + retrieval selection and write `.env`
4. Disable the unwanted default backends in `config/config_retrieval.yaml` (`nlweb_west`, `shopify_mcp` for local-only dev)
5. `nlweb data-load <source> <site-name>` — ingest sample content (use a small RSS feed for first run)
6. `nlweb check` — runs connectivity diagnostics; resolve any red flags
7. `nlweb app` — start the server, hit `http://localhost:8000/`
8. Test `/ask?query=hello&site=<site-name>&streaming=false`

### .env Conventions

NLWeb expects credentials via env vars (never YAML). Common keys (verify live):
- `OPENAI_API_KEY`, `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`
- `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`
- `AZURE_SEARCH_ENDPOINT`, `AZURE_SEARCH_API_KEY`
- `QDRANT_API_KEY` (only for remote Qdrant)
- `SNOWFLAKE_USER`, `SNOWFLAKE_PASSWORD`, `SNOWFLAKE_ACCOUNT`
- `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID`

### Verification Targets

After setup, these should work:
- `curl http://localhost:8000/sites` → JSON list including your loaded site
- `curl 'http://localhost:8000/ask?query=test&site=<your-site>&streaming=false'` → JSON with results
- `curl -X POST http://localhost:8000/mcp -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'` → `ask`, `list_sites`, optionally `who`

### Common Setup Failures

- **`nlweb check` fails on Azure**: usually `AZURE_OPENAI_ENDPOINT` missing trailing slash or wrong deployment name.
- **Embedding dim mismatch on retrieval**: data was loaded with a different embedding provider than runtime config. Either re-ingest or change `preferred_provider` in `config_embedding.yaml`.
- **Server starts but `/ask` returns empty**: site name in the query doesn't match the `site` value used during ingest, or the `sites:` allowlist in `config_nlweb.yaml` excludes it.
- **Slow first request**: cold model loading + `/who` endpoint pinging `nlwm.azurewebsites.net`. Disable `who_endpoint_enabled` for offline dev.

Always re-verify against the latest hello-world doc — the exact env-var names and CLI flags change.

Versions

The database has no other skills named nlweb-setup.

Database snapshot: 13.11.2026 · package version 8531d98a7f12 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100nlweb-schema-org-grounding
    Prepare and structure site content as Schema.org JSON-LD for NLWeb ingestion — covers the supported types (Recipe, Product, Movie, Event, Article, RealEstate, Course, etc.), per-type behavior in NLWeb's tool routing, JSON-LD embedding patterns in HTML, sites.xml registration,…
  • 100nlweb-tools-framework
    Design and implement NLWeb tools — the per-Schema.org-type handlers that turn a query into a specialized response (search, item_details, compare_items, ensemble, recipe_substitution, accompaniment, conversation_search, etc.). Covers `tools.xml`, the ToolSelector router, builtin…
  • 100orchardcore-nlweb
    Skill for the CrestApps Orchard Core NLWeb module (CrestApps.OrchardCore.NLWeb), which lets people and AI agents ask the site questions in natural language with the NLWeb protocol with schema.org answers built by Advanced SEO. Covers the /ask and /await endpoints, protocol…
  • 100init-repo
    Bootstrap a new n8n workflow repo from scratch. Scaffolds directory structure, writes plugin-compatible CLAUDE.md/README/.gitignore/.env.example, runs the n8nac >= 2.3 setup flow (`env add` + `env auth set` + `env use`), pulls node schemas, and verifies setup. Use when starting…
  • 100orchardcore-advanced-seo
    Skill for the CrestApps Orchard Core Advanced SEO module (CrestApps.OrchardCore.Seo), which publishes schema.org structured data as one JSON-LD graph per page and for NLWeb answers. Covers the Schema.org part (SchemaOrgPart, SchemaOrgPartSettings with SchemaType and property…
  • 100bootstrap
    Create, diagnose, and validate a local dev bootstrap. Use when the user asks to clone a repo, install toolchains, install dependencies, and prove the project runs.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.