verify-against-source
/verify-against-source
Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.
- Author
- langchain-ai ↗
- Repository
- docs ↗
- Licence
- MIT
- Last repository change
- 27.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 7
No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.
Original author description
Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.
How it works
- Step 1. Pick the strongest evidence available
- Step 2. Know which repository owns the claim
- Step 3. Verify the claim, not the sentence
- Step 4. Record what you checked, and what you could not
- Related
SKILL.md preview
--- name: verify-against-source description: Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked. --- # Verify a claim against the source The failure this skill prevents: a page states a default, a precedence order, or a field name that reads plausibly, passes Vale and the link checker, and is wrong. Nothing in CI checks whether a sentence about product behavior is true. A reviewer who knows the product catches some of it; the rest reaches readers. Published docs, a README, and a model's recollection are all secondary sources. Prefer evidence you produced in this session. ## Step 1. Pick the strongest evidence available Work down this list and stop at the first rung you can actually reach. Each rung is weaker than the one above it. - **Run the sample.** `make test-code-samples`, or scope it with `FILES="src/code-samples/langchain/foo.py"`. This is the only method that proves a sample works end to end. It needs provider API keys in the environment, so it often fails locally and runs in CI instead. - **Run the half that needs no model.** Most claims are about a library's behavior, not the model's. Stand up the real object in a scratch script and print what it returns. A local MCP server, an adapter, a parser, or a store needs no API key, and the answer it gives is exact rather than inferred. - **Read the library source.** `gh api repos/langchain-ai/<repo>/contents/<path> --jq '.content' | base64 -d`, or `gh api "search/code?q=<symbol>+repo:langchain-ai/<repo>"` to locate the file first. Settles what a type is, what a default is, and which branch actually runs. - **Check the installed package.** `uv run python -c "from x import Y"` confirms an import path resolves at the version this repo pins, which source on `main` cannot tell you. - **Look up the signature.** The `reference-langchain` MCP server exposes `search_api` and `get_symbol` over reference.langchain.com. Right for "what are the parameters of X", not for "what does X do at runtime". ## Step 2. Know which repository owns the claim | Product | Repository | Where to look | |---------|------------|---------------| | LangChain | [langchain-ai/langchain](https://github.com/langchain-ai/langchain) | `libs/langchain_v1/langchain/` for v1, `libs/core/` for core, `libs/partners/` for provider packages | | LangGraph | [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) | `libs/langgraph/`, `libs/prebuilt/`, `libs/checkpoint*/`, `libs/cli/`, `libs/sdk-py/` and `libs/sdk-js/` | | Deep Agents | [langchain-ai/deepagents](https://github.com/langchain-ai/deepagents) | Also the source of the eval matrix the docs publish | | LangSmith SDK | [langchain-ai/langsmith-python](https://github.com/langchain-ai/langsmith-python) and [langchain-ai/langsmith-javascript](https://github.com/langchain-ai/langsmith-javascript) | The Python and JavaScript client libraries | | LangSmith platform | langchain-ai/langchainplus (private) | `smith-backend/` (Python API), `smith-go/` (Go services), `smith-frontend/` (UI labels and flows), `host-backend/`, `lc_config/` for settings and their defaults | | Agent Server | langchain-ai/langgraph-api (private) | Not in the OSS langgraph repo. Also the source of the Agent Server OpenAPI spec PRs | | Helm charts | [langchain-ai/helm](https://github.com/langchain-ai/helm) | `charts/langsmith/values.yaml` for defaults, `templates/_helpers.tpl` for the value-to-environment-variable mapping | | OpenEvals | [langchain-ai/openevals](https://github.com/langchain-ai/openevals) | Prebuilt evaluators | Two traps worth naming: - **A private repo is still readable through `gh`.** `gh api` works on langchainplus and langgraph-api with the usual credentials. GitHub code search is unreliable on the larger ones, so fall back to the git trees API and fetch a path directly. - **langchain-ai/deployments is archived.** Its GitOps content moved into langchainplus. A skill, script, or note that still points at it is stale. A self-hosted LangSmith claim usually spans two repositories: the chart sets a value, and the backend decides what to do with it. Check both before describing precedence, because the chart alone does not tell you which setting wins. ## Step 3. Verify the claim, not the sentence Read the code path that produces the behavior, rather than a name that sounds like it. Three failures that recur: - **A resolution order inferred from a settings table.** Find the query or the conditional that picks a value. An `ORDER BY`, a `COALESCE`, or an `or` expression is the precedence; a list of configurable fields is not. - **A defensive pattern copied from another sample.** Check whether the guarded case can occur. If the constructor only builds an object when a field is present, a `.get` with a default is documenting a state that never exists. - **A version or model identifier from memory.** Fetch it. The repository's own internal consensus is evidence about the repository, not about the provider. For model IDs specifically, see the Model references section of `AGENTS.md`. ## Step 4. Record what you checked, and what you could not Name the file and symbol in the pull request body, not just the conclusion. A reviewer can then disagree with the evidence rather than with an assertion. State the gaps in the same place. Common ones that no amount of source reading closes: - **A UI string.** Source gives you a field and a component; only the running product gives you the rendered label. Describe the behavior rather than quoting a label you have not seen. - **The model-dependent half of a sample.** Say that CI has yet to run it. - **A beta surface.** `langchain.mcp` and similar namespaces can move under a page that was accurate when written. Writing "not verified: the visible label of this control" is a better outcome than a confident sentence a reader discovers is wrong. ## Related `AGENTS.md` holds the rules that apply to every edit, including never fabricating an example and testing code before publishing it. Use `docs-code-samples` when the claim belongs in a runnable sample rather than in prose, and `docs-review` for the style pass once the facts are settled.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · langchain-ai/docs | 100 | MIT |
| leonletto/thrum | 100 | MIT |
| leonletto/thrum | 100 | MIT |
| oelnoury/verify-against-source | 90 | MIT |
Database snapshot: 13.11.2026 · package version 2bbbcf776a63 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100source-driven-development
Use when implementing framework-specific or library-specific code — verify non-trivial APIs against official documentation before writing them, and record the source in task notes, docs, or the PR. - 100verify-work
Use whenever you need to know whether something works or a claim is true: running tests, builds, linters or type checks; reproducing a bug or checking a fix; trying the real path in the running software, a browser or a CLI; measuring performance; and reading what the output… - 82verify
Prove a claim of done, fixed, or passing before you make it. Run the commands and read their output. Read this skill before you say that work is done, fixed, or tested. Read it also after a small change. Use it before a commit or a PR. Use it also when the user asks to write or… - 90sample-write
Write a working code sample or tutorial for an API feature or integration pattern. Use when asked to "write a code sample", "write a tutorial for this feature", or "build an integration example". - 100verify
Verify that a code change actually does what it's supposed to by running the app and observing behavior. Use when asked to verify a PR, confirm a fix works, test a change manually, check that a feature works, or validate local changes before pushing. - 100research-primary-sources
Use when a decision depends on how a library, platform, protocol, or tool behaves, and that behavior can change by version. Also use it when the user says "check the docs", "is this still true?", or "what does the spec say?". Finds the first-party source for the installed…