Skip to content
Skills Library
Programming · Review & audit

verify-against-source

/verify-against-source

Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.

Repository
docs ↗
Licence
MIT
Last repository change
27.10.2026

Assessment and security

100/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
7
Deprecation

No signals detected. Repository changed on 27.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.

Original author description

Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.

How it works

  1. Step 1. Pick the strongest evidence available
  2. Step 2. Know which repository owns the claim
  3. Step 3. Verify the claim, not the sentence
  4. Step 4. Record what you checked, and what you could not
  5. Related
Original SKILL.md headings · 101 lines, 899 words
SKILL.md preview
---
name: verify-against-source
description: Check that a code sample, API signature, default, or behavior claim in the docs is actually true, by running it or reading the product source. Covers which repository owns each product, how to reach the private ones, running the part of a sample that needs no API key, and what to say about a claim you could not verify. Use when writing or reviewing a page that asserts how LangChain, LangGraph, Deep Agents, or LangSmith behaves, and when a PR describes behavior without citing where it was checked.
---

# Verify a claim against the source

The failure this skill prevents: a page states a default, a precedence order, or
a field name that reads plausibly, passes Vale and the link checker, and is
wrong. Nothing in CI checks whether a sentence about product behavior is true.
A reviewer who knows the product catches some of it; the rest reaches readers.

Published docs, a README, and a model's recollection are all secondary sources.
Prefer evidence you produced in this session.

## Step 1. Pick the strongest evidence available

Work down this list and stop at the first rung you can actually reach. Each rung
is weaker than the one above it.

- **Run the sample.** `make test-code-samples`, or scope it with
  `FILES="src/code-samples/langchain/foo.py"`. This is the only method that
  proves a sample works end to end. It needs provider API keys in the
  environment, so it often fails locally and runs in CI instead.
- **Run the half that needs no model.** Most claims are about a library's
  behavior, not the model's. Stand up the real object in a scratch script and
  print what it returns. A local MCP server, an adapter, a parser, or a store
  needs no API key, and the answer it gives is exact rather than inferred.
- **Read the library source.** `gh api repos/langchain-ai/<repo>/contents/<path>
  --jq '.content' | base64 -d`, or `gh api "search/code?q=<symbol>+repo:langchain-ai/<repo>"`
  to locate the file first. Settles what a type is, what a default is, and which
  branch actually runs.
- **Check the installed package.** `uv run python -c "from x import Y"` confirms
  an import path resolves at the version this repo pins, which source on `main`
  cannot tell you.
- **Look up the signature.** The `reference-langchain` MCP server exposes
  `search_api` and `get_symbol` over reference.langchain.com. Right for "what
  are the parameters of X", not for "what does X do at runtime".

## Step 2. Know which repository owns the claim

| Product | Repository | Where to look |
|---------|------------|---------------|
| LangChain | [langchain-ai/langchain](https://github.com/langchain-ai/langchain) | `libs/langchain_v1/langchain/` for v1, `libs/core/` for core, `libs/partners/` for provider packages |
| LangGraph | [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) | `libs/langgraph/`, `libs/prebuilt/`, `libs/checkpoint*/`, `libs/cli/`, `libs/sdk-py/` and `libs/sdk-js/` |
| Deep Agents | [langchain-ai/deepagents](https://github.com/langchain-ai/deepagents) | Also the source of the eval matrix the docs publish |
| LangSmith SDK | [langchain-ai/langsmith-python](https://github.com/langchain-ai/langsmith-python) and [langchain-ai/langsmith-javascript](https://github.com/langchain-ai/langsmith-javascript) | The Python and JavaScript client libraries |
| LangSmith platform | langchain-ai/langchainplus (private) | `smith-backend/` (Python API), `smith-go/` (Go services), `smith-frontend/` (UI labels and flows), `host-backend/`, `lc_config/` for settings and their defaults |
| Agent Server | langchain-ai/langgraph-api (private) | Not in the OSS langgraph repo. Also the source of the Agent Server OpenAPI spec PRs |
| Helm charts | [langchain-ai/helm](https://github.com/langchain-ai/helm) | `charts/langsmith/values.yaml` for defaults, `templates/_helpers.tpl` for the value-to-environment-variable mapping |
| OpenEvals | [langchain-ai/openevals](https://github.com/langchain-ai/openevals) | Prebuilt evaluators |

Two traps worth naming:

- **A private repo is still readable through `gh`.** `gh api` works on
  langchainplus and langgraph-api with the usual credentials. GitHub code search
  is unreliable on the larger ones, so fall back to the git trees API and fetch
  a path directly.
- **langchain-ai/deployments is archived.** Its GitOps content moved into
  langchainplus. A skill, script, or note that still points at it is stale.

A self-hosted LangSmith claim usually spans two repositories: the chart sets a
value, and the backend decides what to do with it. Check both before describing
precedence, because the chart alone does not tell you which setting wins.

## Step 3. Verify the claim, not the sentence

Read the code path that produces the behavior, rather than a name that sounds
like it. Three failures that recur:

- **A resolution order inferred from a settings table.** Find the query or the
  conditional that picks a value. An `ORDER BY`, a `COALESCE`, or an `or`
  expression is the precedence; a list of configurable fields is not.
- **A defensive pattern copied from another sample.** Check whether the guarded
  case can occur. If the constructor only builds an object when a field is
  present, a `.get` with a default is documenting a state that never exists.
- **A version or model identifier from memory.** Fetch it. The repository's own
  internal consensus is evidence about the repository, not about the provider.
  For model IDs specifically, see the Model references section of `AGENTS.md`.

## Step 4. Record what you checked, and what you could not

Name the file and symbol in the pull request body, not just the conclusion. A
reviewer can then disagree with the evidence rather than with an assertion.

State the gaps in the same place. Common ones that no amount of source reading
closes:

- **A UI string.** Source gives you a field and a component; only the running
  product gives you the rendered label. Describe the behavior rather than
  quoting a label you have not seen.
- **The model-dependent half of a sample.** Say that CI has yet to run it.
- **A beta surface.** `langchain.mcp` and similar namespaces can move under a
  page that was accurate when written.

Writing "not verified: the visible label of this control" is a better outcome
than a confident sentence a reader discovers is wrong.

## Related

`AGENTS.md` holds the rules that apply to every edit, including never
fabricating an example and testing code before publishing it. Use
`docs-code-samples` when the claim belongs in a runnable sample rather than in
prose, and `docs-review` for the style pass once the facts are settled.

Versions

RepositoryScoreLicence
This page · langchain-ai/docs100MIT
leonletto/thrum100MIT
leonletto/thrum100MIT
oelnoury/verify-against-source90MIT

Database snapshot: 13.11.2026 · package version 2bbbcf776a63 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100source-driven-development
    Use when implementing framework-specific or library-specific code — verify non-trivial APIs against official documentation before writing them, and record the source in task notes, docs, or the PR.
  • 100verify-work
    Use whenever you need to know whether something works or a claim is true: running tests, builds, linters or type checks; reproducing a bug or checking a fix; trying the real path in the running software, a browser or a CLI; measuring performance; and reading what the output…
  • 82verify
    Prove a claim of done, fixed, or passing before you make it. Run the commands and read their output. Read this skill before you say that work is done, fixed, or tested. Read it also after a small change. Use it before a commit or a PR. Use it also when the user asks to write or…
  • 90sample-write
    Write a working code sample or tutorial for an API feature or integration pattern. Use when asked to "write a code sample", "write a tutorial for this feature", or "build an integration example".
  • 100verify
    Verify that a code change actually does what it's supposed to by running the app and observing behavior. Use when asked to verify a PR, confirm a fix works, test a change manually, check that a feature works, or validate local changes before pushing.
  • 100research-primary-sources
    Use when a decision depends on how a library, platform, protocol, or tool behaves, and that behavior can change by version. Also use it when the user says "check the docs", "is this still true?", or "what does the spec say?". Finds the first-party source for the installed…

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.