Skip to content
Skills Library
AI & agents · Review & audit

modelspy

/modelspy

对任意 OpenAI 兼容接口做行为取证,识别背后真正在跑的模型家族与档位。查中转站/API 代理/代充前端有没有偷换便宜模型、验证端点是不是宣称的那个模型、批量抽测降智时使用:六路探针(分词器指纹、陷阱题库、生成速度 TPOT、自我披露、知识截止、Function calling)按权重投票,输出候选排名 + 置信度 + MATCHES/MISMATCH 判定和 HTML 报告。英文场景:identify which LLM actually serves an endpoint, detect reseller model downgrade, verify "is this really GPT-6", audit OpenAI-compatible API behavior.

Repository
modelspy ↗
Licence
MIT
Last repository change
09.10.2026

Assessment and security

98/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene13 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
4
Deprecation

No signals detected. Repository changed on 09.10.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

对任意 OpenAI 兼容接口做行为取证,识别背后真正在跑的模型家族与档位。查中转站/API 代理/代充前端有没有偷换便宜模型、验证端点是不是宣称的那个模型、批量抽测降智时使用:六路探针(分词器指纹、陷阱题库、生成速度 TPOT、自我披露、知识截止、Function calling)按权重投票,输出候选排名 + 置信度 + MATCHES/MISMATCH 判定和 HTML 报告。英文场景:identify which LLM actually serves an endpoint, detect reseller model downgrade, verify "is this really GPT-6", audit OpenAI-compatible API behavior.

Triggers from the description

is this really GPT-6
Original author description

对任意 OpenAI 兼容接口做行为取证,识别背后真正在跑的模型家族与档位。查中转站/API 代理/代充前端有没有偷换便宜模型、验证端点是不是宣称的那个模型、批量抽测降智时使用:六路探针(分词器指纹、陷阱题库、生成速度 TPOT、自我披露、知识截止、Function calling)按权重投票,输出候选排名 + 置信度 + MATCHES/MISMATCH 判定和 HTML 报告。英文场景:identify which LLM actually serves an endpoint, detect reseller model downgrade, verify "is this really GPT-6", audit OpenAI-compatible API behavior.

How it works

  1. 触发场景
  2. 用法
  3. 工作流程
  4. 输出解读
  5. 局限
Original SKILL.md headings · 75 lines, 264 words
SKILL.md preview
---
name: modelspy
description: '对任意 OpenAI 兼容接口做行为取证,识别背后真正在跑的模型家族与档位。查中转站/API 代理/代充前端有没有偷换便宜模型、验证端点是不是宣称的那个模型、批量抽测降智时使用:六路探针(分词器指纹、陷阱题库、生成速度 TPOT、自我披露、知识截止、Function calling)按权重投票,输出候选排名 + 置信度 + MATCHES/MISMATCH 判定和 HTML 报告。英文场景:identify which LLM actually serves an endpoint, detect reseller model downgrade, verify "is this really GPT-6", audit OpenAI-compatible API behavior.'
disable-model-invocation: false
user-invocable: true
---

# ModelSpy - 模型行为取证

对任意 OpenAI 兼容接口做行为取证,识别背后真正在跑的模型家族与档位,输出带置信度的判定报告。

## 触发场景

- 用户怀疑中转站/代充站把高配模型换成低配
- 用户想验证某个 API 端点跑的是不是宣称的模型
- 用户说"测一下这个模型降没降智"、"帮我查查这个中转"

## 用法

### 第 0 步 · 确保二进制可用

优先使用本技能目录 `scripts/` 下已有的 modelspy 程序(Windows 为 `modelspy.exe`,Linux/macOS 为 `modelspy`)。若不存在,从官方 Release 下载到 `scripts/` 目录并按平台改名:

- Windows amd64:`https://github.com/3378925604-a11y/modelspy/releases/download/v0.1.0/modelspy-windows-amd64.exe` → 保存为 `scripts/modelspy.exe`
- Linux amd64:`https://github.com/3378925604-a11y/modelspy/releases/download/v0.1.0/modelspy-linux-amd64` → 保存为 `scripts/modelspy`,并 `chmod +x`
- macOS arm64:`https://github.com/3378925604-a11y/modelspy/releases/download/v0.1.0/modelspy-darwin-arm64` → 保存为 `scripts/modelspy`,并 `chmod +x`

下载后先运行 `-h` 确认可执行,再进入下一步。

### 第 1 步 · 执行检测

```powershell
& "SKILL_DIR\scripts\modelspy.exe" -base <接口地址> -model <模型名> [-key <API_KEY>] [-expect <宣称模型>] [-json] [-html <报告路径>]
```

### 参数说明

| 参数 | 必填 | 说明 |
|------|------|------|
| `-base` | ✅ | OpenAI 兼容接口地址,如 `https://api.openai.com/v1` |
| `-model` | ✅ | 调用时使用的模型名,如 `gpt-6-astra` |
| `-key` | ❌ | API key(默认读环境变量 `OPENAI_API_KEY`) |
| `-expect` | ❌ | 对方宣称的模型(默认同 `-model`) |
| `-probes` | ❌ | 只跑指定探针,逗号分隔:`identity,tokenizer,capability,cutoff,speed,toolcall` |
| `-json` | ❌ | 输出机器可读 JSON |
| `-html` | ❌ | 导出 HTML 可视化报告 |
| `-timeout` | ❌ | 单次请求超时秒数(默认 90) |
| `-list-probes` | ❌ | 列出全部探针后退出 |
| `-version` | ❌ | 显示版本 |

### 探针说明

| 探针 | 说明 |
|------|------|
| `identity` | 随机措辞询问模型身份,防固定话术背板 |
| `tokenizer` | 固定中英文语料的 prompt_tokens 比值,物理级指纹 |
| `capability` | 陷阱题库(strawberry、9.11 等),temp=0 下区分代际 |
| `cutoff` | 自报训练截止日期与候选模型比对 |
| `speed` | 流式 TPOT(每 token 耗时)分布,小模型快得多 |
| `toolcall` | Function calling 结构完整性检查 |

## 工作流程

1. 确认用户提供了接口地址和模型名
2. 如用户未提供 API key,提示设置环境变量或传 `-key`
3. 执行 modelspy 并等待结果(默认 10 分钟超时)
4. 解读报告:关注置信度排名和 `MATCHES`/`MISMATCH` 判定
5. 如结果置信度低,建议多跑几轮或只跑特定探针

## 输出解读

- `MATCHES`(退出码 0):识别结果与声明一致
- `MISMATCH`(退出码 1):疑似换壳
- 退出码 3:证据不足

## 局限

- 同家族盲区:GPT-6 与 GPT-5.6 分词器相同,主要靠题库与速度区分
- 概率性结论:低置信时多跑几轮
- 对抗性缺口:中转可以"只对疑似测试的请求走真模型"

Versions

The database has no other skills named modelspy.

Identical content appears in 3 other GitHub folders. We show one copy with the highest score.

Database snapshot: 13.11.2026 · package version c8cc97db05cc (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 95openai-compatible-endpoint-pack
    Use when a Codex task needs to configure, diagnose, or migrate an AI coding project to an OpenAI-compatible API endpoint without leaking credentials.
  • 100imagegen
    Generate or edit raster images with an OpenAI-compatible Images API (default model gpt-image-2). Use when the user asks for image generation, image edits, GPT Image, gpt-image-2, or image outputs that should be saved into the current project.
  • 100agent-tool-use-function-calling
    Use this skill when implementing reliable tool invocation using OpenAI-compatible JSON schema function calling specifications.
  • 100provider-onboarding
    Use when the user wants to add support for, wire up, onboard or integrate an LLM provider or model ("add support for X's Y model"), asks how to call a model, what a provider's endpoint, API-key env var, model id, context limit or price is, or where to get an API key, or wants a…
  • 100add-llm-model
    新增 LLM 模型专家,指导如何在本项目中添加新的 LLM 供应商和模型。当需要接入新的大语言模型(如 OpenAI、Codex、DeepSeek 等)时使用。
  • 95preset-skill-openai
    Use when configuring or revising the built-in openai (OpenAI-compatible) TUI preset, including pointing it at another vendor's OpenAI-compatible API or an account pool such as sub2api / subs-pool.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.