dify-knowledge-base-upload
/dify-knowledge-base-upload
Upload local files to a Dify pipeline knowledge base through the published pipeline datasource flow and optionally apply existing metadata fields. Use when the target dataset is a published rag_pipeline/local_file knowledge base or ordinary Dify document create APIs return completed with 0 chunks.
- Author
- tiangong-ai ↗
- Repository
- agent-skills ↗
- Licence
- MIT
- Last repository change
- 20.10.2026
Assessment and security
- Files in this package
- 8
- Scripts
- 1
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 20.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
- 1 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/upload_to_dataset.py
What it does and when to use it
Upload local files to a Dify pipeline knowledge base through the published pipeline datasource flow and optionally apply existing metadata fields. Use when the target dataset is a published rag_pipeline/local_file knowledge base or ordinary Dify document create APIs return completed with 0 chunks.
Original author description
Upload local files to a Dify pipeline knowledge base through the published pipeline datasource flow and optionally apply existing metadata fields. Use when the target dataset is a published rag_pipeline/local_file knowledge base or ordinary Dify document create APIs return completed with 0 chunks.
How it works
- Use this skill when
- Prepare caller-side inputs
- Run workflow
- Interpret response
- Troubleshoot quickly
- References
- Assets
SKILL.md preview
---
name: dify-knowledge-base-upload
description: Upload local files to a Dify pipeline knowledge base through the published pipeline datasource flow and optionally apply existing metadata fields. Use when the target dataset is a published rag_pipeline/local_file knowledge base or ordinary Dify document create APIs return completed with 0 chunks.
---
# Dify Knowledge Base Upload
## Use this skill when
- The target Dify knowledge base is backed by a published pipeline datasource.
- UI uploads succeed but `document/create-by-file` or `create-by-text` yields `completed` with `0 chunks`.
- You need to upload one local file and optionally write existing metadata fields afterward.
## Prepare caller-side inputs
- Provide a local file path to upload.
- For pure text, save it as `.txt` or `.md` first and upload that file.
- Set caller env values:
- `DIFY_API_BASE_URL`
- `DIFY_DATASET_ID`
- `DIFY_API_KEY`
- Optionally provide pipeline `inputs` JSON if the published pipeline exposes user input variables.
- Optionally provide metadata JSON:
- a flat object mapping existing Dify metadata field names to values, or
- a list of `{ "name": ..., "value": ... }` / `{ "id": ..., "value": ... }` items.
- The skill does not hardcode your metadata schema. Pass the current field names or ids in the JSON you provide at runtime.
- Read `references/env.md` only if you need env or debug details.
## Run workflow
- Use `scripts/upload_to_dataset.py` to:
- discover the published datasource plugin from `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/pipeline/datasource-plugins?is_published=true`
- upload the file to `${DIFY_API_BASE_URL}/datasets/pipeline/file-upload`
- run the published pipeline through `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/pipeline/run`
- poll `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/documents/{batch}/indexing-status`
- fetch `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/documents/{document_id}?metadata=without`
- optionally resolve metadata field names through `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/metadata`
- optionally update metadata through `${DIFY_API_BASE_URL}/datasets/${DIFY_DATASET_ID}/documents/metadata`
- Upload without metadata:
```bash
python3 scripts/upload_to_dataset.py \
--file /path/to/document.pdf
```
- Upload with pipeline inputs:
```bash
python3 scripts/upload_to_dataset.py \
--file /path/to/document.pdf \
--inputs-json assets/example-pipeline-inputs.json
```
- Upload and then apply metadata:
```bash
python3 scripts/upload_to_dataset.py \
--file /path/to/document.pdf \
--metadata-json assets/example-metadata.json
```
## Interpret response
- Success prints JSON with `batch`, `document_id`, `file_upload_response`, `pipeline_run_response`, `indexing_status_response`, `document_response`, optional `metadata_response`, and `validation`.
- `validation.ok` is only true when:
- `indexing_status = "completed"`
- `total_segments > 0`
- `segment_count > 0`
- If `tokens` is returned by the document detail API, it should also be greater than `0`. Some pipeline deployments leave `tokens = null`; the script reports that as a warning instead of a hard failure.
- The script exits non-zero if the required checks fail.
- If `--metadata-json` is omitted, the script uploads the file only and skips metadata API calls.
- Metadata keys must already exist in the target dataset. Unknown names fail fast before the metadata write request.
- `assets/example-metadata.json` is only a template. Replace its keys with your own existing Dify metadata field names before live use.
- Use `--dry-run` to validate local files and request shape without calling Dify.
## Troubleshoot quickly
- If auth fails, verify `DIFY_API_KEY` and `Authorization` header format.
- If datasource discovery fails, verify the dataset has a published pipeline datasource and the key belongs to that dataset.
- If the pipeline run fails with missing input errors, inspect `user_input_variables` from `discovery_response` and pass the missing keys in `--inputs-json`.
- If the published datasource is not `local_file`, this skill is not the right uploader.
- If metadata resolution fails, rename keys in the metadata JSON to the exact field names defined in Dify.
- If you only have raw text, save it to `.txt` or `.md` and upload that file.
- In the current tested deployment, `.md` uploaded successfully while `.txt` hit a Dify-side indexing `400`. If plain text fails as `.txt`, retry as `.md`.
## References
- `references/env.md`
- `references/request-response.md`
- `references/testing.md`
## Assets
- `assets/example-pipeline-inputs.json`
- `assets/example-metadata.json`
Files in this package (8)
- SKILL.md 5 KB
- agents/openai.yaml 1 KB
- assets/example-metadata.json 1 KB
- assets/example-pipeline-inputs.json 1 KB
- references/env.md 2 KB
- references/request-response.md 4 KB
- references/testing.md 2 KB
- scripts/upload_to_dataset.py 31 KB
Versions
The database has no other skills named dify-knowledge-base-upload.
Identical content appears in 2 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version d918b0f0cb05 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100dify-knowledge-base-search
Dify dataset retrieve API for knowledge base chunk search/testing. Use when integrating or debugging Dify knowledge base retrieval requests, retrieval_model options, or response shaping. - 100dify
Dify is an open-source platform for building LLM apps such as chatbots, agents, workflows and RAG knowledge bases, and it exposes every published app as a REST API. Use when a user asks to self-host Dify with Docker Compose, configure Dify environment variables, call a Dify app… - 100dify-workflow
设计、生成并部署 Dify 应用 DSL (workflow/advanced-chat)。当用户要求构建 Dify 工作流、生成/更新 DSL 文件或通过 Admin API 部署到 Dify 实例时使用。 - 100validate-dify-dsl
Validates the core nodes of a Dify DSL YAML file (workflow / chatflow export) against graphon, the graph engine schemas Dify itself uses. Reports invalid fields per node. Scope is core nodes only — triggers, knowledge-retrieval, agent, human-input, edges, and graph structure are… - 100skills-to-dify-workflow
Converts Claude Code skill folders (SKILL.md + scripts/ + references/ + assets/) into Dify-importable workflow DSL (YAML files). Analyzes the entire skill directory structure to map scripts to code nodes, references to LLM prompts, and assets to templates. Also generates a React… - 100dify-chatflow-builder
用 Dify Chatflow / Workflow 搭建生产级多轮对话应用的方法与踩坑库,可配合对话逐步指导搭建。当用户说要搭一个 Dify 对话流、工作流或智能体,或需要设计状态机与多场景分诊、配置节点与变量、编写节点级提示词、排查运行报错、接入 HTTP / Swagger / MCP 接口与飞书、实现富媒体组件返回、优化响应延迟,或在对话流、工作流、Agent 三种形态之间选型时使用。