idea-evaluator
/idea-evaluator
Ocenia wkład, dowody i wykonalność pomysłu badawczego oraz zwraca decyzję: kontynuować, najpierw sprawdzić, odłożyć lub zatrzymać.
- Autor
- AfonsoZhang ↗
- Repozytorium
- auto-research-system ↗
- Licencja
- CC-BY-4.0
- Ostatnia zmiana repozytorium
- 02.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 12
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 02.11.2026, bez odwołań do wycofanych modeli i API.
CC-BY-4.0, otwarta.
Co robi i kiedy
Ocenia wkład, dowody i wykonalność pomysłu badawczego oraz zwraca decyzję: kontynuować, najpierw sprawdzić, odłożyć lub zatrzymać.
Oryginalny opis (EN)
Evaluate a research idea’s contribution, evidence, value and feasibility; identify a decisive next check and return Continue, Validate First, Park or Stop Current Version. Use for topic selection, candidate comparison, novelty/feasibility assessment or revising an existing idea evaluation.
Jak działa
- Scope
- Procedure
- Delivery
Podgląd SKILL.md
--- name: idea-evaluator description: "Evaluate a research idea’s contribution, evidence, value and feasibility; identify a decisive next check and return Continue, Validate First, Park or Stop Current Version. Use for topic selection, candidate comparison, novelty/feasibility assessment or revising an existing idea evaluation." license: CC-BY-4.0 --- # Idea Evaluator Apply the shared [working-language policy](../shared-references/research-autonomy.md#working-language) to internal work and user reports. Evaluate whether an idea is worth developing, what remains uncertain and which next step would resolve it. Read the [shared topic-selection policy](../shared-references/topic-selection-policy.md) first. It governs contribution scale, evidence, resources and the four decisions; this skill does not simulate an acceptance decision from a top-venue reviewer. ## Scope Use for a draft idea, candidate comparison, feasibility check or scope revision. For a small revision, read the existing evaluation and update affected judgments only. An existing paper review or writing task stays with the applicable writing skill; it does not restart topic selection. Brainstorming from scratch can use the installed brainstorming skills if within the requested task. ## Procedure ### 1. Clarify the contribution Restate the question, intended contribution and scientific/practical use. Include mechanism improvements, combinations, new settings, evaluation/resources, systematic analysis, failure boundaries, negative findings and informative replication. If the claim is vague, propose a precise interpretation and identify the missing detail; do not infer that an unclear pitch is an invalid idea. Apply [question value and hypothesis relationships](../shared-references/topic-selection-policy.md#question-value-and-hypothesis-relationships) to distinguish a substantive attribution question from a supporting ablation, and competing explanations from mechanisms that can coexist or form a causal chain. Use the actual research method to choose an assessment framework. For non-STEM work, consult `references/domain-evaluation-frameworks.md`; retain its relevant methodological checks, but use the shared evidence/decision rules rather than any inherited score or venue heuristic as an automatic rejection threshold. ### 2. Establish the delta and investigate blockers Read `references/fatal-flaws.md` as a checklist of possible problems. Record the important specific issues, evidence status and possible resolution. Do not turn their count into a decision or infer a fatal problem from missing information. Ground novelty judgments in actual retrieval. Use relevant keyword groups and follow closest work; reuse the publication records governed by [publication-policy.md](../shared-references/publication-policy.md). Read the relevant passages before judging method/result overlap. A different title or one differing axis alone is not enough: identify the meaningful added knowledge or capability. Report limited retrieval as limited retrieval, not proof of novelty. Separate invalid measurements, underpowered/inconclusive results and credible refutation. A baseline tie/loss alone cannot force rejection. Reliable evidence against a claimed mechanism requires revising that claim, followed by a separate assessment of any valuable negative finding or other remaining contribution. ### 3. Assess value and resources separately Explain who can use the contribution and what decision or understanding it changes. A narrow scientific audience is legitimate. For method ideas, optionally use the Higher/Faster/Stronger/Cheaper/Broader lenses in `references/five-dimensions.md`. They are not required achievements for an audit, replication or negative result. Do not require two high scores, SOTA, a minimum speedup or broad generality. Read `references/lifecycle-capability-matching.md` for resource planning. Specify the scientifically appropriate experiment first, identify the user's actual resources and gaps, and recommend how to acquire missing essentials or compare valid alternatives. Ask about material scope/cost choices. Do not constrain the research to the equipment currently available or infer capability from a category. ### 4. Identify the smallest decisive validation For a central uncertainty, state the claim, relevant baseline/control, observations that would support/refute it, an inconclusive outcome and the budget/stop condition. Reuse existing experiment-plan artifacts. Do not automatically run the experiment. Select replication and coverage according to the claim, not fixed counts. A pilot can justify stopping expenditure or revising the claim without rejecting a field. ### 5. Optional strategic upside Use `references/paradigm-shift-probe.md` when broader strategic exploration is useful or requested. Disruptive potential, competitive head start and novelty of framing are bonuses. Their absence never blocks a focused contribution. A draft conclusion is a hypothesis about what the work could establish, not invented data. ### 6. Decide Apply the shared four decisions: **CONTINUE**, **VALIDATE_FIRST**, **PARK**, **STOP_CURRENT_VERSION**. State the evidential reason and next action; do not translate scores or issue counts into verdicts. STOP_CURRENT_VERSION applies to the present claim/scope and needs the exact covered/refuted claim plus consideration of worthwhile remaining contributions. ## Delivery Scale the output to the request. A full evaluation may use: 1. Question, contribution type and candidate delta. 2. Closest works: publication/version status, compared claim, evidence locator, meaningful difference or unresolved comparison. 3. Research value, evidence strength and important limitations, separated. 4. Preferred experiment/resources, actual access and missing prerequisites. 5. Common decision and its reason. 6. Next decisive action and its continuation/stop/resumption conditions. Optional dimension ratings must distinguish measured evidence, mechanism-based expectation and unknown/not-applicable. They are not acceptance probabilities or gates. `references/worked-examples.md` illustrates field calibration and edge cases. Before delivery, check that negative judgments are sourced, unknowns stay unknown, attribution claims stay within what the comparisons identify, resource claims reflect known facts, and the decision follows the shared policy. Fix errors in the evaluation instead of punishing the research idea for a reporting error. Complete this call without automatically starting another skill, review, team or experiment.
Pliki w paczce (12)
- SKILL.md 6 KB
- references/domain-evaluation-frameworks.md 9 KB
- references/fatal-flaws.md 4 KB
- references/five-dimensions.md 3 KB
- references/lifecycle-capability-matching.md 3 KB
- references/paradigm-elephant.md 5 KB
- references/paradigm-examples.md 5 KB
- references/paradigm-first-principles.md 5 KB
- references/paradigm-hamming.md 5 KB
- references/paradigm-shift-probe.md 2 KB
- references/paradigm-technology-cycle.md 5 KB
- references/worked-examples.md 3 KB
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · AfonsoZhang/auto-research-system | 100 | CC-BY-4.0 |
| HKUSTDial/Supervisor-Skills | 100 | CC-BY-4.0 |
Migawka bazy z 13.11.2026 · wersja paczki f1a37dc7b689 (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 90novelty-gate
Sprawdza nowość i wykonalność pomysłu badawczego przed eksperymentem, szukając najbliższych prac, i proponuje najtańszy test. - 100sciatlas-idea-evaluate
Ocenia nowość, wykonalność i spójność pomysłu badawczego na podstawie wyników wyszukiwania prac w SciAtlas. - 90novelty-audit
Ocenia, czy główny wkład pomysłu badawczego jest nowy na tle najbliższych prac i sprawdza nakładanie się z literaturą. - 100novelty-check
Sprawdza nowość pomysłu badawczego na tle najnowszej literatury, zanim zacznie się jego wdrażanie. - 100novelty-check
Sprawdza nowość pomysłu badawczego na tle najnowszej literatury, zanim zacznie się jego realizację. - 100novelty-check
Sprawdza nowość pomysłu badawczego na tle najnowszej literatury, zanim zacznie się jego wdrażanie.