Przejdź do treści
Biblioteka skilli
Nauka i badania · Analiza

source-evaluation

/source-evaluation

Ocenia wiarygodność źródeł, wykrywa spam SEO i sprawdza kondycję bibliotek oraz repozytoriów przed ich wyborem.

Repozytorium
fractional-cto ↗
Licencja
MIT
Ostatnia zmiana repozytorium
25.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
3
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 25.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.

Co robi i kiedy

Ocenia wiarygodność źródeł, wykrywa spam SEO i sprawdza kondycję bibliotek oraz repozytoriów przed ich wyborem.

Oryginalny opis (EN)

This skill should be used when evaluating source credibility, deciding which search results to trust, choosing between search providers, detecting SEO spam or content farms, selecting domain-specific sources (academic, medical, legal, technical), evaluating software packages or libraries, comparing tools or technologies, assessing GitHub repo health, checking adoption metrics, or when research quality depends on retrieval quality. Covers the source credibility taxonomy (T1-T6 tiers), CRAAP framework adaptation, multi-provider search strategy, artifact evaluation framework (health/adoption/auth

Jak działa

  1. Source Credibility Tiers
  2. The CRAAP Framework — Automated Signals
  3. Multi-Provider Search Strategy
  4. SEO Spam Detection
  5. Artifact Evaluation
  6. Retrieval Best Practices
  7. Retrieval Anti-Patterns
  8. Reference Files
Sekcje z nagłówków SKILL.md, w oryginale · 153 linii, 1 520 słów
Podgląd SKILL.md
---
name: source-evaluation
description: "This skill should be used when evaluating source credibility, deciding which search results to trust, choosing between search providers, detecting SEO spam or content farms, selecting domain-specific sources (academic, medical, legal, technical), evaluating software packages or libraries, comparing tools or technologies, assessing GitHub repo health, checking adoption metrics, or when research quality depends on retrieval quality. Covers the source credibility taxonomy (T1-T6 tiers), CRAAP framework adaptation, multi-provider search strategy, artifact evaluation framework (health/adoption/authority signals for packages, repos, APIs, standards, technologies), and source quality anti-patterns."
version: 1.2.0
---

# Source Evaluation

Source quality is the primary bottleneck in research agent pipelines. Research on deep research agent trajectories found that over 57% of source errors occur in early retrieval stages, where initial fabrication acts as the primary catalyst for cascading downstream errors (arXiv 2601.22984). A single bad source in the first retrieval round contaminates the entire research trajectory.

## Source Credibility Tiers

Every source encountered during research falls into one of six tiers. Always prefer higher-tier sources and cite the tier when reporting findings.

| Tier | Source Type | Examples | Trust Level |
|------|-----------|----------|-------------|
| **T1 — Primary** | Peer-reviewed journals, official specs, primary datasets | Nature, Science, IEEE, IETF RFCs, W3C specs | Highest |
| **T2 — Institutional** | Government agencies, established research institutions | NIH, WHO, NIST, ACM Digital Library | High |
| **T3 — Expert** | Named expert blogs, conference proceedings, major tech engineering blogs | Anthropic blog, Google Research, NeurIPS/ICML papers | Moderate-High |
| **T4 — Quality Editorial** | Major publications with editorial review | MIT Technology Review, Ars Technica, The Verge | Moderate |
| **T5 — Community** | Well-moderated forums, high-reputation answers | Stack Overflow (high-score), GitHub discussions | Low-Moderate |
| **T6 — Unverified** | Content farms, SEO-optimized articles, anonymous posts, AI-generated content | Medium listicles, affiliate blogs, uncredited tutorials | Do not cite |

**Rule:** Never cite T6 sources. Prefer T1-T3 for factual claims. Use T4-T5 for context and community consensus only.

## The CRAAP Framework — Automated Signals

Adapted from the CRAAP framework (CSU Chico), five dimensions for evaluating sources:

| Dimension | What to Check | Red Flags |
|-----------|--------------|-----------|
| **Currency** | Publication date, last-modified headers | No date visible, information predates major changes in the field |
| **Relevance** | Does it address the specific research question? | Tangential coverage, keyword-stuffed but shallow |
| **Authority** | Who published it? Credentials? | Anonymous author, no institutional affiliation, no citations |
| **Accuracy** | Are claims sourced? Can they be verified? | No inline citations, contradicts known facts, round numbers without source |
| **Purpose** | Is it informing, selling, or persuading? | High ad density, affiliate links, promotional language |

Note: CRAAP evaluates surface features. Use it as an initial filter, not the sole credibility signal (Stanford research found reliance on CRAAP alone makes researchers susceptible to misinformation).

## Multi-Provider Search Strategy

Different search providers excel in different domains. Route queries to the appropriate provider:

| Provider | Best For | Limitations |
|----------|---------|-------------|
| **WebSearch (general)** | Broad topics, recent events, technical documentation | May surface SEO-optimized content |
| **arXiv / Semantic Scholar** | Academic ML/AI research, preprints | Not peer-reviewed, may be superseded |
| **PubMed** | Medical, biomedical, clinical research | Limited to biomedical domain |
| **Official documentation** | API specs, library usage, framework guides | May lag behind actual behavior |
| **GitHub** | Code examples, implementation patterns, issue discussions | Quality varies widely |

**Strategy:** Start with domain-appropriate providers. Use general web search to fill gaps. Cross-reference findings across multiple providers when possible.

## SEO Spam Detection

Red flags that indicate low-quality, SEO-optimized content:

- **Listicle format** with no depth ("Top 10 ways to...")
- **Keyword stuffing** — the search term appears unnaturally often
- **No author attribution** or author has no verifiable expertise
- **High ad-to-content ratio**
- **Recycled/syndicated content** appearing verbatim across multiple domains
- **AI-generated markers** — generic phrasing, lack of specific examples, overly smooth prose
- **Affiliate links** embedded throughout

When a source triggers 2+ red flags, discard it and search for a higher-quality alternative.

## Artifact Evaluation

Research often involves evaluating non-content artifacts — packages, tools, technologies, standards, organizations. These require different signals than content sources. Every artifact has three signal dimensions:

| Dimension | What It Measures | Key Question |
|-----------|-----------------|--------------|
| **Health** | Is it alive and maintained? | When was the last meaningful activity? |
| **Adoption** | Does anyone actually use it? | What are the real usage numbers? |
| **Authority** | Who's behind it and are they credible? | Is this backed by a credible entity? |

### Artifact Types and Key Signals

| Artifact Type | Health Signals | Adoption Signals | Authority Signals |
|---------------|---------------|-----------------|-------------------|
| **Software packages** | Last commit, release frequency, open issue response time | Downloads (npm weekly, PyPI monthly), dependents count | Maintainer reputation, organizational backing, license |
| **GitHub repos** | Commit frequency, PR merge time, stale issue ratio | Stars, forks, contributor count | Bus factor (>1 critical), corporate sponsor, notable users |
| **APIs/Services** | Uptime history, changelog frequency, deprecation notices | Customer logos, integration count, community size | Company funding, revenue stability, enterprise adoption |
| **Standards/Specs** | Last revision date, errata activity | Implementation count, conformance test suites | Standards body status (draft/proposed/standard), industry backing |
| **Technologies** | Release cadence, roadmap activity, CVE response time | Stack Overflow survey ranking, job postings, TIOBE/RedMonk index | Backing organization, governance model, ecosystem size |
| **Architectural patterns** | Recent case studies, active community discussion | Industry adoption breadth, conference talk frequency | Documented at-scale deployments, known failure case studies |
| **People/Authors** | Recent publication activity | Citation count, h-index, follower count | Institutional affiliation, industry role, peer recognition |
| **Companies/Orgs** | Recent funding, hiring activity, product releases | Revenue, customer count, market share | Investor quality, leadership track record, industry awards |
| **Communities** | Messages per week, new member rate | Member count, active member ratio | Moderation quality, notable members, signal-to-noise ratio |
| **Datasets/Benchmarks** | Last update, known issues addressed | Citation count, leaderboard participation | Creator credentials, methodology transparency, peer review |
| **Claims/Statistics** | Date of study, methodology recency | Citation count, replication status | Funding source, sample size, peer review, original source |

### Programmatic Stat Verification

When evaluating artifacts, use APIs for exact stats instead of search snippets:

| Ecosystem | API Command | Returns |
|-----------|-------------|---------|
| **GitHub** | `gh api repos/{owner}/{name}` | Stars, forks, license, language, last update, open issues |
| **GitHub releases** | `gh api repos/{owner}/{name}/releases/latest` | Latest version tag, release date |
| **npm** | `curl api.npmjs.org/downloads/point/last-week/{pkg}` | Exact weekly downloads |
| **PyPI** | `curl pypistats.org/api/packages/{pkg}/recent` | Recent download counts |
| **crates.io** | `curl crates.io/api/v1/crates/{crate}` | Downloads, version, recent downloads |
| **RubyGems** | `curl rubygems.org/api/v1/gems/{gem}.json` | Downloads, latest version |
| **Maven** | Search `site:mvnrepository.com {artifact}` | Usage stats page |

These APIs return ground truth. Search snippets for these stats are unreliable.

### Red Flags by Artifact Type

**Software packages and repos:**
- Last commit >6 months ago with open issues unanswered
- Fewer than 50 stars with no organizational backing
- Single maintainer (bus factor = 1) for critical dependency
- No tests, no CI, no changelog
- License incompatible with intended use

**Technologies and standards:**
- No major release in 12+ months
- Declining Stack Overflow activity trend
- Abandoned by original backing organization
- No conformance test suite (for standards)

**Claims and statistics:**
- No original source cited (circular citation)
- Study funded by party with commercial interest in the outcome
- Sample size <100 for quantitative claims
- No methodology description

**General rule:** When an artifact triggers 2+ red flags, flag it explicitly in the research output. Do not recommend it without noting the risks.

For detailed per-artifact-type evaluation guides and how to check each signal programmatically, consult `references/artifact-signals.md`.

## Retrieval Best Practices

1. **Front-load quality** — The first retrieval round disproportionately determines research quality due to the saturation bottleneck (agents fixate on early results). Start with high-authority sources.
2. **Search iteratively** — Refine search queries based on initial results. First search identifies terminology; second search uses domain-specific terms.
3. **Diversify sources** — Do not rely on a single provider or a single source for any claim. Cross-reference across independent sources.
4. **Verify fetched content** — After WebFetch, scan the content for authority signals (author credentials, citations, publication venue) before incorporating.
5. **Track provenance** — Record which source produced which claim. This metadata is essential for citation and for tracing errors.

## Retrieval Anti-Patterns

| Anti-Pattern | Problem | Fix |
|-------------|---------|-----|
| **Single-provider dependency** | All searches go through one provider | Route by domain; use multiple providers |
| **First-result trust** | Accepting the top search result without evaluation | Evaluate credibility tier before incorporating |
| **Equal credibility** | Treating a blog post the same as a journal paper | Apply tier system; weight higher-tier sources |
| **Ignoring retrieval failures** | Silent fallback when search returns nothing useful | Log the gap; try alternative queries or providers |
| **Breadth without depth** | Fetching 20 URLs but reading none carefully | Fetch fewer sources; read each thoroughly |

## Reference Files

For detailed provider comparison, domain-specific source guides, and artifact evaluation:
- **`references/provider-comparison.md`** — Detailed comparison of search providers with API specifics, rate limits, and optimal use cases
- **`references/artifact-signals.md`** — Per-artifact-type evaluation guides with health/adoption/authority thresholds, how to check each signal, and the quick evaluation checklist
Pliki w paczce (3)
  • SKILL.md 11 KB
  • references/artifact-signals.md 11 KB
  • references/provider-comparison.md 6 KB

Wersje

RepozytoriumWynikLicencja
Ta strona · oborchers/fractional-cto100MIT
Amey-Thakur/AI-SKILLS90MIT

Migawka bazy z 13.11.2026 · wersja paczki a0a9c0d55e9f (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100source-evaluation-bibliography
    Ocenia jakość źródeł testem CRAAP, buduje adnotowane bibliografie i prowadzi kolekcje referencji, w tym klasyfikację źródeł i dokumentację badań.
  • 100web-researcher
    Bada temat online, zbiera informacje z wielu źródeł i ocenia ich wiarygodność.
  • 100source-credibility-assessment
    Ocenia, ile wagi zasługuje źródło przed użyciem twierdzenia, sprawdzając wiarygodność, pochodzenie, aktualność i możliwe konflikty interesów.
  • 90validating-source-credibility
    Weryfikuje wiarygodność źródeł w pracy badawczej, tak aby wynik był możliwy do prześledzenia, niezależnie sprawdzalny i bezpieczny do przekazania innemu agentowi.
  • 100research-claim-map
    Sprawdza twierdzenia przez triangulację źródeł, ocenia jakość dowodów i wiarygodność źródeł oraz podaje wynik z poziomem pewności.
  • 100source-research
    Szybko bada temat, wyszukuje i ocenia źródła oraz tworzy zwięzłe podsumowanie z cytatami.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.