llm-fallback-chains
/llm-fallback-chains
Projektuje łańcuchy zapasowych dostawców LLM i stopniowe obniżanie jakości, gdy usługa AI jest niedostępna.
- Autor
- latestaiagents ↗
- Repozytorium
- agent-skills ↗
- Licencja
- MIT
- Ostatnia zmiana repozytorium
- 04.11.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 04.11.2026, bez odwołań do wycofanych modeli i API.
MIT, otwarta: można używać i zmieniać, z zachowaniem informacji o autorze.
Co robi i kiedy
Projektuje łańcuchy zapasowych dostawców LLM i stopniowe obniżanie jakości, gdy usługa AI jest niedostępna.
Oryginalny opis (EN)
Use this skill when implementing fallback strategies for LLM applications. Activate when the user needs graceful degradation for AI services, multi-provider failover, handling LLM outages, or building resilient AI systems.
Jak działa
- When to Use
- Fallback Architecture
- Fallback Chain Implementation
- Multi-Provider Setup
- Graceful Degradation Strategies
- Health Monitoring
- Alerting on Fallbacks
- Best Practices
Podgląd SKILL.md
---
name: llm-fallback-chains
description: |
Use this skill when implementing fallback strategies for LLM applications. Activate when the user needs
graceful degradation for AI services, multi-provider failover, handling LLM outages, or building
resilient AI systems.
---
# LLM Fallback Chains
Build resilient AI systems that gracefully handle failures across providers and models.
## When to Use
- Primary LLM provider experiences outages
- Need to maintain service during API issues
- Building high-availability AI systems
- Implementing cost-quality tradeoffs
- Managing multi-provider AI infrastructure
## Fallback Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Request Handler │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Fallback Chain │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Primary │──►│Fallback │──►│Fallback │──►│ Cached │ │
│ │ Model │ │ Model 1 │ │ Model 2 │ │Response │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## Fallback Chain Implementation
```typescript
interface FallbackProvider {
name: string;
model: string;
client: LLMClient;
priority: number;
healthCheck: () => Promise<boolean>;
isAvailable: boolean;
lastFailure?: Date;
failureCount: number;
}
interface FallbackConfig {
maxRetries: number;
retryDelayMs: number;
circuitBreakerThreshold: number;
circuitBreakerResetMs: number;
}
class FallbackChain {
private providers: FallbackProvider[] = [];
private config: FallbackConfig;
constructor(config: FallbackConfig) {
this.config = config;
}
addProvider(provider: Omit<FallbackProvider, 'isAvailable' | 'failureCount'>): void {
this.providers.push({
...provider,
isAvailable: true,
failureCount: 0
});
this.providers.sort((a, b) => a.priority - b.priority);
}
async complete(params: CompletionParams): Promise<CompletionResponse> {
const availableProviders = this.providers.filter(p =>
p.isAvailable || this.shouldRetryProvider(p)
);
if (availableProviders.length === 0) {
throw new Error('All providers unavailable');
}
let lastError: Error | null = null;
for (const provider of availableProviders) {
try {
console.log(`Trying provider: ${provider.name}`);
const response = await this.executeWithTimeout(provider, params);
// Success - reset failure count
provider.failureCount = 0;
provider.isAvailable = true;
return response;
} catch (error) {
lastError = error as Error;
console.error(`Provider ${provider.name} failed:`, error);
this.recordFailure(provider);
if (!this.isRetryableError(error)) {
throw error; // Don't try other providers for non-retryable errors
}
}
}
throw lastError || new Error('All providers failed');
}
private shouldRetryProvider(provider: FallbackProvider): boolean {
if (!provider.lastFailure) return true;
const timeSinceFailure = Date.now() - provider.lastFailure.getTime();
return timeSinceFailure > this.config.circuitBreakerResetMs;
}
private recordFailure(provider: FallbackProvider): void {
provider.failureCount++;
provider.lastFailure = new Date();
if (provider.failureCount >= this.config.circuitBreakerThreshold) {
provider.isAvailable = false;
console.warn(`Circuit breaker opened for ${provider.name}`);
}
}
private isRetryableError(error: any): boolean {
// Rate limits, timeouts, and server errors are retryable
if (error.status === 429) return true;
if (error.status >= 500) return true;
if (error.code === 'ETIMEDOUT' || error.code === 'ECONNRESET') return true;
return false;
}
private async executeWithTimeout(
provider: FallbackProvider,
params: CompletionParams
): Promise<CompletionResponse> {
const timeoutMs = 30000;
return Promise.race([
provider.client.complete({ ...params, model: provider.model }),
new Promise<never>((_, reject) =>
setTimeout(() => reject(new Error('Timeout')), timeoutMs)
)
]);
}
}
```
## Multi-Provider Setup
```typescript
// Model IDs come from config. Set each to a current ID from the provider's
// models page; IDs are retired over time, so don't hard-code them in logic.
const MODELS = {
anthropicMain: process.env.ANTHROPIC_MAIN_MODEL!,
anthropicSmall: process.env.ANTHROPIC_SMALL_MODEL!,
openaiMain: process.env.OPENAI_MAIN_MODEL!,
openaiSmall: process.env.OPENAI_SMALL_MODEL!,
};
const fallbackChain = new FallbackChain({
maxRetries: 3,
retryDelayMs: 1000,
circuitBreakerThreshold: 5,
circuitBreakerResetMs: 60000
});
// Primary: main Claude model
fallbackChain.addProvider({
name: 'anthropic-sonnet',
model: MODELS.anthropicMain,
client: new AnthropicClient(process.env.ANTHROPIC_API_KEY!),
priority: 1,
healthCheck: () => anthropicHealthCheck()
});
// Fallback 1: main OpenAI model
fallbackChain.addProvider({
name: 'openai-main',
model: MODELS.openaiMain,
client: new OpenAIClient(process.env.OPENAI_API_KEY!),
priority: 2,
healthCheck: () => openaiHealthCheck()
});
// Fallback 2: smaller Claude model (cheaper, faster)
fallbackChain.addProvider({
name: 'anthropic-small',
model: MODELS.anthropicSmall,
client: new AnthropicClient(process.env.ANTHROPIC_API_KEY!),
priority: 3,
healthCheck: () => anthropicHealthCheck()
});
// Fallback 3: smaller OpenAI model
fallbackChain.addProvider({
name: 'openai-small',
model: MODELS.openaiSmall,
client: new OpenAIClient(process.env.OPENAI_API_KEY!),
priority: 4,
healthCheck: () => openaiHealthCheck()
});
```
## Graceful Degradation Strategies
### Strategy 1: Quality Degradation
```typescript
interface DegradationLevel {
level: number;
model: string;
maxTokens: number;
features: string[];
}
const degradationLevels: DegradationLevel[] = [
// Model IDs from config (see MODELS above); LARGE_MODEL_ID is your top tier
{ level: 0, model: LARGE_MODEL_ID, maxTokens: 4000, features: ['full', 'reasoning', 'creativity'] },
{ level: 1, model: MODELS.anthropicMain, maxTokens: 2000, features: ['full', 'reasoning'] },
{ level: 2, model: MODELS.anthropicSmall, maxTokens: 1000, features: ['basic'] },
{ level: 3, model: MODELS.openaiSmall, maxTokens: 500, features: ['minimal'] },
];
class GracefulDegrader {
private currentLevel = 0;
async execute(task: string): Promise<{ response: string; degraded: boolean }> {
for (let level = this.currentLevel; level < degradationLevels.length; level++) {
try {
const config = degradationLevels[level];
const response = await this.callModel(config, task);
return {
response,
degraded: level > 0
};
} catch (error) {
console.log(`Level ${level} failed, degrading...`);
this.currentLevel = level + 1;
}
}
// Final fallback: cached/static response
return {
response: this.getCachedResponse(task),
degraded: true
};
}
private getCachedResponse(task: string): string {
return 'We are experiencing high demand. Please try again shortly.';
}
}
```
### Strategy 2: Feature-Based Fallback
```typescript
interface FeatureConfig {
name: string;
requiredCapabilities: string[];
fallbackBehavior: 'skip' | 'simplify' | 'cache';
}
class FeatureFallback {
private featureConfigs: FeatureConfig[] = [
{ name: 'streaming', requiredCapabilities: ['sse'], fallbackBehavior: 'skip' },
{ name: 'functionCalling', requiredCapabilities: ['tools'], fallbackBehavior: 'simplify' },
{ name: 'imageAnalysis', requiredCapabilities: ['vision'], fallbackBehavior: 'skip' },
];
async executeWithFeatures(
task: string,
requestedFeatures: string[],
provider: LLMProvider
): Promise<ExecutionResult> {
const supportedFeatures = provider.capabilities;
const enabledFeatures: string[] = [];
const skippedFeatures: string[] = [];
for (const feature of requestedFeatures) {
const config = this.featureConfigs.find(f => f.name === feature);
if (!config) continue;
const supported = config.requiredCapabilities.every(
cap => supportedFeatures.includes(cap)
);
if (supported) {
enabledFeatures.push(feature);
} else {
skippedFeatures.push(feature);
console.log(`Feature ${feature} unavailable, using fallback: ${config.fallbackBehavior}`);
}
}
return this.execute(task, enabledFeatures, skippedFeatures);
}
}
```
### Strategy 3: Cached Response Fallback
```typescript
class CacheFallback {
private cache: ResponseCache;
async executeWithCache(
task: string,
chain: FallbackChain
): Promise<{ response: string; fromCache: boolean }> {
try {
// Try live response first
const response = await chain.complete({ messages: [{ role: 'user', content: task }] });
// Cache successful responses
await this.cache.set(task, response.content);
return { response: response.content, fromCache: false };
} catch (error) {
// Try cache on failure
const cached = await this.cache.get(task);
if (cached) {
console.log('Using cached response due to API failure');
return { response: cached, fromCache: true };
}
throw error;
}
}
}
```
## Health Monitoring
```typescript
class ProviderHealthMonitor {
private healthStatus = new Map<string, {
isHealthy: boolean;
lastCheck: Date;
consecutiveFailures: number;
latencyMs: number;
}>();
async checkHealth(provider: FallbackProvider): Promise<boolean> {
const startTime = Date.now();
try {
const healthy = await provider.healthCheck();
const latency = Date.now() - startTime;
this.healthStatus.set(provider.name, {
isHealthy: healthy,
lastCheck: new Date(),
consecutiveFailures: healthy ? 0 : (this.healthStatus.get(provider.name)?.consecutiveFailures || 0) + 1,
latencyMs: latency
});
return healthy;
} catch (error) {
this.healthStatus.set(provider.name, {
isHealthy: false,
lastCheck: new Date(),
consecutiveFailures: (this.healthStatus.get(provider.name)?.consecutiveFailures || 0) + 1,
latencyMs: -1
});
return false;
}
}
// Run periodic health checks
startMonitoring(providers: FallbackProvider[], intervalMs: number = 30000): void {
setInterval(async () => {
for (const provider of providers) {
await this.checkHealth(provider);
}
}, intervalMs);
}
getStatus(): Record<string, { healthy: boolean; latency: number }> {
const status: Record<string, { healthy: boolean; latency: number }> = {};
for (const [name, data] of this.healthStatus) {
status[name] = {
healthy: data.isHealthy,
latency: data.latencyMs
};
}
return status;
}
}
```
## Alerting on Fallbacks
```typescript
class FallbackAlerts {
async onFallback(
primaryProvider: string,
fallbackProvider: string,
error: Error
): Promise<void> {
// Log the event
console.warn(`Fallback activated: ${primaryProvider} -> ${fallbackProvider}`);
console.warn(`Reason: ${error.message}`);
// Send alert if primary fails repeatedly
const recentFallbacks = await this.getRecentFallbacks(primaryProvider);
if (recentFallbacks > 5) {
await this.sendAlert({
severity: 'high',
message: `Primary provider ${primaryProvider} has failed ${recentFallbacks} times in the last hour`,
action: 'Investigate provider status'
});
}
}
async onAllProvidersFailed(): Promise<void> {
await this.sendAlert({
severity: 'critical',
message: 'All LLM providers are failing - service degraded',
action: 'Immediate investigation required'
});
}
}
```
## Best Practices
1. **Order by priority** - Best quality first, cheapest last
2. **Use circuit breakers** - Don't hammer failing providers
3. **Monitor health proactively** - Don't wait for failures
4. **Cache where possible** - Final fallback for availability
5. **Alert on degradation** - Know when you're running degraded
6. **Test failover regularly** - Chaos engineering for AI
7. **Document SLAs** - Know what you're guaranteeing
Wersje
W bazie nie ma innych skilli o nazwie llm-fallback-chains.
Migawka bazy z 13.11.2026 · wersja paczki 956146c89a7e (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 0 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100openrouter-fallback-config
Konfiguruje łańcuchy zapasowych modeli w OpenRouter dla wysokiej dostępności, przy budowie odpornych na awarie systemów LLM z przełączaniem awaryjnym. - 100llm-gateway-and-routing
Kieruje ruch LLM aplikacji między dostawcami: podział żądań na poziomy, łańcuchy awaryjne, przełączanie świadome kondycji, planowanie wycofywania modeli i odpowiedzi ograniczone schematem. - 100llm-gateway-and-routing
Kieruje ruch LLM aplikacji między dostawcami: podział żądań na poziomy, łańcuchy awaryjne, przełączanie świadome kondycji, planowanie wycofywania modeli i odpowiedzi ograniczone schematem. - 90add-llm-fallbacks
Uodparnia aplikację LLM na błędy dostawców, limity i przestoje, dodając ponawianie, timeouty i zapasowe modele. - 100langchain
Pomaga budować aplikacje oparte na LLM we frameworku LangChain: agentów ReAct, łańcuchy, RAG, wywoływanie narzędzi, pamięć i wyszukiwanie w bazach wektorowych dla wielu dostawców modeli. - 100langchain-core-workflow-a
Buduje łańcuchy i prompty LangChain dla ustrukturyzowanych przepływów LLM: szablony promptów, łańcuchy LCEL i sekwencyjne potoki przetwarzania.