backtesting-configuration
/backtesting-configuration
Konfiguruje parametry TimeSeriesFold do backtestingu według scenariusza wdrożenia, mapując wymagania biznesowe (częstotliwość retrenowania, horyzont prognozy, budżet danych) na strategię walidacji krzyżowej.
- Autor
- skforecast ↗
- Repozytorium
- skforecast-ai ↗
- Licencja
- Apache-2.0
- Ostatnia zmiana repozytorium
- 16.10.2026
Ocena i bezpieczeństwo
- Pliki w paczce
- 1
- Skrypty
- 0
- Komentarze HTML
- 0
- Adresy URL w SKILL.md
- 0
Brak sygnałów. Repozytorium zmienione 16.10.2026, bez odwołań do wycofanych modeli i API.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
Co robi i kiedy
Konfiguruje parametry TimeSeriesFold do backtestingu według scenariusza wdrożenia, mapując wymagania biznesowe (częstotliwość retrenowania, horyzont prognozy, budżet danych) na strategię walidacji krzyżowej.
Oryginalny opis (EN)
Configures TimeSeriesFold parameters for backtesting based on deployment scenarios. Maps business requirements (retraining frequency, forecast horizon, data budget) to cross-validation strategy parameters. Use when the user describes how they plan to deploy or evaluate a model.
Jak działa
- When to Use
- Stop Conditions
- TimeSeriesFold Parameters
- Constraints
- Business Scenario Mapping
- Examples
Podgląd SKILL.md
---
name: backtesting-configuration
description: >
Configures TimeSeriesFold parameters for backtesting based on deployment
scenarios. Maps business requirements (retraining frequency, forecast
horizon, data budget) to cross-validation strategy parameters.
Use when the user describes how they plan to deploy or evaluate a model.
---
# Backtesting Configuration
## When to Use
Use this skill to translate a deployment scenario (retraining cadence, forecast
horizon, data budget, ingestion delay) into `TimeSeriesFold` parameters.
`TimeSeriesFold` is the cross-validation strategy passed via the `cv` argument to
`backtesting_forecaster` (and its multi-series / stats variants) and to the
hyperparameter search functions.
```python
from skforecast.model_selection import backtesting_forecaster, TimeSeriesFold
cv = TimeSeriesFold(
steps=7, # forecast horizon
initial_train_size=365, # first training window (required)
refit=False, # train once (default)
fixed_train_size=True, # rolling window (default)
gap=0,
)
metric, predictions = backtesting_forecaster(
forecaster=forecaster,
y=data['target'],
cv=cv,
metric='mean_absolute_error',
)
```
For fast hyperparameter tuning where multi-step realism is not required, use
`OneStepAheadFold` instead: it validates one step ahead (no recursive
prediction), so it is much faster but less representative of multi-step
performance. Use `TimeSeriesFold` for realistic multi-step backtesting.
### Related skills
- **Prerequisite**: `forecasting-single-series` / `forecasting-multiple-series` (have a fitted forecaster before backtesting)
- **Alongside**: `metric-selection` (choose the metric(s) backtesting reports)
- **Alongside**: `hyperparameter-optimization` (the same `cv` object drives the search functions)
- **Next**: `prediction-intervals` (add `interval=` / `interval_method=` to backtest uncertainty)
## Stop Conditions
Scan before writing code. Each row lists a rule, the symptom when it is broken, and the recovery. Full pitfall catalog: the `troubleshooting-common-errors` skill.
| Rule | Symptom | Recovery |
|------|---------|----------|
| `initial_train_size` must be provided (the API default is `None`, which only works when reusing an already-fitted forecaster) | `ValueError` / no training in the first fold | Pass an int, date string, or Timestamp, e.g. `initial_train_size=len(y) - 100` |
| `initial_train_size` does not accept a float fraction | `ValueError`: must be int, date string, Timestamp, or None | Convert fractions to an int, e.g. `int(len(data) * 0.7)` |
| Configuration must yield at least 2 folds | Single-fold or empty backtest | Ensure `initial_train_size + gap + 2 * steps <= n_observations` |
| `refit=True` retrains every fold (slow path); the default is `refit=False` | Backtest much slower than expected | Use `refit=False` or an int cadence (e.g. `refit=7`) unless per-fold retraining is required |
| `gap`, `steps`, and `fold_stride` count observations, not calendar time | Off-by-frequency delays | Convert the delay to the series frequency (e.g. 2 days at daily freq = `gap=2`) |
## TimeSeriesFold Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `initial_train_size` | int, str, pd.Timestamp | None (required) | Observations for initial training. Int = count, str/Timestamp = last training date. A common starting point is `int(len(data) * 0.7)`. |
| `refit` | bool, int | False | Refit every fold (True), train once (False, default), or every n folds (int). |
| `fixed_train_size` | bool | True | Rolling fixed window (True, default) vs expanding window (False). |
| `gap` | int | 0 | Observations between training end and test start. |
| `fold_stride` | int, None | None (= steps) | Distance between consecutive test set starts. |
| `skip_folds` | int, list, None | None | Skip folds to reduce compute. Int = keep every n-th. |
| `allow_incomplete_fold` | bool | True | Allow final fold with fewer observations than steps. |
## Constraints
- Must produce **at least 2 folds**: `initial_train_size + gap + 2 * steps <= n_observations`
- **Hard minimum**: `initial_train_size` must be greater than `forecaster.window_size`, the number of past observations needed to build one training row (`max(max_lag, max_size_window_features)` plus the `differentiation` order). Direct forecasters (`ForecasterDirect`, `ForecasterDirectMultiVariate`, `ForecasterRnn`) need at least `window_size + steps`, using the forecaster's `steps`. In multi-series forecasters, each series with data in the first window needs more than `window_size` observations there.
- Forecasters without lags: `ForecasterEquivalentDate` has `window_size = offset * n_offsets` (same rule); in `ForecasterStats` the estimator sets the minimum (`window_size` is fixed at 1); `ForecasterFoundation` accepts any value ≥ 1 (`context_length` is a maximum, not a minimum).
- **Recommended**: leave a margin above the hard minimum so the model has enough rows to learn from (e.g. two or more seasonal cycles), while keeping at least 2 folds
- `gap` simulates real-world delay between data availability and forecast usage
- When `fixed_train_size=True`, the window rolls forward (oldest data discarded). Use for concept drift or when old data is less relevant.
- When `fixed_train_size=False`, the window expands (all history retained). Use when more data always helps.
## Business Scenario Mapping
### Retraining frequency
| Scenario | Configuration |
|----------|--------------|
| Retrain every time new data arrives | `refit=True` |
| Retrain weekly (with daily forecasts, steps=1) | `refit=7` |
| Retrain monthly (with daily forecasts, steps=7) | `refit=4` (every 4 folds × 7 steps ≈ monthly) |
| Never retrain / train once, evaluate across time | `refit=False` (`fixed_train_size` has no effect without refit) |
### Data freshness vs volume
| Scenario | Configuration |
|----------|--------------|
| Recent data more relevant (concept drift) | `fixed_train_size=True` |
| All historical data valuable | `fixed_train_size=False` (expanding) |
| Limited compute budget | `refit=False` or `skip_folds=3` (keep every 3rd fold) |
### Deployment gap
| Scenario | Configuration |
|----------|--------------|
| Real-time predictions (no delay) | `gap=0` |
| 1-day delay between data collection and forecast | `gap=1` (if freq=daily) |
| Forecast must be ready before weekend (2-day gap) | `gap=2` |
### Initial training size
| Scenario | Configuration |
|----------|--------------|
| Default (balanced) | `int(len(data) * 0.7)` |
| Maximize evaluation coverage | Hard minimum (`window_size + 1`; `window_size + steps` for direct) plus a learning margin, see Constraints |
| Maximize training data | `n_observations - gap - 2 * steps` (minimum 2 folds) |
| Start from specific date | `initial_train_size="2023-01-01"` |
| Conservative (large training set) | `int(len(data) * 0.8)` |
### Fold stride (test set overlap)
| Scenario | Configuration |
|----------|--------------|
| Non-overlapping evaluation (default) | `fold_stride=None` (equals steps) |
| Sliding window evaluation (1-step shift) | `fold_stride=1` |
| Sparse evaluation (save compute) | `fold_stride=steps * 2` |
## Examples
### "I retrain my model every Monday and forecast the next 7 days"
```
refit = True
fixed_train_size = False # Keep all history
gap = 0 # No delay
fold_stride = None # Non-overlapping weeks
```
### "I want to simulate deploying once and seeing how the model degrades"
```
refit = False # Train once; fixed_train_size has no effect without refit
gap = 0
```
### "There's a 2-day lag between data ingestion and when forecasts are needed"
```
gap = 2
refit = True
```
### "I have limited compute — evaluate every other week"
```
refit = True
skip_folds = 2 # Keep every 2nd fold
```
### "I want maximum evaluation coverage with a 12-step horizon"
```
# Hard minimum: forecaster.window_size + 1 (window_size + 12 for ForecasterDirect with steps=12)
initial_train_size = forecaster.window_size + margin # margin: rows to learn from, e.g. 2+ seasonal cycles
refit = False # Faster; trains once, fixed_train_size has no effect
fold_stride = 1 # Sliding window (many overlapping folds)
```
Wersje
| Repozytorium | Wynik | Licencja |
|---|---|---|
| Ta strona · skforecast/skforecast-ai | 100 | Apache-2.0 |
| skforecast/skforecast | 100 | BSD-3-Clause |
Identyczna treść występuje jeszcze w 1 innych folderach na GitHubie. Tu pokazujemy jedną, najlepiej ocenioną kopię.
Migawka bazy z 13.11.2026 · wersja paczki 424c061094ed (skrót SHA-256 całej paczki)
Pobrania w ostatnich 30 dniach: 1 · ranking
Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.
Podobne skille
- 100backtesting-py-oracle
Konfiguruje backtesting.py do walidacji wyroczni SQL i backtestingu wzorców na range barach. - 90backtesting-frameworks
Stosuje frameworki backtestingu w powtarzalnych lokalnych przepływach danych, z jawnymi założeniami i walidacją. - 100metric-selection
Pomaga dobrać miary oceny dla zadania prognozowania według typu prognozy, charakteru danych i agregacji wielu serii, w tym parametr metric w backtestingu i wyszukiwaniu hiperparametrów. - 100metric-selection
Pomaga wybrać metrykę oceny prognoz dla zadania prognozowania na podstawie typu prognozy i danych. - 100backtesting-alt-data-strategies-with-realistic-availability-lag
Przy backteście strategii na danych alternatywnych indeksuje każdą obserwację datą publikacji, bo różnica między publikacją a zdarzeniem daje nierealne wyniki.