backtesting-configuration
/backtesting-configuration
Configures TimeSeriesFold parameters for backtesting based on deployment scenarios. Maps business requirements (retraining frequency, forecast horizon, data budget) to cross-validation strategy parameters. Use when the user describes how they plan to deploy or evaluate a model.
- Author
- skforecast ↗
- Repository
- skforecast-ai ↗
- Licence
- Apache-2.0
- Last repository change
- 16.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 16.10.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Configures TimeSeriesFold parameters for backtesting based on deployment scenarios. Maps business requirements (retraining frequency, forecast horizon, data budget) to cross-validation strategy parameters. Use when the user describes how they plan to deploy or evaluate a model.
Original author description
Configures TimeSeriesFold parameters for backtesting based on deployment scenarios. Maps business requirements (retraining frequency, forecast horizon, data budget) to cross-validation strategy parameters. Use when the user describes how they plan to deploy or evaluate a model.
How it works
- When to Use
- Stop Conditions
- TimeSeriesFold Parameters
- Constraints
- Business Scenario Mapping
- Examples
SKILL.md preview
---
name: backtesting-configuration
description: >
Configures TimeSeriesFold parameters for backtesting based on deployment
scenarios. Maps business requirements (retraining frequency, forecast
horizon, data budget) to cross-validation strategy parameters.
Use when the user describes how they plan to deploy or evaluate a model.
---
# Backtesting Configuration
## When to Use
Use this skill to translate a deployment scenario (retraining cadence, forecast
horizon, data budget, ingestion delay) into `TimeSeriesFold` parameters.
`TimeSeriesFold` is the cross-validation strategy passed via the `cv` argument to
`backtesting_forecaster` (and its multi-series / stats variants) and to the
hyperparameter search functions.
```python
from skforecast.model_selection import backtesting_forecaster, TimeSeriesFold
cv = TimeSeriesFold(
steps=7, # forecast horizon
initial_train_size=365, # first training window (required)
refit=False, # train once (default)
fixed_train_size=True, # rolling window (default)
gap=0,
)
metric, predictions = backtesting_forecaster(
forecaster=forecaster,
y=data['target'],
cv=cv,
metric='mean_absolute_error',
)
```
For fast hyperparameter tuning where multi-step realism is not required, use
`OneStepAheadFold` instead: it validates one step ahead (no recursive
prediction), so it is much faster but less representative of multi-step
performance. Use `TimeSeriesFold` for realistic multi-step backtesting.
### Related skills
- **Prerequisite**: `forecasting-single-series` / `forecasting-multiple-series` (have a fitted forecaster before backtesting)
- **Alongside**: `metric-selection` (choose the metric(s) backtesting reports)
- **Alongside**: `hyperparameter-optimization` (the same `cv` object drives the search functions)
- **Next**: `prediction-intervals` (add `interval=` / `interval_method=` to backtest uncertainty)
## Stop Conditions
Scan before writing code. Each row lists a rule, the symptom when it is broken, and the recovery. Full pitfall catalog: the `troubleshooting-common-errors` skill.
| Rule | Symptom | Recovery |
|------|---------|----------|
| `initial_train_size` must be provided (the API default is `None`, which only works when reusing an already-fitted forecaster) | `ValueError` / no training in the first fold | Pass an int, date string, or Timestamp, e.g. `initial_train_size=len(y) - 100` |
| `initial_train_size` does not accept a float fraction | `ValueError`: must be int, date string, Timestamp, or None | Convert fractions to an int, e.g. `int(len(data) * 0.7)` |
| Configuration must yield at least 2 folds | Single-fold or empty backtest | Ensure `initial_train_size + gap + 2 * steps <= n_observations` |
| `refit=True` retrains every fold (slow path); the default is `refit=False` | Backtest much slower than expected | Use `refit=False` or an int cadence (e.g. `refit=7`) unless per-fold retraining is required |
| `gap`, `steps`, and `fold_stride` count observations, not calendar time | Off-by-frequency delays | Convert the delay to the series frequency (e.g. 2 days at daily freq = `gap=2`) |
## TimeSeriesFold Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `initial_train_size` | int, str, pd.Timestamp | None (required) | Observations for initial training. Int = count, str/Timestamp = last training date. A common starting point is `int(len(data) * 0.7)`. |
| `refit` | bool, int | False | Refit every fold (True), train once (False, default), or every n folds (int). |
| `fixed_train_size` | bool | True | Rolling fixed window (True, default) vs expanding window (False). |
| `gap` | int | 0 | Observations between training end and test start. |
| `fold_stride` | int, None | None (= steps) | Distance between consecutive test set starts. |
| `skip_folds` | int, list, None | None | Skip folds to reduce compute. Int = keep every n-th. |
| `allow_incomplete_fold` | bool | True | Allow final fold with fewer observations than steps. |
## Constraints
- Must produce **at least 2 folds**: `initial_train_size + gap + 2 * steps <= n_observations`
- **Hard minimum**: `initial_train_size` must be greater than `forecaster.window_size`, the number of past observations needed to build one training row (`max(max_lag, max_size_window_features)` plus the `differentiation` order). Direct forecasters (`ForecasterDirect`, `ForecasterDirectMultiVariate`, `ForecasterRnn`) need at least `window_size + steps`, using the forecaster's `steps`. In multi-series forecasters, each series with data in the first window needs more than `window_size` observations there.
- Forecasters without lags: `ForecasterEquivalentDate` has `window_size = offset * n_offsets` (same rule); in `ForecasterStats` the estimator sets the minimum (`window_size` is fixed at 1); `ForecasterFoundation` accepts any value ≥ 1 (`context_length` is a maximum, not a minimum).
- **Recommended**: leave a margin above the hard minimum so the model has enough rows to learn from (e.g. two or more seasonal cycles), while keeping at least 2 folds
- `gap` simulates real-world delay between data availability and forecast usage
- When `fixed_train_size=True`, the window rolls forward (oldest data discarded). Use for concept drift or when old data is less relevant.
- When `fixed_train_size=False`, the window expands (all history retained). Use when more data always helps.
## Business Scenario Mapping
### Retraining frequency
| Scenario | Configuration |
|----------|--------------|
| Retrain every time new data arrives | `refit=True` |
| Retrain weekly (with daily forecasts, steps=1) | `refit=7` |
| Retrain monthly (with daily forecasts, steps=7) | `refit=4` (every 4 folds × 7 steps ≈ monthly) |
| Never retrain / train once, evaluate across time | `refit=False` (`fixed_train_size` has no effect without refit) |
### Data freshness vs volume
| Scenario | Configuration |
|----------|--------------|
| Recent data more relevant (concept drift) | `fixed_train_size=True` |
| All historical data valuable | `fixed_train_size=False` (expanding) |
| Limited compute budget | `refit=False` or `skip_folds=3` (keep every 3rd fold) |
### Deployment gap
| Scenario | Configuration |
|----------|--------------|
| Real-time predictions (no delay) | `gap=0` |
| 1-day delay between data collection and forecast | `gap=1` (if freq=daily) |
| Forecast must be ready before weekend (2-day gap) | `gap=2` |
### Initial training size
| Scenario | Configuration |
|----------|--------------|
| Default (balanced) | `int(len(data) * 0.7)` |
| Maximize evaluation coverage | Hard minimum (`window_size + 1`; `window_size + steps` for direct) plus a learning margin, see Constraints |
| Maximize training data | `n_observations - gap - 2 * steps` (minimum 2 folds) |
| Start from specific date | `initial_train_size="2023-01-01"` |
| Conservative (large training set) | `int(len(data) * 0.8)` |
### Fold stride (test set overlap)
| Scenario | Configuration |
|----------|--------------|
| Non-overlapping evaluation (default) | `fold_stride=None` (equals steps) |
| Sliding window evaluation (1-step shift) | `fold_stride=1` |
| Sparse evaluation (save compute) | `fold_stride=steps * 2` |
## Examples
### "I retrain my model every Monday and forecast the next 7 days"
```
refit = True
fixed_train_size = False # Keep all history
gap = 0 # No delay
fold_stride = None # Non-overlapping weeks
```
### "I want to simulate deploying once and seeing how the model degrades"
```
refit = False # Train once; fixed_train_size has no effect without refit
gap = 0
```
### "There's a 2-day lag between data ingestion and when forecasts are needed"
```
gap = 2
refit = True
```
### "I have limited compute — evaluate every other week"
```
refit = True
skip_folds = 2 # Keep every 2nd fold
```
### "I want maximum evaluation coverage with a 12-step horizon"
```
# Hard minimum: forecaster.window_size + 1 (window_size + 12 for ForecasterDirect with steps=12)
initial_train_size = forecaster.window_size + margin # margin: rows to learn from, e.g. 2+ seasonal cycles
refit = False # Faster; trains once, fixed_train_size has no effect
fold_stride = 1 # Sliding window (many overlapping folds)
```
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · skforecast/skforecast-ai | 100 | Apache-2.0 |
| skforecast/skforecast | 100 | BSD-3-Clause |
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version 424c061094ed (SHA-256 digest of the entire package)
Downloads in the last 30 days: 1 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100backtesting-py-oracle
backtesting.py configuration for SQL oracle validation and range bar pattern backtesting. Use when running backtesting.py against. - 90backtesting-frameworks
Apply backtesting frameworks in reproducible local data workflows with version-aware APIs, explicit assumptions, and validation. Use when the user chooses backtesting frameworks or its strengths fit the task. - 100metric-selection
Guides selection of the most appropriate evaluation metric(s) for a forecasting task based on forecaster type, prediction output type, data characteristics, and multi-series aggregation needs. Use when the user asks which metric to use, how to evaluate forecast quality, or needs… - 100metric-selection
Guides selection of the most appropriate evaluation metric(s) for a forecasting task based on forecaster type, prediction output type, data characteristics, and multi-series aggregation needs. Use when the user asks which metric to use, how to evaluate forecast quality, or needs… - 100backtesting-alt-data-strategies-with-realistic-availability-lag
Use when backtesting an alternative data strategy, to key every observation on its publication date rather than its event date, because the gap between the two is where impossible backtest results come from.