Long-horizon Forecasting with FTN
FTN (Forecasted Trajectory Neighbors) is an instance-based (good old KNN) approach for improving multi-step forecasts, especially for long horizons. It’s primarily designed to correct i) error propagations along the horizon (in recursive-based approaches), and ii) the implicit independence assumption of direct (1 model per horizon) forecasting approaches. Not suitable for MIMO (e.g. neural nets), except when the horizon is quite large.
This notebook explores how to couple FTN with NHITS for long horizon forecasting
Loading LongHorizon’s ETTm2 dataset
Fitting a NHITS model
Fitting FTN
Getting forecasts from NHITS and post-processing them using FTN
Evaluating all models
[1]:
import warnings
warnings.filterwarnings("ignore")
If necessary, install the package using pip:
[2]:
# !pip install metaforecast -U
1. Data preparation
Let’s start by loading the dataset. This tutorial uses the ETTm2 dataset available on datasetsforecast.
We also set the forecasting horizon and input size (number of lags) to 360, 6 hours of data.
[3]:
import pandas as pd
from datasetsforecast.long_horizon import LongHorizon
# ade is best suited for short-term forecasting
horizon = 360
n_lags = 360
df, *_ = LongHorizon.load('.',group='ETTm2')
df['ds'] = pd.to_datetime(df['ds'])
Split the dataset into training and testing sets:
[4]:
df_by_unq = df.groupby('unique_id')
train_l, test_l = [], []
for g, df_ in df_by_unq:
df_ = df_.sort_values('ds')
train_df_g = df_.head(-horizon)
test_df_g = df_.tail(horizon)
train_l.append(train_df_g)
test_l.append(test_df_g)
train_df = pd.concat(train_l).reset_index(drop=True)
test_df = pd.concat(test_l).reset_index(drop=True)
train_df.query('unique_id=="HUFL"').tail()
[4]:
| unique_id | ds | y | |
|---|---|---|---|
| 57235 | HUFL | 2018-02-17 04:45:00 | -2.265949 |
| 57236 | HUFL | 2018-02-17 05:00:00 | -2.001912 |
| 57237 | HUFL | 2018-02-17 05:15:00 | -1.945934 |
| 57238 | HUFL | 2018-02-17 05:30:00 | -2.089988 |
| 57239 | HUFL | 2018-02-17 05:45:00 | -2.145967 |
[5]:
test_df.query('unique_id=="HUFL"').head()
[5]:
| unique_id | ds | y | |
|---|---|---|---|
| 0 | HUFL | 2018-02-17 06:00:00 | -1.881931 |
| 1 | HUFL | 2018-02-17 06:15:00 | -1.953862 |
| 2 | HUFL | 2018-02-17 06:30:00 | -1.945934 |
| 3 | HUFL | 2018-02-17 06:45:00 | -1.857858 |
| 4 | HUFL | 2018-02-17 07:00:00 | -2.033914 |
2. Model setup and fitting
We focus on NHITS, which has been shown to excel on long-horizon forecasting problems.
Default configuration for simplicity
[6]:
from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS
CONFIG = {
'max_steps': 1000,
'input_size': n_lags,
'h': horizon,
'enable_checkpointing': True,
'accelerator': 'cpu'}
models = [NHITS(start_padding_enabled=True, **CONFIG),]
nf = NeuralForecast(models=models, freq='15min')
2024-10-10 22:30:00,319 INFO util.py:154 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.
2024-10-10 22:30:00,370 INFO util.py:154 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.
INFO:lightning_fabric.utilities.seed:Seed set to 1
[7]:
%%capture
nf.fit(df=train_df)
INFO:pytorch_lightning.utilities.rank_zero:GPU available: True (mps), used: False
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
INFO:pytorch_lightning.callbacks.model_summary:
| Name | Type | Params | Mode
-------------------------------------------------------
0 | loss | MAE | 0 | train
1 | padder_train | ConstantPad1d | 0 | train
2 | scaler | TemporalNorm | 0 | train
3 | blocks | ModuleList | 3.6 M | train
-------------------------------------------------------
3.6 M Trainable params
0 Non-trainable params
3.6 M Total params
14.445 Total estimated model params size (MB)
INFO:pytorch_lightning.utilities.rank_zero:`Trainer.fit` stopped: `max_steps=1000` reached.
3. Fitting FTN
Now, we can fit FTN.
This process is essentially fitting a KNN for each unique_id in the dataset.
We apply an exponentially weighted average to smooth the time series for KNN estimation (apply_ewm=True)
[8]:
from metaforecast.longhorizon.ftn import MLForecastFTN as FTN
ftn = FTN(horizon=horizon,
n_neighbors=150,
apply_ewm=True)
[9]:
ftn.fit(train_df)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00, 6.66it/s]
[10]:
fcst_nf = nf.predict()
fcst_ftn = ftn.predict(fcst_nf)
INFO:pytorch_lightning.utilities.rank_zero:GPU available: True (mps), used: False
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 90.88it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 22.70it/s]
[11]:
fcst_ftn.head()
[11]:
| ds | NHITS | NHITS(FTN) | |
|---|---|---|---|
| unique_id | |||
| HUFL | 2018-02-17 06:00:00 | -2.155212 | -1.672768 |
| HUFL | 2018-02-17 06:15:00 | -2.148897 | -1.684195 |
| HUFL | 2018-02-17 06:30:00 | -2.134080 | -1.694016 |
| HUFL | 2018-02-17 06:45:00 | -2.112995 | -1.702521 |
| HUFL | 2018-02-17 07:00:00 | -2.088157 | -1.709317 |
Below are the weights of each model (equal across all unique ids because weight_by_uid=False)
Then, we refit the neural networks are get the test forecasts
4. Evaluation
Finally, we compare all approaches
[12]:
test_df = test_df.merge(fcst_ftn, on=['unique_id','ds'], how="left")
[13]:
from neuralforecast.losses.numpy import smape
from datasetsforecast.evaluation import accuracy
evaluation_df = accuracy(test_df, [smape], agg_by=['unique_id'])
[14]:
eval_df = evaluation_df.drop(columns=['metric','unique_id'])
eval_df
[14]:
| NHITS | NHITS(FTN) | |
|---|---|---|
| 0 | 0.250525 | 0.198709 |
| 1 | 0.276969 | 0.265187 |
| 2 | 0.048266 | 0.044834 |
| 3 | 0.848007 | 0.446870 |
| 4 | 0.226960 | 0.237771 |
| 5 | 0.231628 | 0.188907 |
| 6 | 0.175302 | 0.154011 |
[15]:
eval_df.mean().sort_values()
[15]:
NHITS(FTN) 0.219470
NHITS 0.293951
dtype: float64