make_future_df with lagged regressors

[gaurav1195]

Hi

I could not find a example on using make_future_df with regressors on the entire internet. Can somebody give an example on how to do this?

model = NeuralProphet(n_lags=3, n_forecasts=1, daily_seasonality=True, epochs=10)
model.add_lagged_regressor("TMAX")
model.add_lagged_regressor("TMIN")
metrics = model.fit(df_req, validate_each_epoch=True,
valid_p=0.2, freq='D', plot_live_loss=True)
future = model.make_future_dataframe(df_req, regressors_df=df_req_test)

ds | y | TMAX | TMIN | t | y_scaled |  
15-12-2003 | 0.01 | 0.549315 | 0.652727 | 1.249817 | 0.122857
16-12-2003 | 0.07 | 0.615068 | 0.696364 | 1.249878 | 0.26
17-12-2003 | 0 | 0.626027 | 0.638182 | 1.249939 | 0.1
18-12-2003 | NaN | NaN | NaN | 1.25 | NaN

My regressors TMAX and TMIN are showing NaNs in future df. Also same future df comes out when I don't use regressors_df in make_future_df

[ourownstory]

hi @gaurav1195 ,

This is correct. if you are using lagged regressors, they are not known of in the future. Thus, they should be NaN. There is no need to add them, as you should already have them (up to the recorded present time) in your df_req.

[davilineni]

Hi! I have the same problem.

I'm trying to use regressors and I'm not able to create the future data frame.
I provide you with an essay example:

`

data = {
'ds': pd.date_range('2023-01-01', '2023-01-10'),
'y': [10, 15, 20, 25, 30, 35, 40, 45, 50, 55],
'lagged_regressor': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'lagged_regressor2': [5, 8, 3, 6, 4, 6, 1, 8, 19, 1]
}

df = pd.DataFrame(data)

model = NeuralProphet()

model.add_lagged_regressor('lagged_regressor').add_lagged_regressor('lagged_regressor2')

df_train = df.iloc[:8]
df_test = df.iloc[8:]
model.fit(df_train)

df_future = model.make_future_dataframe(df=df_train[['ds','y']], n_historic_predictions=True, regressors_df=df_train[['ds','lagged_regressor','lagged_regressor2']],periods=4)
print(df_future)`

And it gives the following error in the last line:

KeyError Traceback (most recent call last)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandas/core/indexes/base.py:3621, in Index.get_loc(self, key, method, tolerance)
3620 try:
-> 3621 return self._engine.get_loc(casted_key)
3622 except KeyError as err:

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandas/_libs/index.pyx:136, in pandas._libs.index.IndexEngine.get_loc()

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandas/_libs/index.pyx:163, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:5198, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:5206, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'lagged_regressor'

The above exception was the direct cause of the following exception:

KeyError Traceback (most recent call last)
File :26
23 model.fit(df_train)
25 # Crear el DataFrame futuro con las fechas para las cuales haremos predicciones
---> 26 df_future = model.make_future_dataframe(df=df_train[['ds','y']], n_historic_predictions=True, regressors_df=df_train[['ds','lagged_regressor','lagged_regressor2']],periods=4)
27 print(df_future)

Do you know what I'm doing wrong?