Questions: Time Series Forecasting

Question 1

A data scientist randomly splits two years of hourly sales data 80/20 into train and test sets, trains an LSTM, and reports excellent test accuracy. What is the fundamental problem?

• A. LSTMs are not appropriate for sales data — a simpler ARIMA model should have been used
• B. The test set likely contains timestamps from before the end of the training set, so the model was effectively trained on 'future' information it would not have in production
• C. An 80/20 split does not provide enough training data for a neural network
• D. Sales data is too noisy for any forecasting model to achieve high accuracy

Answer: (B)

Random splitting breaks temporal causality. When timestamps are randomly distributed across folds, the training set may contain observations *after* some test observations. During training, the model learns patterns that include 'future' values it would never have in a real deployment, producing artificially inflated accuracy. The correct approach — train on the past, evaluate on the future — simulates how the model will actually be used. Walk-forward (rolling-origin) validation is the standard method.

Question 2

A naive baseline that predicts the last observed value ('predict t+1 = t') outperforms a carefully tuned LSTM on a stationary demand series. What does this most likely indicate?

• A. LSTMs are computationally too slow for real-time forecasting
• B. The naive baseline is guaranteed to have lower RMSE by mathematical construction
• C. The series has strong autocorrelation and is well-behaved; the LSTM likely overfit to noise in training, failing to add value over a simple recency heuristic
• D. LSTMs require more than one year of training data to outperform naive baselines

Answer: (C)

Naive baselines exploit autocorrelation: in many series, the most recent value is a strong predictor of the next. A model that overfits to training noise, or one that is overcomplicated relative to the series' structure, can perform worse than simply predicting yesterday's value. Comparing against naive baselines before claiming model improvement is essential discipline — if you can't beat 'last value' or 'same time last year,' the sophisticated model has added no value. This is not a quirk; many published models fail this test.

Question 3

Computing normalization statistics (mean and standard deviation) over the entire dataset — including the test period — before splitting is a valid preprocessing step for time series forecasting.

• T. True
• F. False

Answer: False

This is data leakage. Using statistics from the entire dataset (including the test period) to normalize the training data means the training set is subtly contaminated with information from the future — the model learns the future mean and variance through the scaling. In production, you only have access to past data, so normalization must be computed on the training period only and then applied with those fixed statistics to the test period. This is one of the most common and insidious mistakes in time series preprocessing.

Question 4

Walk-forward (rolling-origin) validation is more appropriate than k-fold cross-validation for evaluating time series forecasting models.

• T. True
• F. False

Answer: True

k-fold cross-validation randomly assigns observations to folds, which breaks temporal ordering and introduces future information into training folds — the equivalent of time travel. Walk-forward validation preserves causality: train on all data up to time t, forecast from t+1 to t+h, advance the window, and repeat. This accurately simulates how the model will be used in deployment and produces realistic performance estimates. k-fold is appropriate for i.i.d. data; time series data is structurally incompatible with it.

Question 5

Explain why non-stationarity (trend and seasonality) must be diagnosed before fitting a classical forecasting model, and what happens if it is ignored.

Model answer: Classical models like ARIMA assume stationarity — constant mean and variance over time. Trend creates a non-constant mean (the series drifts); seasonality creates structured periodicity that violates the assumption of a constant unconditional distribution. If you fit ARIMA to a trending, seasonal series without addressing non-stationarity, the model estimates unstable parameters, produces unreliable confidence intervals, and typically forecasts by extrapolating a short-term local pattern rather than capturing the true long-run behavior. For example, a model fit to the last few months of a rising trend may forecast continued linear growth but miss the seasonal trough that follows. Standard remedies include differencing (to remove trend), seasonal differencing or decomposition (to remove seasonality), or using models that explicitly handle these components like SARIMA or Prophet.

Stationarity is the foundation for reliable statistical inference in time series. Even neural models (LSTMs, Transformers) benefit from understanding trend and seasonality — features encoding time of year, day of week, and trend direction often improve performance dramatically, because the model can learn seasonal effects explicitly rather than having to discover them from raw timestamps.