Yesterday Predicts Today: A Time-Series Forecasting and Autocorrelation Case Study

The research decision

The operational question is:

How should the nonprofit forecast daily meal requests when demand contains a long-term trend, recurring short-term fluctuation, and persistence from one day to the next?

The distinction matters because time-series analysis is designed for outcomes measured repeatedly across ordered time periods and can be used for forecasting from past observations. Tabachnick and Fidell (2013) describe time-series analysis as analysis of a dependent variable observed over many time periods, with forecasting and intervention evaluation among its principal applications.

Meier et al. (2014) similarly treat a time series as an ordered sequence in which analysts first inspect chronological patterns, distinguish longer-term movement from short-term fluctuation, and then construct forecasts appropriate to that structure.

The synthetic dataset

The synthetic series contains three intentional features:

A time plot is the appropriate starting point for ordered measurements because it displays both the values and their chronological sequence. Moore et al. (2021) recommend graphical examination of data and explicitly include time plots for displaying quantitative measurements across time.

What the time plot shows

The raw series does not wander randomly around a fixed level. Three features are visible.

1. Meal requests generally rise over the 180 days.
2. There is a recurring within-week pattern.
3. Neighboring observations tend to move together: unusually busy days are often followed by another relatively busy day, while quieter periods tend to persist briefly.

Meier et al. (2014) distinguish this kind of short-term variation from the longer-term trend. Their time-series treatment uses moving averages to filter recurring short-term fluctuation so that the underlying trend can be examined more clearly. When the short-term cycle has a known length, the moving-average window can be matched to that cycle (Meier et al., 2014).

For this daily series, a 7-day centered moving average is therefore a useful descriptive device. It is not the final forecasting model; it is a way to make the longer-term movement easier to see.

The initial analyst's model

The analyst begins with the ordinary regression:

Y_t = β₀ + β₁t + e_t

where Y_t is daily meal requests and t is the day number.

Fitted to the first 166 observations, the synthetic analysis gives:

The fitted slope is consistent with an upward trajectory in the synthetic data. But the model's residuals are strongly related across neighboring days.

Linear regression assumes, among other conditions, that prediction errors are independent of one another. Meier et al. (2014) list independence of errors as one of the assumptions underlying simple linear regression and caution that regression analysis becomes less reliable when its assumptions are not met.

Lovric (2011) makes the consequence more specific. When regression disturbances are autocorrelated, ordinary least squares is no longer efficient under the usual conditions, and standard errors calculated under the assumption of no autocorrelation can be wrong. The resulting t statistics and statistical inference can therefore be misleading.

Autocorrelation: why yesterday can predict today

Autocorrelation asks whether values—or, more importantly for regression diagnostics, disturbances—remain related across time.

The synthetic raw series has a correlation of approximately 0.739 between today's and yesterday's meal requests. That raw lag relationship combines trend, weekly pattern, and serial persistence, so it should not automatically be interpreted as a pure autoregressive effect.

The more revealing result is the initial regression's residual lag-1 autocorrelation of 0.584. Even after fitting the overall time trend, a positive unexplained deviation today tends to be followed by another positive deviation tomorrow.

Lovric (2011) describes the simplest representation of this pattern as a first-order autoregressive disturbance:

u_t = ρu_t−1 + ε_t

where the current disturbance depends on its lagged value plus a new random innovation. In a stationary AR(1) process, dependence weakens as observations become farther apart in time (Lovric, 2011).

Weekly fluctuation is not the same as autocorrelation

A second problem appears if the analyst responds by merely adding day-of-week indicators.

That expanded ordinary regression does represent the recurring weekly pattern, but its lag-1 residual autocorrelation is still approximately 0.650 in the synthetic analysis.

The two structures answer different questions.

Modeling one does not guarantee that the other disappears.

Tabachnick and Fidell (2013) distinguish local autocorrelation from seasonal autocorrelation. Seasonal autocorrelation appears at predictably spaced lags—for example, lag 7 in daily observations—whereas local autocorrelation describes shorter-range dependence. Their time-series workflow uses time plots together with autocorrelation and partial-autocorrelation information to identify these patterns.

A more appropriate time-series model

For this case, the revised model keeps the substantively useful deterministic components—a linear time trend and day-of-week indicators—but allows the disturbances to follow an AR(1) process.

Conceptually:

Y_t = β₀ + β₁t + day-of-week effects + u_t

with

u_t = ρu_t−1 + ε_t.

Lovric (2011) explicitly discusses AR(1) disturbances as a basic model for autocorrelation in regression and also describes methods designed to account for serial correlation rather than calculating ordinary least-squares inference under a false independence assumption.

Tabachnick and Fidell (2013) develop the broader Box–Jenkins ARIMA framework for time-domain analysis. In that framework, analysts identify the time-series structure, estimate a model, and diagnose whether the remaining residuals behave sufficiently like random error. Trend, local autocorrelation, seasonal components, interventions, and additional predictors can all be incorporated when justified by the observed series and the research problem.

The present case stays deliberately simple: there is no need to introduce a more elaborate forecasting algorithm when a trend, a known weekly pattern, and first-order serial persistence describe the synthetic design.

What changes after modeling serial dependence?

The fitted time-series regression estimates the AR(1) disturbance coefficient at approximately:

ρ̂ = 0.650

That estimate is substantively important. It indicates strong positive short-range persistence in the synthetic disturbances.

But estimating an autoregressive coefficient is not enough. The revised model must also be checked to see whether meaningful serial structure remains in its residuals.

A Ljung–Box residual check gives:

The diagnostic principle, however, comes directly from the cited sources. Tabachnick and Fidell (2013) describe autocorrelation checks of residuals as tools for assessing departure of model residuals from random error. Significant residual autocorrelation indicates discrepancies between the fitted time-series model and the observed data.

Here, the revised residuals no longer show the obvious serial pattern left by the initial regression. That is evidence that the simple trend-plus-weekly-pattern model with AR(1) disturbances has captured the main dependence intentionally built into the synthetic series.

Forecasting the next 14 days

Forecasting is not simply extending a line.

Meier et al. (2014) show that when a time series contains short-term fluctuation, analysts should first identify that fluctuation, distinguish it from the longer-term trend, and incorporate the short-term pattern appropriately when producing forecasts. Their examples use moving averages and regression to separate trend from recurring variation.

The revised model instead handles the same general problem within a time-series regression: it forecasts from the estimated long-term trend, the known day-of-week position, and the serial state inherited from the immediately preceding observations.

Using the final 14 days as a chronological holdout gives:

The time-series model forecasts better in this particular synthetic holdout.

Forecast uncertainty matters

A point forecast is only one possible future value.

The time-series model therefore produces forecast intervals as well as predicted daily requests. Tabachnick and Fidell's (2013) discussion of time-series software and analysis includes predicted values, forecast values, and confidence limits around predictions as standard outputs of ARIMA modeling. Meier et al. (2014) likewise demonstrate interval estimation around forecasts rather than treating projected values as certain.

Forecast uncertainty should normally increase as the forecast moves farther beyond the observed series because future disturbances have not yet been observed. A useful operational report should therefore present ranges alongside expected meal demand rather than staffing to a single predicted count as though it were known in advance.

A practical diagnostic sequence

For a regularly measured institutional or business outcome, the analysis should proceed from chronological structure to model adequacy rather than from a coefficient table directly to interpretation.

1. Preserve the time order. Record the spacing of observations and identify gaps, irregular measurement intervals, or changes in data collection.
2. Plot the outcome against time. Look for trend, abrupt changes, cycles, unusual observations, and changing variability (Moore et al., 2021; Tabachnick & Fidell, 2013).
3. Separate long-term movement from recurring short-term fluctuation. Moving averages can be useful descriptively when the short-term cycle has an identifiable length (Meier et al., 2014).
4. Fit the proposed regression, but inspect its residuals chronologically. Independence should not be assumed merely because ordinary regression software produced estimates (Meier et al., 2014; Lovric, 2011).
5. Examine autocorrelation and relevant lags. Determine whether neighboring or seasonal residuals remain related (Lovric, 2011; Tabachnick & Fidell, 2013).
6. Use a time-series specification that matches the observed structure. AR, MA, ARMA/ARIMA, seasonal components, or regression with serially correlated errors are among the approaches described in the cited sources; complexity should follow evidence in the series rather than be added automatically (Lovric, 2011; Tabachnick & Fidell, 2013).
7. Diagnose the fitted residuals. Remaining autocorrelation suggests that important temporal structure is still unmodeled (Tabachnick & Fidell, 2013).
8. Forecast with uncertainty. Report prediction ranges as well as point forecasts and distinguish model-based projections from guaranteed future outcomes (Meier et al., 2014; Tabachnick & Fidell, 2013).

What the ordinary regression got wrong

The initial regression did identify the broad upward trend. Its failure was more specific: it treated remaining observations as independent when they were not.

That matters because regression inference depends on the behavior of the disturbances. With serially correlated errors, conventional standard errors can be incorrect, making ordinary significance tests unreliable (Lovric, 2011).

The lesson is therefore not that linear regression is prohibited for chronological data. Time-series methods can themselves contain regression components. The issue is whether the model represents the temporal structure that remains after the systematic predictors have been included.

Limitations of the case

Conclusion

The initial analyst saw an upward trend and reached for ordinary regression. The more important feature was hidden in what that regression left behind.

Daily meal requests were chronologically structured. They contained a recurring weekly fluctuation, and disturbances persisted from one day to the next. Once that dependence was modeled, lag-1 residual autocorrelation fell from 0.584 to −0.049, residual diagnostic tests no longer indicated obvious serial structure, and the synthetic 14-day forecast error decreased.

Time is not merely another column in the dataset. In a time series, the ordering can be part of the model.

References

Lovric, M. (Ed.). (2011). International encyclopedia of statistical science. Springer. https://doi.org/10.1007/978-3-642-04898-2

Meier, K. J., Brudney, J. L., & Bohte, J. (2014). Applied statistics for public and nonprofit administration (9th ed.). Cengage Learning.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2021). Introduction to the practice of statistics (10th ed.). Macmillan Learning.

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson.