How can you test for autocorrelation in ML?

I have been working on process control, modelling and optimization for many years, mostly on first principle based models or empirical dynamic models for refining and petrochemical process. Sometime it is extremely difficult to create such first principle based models. Now more and more people try to use ML to create models when first principle models are not available. To some users, ML is a blackbox modelling tool, dumping in operation data, voilà, a model with much better prediction capacity than the OLS is created. What a wonderful world! Really? Operational data are typically highly autocorrelated. Using ML such way will not generate a usable model at all. The model predictions will drift away from true process values.

In my years of practical experience as a data scientist, one key takeaway has always been that overlooking autocorrelation can lead to deceptive model performances. I've witnessed projects where ignoring autocorrelation resulted in a model that appeared to have high accuracy during the training phase. However, when deployed in real-world scenarios, these models failed to generalise effectively. In essence, the models were merely "memorising" the training data due to the underlying autocorrelation, thus misleading the team into thinking it was performing better than it truly was.

Autocorrelation is quite common, in time series data. It occurs when a variable exhibits a correlation with itself across time periods. Do you know how sometimes things seem to follow a pattern over time? Well, think of autocorrelation as something that tells us if the weather was high yesterday it's likely to be high today. If it was low then it's likely to remain low. It's like a consistent trend. Now, negative autocorrelation is the opposite. So, if you had high weather yesterday, expect a low one today. If you ignore the autocorrelation when building a model to predict future weather, it is like trying to guess the next day's weather without considering the past values.

To test for autocorrelation visually, you can use heatmaps that display autocorrelation values calculated from different segments of your time series data. By organizing these values in a heatmap, strong negative autocorrelation through to strong positive autocorrelation, you can easily observe how previous values in the series influence subsequent ones. This visual method allows for quick identification of patterns, such as consistent trends or changes over time, helping you understand the temporal dynamics and dependencies within your data, which is crucial for effective analysis and forecasting.

ACF plots are one of the widely used visualization to analyze auto-correlation in time-series. Autocorrelation basically is a measure of the relationship between a variable's current value and its past values. ACF plots helps us to understand statistically if a particular time series is stationary or non-stationary. ACF values drop rapidly for stationary series unlike for a non-stationary series. Making a series stationary is a important step for time-series forecasting or analysis as it involves removing of trend and seasonal components.

Testing the autocorrelation is fairly straightforward in Python. One can use the acf function from statsmodels library. The autocorrelation can also be plotted using plot_acf for visulization. I wrote this because I started to learn Python, which has a lot free cool libraries to use :)

Addressing Autocorrelation in ML involves: - Using Autoregressive Models such as ARIMA, SARIMA for modelling autocorrelation in Time-series data - RNNs, LSTM, networks are suitable for modelling sequences - Residual Analysis to check the differences between predicted and actual to check the correlation - Bootstrapping techniques for mitigating the impact of autocorrelation - Creating log features - incorporating lagged value of the target variable

I have seen an industrial report using radomization to make a highly autocorrelated time series not autocorrelated anymore. Indeed, randomization can only "hide" the autocorrelation character without changing the nature of the time series. Typically a kind of autoregressive models can be used to model highly autocorrelated signals (most industrial signals are). However, drifting can still happen for AR models. Therefore it is important to have a bias term that can be corrected peredically with lab samples. If the lab sample is fast enough to allow quick bias update, you even do not need fancy models for physical property prediction, the OLS will do the job.

Always clean the data before start modelling work, particularly large quantity of data are involved. Outliers can distort the model. Always perform the residual analysis, which can tell you if the underlying singal is fully explained by the model. Select the independent variables as the model prediction variables. The same process can have different independent process variables due to different control strategies are selected. For example, a industrial crude heater, only 2 out of process variables, FG, crude rate and COT, can be independent and the third one is dependent.