Time-Series Transformation Toolkit: Feature Engineering for Predictive Analytics

 

Time-Series Transformation Toolkit: Advanced Feature Engineering for Predictive Analytics
Image by Editor | ChatGPT

Introduction

In time series analysis and forecasting, transforming data is often necessary to uncover underlying patterns, stabilize properties like variance, and improve the performance of predictive models. For example, a time series describing product sales might show strong weekly seasonality and the impact of promotional events. In such cases, transforming raw timestamps into categorical features, such as day of the week or holiday flags, might help models capture temporal dependencies and context more effectively.

This article demonstrates a moderately advanced, feature-engineering approach to constructing meaningful temporal features and applying various transformations for predictive analytics.

We’ll explore how to:

• Add multiple lagging features to a time series.
• Incorporate rolling statistics like a rolling mean over a sliding time window.
• Apply differencing to capture variations in counts across a time interval.

A Gentle Hands-On Dive

We will use the Bike Sharing Dataset, a common time series dataset that contains daily recordings with features like date (dteday), daily bike rental count (cnt), average temperature (temp), day of the week (weekday), whether the day is a holiday (holiday), and whether it is a working day (workingday).

import pandas as pd   url = "https://raw.githubusercontent.com/deep-learning-with-pytorch/dlwpt-code/master/data/p1ch4/bike-sharing-dataset/day.csv" df = pd.read_csv(url, parse_dates=['dteday'])   df[['dteday', 'cnt', 'temp', 'weekday', 'holiday', 'workingday']].head()

In time series data, before any preprocessing and predictive tasks, it is important to set the date-time attribute as the index. In this case, that honor will be granted to the dteday attribute, and this is how it is done in Pandas:

df['date'] = pd.to_datetime(df['dteday']) df.set_index('date', inplace=True)

We’ll also perform a simple feature engineering task (not quite advanced yet): determining if a date is a weekend and extracting the month.

df['is_weekend'] = df['weekday'].isin([5, 6]).astype(int) df['month'] = df.index.month

Adding lag features is a feature engineering technique used on time series data to incorporate some “short-term memory” of past records in a given record. This way, values for attributes like the rental count on previous days can be used as predictor attributes.

df['cnt_lag1'] = df['cnt'].shift(1) df['cnt_lag2'] = df['cnt'].shift(2) df['cnt_lag7'] = df['cnt'].shift(7)

Importantly, the shift(n) function does not calculate an average value for the specified attribute over the past n days or time instants: it just takes the value that the attribute had n time instants before.

Another feature engineering technique that is very useful in time series forecasting is the so-called rolling statistics, which use a sliding time window to calculate a mean or any other aggregate value over the period defined by that window. For instance, the code below adds two attributes to the dataset: one with the 7-day rolling mean — i.e., the mean of the previous seven days’ values for a given attribute — and a 7-day rolling standard deviation.

df['cnt_roll7_mean'] = df['cnt'].shift(1).rolling(window=7).mean() df['cnt_roll7_std'] = df['cnt'].shift(1).rolling(window=7).std()

Rolling statistics help gain insight into how a value like rental count behaves over time, helping to easily identify trends and variability patterns.

Moreover, differencing, consisting of calculating the difference between the present value of an attribute and its value n times back, is also useful for revealing how values change over time, beyond merely looking at their raw magnitude.
This can be easily done by using the shift(n) function again combined with a column-level subtraction, as follows:

df['cnt_diff1'] = df['cnt'] - df['cnt'].shift(1) df['cnt_diff7'] = df['cnt'] - df['cnt'].shift(7)

Notice that using the three feature transformations explored above results in the appearance of some missing values (NaN) due to shifting and rolling over the first few instances of the dataset, where there is insufficient past information to perform the desired transformations. You may need to decide how to handle them, for instance, by simply removing those rows from the dataset (if the time series is large enough, removing the first few rows generally shouldn’t affect predictive performance).

df_clean = df.dropna(subset=[     'cnt_lag1', 'cnt_lag2', 'cnt_lag7',     'cnt_roll7_mean', 'cnt_roll7_std',     'cnt_diff1', 'cnt_diff7' ])

And so, we’ve ended up with a time series dataset that contains plenty of useful, additional information for predictive analysis as a result of some transformation-driven feature engineering operations. Great job!

Conclusion

This article demonstrated some strategies to extract and unlock meaningful temporal features in time series data using lagging, rolling statistics, and differencing. When applied properly, these strategies will turn your raw time series data into a much better fit for predictive analysis processes, particularly when building machine learning models for forecasting.