Get Creative with AI Forecasting in Changing Economic Conditions

The shift in consumer habits and geopolitical crises have rendered data patterns collected pre-COVID obsolete. This has prompted AI/ML model owners to retrain their legacy models using data from the post-COVID era, while adapting to continually fluctuating market trends and thinking creatively about forecasting. In this blog, we’ll review the new DataRobot Time Series clustering feature, which gives you a creative edge to build time series forecasting models by automatically grouping series that are identical to each other and then building models tailored to these groups. 

Managing Through Socio-Economic Disruption

In the last few years, businesses have experienced disruptions and uncertainty on an unprecedented scale. The situation is even more challenging for companies in industries that use historical data to give them visibility into future operations, staffing, and sales forecasting.

Retail is just one of the industries reeling from the effects of COVID-induced change. Others include supply chain disruptions for manufacturers, staffing shortages for hospitals or distribution centers and many more.

New research at MIT Sloan into consumer behavior during COVID-19 reveals that 54% of shoppers bought from brands that were new to them—32% said they did so because their “favorite brand was out of stock”.

Unlocking New Business Opportunities with AI Forecasting

Solving time-dependent business challenges requires an in-depth understanding of various special algorithms that rely on historical, dynamic data to make forecasts. These forecasts can be at varying levels of granularity, such as hourly, daily, weekly, or monthly, and can include a diverse set of multi-modal attributes. However, hand-coding, testing, evaluating and deploying highly accurate models is a tedious and time-consuming process. Manually scaling out this process to thousands of stores or SKUs at once and then monitoring them, for example, is a nightmarish experience for data scientists.

In fact, 87% of organizations struggle with long deployment timelines.

Building robust and highly accurate models at scale is very crucial in a use case where every percent increase in accuracy can lead to millions of dollars in savings or revenue. 

DataRobot AI Platform offers an out-of-the-box, end-to-end Time Series Clustering feature that augments your AI forecasting by identifying groups or clusters of series with identical behavior. This new capability builds on Segmented Modeling—a functionality where you can manually choose how you want to group together your series. Time Series Clustering takes it a step further, allowing you to automatically detect new ways to segment your series.

Time Series Clustering significantly enhances your capability to build high performing models by grouping together series (e.g., retail stores) based on similar behavior, and then use these groups as segments to the Segmented Modeling workflow. This automation drastically reduces model building, testing, evaluation and deployment time, promotes creativity, and enables rapid experimentation for time-sensitive use cases. With Time Series Clustering, you no longer need to manually run time series clustering projects outside of the DataRobot platform and then merge them with your Segmented Modeling workflow on the platform.

What’s Under the Hood of AI-Driven Forecasting?

For this blog, we will be tackling a use case that forecasts sales across multiple retail stores in the U.S. and demonstrate how this can be done at speed and scale using DataRobot.

The dataset consist of sales data collected for multiple retail stores across North America. Our goal is to predict sales for each of these stores as accurately as we can within a short span of time.

1. Improved Productivity 

Time Series Clustering can be used in two ways:

• As a part of the Segmented Modeling workflow where the clusters identified are your new Segment IDs, thus leading to more accurate Time Series models.
• As an independent project where you can choose to run clustering on top of a Multi-Series dataset and identify series that are behaving similar to each other to get counter-intuitive but logical insights.

Here, we will focus on how Time Series Clustering fits into the Segmented Modeling workflow using a simple yet highly relevant Multi-Series Sales Forecasting example. 

The Dataset

Within DataRobot, you can store all your datasets in the AI Catalog and share it with your team. You can also connect to Snowflake, Azure, Redshift and many other databases. We are using a multimodal dataset to predict sales across 10 different stores.

Multimodal data lets you simultaneously ingest and process various data types, such as images, text, and numeric data, quite seamlessly. So, next time, you won’t have to think twice before combining customer review data along with your store sales.

Next, you can create a supervised, time aware project to predict sales, and select “stores” as your series ID. 

2. All in One! Seamless Integration of Time Series Clustering and Segmented Modeling

On this new project, once you click on “Segmentation Method,” you will see the option to choose existing or new time series clusters as Segment IDs. We will click on the highlighted option that lets us build a whole new clustering model.

You can choose one or more features to be used for clustering. In this case, we are selecting “Sales,” in addition to the primary Date column and store (our series identifier).

As a next step let’s choose the appropriate Clustering Model.

In this case, the DataRobot platform recommends using the model that has split our 10 stores into two clusters. A high Silhouette score indicates that the two clusters have distinct properties.

You can either choose the recommended clustering model or any other model with a different number of groups or clusters and thus carry out more experiments.

3. Valuable Insights at Your Fingertips 

It seems that the clustering has identified the stores in Savannah, Georgia and Louisville, Kentucky to have similar sales behavior, despite being in completely different parts of the country. Maybe both these stores were located close to a big university? This is where your domain expertise on the data and the business use case would play a key role in making informed decisions based on these model insights.

The rest of the stores seem to have similar sales characteristics and, hence, are grouped together. This insight is the key to creating and experimenting with new segments that could lead to higher accuracy. All of this without writing a single line of code.

4. New AI Experiments with a Few Clicks

Now you can create a segmentation project on top of the existing clustering project. This is a great example of using AI on top of AI (or DataRobot on top of DataRobot). With a single click, you can kick off a segmentation model workflow with the clusters as the Segment IDs. 

The Segmented Modeling project has created model leaderboards for each of the two segments corresponding to the two clusters minted above. Each of these can be explored just like any other AutoML or AutoTS projects would be within DataRobot.

5. Clear Path into Production

With a single click in the Predict tab, you can deploy this combination of clustering and segmentation into production and start making predictions.

6. Powerful Model Monitoring

Once the model is deployed into production, you can view the deployment assets, such as the prediction environment, approval status, and build environment, as well as the audit trail for any model replacements.

You can deploy a time series clustering and segmentation model from scratch in DataRobot! This took me less than 45 minutes end to end, and I was able to experiment with using different permutations and combinations of clusters and segments.

Start Today 

Go beyond the basics and apply advanced, AI-driven forecasting models to the most critical parts of your operations with DataRobot Automated Time Series. Help your organization thrive in the face of continuous turbulence by rapidly delivering powerful, AI-driven forecasts at scale. 

Access public documentation to get more technical details about recently released features.