Feature Impact. How to Customize the Sample Size in DataRobot

May 13, 2020
by
· 2 min read

This post was originally part of the DataRobot Community. Visit now to browse discussions and ask questions about DataRobot, AI Platform, data science, and more.

This article explains how to customize the sample size when calculating Feature Impact scores.

Overview

Feature Impact is calculated using permutation importance which has proven to be an excellent measure for determining which features affect a model the most. Having said that, permutation importance is computationally expensive to calculate, which is why DataRobot only uses a subset of the data to do the actual math.

To minimize the chances of having a suboptimal Feature Impact calculation, you can choose the sample size that will be used in the calculations. Currently, the maximum sample you can choose is 100,000 rows.

Defining Custom Sample Size

To choose your own custom sample size for Feature Impact calculations, navigate to the Feature Impact tab for the model of interest (Figure 1).

Figure 1. Feature Impact Tab
Figure 1. Feature Impact Tab

In the Feature Impact tab click the + icon (to the right of the Sample Size field).

Figure 2. Sample size field
Figure 2. Sample Size Field

Now you can choose the appropriate sample size. There are four different ways to do that:

  • Specify the percent of rows to use.
  • Specify the absolute number of rows to use.
  • Choose from the three predefined “Snap To” options (quick, half, max).
    Use the sample size slider.

All of these options are shown in Figure 3.

Figure 3. Feature Impact sample size
Figure 3. Feature Impact Sample Size

After you have finished setting up, just click Set Sample and Enable Feature Impact to kick off the calculations.

Recalculating Feature Impact

Feature Impact can be recalculated with a new sample size as many times as needed. To do this, go to the Feature Impact tab of the model and click Adjust sample size as shown in Figure 4.

Figure 4. Adjust Sample Size
Figure 4. Adjust Sample Size

This will display a window with the same options (as shown in Figure 3). Specify a new sample size and start the calculations again.

More Information

See the in-app Platform Documentation for Feature Impact.

Pathfinder
Explore Our Marketplace of AI Use Cases
Visit Now
About the author
Linda Haviland
Linda Haviland

Community Manager

Meet Linda Haviland
  • Listen to the blog
     
  • Share this post
    Subscribe to DataRobot Blog
    Newsletter Subscription
    Subscribe to our Blog