In this article:
1. General Overview of the Forecasting Feature
1.1. How It Works
1.2. How to Interpret It
2. Understanding the Settings
2.1. Certainty
2.2. Scale and Return Scale Parameters
2.2.1. The Return Scale Parameter
2.3. Adjusting Your Data Set
2.3.1. Primary Dimension
2.3.2. Secondary Dimension
2.3.3. How to Work with Them?
3. Understanding the Advanced Settings
(1) Samples
(2) Additional Cards
(3) Remove Values <
(4) Remove Values >
(5) Historical Data
(6) Keep Outliers
(7) Forecast Children Only
(8) Forecast All Children
(9) Forecasting Output
4. Disclaimer
5. FAQ
As the name suggests, the Forecast feature is used to forecast the duration of the corresponding initiative. To see it in action, open an initiative anywhere in the account and go to the tab called “Forecast.”
1. General Overview of the Forecasting Feature
The forecasting feature in Businessmap is based on a Monte Carlo algorithm. To obtain a forecast, you need to have at least one child card (living in a Cards workflow) linked to the initiative you are running the forecast for.
1.1. How It Works
When the simulation starts, it will find all child cards of the initiative that live on the lowest level (in a Cards workflow). It will also find similar cards that were finished in the past. Then, it will run thousands of samples, trying to project the duration of the whole initiative. The result will be visualized in a horizontal bar chart.
The graph provides four estimations (all measured in days):
- Optimistic
- Realistic
- Pessimistic
- Very Pessimistic
1.2. How to Interpret It
Forecasts should be interpreted based on the status of the initiative:
- Not started — for initiatives that have not been started, the forecast should be read as: “Once the initiative is started, it will take between X and Y days to be finished.”
- In progress — forecasts for initiatives that are currently in progress should be read as: “This initiative will take between X and Y days to be finished.”
- Completed — you can use the forecast of a completed initiative to calibrate your forecasting skills by obtaining an accurate return scale value (read more about it below) to use when forecasting future and current initiatives.
2. Understanding the Settings
The foundation of any successful forecast is a proper data set. The forecast settings let you fine-tune your data set to ensure accurate results.
There are four primary settings:
- Certainty
- Scale
- Primary Dimension
- Secondary Dimension
2.1. Certainty
This parameter instructs the algorithm to use as input data the historical cycle times up to the “Certainty” percentile. The cycle time values above that threshold will be ignored. The lower the percentage is, the shorter the estimated duration is going to be. However, the likelihood of getting it is also going to be lower.
To learn more about cycle time percentiles, please review the Cycle Time Scatterplot article.
2.2. Scale and Return Scale Parameters
The scale parameter is the most important parameter, as it accommodates many factors that can influence the forecast. The lower the scale value, the longer the duration.
A value of 1 assumes the following:
- All child cards are “normal,” meaning they are likely to take the same time as your historical data.
- The child cards are worked sequentially*, one after the other.
* The forecasting feature assumes a sequential execution of tasks, where a new one starts only after the previous card has been completed. It is based only on the times of the completed cards from the historical data and does not consider the parallel execution of cards nor the throughput of the workflow that is expected when a team of multiple people is working on the same initiative/project. This is a key difference from the Monte Carlo (When) simulation in the analytics module, where the throughput is embedded into the forecasting as all forecasted cards are on the same board. In contrast, initiative forecasting may have child cards located on different boards.
It is obvious that the assumptions above will not be fulfilled all the time. In that case, change the scale value to reflect your reality.
An example of a property that affects the scale parameter is “complexity.” If you know with a good certainty that the upcoming work is more complex than the typical task, you may want to set a low value (e.g. 0.5) to indicate that the initiative is likely to take longer. A scale parameter of 0.5 will increase the duration of the forecast by approximately 50%.
Another example: If you know that many people will work on the child cards simultaneously, you may want to increase the value of the scale parameter to reflect the desired level of parallelism. If you set the value to two, the forecasted duration will be approximately half the normal one. One caveat to that, though! If ten people work on the initiative, but they are all allocating only 10% of their time to it, the scale parameter should still be 1 and not 10.
This is where the return scale parameter is essential in helping you adjust your scale parameter to obtain a more accurate result.
2.2.1. The Return Scale Parameter
When you are running a forecast for a completed initiative, you can obtain the return scale value. This is another option you can use to calibrate your forecasting skills. It is available only for finished initiatives and its purpose is to show you what scale parameter would have resulted in a correct forecast. Use this option to teach yourself what scale values to use in the future.
Let's illustrate how to use it with a practical example:
Let us say you have 10 people working on an initiative called "Project 1," and it gets completed in 100 days (cycle time = 100 days).
You have a similar project ("Project 2") coming up, and you would like to use the "Return Scale" functionality to forecast the upcoming project.
The system returns the following values for Project 1:
Optimistic - 0.75 (75 days of cycle time)
Realistic - 1 (100 days of cycle time) (the actual cycle time that was needed to complete "Project 1")
Pessimistic - 1.25 (125 days of cycle time)
Very Pessimistic - 1.5 (150 days of cycle time)
All the days above are taken into consideration if the 'scale' is at the default value of 1.
As mentioned, "Project 2" is similar to “Project 1,” but for the duration of this new project, you are left with only 6 people, as the other people are out of the office.
Having this in mind, you know that it would be impossible to realistically finish "Project 2" in 100 days.
What you would want to do in that case is to use the Optimistic scale parameter of 0.75 from the previous project, and now you would have the following approximate values:
Optimistic - 100 days of cycle time (the realistic cycle time in which the previous initiative was completed). This value now becomes optimistic because of the lowered number of people working on the new project.
Realistic - 125 days of cycle time
Pessimistic - 150 days of cycle time
Very Pessimistic - 200 days of cycle time
After "Project 2" was completed, you have a new "Project 3," which is also identical to “Project 1,” however, this time, all of the 10 people are working on it, and you even have 5 more people that will be helping with it.
This time you know that the job would be completed in less than 100 days because you have more people working on it, so now you can use a pessimistic scale parameter of 1.25 for "Project 3" and get the following approximate results:
Optimistic - 55 days of cycle time
Realistic - 75 days of cycle time - (realistically, you should be able to finish this project in less than 100 days)
Pessimistic - 100 days of cycle time (this was the actual cycle time of the completed "Project 1)
Very Pessimistic - 125 days of cycle time
With respect to sequencing, if you expect to have gaps between finishing one child card and starting the next, set a lower value for the scale parameter.
Quite often, all of these factors will take effect to some extent, and that’s the “art” of forecasting - you need to develop a good intuition about the “scale” parameter. This can only happen if you know your work and process really well. The rest is trial and error.
2.3. Adjusting Your Data Set
2.3.1. Primary Dimension
As mentioned above, the algorithm will find similar cards that have been completed and will build the forecast based on this data set. Use the primary dimension to instruct the algorithm on what the most significant factor to search by is.
For example, if you know that work type A inherently takes longer than work type B, set “type” as a primary dimension.
2.3.2. Secondary Dimension
The secondary dimension is the same as the primary dimension, it just adds another slice to the data.
For example, if you know that user A will be much faster than user B, set “owner” as the secondary dimension.
2.3.3. How to Work with Them?
Forecasts are heavily dependent on the Primary and Secondary dimension parameters. When the historical data is collected, it determines the corresponding data sets based on these two parameters.
For example:
If you've configured the primary dimension to be Owner and the secondary to be Type, it will generate a list of all possible combinations (based on what child cards are linked to the initiative) and then you will get historical data for each pair separately. If you have two child cards with owners Owner1 and Owner2 and types Type1 and Type2, the forecast calculation would need to collect the following groups of historical data:
- Cards that are finished and had the Owner1/Type1 parameters (data set 1)
- Cards that are finished and had the Owner1/Type2 parameters (data set 2)
- Cards that are finished and had the Owner2/Type1 parameters (data set 3)
- Cards that are finished and had the Owner2/Type2 parameters (data set 4)
If there are more cards and parameters, the number of historical data sets might grow significantly.
We have the lowest possible limit of 5 entries per one of these data sets. This means that the forecast will ignore data sets with less than 5 historical items, as they would be statistically invalid.
Hint: If your data sets do not have enough cards, you can set both Primary and Secondary Dimensions to the same parameter, such as Card Type.
3. Understanding the Advanced Settings
The Forecast feature provides the following advanced settings:
- Samples
- Additional cards
- Remove values <
- Remove values >
- Historical data
- Keep outliers
- Forecast children only
- Forecast all children
(1) Samples
By default, the simulation will run 1000 samples (it’s a Monte Carlo simulation). Change this parameter if you need a different number of samples.
(2) Additional Cards
Sometimes scope gets added to an initiative/project at a later stage. If you want to account for such additional work, enter how many child cards you expect to be attached to this initiative in the future.
The additional cards will be distributed proportionally with respect to the primary and secondary dimensions.
(3) Remove Values <
If you know that all child cards will take X days or more, you can exclude historical data with smaller cycle times**. E.g. if you set this value to 2, historical cycle times smaller than two days will be excluded from the data set. This will usually result in a longer duration forecasted.
(4) Remove Values >
If you know that all child cards will take X days or less, you can exclude historical data with greater cycle times**. E.g. if you set this value to 10, historical cycle times greater than ten days will be excluded from the data set. This will usually result in a shorter duration forecasted.
** In the forecasting feature, cycle time is measured between the last date a card has transitioned to In Progress and the last date it was moved to Done.
(5) Historical Data
By default, the simulation will get the last 100 finished cards and will use them to build the forecast. If you want to go farther back in time, you can increase this value. However, this is rarely recommended, as the distant past is rarely similar to the present.
(6) Keep Outliers
By default, the algorithm will clean up the outliers from your data set. If you need all data points in the data set, check this option.
(7) Forecast Children Only
This option is useful if you want to forecast an initiative without considering all other cards on the board. This can be used for expedited requests, as it will assume that no other work blocks the execution of the child cards. Another scenario is when the people working on the initiative have no other assignments and will not be affected by the existing work.
(8) Forecast All Children
This option is useful if you want to calibrate your forecasting skills. The parameter makes sense when you want to compare the forecast with the actual outcome. In fact, if you open the forecast tab of an already finished initiative, this option is selected and cannot be unchecked.
(9) Forecasting Output
The Forecasting Output offers a summary of the forecast results. It includes information about the cards/initiatives that were included/excluded from the forecast, as well as outliers that were removed, and details about the simulation (sample size, standard deviation, etc.). If the simulation was not successful, you will also find troubleshooting tips in the Forecasting Output panel (e.g. “Try increasing the value of the ‘Historical Data’ parameter”).
Important: The number of cards to be forecasted includes all cards that are located before the current one on the board.
4. Disclaimer
The accuracy of the forecasting feature depends on the quality of your historical data. If you have a stable, predictable process, your forecasts will be very accurate. However, if your cycle times fluctuate too much, you allow work to age indefinitely, or you don't set WIP limits and continuously start new work without finishing other tasks first, your forecasts will be inaccurate.
5. FAQ
Does the size of the cards matter?
Generally speaking no. What matters is how much time the cards actually take on the board. If two cards take the same time to finish (cycle time), they will mean the same to the algorithm, even if their sizes are very different.
Does the algorithm take into account the sequence of the initiatives on a timeline?
In the scenario where you have a parent card with two or more child initiatives on a timeline, the algorithm will not account for the sequencing. This is a known limitation, which will be addressed in future releases.