Table of Contents

**Model browser** is used to select and browse simulation results of different target variables. Models are organized in a tree-like structure. Top-level nodes represent post-processed modeling results, while nested items represent one or more raw models.

When a raw model is selected you can use **Model rank** to access lower model ranks outperformed by the top-ranked model. One of the reasons for inspecting lower model ranks is that you can average them in the post-process panel.

**View > Simulation results > Plot** is a time series chart used to analyse models visually. It is interactive like all other charts in GMDH Shell. You can view values, hold and move the curve, zoom chart, and save the chart to a file. When a model is selected in the Model browser the Plot outputs it immediately.

The chart has the following legend:

**Predicted**, i.e. model predictions are red.

**Model fit** are model values fitted to the data, they are blue.

**Actual data**, i.e. the data available from the initial dataset is gray.

**Confidence band** is the 95% confidence band calculated for predictions. Confidence band calculation uses model values fitted to the data (blue curve), it equals to two standard deviations (2*sigma) of model residuals.

If predicted values have no their own IDs or future timestamps cannot be automatically generated, the forecast values will be marked with *+1*, *+2*, *+3* on the horizontal axes. The Plot outputs post-processed (final) predictions using a thick line with dots. Raw models are plotted with a thin line. Raw model plot is only accessible when a raw model is selected in the Model browser.

**View > Simulation results > Raw model** is used to view raw model selected in the model browser as an equation or a number of equations. GMDH Shell predictions are obtained after applying post-process transformations to raw model predictions.

**Model ID** informs which raw model is selected in the Model browser. The format is `[model name][space]#[model rank]`

.

**Model complexity** informs about the number of coefficients in the model and the number of layers. For example, `13 (layer 5)`

means the model has 5 layers and 13 coefficients.

**Criterion value** informs about the value of Validation criterion configured in the Solver module. Top-ranked model has the smallest criterion value.

Model formula is one or several expressions that relate input variables and predicted variable. Each expression is composed from coefficients and input variable names. The names include one line transformations applied by the preprocessor, so they can be quite long. For example, model formula in the above screenshot contains the following variable names: “H1020.Qty@10”, “H1020.Qty@12”, “|id, workdaysperperiod”, “H1020.Qty@6”.

**Accuracy** is a panel accessible in the menu **View > Simulation results > Accuracy** that shows different accuracy metrics for the model selected in the Model browser.

**Target** shows the name of the model selected in the Model browser.

The table nested in the panel indicates in the upper left corner if it is the **Raw model results** or the **Post-processed results**. Other two header cells are **Model fit** and **Predictions**. The column **Model fit** contains accuracy measures calculated for observations used to crate the model. The column **Predictions** contains accuracy measures calculated for withheld observations, therefore if predicted values have no actual values this column is empty. **Number of observations** in the table informs about the number of actual observations both in the fitted and the withheld subsamples.

**Error measure** is used to choose a metric for calculation of the mean and the root mean errors. Available metrics are the **Absolute**, which outputs mean error values “as is”, the **Range percentage**, i.e. percentage of magnitude of predicted variable, and the **Target percentage**, where for each model value we calculate percentage deviation from the actual value and then the percentage deviations are averaged.

Calculation of predicted variable magnitude involves only the observations used for model training and testing.

Error measure | Mean | Root mean square |

Absolute | Mean absolute error (MAE) | Root mean square error (RMSE) |

Range percentage | Normalized mean absolute error (NMAE) | Normalized root mean square error (NRMSE) |

Target percentage | Mean absolute percentage error (MAPE) | Root mean square percentage error (RMSPE) |

Other accuracy measures available in the table:

**Max. negative error**, the largest underestimation

**Max. positive error**, the largest overestimation

**Residual sum**

**Standard deviation of residuals**

**Coefficient of determination (R²)**

**Correlation** coefficient

Accessible in the menu **View > Simulation results > Importance** the panel helps to understand the value of each input variable in a model.

To calculate importance of variables we replace variables in the model with their mean value one by one and measure the root mean squared error (RMSE) of the “new” model. Original model error is considered a zero percent impact on RMSE and 100% impact is a case where all variables are replaced with their mean.

The impact can easily exceed 100% in the case the variable in a model is multiplied by another variable or squared. A small negative percentage can also happen if a variable is merely useless for the model.

Importance table has the following columns:

**Replaced with mean** column contains the name of a variable tested by replacing with its mean value and two reference tests in the bottom of the list.

**Impact on RMSE** is a percentage value that helps to compare variables and reference values, it is calculated as

`Impact = (R_var - R_ori)/(R_all - R_ori)*100%`

where `R_var`

is RMSE of the variable we consider, `R_ori`

is zero-impact RMSE and `R_all`

is RMSE of a model where all variables are replaced with mean.

**Bars** column visualises difference between the variables.

**RMSE** is a root mean square error obtained for the model in a result of the importance test.

**Table** is a panel accessible in the menu **View > Simulation results > Table**, it is used to output simulation results shown in the Plot as a table of numeric values.

Table has a toolbar with tree buttons:

**Settings** button opens the Settings dialog box.

**Export to Excel** exports the table to Excel spreadsheet located in the project directory.

**Transpose** button changes table arrangement from vertical to horizontal and vice versa.

Settings is a dialog box accessible via the Settings button in the panel toolbar. The dialog box is used to configure the output of the table.

**Ordinal numbers** adds enumeration of observations and predictions to the table with the heading **#**.

**Confidence band** adds **Upper** and **Lower** margins of the confidence band.

**Override** is used to change predicted values manually by typing into the Override cells.

**Residuals** is used to add differences between actual and model values. If the model operates with categorical targets, residuals are replaced with Hit and Miss marks.

Other entries of the table are:

**ID** values are unique identifiers or timestamps.

**Actual** values are actual values of target (predicted) variable.

**Predictions** are post-processed predictions of the target variable.

**Raw model** heading and values replace **Predictions** in the table if a Raw model is selected in the Model browser.

**Forecast report** accessible in the menu **View > Simulation results > Forecast report** contains forecast of all target variables in one table.

You can use the configuration button to change appearance of the report. Also, you can save the report to XLS file or print. To save the report in the PDF, you should use a third party virtual PDF printer.

**Inventory report** is accessible in the menu **View > Simulation results > Inventory report**. It is used to generate purchase orders for inventory replenishment.

The report aggregates demand forecast calculated by GMDH Shell and inventory information such as quantity on hands, lead time, on order quantity, item code and description.

There are three sections in the table: **Inventory**, **Replenishment calculation** and **Demand forecast**

**Inventory** section shows available information about the stock:

**Description**of the items.**Item code****In stock**- the number of item units on-hands.**Lead time**- the time (in months) between the placement of an order and delivery.**On order**- the number of already ordered units of the item.

**Replenishment calculation** section:

**Safe periods**is the number of periods without replenishment in which there are no stockouts according to the forecast.**To order**is the quantity that should be ordered according to**Demand forecast**,**Lead time**, inventory**On hands**and**On order**.**Upper forecast**is the upper bound of the forecast calculated as the sum of the forecast and the 95% confidence band.

**Demand forecast** section shows forecast by periods. **Safe periods** are green. Periods with shortages are red.

**Default lead time** is used to set the lead time for all items that have no lead time information in the Inventory section of the Inventory report.

**Export table** is used to export the whole Inventory report to a spreadsheet.

**Export order** is used to export the items and quantities that have to be replenished.