Model browser is used to select and browse simulation results of different target variables. Models are organized in a tree-like structure. Top-level nodes represent post-processed modeling results, while nested items represent one or more raw models.
When a raw model is selected you can use Model rank to access lower model ranks outperformed by the top-ranked model. One of the reasons for inspecting lower model ranks is that you can average them in the post-process panel.
View > Simulation results > Plot is a time series chart used to analyse models visually. It is interactive like all other charts in GMDH Shell. You can view values, hold and move the curve, zoom chart, and save the chart to a file. When a model is selected in the Model browser the Plot outputs it immediately.
The chart has the following legend:
Predicted, i.e. model predictions are red.
Model fit are model values fitted to the data, they are blue.
Actual data, i.e. the data available from the initial dataset is gray.
Confidence band is the 95% confidence band calculated for predictions. Confidence band calculation uses model values fitted to the data (blue curve), it equals to two standard deviations (2*sigma) of model residuals.
If predicted values have no their own IDs or future timestamps cannot be automatically generated, the forecast values will be marked with +1, +2, +3 on the horizontal axes. The Plot outputs post-processed (final) predictions using a thick line with dots. Raw models are plotted with a thin line. Raw model plot is only accessible when a raw model is selected in the Model browser.
View > Simulation results > Raw model is used to view raw model selected in the model browser as an equation or a number of equations. GMDH Shell predictions are obtained after applying post-process transformations to raw model predictions.
Model ID informs which raw model is selected in the Model browser. The format is
[model name][space]#[model rank].
Model complexity informs about the number of coefficients in the model and the number of layers. For example,
13 (layer 5) means the model has 5 layers and 13 coefficients.
Criterion value informs about the value of Validation criterion configured in the Solver module. Top-ranked model has the smallest criterion value.
Model formula is one or several expressions that relate input variables and predicted variable. Each expression is composed from coefficients and input variable names. The names include one line transformations applied by the preprocessor, so they can be quite long. For example, model formula in the above screenshot contains the following variable names: “H1020.Qty@10”, “H1020.Qty@12”, “|id, workdaysperperiod”, “H1020.Qty@6”.
Accuracy is a panel accessible in the menu View > Simulation results > Accuracy that shows different accuracy metrics for the model selected in the Model browser.
Target shows the name of the model selected in the Model browser.
The table nested in the panel indicates in the upper left corner if it is the Raw model results or the Post-processed results. Other two header cells are Model fit and Predictions. The column Model fit contains accuracy measures calculated for observations used to crate the model. The column Predictions contains accuracy measures calculated for withheld observations, therefore if predicted values have no actual values this column is empty. Number of observations in the table informs about the number of actual observations both in the fitted and the withheld subsamples.
Error measure is used to choose a metric for calculation of the mean and the root mean errors. Available metrics are the Absolute, which outputs mean error values “as is”, the Range percentage, i.e. percentage of magnitude of predicted variable, and the Target percentage, where for each model value we calculate percentage deviation from the actual value and then the percentage deviations are averaged.
Calculation of predicted variable magnitude involves only the observations used for model training and testing.
|Error measure||Mean||Root mean square|
|Absolute||Mean absolute error (MAE)||Root mean square error (RMSE)|
|Range percentage||Normalized mean absolute error (NMAE)||Normalized root mean square error (NRMSE)|
|Target percentage||Mean absolute percentage error (MAPE)||Root mean square percentage error (RMSPE)|
Other accuracy measures available in the table:
The Importance panel accessible in the menu View > Simulation results > Importance shows the absolute number of times certain model component was used in the set of obtained models. In case of a linear neuron function it represents the importance of input variables directly.
Table is a panel accessible in the menu View > Simulation results > Table, it is used to output simulation results shown in the Plot as a table of numeric values.
Table has a toolbar with tree buttons:
Settings button opens the Settings dialog box.
Export to Excel exports the table to Excel spreadsheet located in the project directory.
Transpose button changes table arrangement from vertical to horizontal and vice versa.
Settings is a dialog box accessible via the Settings button in the panel toolbar. The dialog box is used to configure the output of the table.
Ordinal numbers adds enumeration of observations and predictions to the table with the heading #.
Confidence band adds Upper and Lower margins of the confidence band.
Override is used to change predicted values manually by typing into the Override cells.
Residuals is used to add differences between actual and model values. If the model operates with categorical targets, residuals are replaced with Hit and Miss marks.
Other entries of the table are:
ID values are unique identifiers or timestamps.
Actual values are actual values of target (predicted) variable.
Predictions are post-processed predictions of the target variable.
Raw model heading and values replace Predictions in the table if a Raw model is selected in the Model browser.
Forecast report accessible in the menu View > Simulation results > Forecast report contains forecast of all target variables in one table.
You can use the configuration button to change appearance of the report. Also, you can save the report to xls file or print. To save the report in the pdf format, you should use a third party virtual pdf printer.