Performance Statistics (.rms)

Performance Statistics (.rms)

The RMS file records the performance statistics of all models (which are usually ensemble models) in the grid, one for each architecture examined. For ANNE, each architecture is defined by the number of neurons and the number of inputs; for SVM, it is different numbers of inputs. Those parameters are specified on the respective model method's Settings page.

Performance statistics for each model are presented in three columns, corresponding to the performance on the training set, the verify set, and the test set of data points. Methods other than ANNE do not have the concept of “neurons”, however, for maximum format compatibility; we report a “Neurons” entry for all methods.

There are a few exceptions. For SVME models, we do not show the performance statistics for the verify set. For KPLS models, there is only one ensemble because we use all descriptors for KPLS model building. There is no ensemble for MLR because there is no verification set. We simply show the statistics of MLRs with different inputs.

Performance for regression models is expressed given in terms of RMSE (root mean square error), MAE (mean absolute error), etc. Classification performance is expressed in terms of sensitivity (SENS), specificity (SPEC), Youden's index (YOUDEN), Matthews' correlation coefficient (MCC) and false rate (FALSERATE). Missing values (e.g., RMSE for classification models or false rate for regression models) are indicated by entries of "-9.77E+02".

The RMS file is loaded when you open a modeling project. The following is a sample RMS file that has been modified for clarity; ellipses (...) indicate multiple omitted lines .

ADMET Modeler(TM) 4.1.0047 Ensemble model performance file:

f:\admetpredictorvb\tutorial\example 1\model_0001\Demo2D 4/30/2010 11:35:41 AM

# --------------------------------------------------------------------------------

ProcessType    1    0

Neurons(min, max, step):    1    2    1

Inputs (min, max, step):    8    32    8

Neurons =1

Inputs =8

RMSE  =    1.24E+00    1.18E+00    1.52E+00

MAE  =    9.49E-01    1.03E+00    1.12E+00

SRCC  =    7.41E-01    6.97E-01    7.41E-01

RSQD  =    6.33E-01    5.74E-01    6.34E-01

QSQD  =    2.09E-01    7.10E-01    5.92E-01

SLOPE =    1.03E+00    9.90E-01    1.35E+00

INTERCEPT=    -4.37E-02    5.29E-02    -6.27E-01

FALSERATE=    -9.77E+02    -9.77E+02    -9.77E+02

# --------------------------------------------------------------------------------

Neurons =1

Inputs =16

RMSE  =    1.01E+00    9.23E-01    1.01E+00

MAE  =    7.52E-01    8.66E-01    8.06E-01

SRCC  =    7.97E-01    7.44E-01    7.90E-01

RSQD  =    7.65E-01    7.08E-01    8.40E-01

QSQD  =    3.44E-01    7.86E-01    8.17E-01

SLOPE =    1.03E+00    9.89E-01    1.20E+00