3.4.3. Descriptor Selection
A modeler can use any combination of qualified descriptors to train a model, but not all qualified descriptors are equally well-correlated with the target property. The presence of irrelevant descriptors will make training more difficult, because the learning algorithm has to distinguish between the real information and the noise. In addition, processing more descriptors will increase the computational cost. To build better models and to reduce the computational cost, it is desirable to select descriptors that are the most informative regarding the target property. In the field of chemometrics, this is known as “supervised descriptor selection”.
Sensitivity analysis refers to various ways to sort descriptors by their relevant information content. ADMET Modeler provides several options: input-gradient, truncated linear analysis, iterative truncated linear analysis and by genetic algorithm (GA). This is closely related to but functionally distinct from the Descriptor Sensitivity Analysis (DSA) applied to model outputs.
Sorting the descriptors allows you to begin training models with a small number of highly sensitive descriptors and then incrementally add lower-ranked descriptors until an optimal set is determined. Which works best in any particular case depends on the data set, the descriptors available and the endpoint of interest. Note that sensitivity analysis is applied only to the training pool; the test set is not used in any way.
In Truncated Linear Analysis (TLA), a multiple linear regression model is built with the autoscaled descriptors and the absolute value of each coefficient is used as a measure of the respective descriptor's influence on the target property. The descriptors are sorted on that basis, so that the one with the most influence is included in every architecture, the second most influential one in included in every architecture with more than one input, etc.
Iterative Truncated Linear Analysis (ITLA) involves replacing all descriptors and the dependent variable values by their Spearman ranks across all observations. TLA is then run on the ranks, and the lowest ranking descriptor is assigned a sensitivity of 1.0 and removed from the descriptor pool. TLA is run again with this descriptor removed, and the lowest ranking descriptor is again removed from the pool and assigned a sensitivity of 2.0. This process is repeated until all descriptors have been recursively removed from the descriptor pool. The sensitivity values assigned each descriptor reflect the order in which they were removed from the descriptor pool during this process.
For the Input Gradient method, multiple neural networks are trained with different random starting weights and all of the descriptors under consideration. The average derivatives of each descriptor with respect to the network output are then calculated analytically and ranked by their absolute values. This ranking identifies the “best” descriptors from the larger pool and incorporates them preferentially into the simpler models. This process is repeated for each number of neurons specified, so descriptor sensitivities - and, hence, their order of inclusion in models - may vary from row to row in the performance grid. This contrasts with TLA and ITLA, where the sensitivities are the same for all architectures.
A Genetic Algorithm (GA) method is provided to explore different descriptor combinations, evaluating each based on its ability to model the training and verification sets. A population of models (chromosomes), each one consisting of a different descriptor combination is first randomly generated, and then genetic mutation and crossover operations are performed on members of the population. If a mutation or crossover yields a non-duplicate descriptor combination performing better than the worst member of the population, it replaces the worst member. Kernel PLS is used as the regression method for training/evaluating each descriptor combination.
For ANNE models, the GA may be run once per cell, once per row, or once for the entire grid. In the first case, descriptors are chosen based on the member of the GA population that yields the best cross-validated statistic. Those descriptors are assigned the highest sensitivity and all descriptors are then added additional sensitivity increments based on their frequency of occurrence in the GA population weighted by the cross-validated fitness score of each chromosome in which they occur. In the case of row and grid-based runs, sensitivities of all descriptors are calculated from the sum of cross-validated fitness scores of each chromosome in which they appear.
For cell and row-based GA runs, the number of KPLS components is taken as the number of neurons of the row. Otherwise, for grid-based runs or non-ANNE models, the number of KPLS components is set to 5. For cell-based runs, the number of descriptors selected for each member of the GA population is set to the number of descriptors for that cell. For row and grid-based runs, this is set to 50% of the descriptor pool.
The speed of this method depends on the number of steps that are run, but is typically somewhat slower than the Input Gradient method. Cell-based mode is the slowest setting, but generally gives improved results. In-house experience with this method indicates that model quality in terms of RMSE is usually similar to Truncated Linear Analysis or Input Gradient, but that good models can be found using significantly fewer descriptors than with other methods. Note that, because KPLS is used as the regression method, run time and memory overhead will scale as the square of the number of observations. However, when the number of observations exceeds the Maximum number of observations setting in the Descriptor Selection dialog, then a subset of the observations for running the GA is chosen based on that setting using the K-means algorithm to avert the high memory requirements of the method.