2.2.5. Out-of-Scope Factor

The interpretation of ADMET Risk rules becomes uncertain when the model prediction for a particular compound is out-of-scope because that prediction is less reliable, and the Out-of-Scope Factor introduced in ADMET Predictor 10 helps you deal semi-quantitatively with that uncertainty. Basically, it treats any rule dependent on a prediction that has gone out-of-scope as having been triggered if it could be triggered by that prediction being in scope, but then attenuates the resulting incremental increase in the Risk score by an Out-of-Scope Factor, which is 0.5 by default.

The Risk rule for fraction unbound in plasma (fu), for example, has a weight of 1.0 and is given by:

fu:     fup_hum < [4,6]

where the interval [4,6] defines a soft threshold. Suppose the fup_hum prediction is in-scope. If it is 4 or less, the rule is not triggered, and ADMET_Risk is not incremented at all. If, on the other hand, it is 6 or greater, the Risk is incremented by 1. If it lies somewhere in between, the Risk is incremented proportionately - an in-scope prediction of 4.5, for example, would yield a Risk increment of 1.0 * (4.5 - 4)/(6 - 4) = 0.25. In either of the latter two cases, the fu mnemonic would be added to the ADMET_Code list.

Suppose, on the other hand, that the hum_fup% prediction is out-of-scope. In that case ADMET_Risk gets incremented regardless of what the predicted value is but softened to the degree specified by the Out-of-Scope Factor: by default, that would be 0.5 * 1.0 = 0.5. Here, too, the fu mnemonic would be added to the ADMET_Code list, but with a "+" or "-" flag appended (i.e., as fu+ or fu-) to indicate that it was triggered by an out-of-scope condition rather than by the prediction per se: "+" is appended if the predicted value would have triggered the rule in the absence of the out-of-scope condition (here, had it been >4) and "-" is appended otherwise.

The hERG rule is a more complicated example. It also has a weight of 1.0 but is defined by:

hERG: hERG_Filter = Yes AND hERG_pIC50 > [5.5,6.0]

Because the rule has two conditions connected by a logical AND, an in-scope hERG_Filter prediction of No prevents anything being added to the TOX_Risk or ADMET_Risk scores regardless of the corresponding hERG_pIC50 prediction. A hERG_Filter prediction of Yes and a hERG_pIC50 prediction of 5.7, on the other hand, increments the TOX_Risk and ADMET_Risk scores by 0.4 = (5.7 - 5.5)/(6.0 - 5.5) - provided both predictions are in-scope. In addition, the hERG mnemonic would be added to the TOX_Code and ADMET_Code lists.

Suppose, on the other hand, that the hERG_Filter prediction for a particular compound is out of scope. In that case the first condition will be treated as having been violated regardless of whether the actual prediction is Yes or No, and any Risk penalty that results will be by default multiplied by the Out-of-Scope Factor, i.e., 0.5 by default. If the corresponding hERG_pIC50 prediction is an in-scope value less than or equal to 5.5, however, there will be no incremental Risk to multiply and nothing will be added to the Code lists.

If, however, the hERG_pIC50 prediction is an in-scope value of 5.7, then an out-of-scope hERG_Filter prediction will yield a Risk increment of 0.5 * 0.4 = 0.2 and a hERG+ mnemonic will be added to the TOX_Code and ADMET_Code lists if the hERG_Filter prediction is Yes. An out-of-scope hERG_Filter prediction of No, on the other hand, will cause a hERG- mnemonic to be added, since the rule would not have been triggered had the prediction been in-scope.

Similarly, if a hERG_pIC50 prediction is out-of-scope, the condition dependent upon it will be treated as having been violated, regardless of what the value of that prediction is. If both hERG_Filter and hERG_pIC50 predictions are out-of scope in this example, the incremental Risk will by default be 0.5 * 1.0 = 0.5.

The Out-of-Scope Factor can be changed in the Settings dialog. Changing it to 0.0 obviates any effect on Risk Scores, but "+" and "–" flags will still be appended to the mnemonics in the Risk Codes. The hERG example above can be used to illustrate the effect of that change, as follows.

In this case when the hERG_Filter prediction is an out-of-scope Yes and the corresponding hERG_pIC50 prediction is an in-scope 5.7, the Risk increment will be 0.4 (the same as if the hERG_Filter prediction had been in-scope) but the output mnemonic would be hERG+. If, on the other hand, the hERG_Filter prediction was an out-of-scope No, the Risk increment would be 0 and the output mnemonic would be hERG-.

Examples of these various hERG scenarios are summarized in the Table below for hERG_pIC50 = 5.7.

Accommodating determining descriptors

Classification models can and are often are used as "gatekeepers" in ADMET Predictor by specifying them as determining descriptors for regression models. The CYP1A2_CLint model is a case in point: all examples in its training pool are necessarily substrates, so predictions for structures expected to not be substrates are necessarily suspect. Hence if CYP1A2_Substr = No for a molecule, CYP1A2_CLint is set to NonSubstrate, which gets treated as 0 when it comes to numerical evaluations.

Unfortunately, that means that a rule of the form:

1A2: CYP1A2_CLint > [20,40] AND CYP1A2_Substr = No

will be treated as in-scope in terms of Risk rules even if a No prediction is out-of-scope and the CYP1A2_CLint prediction would have been, say, 60 had the predicted substrate classification been Yes. That is because the determining descriptor logic resets the output CYP1A2_CLint value to 0, which is below the threshold. The Out-of-Scope Factor is not triggered in this case because the conditions are connected by a logical AND (see above).

Fortunately, there is a way around the problem, though the solution does look odd at first glance. The CYP1A2 rule actually found in the ADMET_Risk and [CYP_Risk] model files, for example, is:

1A2: CYP1A2_CLint > [20,40] OR  ( CYP1A2_Substr = No AND CYP1A2_Substr = Yes )

Obviously the second half of the rule can never be literally true in and of itself. Because the halves are connected by a logical OR, however, an out-of-scope CYP1A2_Substr prediction will trigger an appropriate application of the Out-of-Scope Factor regardless of what happens with regard to the first half of the rule.

You should consider using a similar formulation if you have occasion to create a Risk rule for one of your own models that makes use of a determining descriptor as a "gatekeeper."