Uncovering Internal Prediction Mechanisms of Transformer-Based Chemical Foundation Models

However, efforts to understand how these models learn and how specific predictions can be explained are still limited in chemistry, which hinders trust in these predictions.

To overcome both the current widespread approach of considering supervised deep learning models as black boxes and the limitations of explainable deep learning based on feature attribution, we introduce two general methods from the field of mechanistic interpretability to molecular property prediction. Specifically, we leverage ablation and adapt existing techniques into the regression lens to inspect model predictions of the Transformer-based ChemBERTa foundation model.