Robustness in sum-product networks: from measurement to ensembles

Detail is presented of algorithms and complexity results for common inference tasks. We introduce the concept of robustness as a metric for prediction reliability, obtained through perturbing the weights of the SPN within a credal using CSPNs. Experiments are performed, using standard categorical datasets and a real world case study, that show empirically that CSPNs can distinguish between reliable and unreliable classifications of SPNs.

Thus robustness can be seen as providing an important tool for the analysis of such models. <br><br>An extension of CSPNs to facilitate robustness analysis over datasets containing continuous variables is achieved through altering the leaf nodes to propagate density values. Experiments across several continuous datasets are used to demonstrate that CSPNs are still an effective tool for measuring model robustness, with conclusions made using categorical data continuing to hold in the presence of continuous data.

We introduce the concept of adding weights to the children of product nodes in the base SPN structure as exponents to the value computed for each child. A number of methods for calculating this method during the learning process are investigated alongside methods of scaling such values. Some modest but limited potential is observed for gaining accuracy at the risk of losing model explainability.<br><br>We then expand on our work on robustness measurements by investigating their utility for deferring classification across an ensemble of classifiers.

We demonstrate that performance gains can be obtained with such an approach in an ad-hoc hierarchical setting. From this, we develop a new method of ensemble learning using SPNs through the systematic creation of a hierarchy of learned classifiers. In testing time, this hierarchical approach defers the classification of the ensemble model to the hierarchical layer deemed most confident according to its robustness value computed by a CSPN.

A proof is presented to show that our approach can only improve classification accuracy with respect to the initial classifier in the ensemble hierarchy. This proof is given empirical weight through multiple experiments using a large selection of standard categorical datasets. Further to this, the behaviour of the hierarchical SPN continues to be observed with variations to the number of layers and strongest learners of the hierarchy.

This approach is shown to be more powerful than a number of state of the art ensemble-strategy competitors.