SDSS QSO DR7 and DR9

Given the importance of photometric redshift estimation, we adopt it as our case study. We synthetically created 4520 features by combining magnitudes, errors, radii, and ellipticities of quasars, taken from the Sloan Digital Sky Survey (SDSS). We apply a forward selection process, a recursive method in which a huge number of feature sets is tested through a k-Nearest-Neighbours algorithm, leading to a tree of feature sets.

The branches of the feature tree are then used to perform experiments with the random forest, in order to validate the best set with an alternative model. We demonstrate that the sets of features determined with our approach improve the performances of the regression models significantly when compared to the performance of the classic features from the literature. The found features are unexpected and surprising, being very different from the classic features.

Therefore, a method to interpret some of the found features in a physical context is presented. The feature selection methodology described here is very general and can be used to improve the performance of machine learning models for any regression or classification task.