Supervised Random Feature Regression via Projection Pursuit

RF methods offer computational efficiency but are often limited by their reliance on fixed, data-independent feature representations. NNs achieve strong expressive power through deep architectures, yet at the cost of greater computational complexity, reduced interpretability, and sensitivity to hyperparameter choices—particularly in limited-data settings.

This paper proposes a <b>supervised random feature framework based on projection pursuit</b> that bridges these two paradigms. The core estimator, <b>SRF-I</b>, replaces the fixed activation function in classical RF with data-adaptive univariate basis functions learned along random projections in a supervised manner. Building on SRF-I, we develop <b>SRF-II</b>, a blockwise hierarchical extension that aggregates block-level predictions through a projection pursuit regression model, enhancing both scalability and modeling flexibility.