A Fusion of Stacking with Dynamic Integration

In this paper we present a novel method that fuses the ensemble meta-techniques of Stack-ing and Dynamic Integration (DI) for regres-sion problems, without adding any major computational overhead. The intention of the technique is to benefit from the varying per-formance of Stacking and DI for different da-ta sets, in order to provide a more robust technique. We detail an empirical analysis of the technique referred to as weighted Meta –Combiner (wMetaComb) and compare its performance to Stacking and the DI technique of Dynamic Weighting with Selection. The empirical analysis consisted of four sets of experiments where each experiment recorded the cross-fold evaluation of each technique for a large number of diverse data sets, where each base model is created using random feature selection and the same base learning al-gorithm. Each experiment differed in terms of the latter base learning algorithm used. We demonstrate that for each evaluation, wMetaComb was able to outperform DI and Stacking for each experiment and as such fuses the two underlying mechanisms suc-cessfully.