The combination of multiple classifiers, commonly referred to as a classifier ensemble, has previously demonstrated the ability to improve classification accuracy in many application domains. As a result this area has attracted significant amount of research in recent years. The aim of this paper has therefore been to provide a state of the art review of the most well-known ensemble techniques with the main focus on bagging, boosting and stacking and to trace the recent attempts which have been made to improve their performance. Within this paper we present and compare an updated view on the different modifications of these techniques which have specifically aimed to address some of the drawbacks of these methods namely the low diversity problem in bagging or the over-fitting problem in boosting. In addition we provide a review of different ensemble selection methods based on both static and dynamic approach. We present some new directions which have been adopted in the area of classifier ensembles from a range of recently published studies. In order to provide better understanding on how the ensembles work some existing theoretical studies have been reviewed in the paper.