CatBoost and random forest algorithms in binary classification tasks

Abstract

Among the many ML techniques, ensemble learning methods have grabbed significant attention due to their enhanced predictive performance achieved by combining multiple learning algorithms. Notably, ensemble methods like Random Forest, CatBoost have demonstrated superior capabilities in various predictive tasks, including numerous Kaggle competitions. In this study, I conduct an analysis of these two ensemble learning algorithms within the context of a binary classification task. This paper details the dataset selected, followed by the development of classification models using Random Forest, and CatBoost algorithms. I systematically evaluate and compare the performance of these models, analyzing the impact of hyperparameter optimization by Bayesian Optimization, model suitability based on features present, for both algorithms. The findings offer insights into the suitability of specific preprocessing techniques and model selections for each dataset, contributing to the optimization of ensemble learning applica tions in classification tasks.