Brújula Home

Institutional repository of the Universidad Loyola

View Item 
  •   Brújula Home
  • PRODUCCIÓN CIENTÍFICA Y TRANSFERENCIA
  • Departamento Métodos Cuantitativos
  • Artículos
  • View Item
  •   Brújula Home
  • PRODUCCIÓN CIENTÍFICA Y TRANSFERENCIA
  • Departamento Métodos Cuantitativos
  • Artículos
  • View Item
    • español
    • English
JavaScript is disabled for your browser. Some features of this site may not work without it.

Browse

All of BrújulaCommunities and CollectionsAuthorsTitlesKeywordsAuthor profilesThis CollectionAuthorsTitlesKeywords

My Account

Login

Statistics

View Usage Statistics

Añadido Recientemente

Novedades
Repository
How to publish
Visibility
FAQs

Global and Diverse Ensemble Model for Regression

Author:
Durán-Rosal, Antonio Manuel; Ashley, Thomas; Pérez-Rodríguez, Javier; Fernández-Navarro, FranciscoUniversidad Loyola Authority
URI:
https://hdl.handle.net/20.500.12412/6725
ISSN:
0925-2312
DOI:
10.1016/j.neucom.2025.130520
Date:
2025-05-28
Keyword(s):

Ensemble learning

Bagging

Stacking

Diversity

Abstract:

Diversity is a fundamental component in ensemble methods, crucial for enhancing the overall performance and robustness of predictive models. In bagging and boosting, diversity is implicitly generated through the data sampling process. In stacking, diversity is introduced by incorporating heterogeneous machine learning models as base learners, and an error function is created to focus on minimizing the meta-learner’s overall errors. Some models promote diversity directly within the error function; however, they often prioritize generating competitive and diverse individual learners, neglecting the necessity of creating an ensemble that is collectively accurate, as seen in stacking, whilst ensuring diversity among its members. Motivated by this point, two ensemble models are proposed in this manuscript, named Global and Diverse Ensemble Methods. These models incorporate implicit diversity through data, diversity in the heterogeneity of base learners, and an error function designed to produce a competitive overall ensemble with diverse individuals. The two diversity proposals included in the models are negative correlation and squared Pearson correlation. In both cases, the error function incorporates the minimization of the ensemble’s overall error (global error measure) in addition to promoting diversity. The proposed methods have been rigorously tested on 45 publicly available regression datasets using shallow base learners, as well as on 7 additional datasets using deep base learners, yielding very promising results. These findings underscore the importance of integrating these diversity-promoting elements and minimizing the global errors of the ensemble, rather than focusing solely on the errors of individual base learners, in the design of ensemble methods.

Diversity is a fundamental component in ensemble methods, crucial for enhancing the overall performance and robustness of predictive models. In bagging and boosting, diversity is implicitly generated through the data sampling process. In stacking, diversity is introduced by incorporating heterogeneous machine learning models as base learners, and an error function is created to focus on minimizing the meta-learner’s overall errors. Some models promote diversity directly within the error function; however, they often prioritize generating competitive and diverse individual learners, neglecting the necessity of creating an ensemble that is collectively accurate, as seen in stacking, whilst ensuring diversity among its members. Motivated by this point, two ensemble models are proposed in this manuscript, named Global and Diverse Ensemble Methods. These models incorporate implicit diversity through data, diversity in the heterogeneity of base learners, and an error function designed to produce a competitive overall ensemble with diverse individuals. The two diversity proposals included in the models are negative correlation and squared Pearson correlation. In both cases, the error function incorporates the minimization of the ensemble’s overall error (global error measure) in addition to promoting diversity. The proposed methods have been rigorously tested on 45 publicly available regression datasets using shallow base learners, as well as on 7 additional datasets using deep base learners, yielding very promising results. These findings underscore the importance of integrating these diversity-promoting elements and minimizing the global errors of the ensemble, rather than focusing solely on the errors of individual base learners, in the design of ensemble methods.

 

Es la versión preprint del artículo. Se puede consultar la versión final en https://doi.org/10.1016/j.neucom.2025.130520

Es la versión preprint del artículo. Se puede consultar la versión final en https://doi.org/10.1016/j.neucom.2025.130520

 
Show full item record
Collections
  • Artículos
Files in this item
Thumbnail
Global_and_Diverse_Ensemble_Model_for_Regression.pdf (1.721Mb)
Share
Export to Mendeley
Statistics
Usage statistics
Metrics and citations  
Go to Brújula home

Universidad Loyola

Library

Contact

Facebook Loyola BibliotecaTwitter Loyola Biblioteca

The content of the Repository is protected with a Creative Commons license:

Attribution-NonCommercial-NoDerivatives 4.0 Internacional

Creative Commons Image