DualBoost: Handling Missing Values with Feature Weights and Weak Classifiers that Abstain

ABSTRACT

Missing values in real world datasets are a common issue. Handling missing values is one of the most key aspects in data mining, as it can seriously impact the performance of predictive models. In this paper we proposed a unified Boosting framework that consolidates model construction and missing value handling. At each Boosting iteration, weights are assigned to both the samples and features. The sample weights make difficult samples become the learning focus, while the feature weights enable critical features to be compensated by less critical features when they are unavailable. A weak classifier that abstains (i.e, produce no prediction when required feature value is missing) is learned on a data subset determined by the feature weights. Experimental results demonstrate the efficacy and robustness of the proposed method over existing Boosting algorithms.

References

1. Kevin Bache and Moshe Lichman. 2013. UCI machine learning repository.
2. Haixia Chen, Yuping Du, and Kai Jiang. 2012. Classification of incomplete data using classifier ensembles. In Systems and Informatics (ICSAI), 2012 International Conference on. IEEE, 2229--2232.
3. Yoav Freund and Robert E Schapire. 1996. Experiments with a new boosting algorithm. In ICML, Vol. 96. 148--156.
4. O. L. Mangasarian and W. H. Wolberg. 1990. Cancer diagnosis via linear programming. SIAM News , Vol. 23, 5 (1990), 1 & 18.
5. Joseph O'Sullivan, John Langford, Rich Caruana, and Avrim Blum. 2000. FeatureBoost: A Meta-Learning Algorithm that Improves Model Robustness.. In ICML . 703--710.
6. Robert E Schapire and Yoram Singer. 1999. Improved boosting algorithms using confidence-rated predictions. Machine learning , Vol. 37, 3 (1999), 297--336.
7. Kristen A Severson, Brinda Monian, J Christopher Love, and Richard D Braatz. 2017. A method for learning a sparse classifier in the presence of missing data for high-dimensional biological datasets. Bioinformatics , Vol. 33, 18 (2017), 2897--2905.
8. Fabrizio Smeraldi, Michael Defoin-Platel, and Mansoor Saqi. 2010. Handling Missing Features with Boosting Algorithms for Protein--Protein Interaction Prediction. In International Conference on Data Integration in the Life Sciences. Springer, 132--147.
9. Geoffrey I Webb. 1998. The problem of missing values in decision tree grafting. In Australian Joint Conference on Artificial Intelligence. Springer, 273--283.
10. Sabit Anwar Zahin, Chowdhury Farhan Ahmed, and Tahira Alam. 2018. An effective method for classification with missing values. Applied Intelligence (2018), 1--22.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Learning settings

      • Batch learning

    • Machine learning algorithms

      • Feature selection

    • Machine learning approaches

      • Classification and regression trees