Multiclass core vector machine

ABSTRACT

Even though several techniques have been proposed in the literature for achieving multiclass classification using Support Vector Machine(SVM), the scalability aspect of these approaches to handle large data sets still needs much of exploration. Core Vector Machine(CVM) is a technique for scaling up a two class SVM to handle large data sets. In this paper we propose a Multiclass Core Vector Machine(MCVM). Here we formulate the multiclass SVM problem as a Quadratic Programming(QP) problem defining an SVM with vector valued output. This QP problem is then solved using the CVM technique to achieve scalability to handle large data sets. Experiments done with several large synthetic and real world data sets show that the proposed MCVM technique gives good generalization performance as that of SVM at a much lesser computational expense. Further, it is observed that MCVM scales well with the size of the data set.

References

1. Aiolli, F., & Sperduti, A. (2005). Multiclass Classification with Multi-Prototype Support Vector Machines. Journal of Machine Learning Research, 6, 817--850.
2. Bǎdoiu, M., & Clarkson, K. L. (2002). Optimal core sets for balls. In DIMACS workshop on Computational Geometry.
3. Chang, C.-C., & Lin, C.-J. (2001). LIB-SVM: A library for Support Vector Machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
4. Fletcher, R. (2000). Practical methods of optimization, 2nd ed. New York: Wiley-Interscience.
5. Hsu, C.-W., & Lin, C.-J. (2002). A Comparison of Multiclass Support Vector Machines. IEEE Transactions on Neral Networks, 13, 415--425.
6. Kumar, P., Mitchell, J. S. B., & Yildirim, E. A. (2003). Approaximate minimum enclosing balls in high dimensions using core sets. ACM Journal of Experimental Algorithms, 8, Article No. 1.1.
7. Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13, 1443--1472.
8. Smola, A. J., & Schöölkopf, B. (2000). Sparse greedy matrix approximation for machine learning. Proceedings of the Seventeenth International Conference on Machine Learning (pp. 911--918). Stanford, CA, USA.
9. Soman, K. P., Loganathan, R., Vijaya, M. S., Ajay, V., & Shivsubramani, K. (2007). Fast Single-shot Multiclass Proximal Support Vector Machines and Perceptrons. Proceedings of the International Conference on Computing: Theory and Applications (IC-CTA) (pp. 294--298). Kolkata, India.
10. Szedmak, S., & Shawe-Taylor, J. (2005). Multiclass learning at one-class complexity. Technical Report No: 1508, School of Electronics and Computer Science, Southampton, UK.
11. Tsang, I. W., Kwok, J. T., & Cheung, P.-M. (2005). Core Vector Machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6, 363--392.
12. Vapnik, V. N. (1998). Statistical learning theory. John Wiley and Sons.
13. Weston, J., & Watkins, C. (1999). Support Vector Machines for Multi-class Pattern Recognition. Proceedings of the Seventh European Symposium On Artificial Neural Networks (pp. 219--224). Brussels.

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning