AKSDA-MSVM: A GPU-accelerated Multiclass Learning Framework for Multimedia

ABSTRACT

In this paper, a combined nonlinear dimensionality reduction and multiclass classification framework is proposed. Specifically, a novel discriminant analysis (DA) technique, called accelerated kernel subclass discriminant analysis (AKSDA), derives a discriminant subspace, and a linear multiclass support vector machine (MSVM) computes a set of separating hyperplanes in the derived subspace. Moreover, within this framework an approach for accelerating the computation of multiple Gram matrices and an associated late fusion scheme are presented. Experimental evaluation in five multimedia datasets, on tasks such as video event detection and news document classification, shows that the proposed framework achieves excellent results in terms of both training time and generalization performance.

References

1. S. Arestis-Chartampilas, N. Gkalelis, and V. Mezaris. GPU accelerated generalised subclass discriminant analysis for event and concept detection in video. In Proc. ACM Multimedia, pages 1219--1222, Brisbane, Australia, 2015.
2. G. Baudat and F. Anouar. Generalized discriminant analysis using a kernel approach. Neural Comput., 12(10):2385--2404, Oct. 2000.
3. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell., 19(7):711--720, Jul. 1997.
4. D. Cai, X. He, and J. Han. Speed up kernel discriminant analysis. VLDB J., 20(1):21--33, Mar. 2011.
5. C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Trans. on Intelligent Systems and Technology, 2:27:1--27:27, 2011.
6. K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res., 2:265--292, Mar. 2002.
7. R.-E. Fan, K.-W. Chang, C.-J. Hsieh et al. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res., 9:1871--1874, 2008.
8. K. Fukunaga. Introduction to statistical pattern recognition (2nd ed.). Academic Press Professional, Inc., San Diego, CA, USA, 1990.
9. M. G. Genton. Classes of kernels for machine learning: A statistics perspective. J. Mach. Learn. Res., 2:299--312, Mar. 2002.
10. N. Gkalelis and V. Mezaris. Video event detection using generalized subclass discriminant analysis and linear support vector machines. In Proc. ACM ICMR, pages 25:25--25:32, Glasgow, UK, 2014.
11. N. Gkalelis and V. Mezaris. Accelerated nonlinear discriminant analysis. arXiv preprint arXiv:1504.07000, 2015.
12. N. Gkalelis, V. Mezaris, and I. Kompatsiaris. Mixture subclass discriminant analysis. IEEE Signal Process. Lett., 15(5):319--332, May 2011.
13. N. Gkalelis, V. Mezaris, I. Kompatsiaris, and T. Stathaki. Mixture subclass discriminant analysis link to restricted gaussian model and other generalizations. IEEE Trans. Neural Netw. Learn. Syst., 24(1):8--21, Jan 2013.
14. A. Habibian et al. Recommendations for video event recognition using concept vocabularies. In Proc. ACM ICMR, pages 89--96, Dallas, Texas, USA, 2013.
15. T. Hofmann, B. Schölkopf, and A. J. Smola. Kernel methods in machine learning. Ann. Statist., 36(3):1171--1220, 2008.
16. S. S. Keerthi, S. Sundararajan et al. A sequential dual method for large scale multi-class linear SVMs. In Proc. 14th ACM SIGKDD, pages 408--416, Las Vegas, USA, Aug. 2008.
17. J. Kittler, M. Hatef, R. P. W. Duin, and J. Matas. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell., 20(3):226--239, Mar. 1998.
18. H.-T. Lin, C.-J. Lin, and R. C. Weng. A note on platt's probabilistic outputs for support vector machines. Mach. Learn., 68(3):267--276, oct 2007.
19. Y. Liu, Y. Liu, S. Zhong, and K. C. Chan. Semi-supervised manifold ordinal regression for image ranking. In Proc. ACM Multimedia, pages 1393--1396, 2011.
20. S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. R. Mullers. Fisher discriminant analysis with kernels. In Neural Networks for Signal Processing IX, 1999. Proc. of the 1999 IEEE Signal Processing Society Workshop, pages 41--48, 1999.
21. P. Over, G. Awad et al. TRECVID 2012 - an introduction to the goals, tasks, data, evaluation mechanisms and metrics. In Proc. TRECVID 2012 Workshop, Gaithersburg, MD, USA, Nov. 2013.
22. F. Radenović, H. Jégou, and O. Chum. Multiple measurements and joint dimensionality reduction for large scale image search with short vectors. In Proc. ACM ICMR, pages 587--590, Shanghai, China, 2015.
23. S. Sonnenburg, G. Ratsch, C. Schafer, and B. Schölkopf. Large scale multiple kernel learning. J. Mach. Learn. Res., 7:1531--1565, Dec. 2006.
24. A. Tefas, C. Kotropoulos, and I. Pitas. Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication. IEEE Trans. Pattern Anal. Mach. Intell., 23(7):735--746, Jul. 2001.
25. V. Vapnik. Statistical learning theory. New York: Willey, 1998.
26. H. Wang and C. Schmid. Action recognition with improved trajectories. In Proc. IEEE Int. Conf. on Computer Vision (ICCV), pages 3551--3558, Sydney, Australia, Dec. 2013.
27. J. Ye, S. Ji, and J. Chen. Multi-class discriminant kernel learning via convex programming. J. Mach. Learn. Res., 9:719--758, Jun. 2008.
28. D. You, O. C. Hamsici, and A. M. Martinez. Kernel optimization in discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):631--638, Mar. 2008.
29. M. Zhu and A. M. Martinez. Subclass discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell., 28(8):1274--1286, Aug. 2006.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

  • Machine learning

    • Machine learning approaches

• Mathematics of computing

  • Probability and statistics

    • Statistical paradigms

      • Dimensionality reduction

• Theory of computation

  • Design and analysis of algorithms

    • Parallel algorithms