Synthesizing fixed point of views from a spinning omnidirectional ball camera

ABSTRACT

Recent years have seen the emergence of a new way to watch sports, that is, watching them from the ball's point of view (POV) taken with cameras embedded in the ball. The problem, however, is that the ball's POV is unstable since the ball spins. In this paper, we describe how we developed a prototype of the ball camera with an omnidirectional camera and propose an algorithm to fix the ball's POV to an arbitrary direction. Feature points are extracted from an entire omnidirectional image and the pose change between frames is estimated by feature matching and the five-point algorithm. The POV of the current frame is modified to the one of the initial frame by applying the product of estimated rotation matrices between all adjacent frames up to the current one. Our algorithm is not limited to our hardware and can be applied to any omnidirectional videos taken with different camera systems.

References

1. J. Cruz-Mota, I. Bogdanova, B. Paquler, M. Bierlaire, and J. Thiran. Scale invariant feature transform on the sphere: Theory and applications. In International Journal of Computer Vision, Vol.98, No.2, pages 217--241, 2012.
2. R. Funakoshi, Y. Okudera, and H. Koike. Synthesizing pseudo straight view from a spinning camera ball. In Proceedings of the 7th Augmented Human International Conference, No. 30, 2016.
3. K. Horita, H. Sasaki, H. Koike, and K. M. Kitani. Experiencing the ball ' s POV for ballistic sports. In Proceedings of the 4th Augmented Human International Conference, pages 128--133, 2013.
4. S. Kasahara, S. Nagai, and J. Rekimoto. First person omnidirectional video: System design and implications for immersive experience. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, pages 33--42, 2015.
5. K. M. Kitani, K. Horita, and H.Koike. Ballcam!: dynamic view synthesis from spinning cameras. In Adjunct proceedings of the 25th annual ACM Symposium on User Interface Software and Technology, pages 87--88, 2012.
6. D. G. Lowe. Distinctive image features from scale-invariant keypoints. In International Journal of Computer Vision, Vol.60, No.2, pages 91--110, 2004.
7. J. M. Morel and G. Yu. ASIFT: A new framework for fully affine invariant image comparison. In SIAM Journal on Imaging Sciences, Vol.2, No.2, pages 438--469, 2009.
8. D. Nistér. An efficient solution to the five-point relative pose problem. In Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.26, No.6, pages 756--777, 2004.
9. J. Pfeil, K. Hildebrand, B. B. C. Gremzow, and M. Alexa. Throwable panoramic ball camera. In SIGGRAPH Asia 2011 Emerging Technologies, No.4, 2011.
10. H. Taira, Y. Inoue, A. Torii, and M. Okutomi. Robust feature matching for distorted projection by spherical cameras. In IPCJ Transactions on Computer Vision and Applications, Vol.7, pages 84--88, 2015.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

      • Image processing