MagTrack: Enabling Safe Driving Monitoring with Wearable Magnetics

ABSTRACT

"Hands on the wheel, eyes on the road" is the central guideline of safe vehicle driving practices. Many advanced driver assistance systems can effectively detect abnormal vehicle motions. However, these systems often leave insufficient time for drivers to respond to complex road situations, especially when the drivers are distracted. To reduce accidents, it is essential to detect whether a driver complies with safe driving guidelines in real time and provide warnings early before any dangerous maneuvers occur. There are vision-based driver distraction monitoring systems which rely on cameras in high-end vehicles, but their performances are heavily constrained by visibility requirements. In this paper, we present MagTrack, a driver monitoring system that is based on tracking magnetic tags worn by the user. With a single smartwatch and two low-cost magnetic accessories: a hand magnetic ring and a head magnetic eyeglasses clip, our system tracks and classifies a driver's bimanual and head movements simultaneously using both analytical and approximation sensing models. Our approach is robust to driver's postures, vehicles, and environmental changes. We demonstrate that a wide range of activities can be detected by our system, including bimanual steering, visual and manual distractions, and lane changes and turns. In extensive road tests with 500+ instances of driving activities and 500+ minutes of road driving with 10 subjects, MagTrack achieves 87% of precision and 90% of recall rate on the detection of unsafe driving activities.

References

1. N. Aksan, L. Sager, S Hacker, Marini R., Dawson J., Anderson S., and Rizzo M. 2016. Forward Collision Warning: Clues to Optimal Timing of Advisory Warnings. In SAE International Journal of Transportation Safety .
2. Nawal Alioua, Aouatif Amine, Alexandrina Rogozan, Abdelaziz Bensrhair, and Mohammed Rziza. 2016. Driver head pose estimation using efficient descriptor fusion. EURASIP Journal on Image and Video Processing, Vol. 2016, 1 (2016), 2.
3. Amazon. 2019. Driver Alertness System. https://www.amazon.com/dp/B00NAQOVF6/ref=psdc_7459508011_t1_B004F2OEEM.
4. Takashi Aoki, Jonathan Feng-Shun Lin, Dana Kulić, and Gentiane Venture. 2016. Segmentation of human upper body movement using multiple IMU sensors. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 3163--3166.
5. Frederick J Berg, Timothy J Bennett, Alan C Davis, Richard K Riefe, Ronald H Dybalski, and Timothy M Phillips. 2007. Steering system with haptic driver warning. US Patent 7,280,046.
6. Chongguang Bi, Jun Huang, Guoliang Xing, Landu Jiang, Xue Liu, and Minghua Chen. 2017. SafeWatch: A Wearable Hand Motion Tracking System for Improving Driving Safety. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation (IoTDI '17). ACM, New York, NY, USA, 223--232.
7. BioStamp. 2019. BioStamp Wearables Health Monitoring. https://www.mc10inc.com.
8. Dongyao Chen, Kyong-Tak Cho, Sihui Han, Zhizhuo Jin, and Kang G. Shin. 2015. Invisible Sensing of Vehicle Steering with Smartphones. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '15). ACM, New York, NY, USA, 1--13.
9. Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D input using two magnetic sensors. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 237--244.
10. Ke-Yu Chen, Rahul C Shah, Jonathan Huang, and Lama Nachman. 2017. Mago: Mode of Transport Inference Using the Hall-Effect Magnetic Sensor and Accelerometer. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, 2 (2017), 8.
11. Chihwen Cheng, Xueliang Huo, and Maysam Ghovanloo. 2009. Towards a magnetic localization system for 3-D tracking of tongue movements in speech-language therapy. In Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE. IEEE, 563--566.
12. In-Ho Choi, Chan-Hee Jeong, and Yong-Guk Kim. 2016. Tracking a driver's face against extreme head poses and inference of drowsiness using a hidden Markov model. Applied Sciences, Vol. 6, 5 (2016), 137.
13. Ashraf Darwish and Aboul Ella Hassanien. 2011. Wearable and implantable wireless sensor network solutions for healthcare monitoring. Sensors, Vol. 11, 6 (2011), 5561--5595.
14. Kamal El Farouki, Emmanuel Lagarde, Ludivine Orriols, Manuel-Pierre Bouvard, Benjamin Contrand, and Cédric Galéra. 2014. The increased risk of road crashes in attention deficit hyperactivity disorder (ADHD) adult drivers: driven by distraction? Results from a responsibility case-control study. PLoS one, Vol. 9, 12 (2014), e115002.
15. Aydin Farajidavar, Jacob M Block, and Maysam Ghovanloo. 2012. A comprehensive method for magnetic sensor calibration: A precise system for 3-D tracking of the tongue movements. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE. IEEE, 1153--1156.
16. Firehouse. 2019. Fire Technology. https://www.firehouse.com/ tech- comm/ mobile- technology- accessories/ article/ 21000640/ wearable- technology- for- firefighters.
17. Center for Disease Control. 2019. Distracted Driving Statistics. https://www.cdc.gov/motorvehiclesafety/distracted _driving.
18. Pia M Forsman, Bryan J Vila, Robert A Short, Christopher G Mott, and Hans PA Van Dongen. 2013. Efficient driver drowsiness detection at moderate levels of drowsiness. Accident Analysis & Prevention (2013).
19. Valerie Galluzzi, Ted Herman, and Philip Polgreen. 2015. Hand Hygiene Duration and Technique Recognition Using Wrist-worn Sensors. In Proceedings of the 14th International Conference on Information Processing in Sensor Networks (IPSN '15). ACM, New York, NY, USA, 106--117.
20. Mostafa Haghi, Kerstin Thurow, and Regina Stoll. 2017. Wearable devices in medical internet of things: scientific research and commercially available devices. Healthcare informatics research, Vol. 23, 1 (2017), 4--15.
21. Brett O Hall. 2001. Collision avoidance system. US Patent 6,223,125.
22. Xinying Han, Hiroaki Seki, Yoshitsugu Kamiya, and Masatoshi Hikizu. 2010. Wearable handwriting input device using magnetic field: 2nd report: Influence of misalignment of magnet and writing plane. Precision Engineering, Vol. 34, 3 (2010), 425--430.
23. Mark A Hanson, Harry C Powell Jr, Adam T Barth, Kyle Ringgenberg, Benton H Calhoun, James H Aylor, and John Lach. 2009. Body area sensor networks: Challenges and opportunities. Computer, Vol. 42, 1 (2009), 58--65.
24. Qichang He, Wei Li, Xiumin Fan, and Zhimin Fei. 2016. Evaluation of driver fatigue with multi-indicators based on artificial neural network. IET Intelligent Transport Systems, Vol. 10, 8 (2016), 555--561.
25. Kasthuri Jayarajah, Vigneshwaran Subbaraju, Noel Athaide, Lakmal Meeghapola, Andrew Tan, and Archan Misra. 2018. Can multimodal sensing detect and localize transient events?. In Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR IX, Vol. 10635. International Society for Optics and Photonics, 106351A.
26. Qiang Ji and Xiaojie Yang. 2002. Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real-Time Imaging, Vol. 8, 5 (2002), 357--377.
27. Haojian Jin, Zhijian Yang, Swarun Kumar, and Jason I Hong. 2018. Towards wearable everyday body-frame tracking using passive rfids. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, 4 (2018), 145.
28. Derick A Johnson and Mohan M Trivedi. 2011. Driving style recognition using a smartphone as a sensor platform. In Intelligent Transportation Systems (ITSC), 2011 14th International IEEE Conference on. IEEE, 1609--1615.
29. Philip W Kithil, Roger D Jones, and John McCuish. 2001. Driver alertness detection research using capacitive sensor array . Technical Report. SAE Technical Paper.
30. Klic. 2019. Klic Wearables. (2019). https://www.klik.co/en/wearables.
31. T. J. Kline. 2007. Using Efficient Steering Techniques. Journal of traffic safety education, Vol. 55, 1 (2007), 14--16, 22. http://dx.doi.org/
32. Dong-Min Lee. 2001. Method for detecting lane deviation of vehicle. US Patent 6,317,057.
33. John Lee, Daniel McGehee, Timothy Brown, and Michelle L Reyes. 2002. Collision Warning Timing, Driver Distraction, and Driver Response to Imminent Rear-End Collisions in a High-Fidelity Driving Simulator. Human factors (2002).
34. Youngki Lee, Chulhong Min, Chanyou Hwang, Jaeung Lee, Inseok Hwang, Younghyun Ju, Chungkuk Yoo, Miri Moon, Uichin Lee, and Junehwa Song. 2013. Sociophone: Everyday face-to-face interaction monitoring platform using multi-phone sensor fusion. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 375--388.
35. Kun Li, Man Lu, Fenglong Lu, Qin Lv, Li Shang, and Dragan Maksimovic. 2012. Personalized driving behavior monitoring and analysis for emerging hybrid vehicles. In International Conference on Pervasive Computing . Springer, 1--19.
36. BOTTOM LINE. 2012. Get with the times: You're driving all wrong . https://www.nbcnews.com/businessmain/get-times-youre-driving-all-wrong-518710/.
37. Luyang Liu, Hongyu Li, Jian Liu, Cagdas Karatas, Yan Wang, Marco Gruteser, Yingying Chen, and Richard P Martin. 2017. Bigroad: Scaling road data acquisition for dependable self-driving. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 371--384.
38. Maven Machine. 2019. Maven co-pilot. https://mavenmachines.com/maven-co-pilot.
39. Chinmay Manohar, James A Levine, Debashis K Nandy, Ahmed Saad, Chiara Dalla Man, Shelly K McCrady-Spitzer, Rita Basu, Claudio Cobelli, Rickey E Carter, Ananda Basu, et almbox. 2012. The effect of walking on postprandial glycemic excursion in patients with type 1 diabetes and healthy people. Diabetes Care, Vol. 35, 12 (2012), 2493--2499.
40. Uwe Maurer, Asim Smailagic, Daniel P Siewiorek, and Michael Deisher. 2006. Activity recognition and monitoring using multiple sensors on different body positions . Technical Report. Carnegie-Mellon Univ, Pittsburgh, PA, USA.
41. mboxState Farm Insurance. 2017. Things Have Changed Since You Learned to Drive . https://www.statefarm.com/simple-insights/auto-and-vehicles/things-have-changed-since-you-learned-to-drive.
42. mboxTexas Driver Training Division, Texas Driver License Division. 2016. Behind-the-Wheel Instruction Guide .
43. John McDonald. 18--7--2006. Rotation About an Arbitrary Axis. In CGEMS - Computer Graphics Educational Materials, (Ed.). The Eurographics Association.
44. Summit Medical. 2019. Wrist Injury Recovery Exercises. https://www.summitmedicalgroup.com/library/adult_health/sma_wrist_tendonitis_exercises/.
45. Wei Mei, Gan-Lin Shan, and X Rong Li. 2007. Simultaneous tracking and classification: a modularized scheme. IEEE transactions on aerospace and electronic systems, Vol. 43, 2 (2007).
46. Michael I Miller, Anuj Srivastava, and Ulf Grenander. 1995. Conditional-mean estimation via jump-diffusion processes in multiple target tracking/recognition. IEEE Transactions on Signal Processing, Vol. 43, 11 (1995), 2678--2690.
47. Erik Murphy-Chutorian and Mohan Manubhai Trivedi. 2010. Head pose estimation and augmented reality tracking: An integrated system and evaluation for monitoring driver awareness. IEEE Transactions on intelligent transportation systems, Vol. 11, 2 (2010), 300--311.
48. Phuc Nguyen, Hieu Nguyen, Dong Nguyen, Thang N Dinh, Hung M La, and Tam Vu. 2017. ParkSense: automatic parking positioning by leveraging in-vehicle magnetic field variation. IEEE Access, Vol. 5 (2017), 25021--25033.
49. OptAlert. 2019. OptAlert. https://www.optalert.com/explore-products/scientifically-validated-glasses-mining.
50. Jean Christophe Popieul, Philippe Simon, and Pierre Loslever. 2003. Using driver's head movements evolution as a drowsiness indicator. In Intelligent Vehicles Symposium, 2003. Proceedings. IEEE. IEEE, 616--621.
51. Progressive. 2019. Snapshot Driving Monitor. https://www.progressive.com/auto/discounts/snapshot/.
52. Valentin Radu, Catherine Tong, Sourav Bhattacharya, Nicholas D Lane, Cecilia Mascolo, Mahesh K Marina, and Fahim Kawsar. 2018. Multimodal deep learning for activity and context recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, 4 (2018), 157.
53. Sasank Reddy, Min Mun, Jeff Burke, Deborah Estrin, Mark Hansen, and Mani Srivastava. 2010. Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN), Vol. 6, 2 (2010), 13.
54. Consumer Reports. 2019. How To Track Teen Driver. https://www.consumerreports.org/cro/magazine/2014/07/how-to-track-your-teen-driver/index.htm.
55. David Sandberg and Mattias Wahde. 2008. Particle swarm optimization of feedforward neural networks for the detection of drowsy driving. In Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on. IEEE, 788--793.
56. Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, and Youngki Lee. 2015. The case for smartwatch-based diet monitoring. In 2015 IEEE international conference on pervasive computing and communication workshops (PerCom workshops). IEEE, 585--590.
57. Santokh Singh. 2015. Critical reasons for crashes investigated in the national motor vehicle crash causation survey . Technical Report.
58. Alexander V Smirnov, Alexey Kashevnik, Igor Lashkov, Olesya Baraniuc, and Vladimir Parfenov. 2016. Smartphone-based identification of dangerous driving situations: Algorithms and implementation.. In FRUCT. 306--313.
59. Eleuthère Stathopoulos, V Schlageter, B Meyrat, Y Ribaupierre, and P Kucera. 2005. Magnetic pill tracking: a novel non-invasive tool for investigation of human digestive motility. Neurogastroenterology & Motility, Vol. 17, 1 (2005), 148--154.
60. Maria Staubach. 2009. Factors correlated with traffic accidents as a basis for evaluating Advanced Driver Assistance Systems. Accident Analysis & Prevention, Vol. 41, 5 (2009), 1025--1033.
61. Smartcap Technology. 2019. Smart Cap. http://www.smartcaptech.com/life-smart-cap.
62. Simone Verheij, Danya Muilwijk, Johan J.M. Pel, Tischa J.M. van der Cammen, Francesco U.S. Mattace-Raso, and Johannes van der Steen. 2012. Visuomotor Impairment in Early-Stage Alzheimer's Disease: Changes in Relative Timing of Eye and Hand Movements. Journal of Alzheimer's Disease, Vol. 30, 1 (2012), 131--143.
63. Vigo. 2019. Vigo. https://www.wearvigo.com.
64. Vitalacy. 2019. Hand Washing Monitoring. (2019). http://www.hyginex.com/.
65. Esra Vural, Mujdat Cetin, Aytul Ercil, Gwen Littlewort, Marian Bartlett, and Javier Movellan. 2007. Drowsy driver detection through facial movement analysis. Human-computer interaction (2007), 6--18.
66. Eric A Wan and Rudolph Van Der Merwe. 2000. The unscented Kalman filter for nonlinear estimation. In Adaptive Systems for Signal Processing, Communications, and Control Symposium 2000. AS-SPCC. The IEEE 2000. Ieee, 153--158.
67. Yanxin Wang, Johnny Wong, and Andrew Miner. 2004. Anomaly intrusion detection using one class SVM. In Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004. IEEE, 358--364.
68. Wikipedia. 2019 a. Kalman Filtering Basics. https://en.wikipedia.org/wiki/Kalman_filter##Details.
69. Wikipedia. 2019 b. Maximum Likelihood Parameter Estimation. https://en.wikipedia.org/wiki/Maximum_likelihood_estimation.
70. Jie Wu, Kun Li, Yifei Jiang, Qin Lv, Li Shang, and Yihe Sun. 2011. Large-scale battery system development and user-specific driving behavior analysis for emerging electric-drive vehicles. Energies, Vol. 4, 5 (2011), 758--779.
71. Minglin Wu, Sheng Zhang, and Yuhan Dong. 2016. A novel model-based driving behavior recognition system using motion sensors. Sensors, Vol. 16, 10 (2016), 1746.
72. Chuang-Wen You, Nicholas D. Lane, Fanglin Chen, Rui Wang, Zhenyu Chen, Thomas J. Bao, Martha Montes-de Oca, Yuting Cheng, Mu Lin, Lorenzo Torresani, and Andrew T. Campbell. 2013. CarSafe App: Alerting Drowsy and Distracted Drivers Using Dual Cameras on Smartphones. In Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '13). ACM, New York, NY, USA, 13--26.
73. Jiadi Yu, Zhongyang Chen, Yanmin Zhu, Yingying Jennifer Chen, Linghe Kong, and Minglu Li. 2017. Fine-grained abnormal driving behaviors detection and identification with smartphones. IEEE Transactions on Mobile Computing, Vol. 16, 8 (2017), 2198--2212.
74. Jiadi Yu, Hongzi Zhu, Haofu Han, Yingying Jennifer Chen, Jie Yang, Yanmin Zhu, Zhongyang Chen, Guangtao Xue, and Minglu Li. 2016. Senspeed: Sensing driving conditions to estimate vehicle speed in urban environments. IEEE Transactions on Mobile Computing, Vol. 15, 1 (2016), 202--216.
75. ZDNet. 2019. Wearable Cameras for the Police. https://www.zdnet.com/article/22000-london-police-are-getting-wearable-cameras-to-video-crime.
76. Chiara Zecchin, Andrea Facchinetti, Giovanni Sparacino, Chiara Dalla Man, Chinmay Manohar, James A Levine, Ananda Basu, Yogish C Kudva, and Claudio Cobelli. 2013. Physical activity measured by physical activity monitoring system correlates with glucose trends reconstructed from continuous glucose monitoring. Diabetes technology & therapeutics, Vol. 15, 10 (2013), 836--844.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing