skip to main content
research-article

Handwritten Signature Verification Using Wrist-Worn Devices

Published:18 September 2018Publication History
Skip Abstract Section

Abstract

This paper suggests a novel verification system for handwritten signatures. The proposed system is based on capturing motion signals from the sensors of wrist-worn devices, such as smartwatches and fitness trackers, during the signing process, to train a machine learning classifier to determine whether a given signature is genuine or forged. Our system can be used to: (1) Verify signatures written on paper documents, such as checks, credit card receipts and vote by mail ballots. Unlike existing systems for signature verification, our system obtains a high degree of accuracy, without requiring an ad hoc digital signing device. (2) Authenticate a user of a secure system based on "who you are" traits. Unlike existing "motion-based" authentication methods that commonly rely on long-term user behavior, writing a signature is a relatively short-term process. In order to evaluate our system, we collected 1,980 genuine and forged signature recordings from 66 different subjects, captured using a smartwatch device. Applying our signature verification system on the collected dataset, we show that it significantly outperforms two other state-of-the-art systems, obtaining an EER of 2.36% and an AUC of 98.52%.

References

  1. Hervé Abdi and Lynne J Williams. 2010. TukeyâĂźs honestly significant difference (HSD) test. Encyclopedia of Research Design. Thousand Oaks, CA: Sage (2010), 1--5.Google ScholarGoogle Scholar
  2. Luca Ardüser, Pascal Bissig, Philipp Brandes, and Roger Wattenhofer. {n. d.}. Recognizing Text Using Motion Data From a Smartwatch. ({n. d.}).Google ScholarGoogle Scholar
  3. Gonzalo Bailador, Carmen Sanchez-Avila, Javier Guerra-Casanova, and Alberto de Santos Sierra. 2011. Analysis of pattern recognition techniques for in-air signature biometrics. Pattern Recognition 44, 10 (2011), 2468--2478. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Tony Beltramelli and Sebastian Risi. 2015. Deep-Spying: Spying using Smartwatch and Deep Learning. arXiv preprint arXiv:1512.05616 (2015).Google ScholarGoogle Scholar
  5. A blog-to watch. 2015. (2015). http://www.ablogtowatch.com/poll-your-hand-orientation-what-wrist-wear-your-watch/ Accessed: 2017-11-01.Google ScholarGoogle Scholar
  6. David Bolton. {n. d.}. Ditching smartphones for wearables. http://arc.applause.com/2015/10/15/wearables-apple-watch-fitness-tracker/. ({n. d.}). Accessed: 2016-03-18.Google ScholarGoogle Scholar
  7. Christopher Carlson, Toney Chen, Jeremy Cruz, Javid Maghsoudi, Hui Zhao, and John V Monaco. 2015. User Authentication with Android Accelerometer and Gyroscope Sensors. Proceedings of Student-Faculty Research Day, CSIS, Pace University (2015).Google ScholarGoogle Scholar
  8. J Guerra Casanova, C Sánchez Ávila, Alberto de Santos Sierra, G Bailador del Pozo, and V Jara Vera. 2010. A real-time in-air signature biometric technique using a mobile device embedding an accelerometer. In International Conference on Networked Digital Technologies. Springer, 497--503.Google ScholarGoogle ScholarCross RefCross Ref
  9. N. L. Clarke and S. M. Furnell. 2007. Authenticating mobile phone users using keystroke analysis. International Journal of Information Security 6, 1 (2007), 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Scott Davidson, Derrick Smith, Chen Yang, and Sebastian Cheah. 2016. Smartwatch User Identification as a Means of Authentication. Department of Computer Science and Engineering Std (2016).Google ScholarGoogle Scholar
  11. Julian Fierrez, Javier Ortega-Garcia, Daniel Ramos, and Joaquin Gonzalez-Rodriguez. 2007. HMM-based online signature verification: Feature extraction and signature modeling. Pattern recognition letters 28, 16 (2007), 2325--2334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Andreas Fischer, Moises Diaz, Rejean Plamondon, and Miguel A Ferrer. 2015. Robust score normalization for DTW-based online signature verification. In Document Analysis and Recognition (ICDAR), 2015 13th International Conference on. IEEE, 241--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. 2015 AFP Payments Fraud and Control Survey. 2015. (2015). https://nacm.org/pdfs/surveyResults/afp-payments-fraud-results.pdf Accessed: 2016-09-30.Google ScholarGoogle Scholar
  14. Marta Gomez-Barrero, Julian Fierrez, and Javier Galbally. 2016. Variable-length template protection based on homomorphic encryption with application to signature biometrics. In 2016 4th International Conference on Biometrics and Forensics (IWBF). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  15. Javier Guerra-Casanova, Carmen Sánchez Ávila, Gonzalo Bailador, and Alberto de Santos-Sierra. 2011. Time series distances measures to analyze in-air signatures to authenticate users on mobile phones. In Security Technology (ICCST), 2011 IEEE International Carnahan Conference on. IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  16. Jeremy Gummeson, Bodhi Priyantha, and Jie Liu. 2014. An energy harvesting wearable ring platform for gestureinput on surfaces. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 162--175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Luiz G Hafemann, Robert Sabourin, and Luiz S Oliveira. 2015. Offline Handwritten Signature Verification-Literature Review. arXiv preprint arXiv:1507.07909 (2015).Google ScholarGoogle Scholar
  18. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. 2009. The WEKA data mining software: an update. ACM SIGKDD explorations newsletter 11, 1 (2009), 10--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Donato Impedovo and Giuseppe Pirlo. 2008. Automatic signature verification: the state of the art. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 38, 5 (2008), 609--635. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Vahab Iranmanesh, Sharifah Mumtazah Syed Ahmad, Wan Azizun Wan Adnan, Salman Yussof, Olasimbo Ayodeji Arigbabu, and Fahad Layth Malallah. 2014. Online handwritten signature verification using neural network classifier based on principal component analysis. The Scientific World Journal 2014 (2014).Google ScholarGoogle Scholar
  21. Felix Juefei-Xu, Chandrasekhar Bhagavatula, Aaron Jaech, Unni Prasad, and Marios Savvides. 2012. Gait-id on the move: Pace independent human identification using cell phone accelerometer dynamics. In Biometrics: Theory, Applications and Systems (BTAS), 2012 IEEE Fifth International Conference on. IEEE, 8--15.Google ScholarGoogle ScholarCross RefCross Ref
  22. Alisher Kholmatov and Berrin Yanikoglu. 2005. Identity authentication using improved online signature verification method. Pattern recognition letters 26, 15 (2005), 2400--2408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rajesh Kumar, Vir V Phoha, and Rahul Raina. 2016. Authenticating users through their arm movement patterns. arXiv preprint arXiv:1603.02211 (2016).Google ScholarGoogle Scholar
  24. A. Lewis, Y. Li, and M. Xie. 2016. Real time motion-based authentication for smartwatch. In 2016 IEEE Conference on Communications and Network Security (CNS). 380--381.Google ScholarGoogle Scholar
  25. Lingjun Li, Xinxin Zhao, and Guoliang Xue. 2013. Unobservable Re-authentication for Smartphones.. In NDSS.Google ScholarGoogle Scholar
  26. Muhammad Imran Malik and Marcus Liwicki. 2012. From terminology to evaluation: Performance assessment of automatic signature verification systems. In Frontiers in Handwriting Recognition (ICFHR), 2012 International Conference on. IEEE, 613--618. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jani Mantyjarvi, Mikko Lindholm, Elena Vildjiounaite, S-M Makela, and HA Ailisto. 2005. Identifying users of portable devices from gait pattern with accelerometers. In Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP'05). IEEE International Conference on, Vol. 2. IEEE, ii--973.Google ScholarGoogle Scholar
  28. The 10 most likely sensors in a 10-sensor Apple smartwatch. 2014. (2014). https://www.pcworld.com/article/2366126/the-10-most-likely-sensors-in-a-10-sensor-apple-smartwatch.html Accessed: 2016-09-30.Google ScholarGoogle Scholar
  29. Myo. 2016. (2016). https://www.myo.com/ Accessed: 2016-09-30.Google ScholarGoogle Scholar
  30. Mohammad Reza Nilchiyan and Rubiyah Bte Yusof. 2013. Improved wavelet-based online signature verification scheme considering pen scenario information. In Artificial Intelligence, Modelling and Simulation (AIMS), 2013 1st International Conference on. IEEE, 8--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Shahriar Nirjon, Robert Dickerson, John Stankovic, Guobin Shen, and Xiaofan Jiang. 2013. sMFCC: exploiting sparseness in speech for fast acoustic feature extraction on mobile devices--a feasibility study. In Proceedings of the 14th Workshop on Mobile Computing Systems and Applications. ACM, 8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Sampo Ojala, Jari Keinanen, and Jorma Skytta. 2008. Wearable authentication device for transparent login in nomadic applications environment. In Signals, Circuits and Systems, 2008. SCS 2008. 2nd International Conference on. IEEE, 1--6.Google ScholarGoogle Scholar
  33. Vishal M Patel, Rama Chellappa, Deepak Chandra, and Brandon Barbello. 2016. Continuous user authentication on mobile devices: Recent progress and remaining challenges. IEEE Signal Processing Magazine 33, 4 (2016), 49--61.Google ScholarGoogle ScholarCross RefCross Ref
  34. Rejean Plamondon and Guy Lorette. 1989. Automatic signature verification and writer identification?the state of the art. Pattern recognition 22, 2 (1989), 107--131.Google ScholarGoogle Scholar
  35. Réjean Plamondon and Sargur N Srihari. 2000. Online and offline handwriting recognition: a comprehensive survey. Pattern Analysis and Machine Intelligence, IEEE Transactions on 22, 1 (2000), 63--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Juhi Ranjan and Kamin Whitehouse. 2015. Object hallmarks: Identifying object users using wearable wrist sensors. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 51--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. K Ramamohan Rao and Ping Yip. 2014. Discrete cosine transform: algorithms, advantages, applications. Academic press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Saeid Rashidi, Ali Fallah, and Farzad Towhidkhah. 2012. Authentication based on signature verification using position, velocity, acceleration and Jerk signals. In Information Security and Cryptology (ISCISC), 2012 9th International ISC Conference on. IEEE, 26--31.Google ScholarGoogle ScholarCross RefCross Ref
  39. S Rashidi, A Fallah, and F Towhidkhah. 2012. Feature extraction based DCT on dynamic signature verification. Scientia Iranica 19, 6 (2012), 1810--1819.Google ScholarGoogle ScholarCross RefCross Ref
  40. Napa Sae-Bae and Nasir Memon. 2013. A simple and effective method for online signature verification. In Biometrics Special Interest Group (BIOSIG), 2013 International Conference of the. IEEE, 1--12.Google ScholarGoogle Scholar
  41. Abdul Serwadda, Vir V. Phoha, and Zibo Wang. 2013. Which verifiers work?: A benchmark evaluation of touch-based authentication algorithms. IEEE Computer Society.Google ScholarGoogle Scholar
  42. Ho-chul Shin. 2016. WEARABLE DEVICE AND METHOD OF OPERATING THE SAME. (Feb. 4 2016). US Patent 20,160,034,041.Google ScholarGoogle Scholar
  43. 2013 ABA Deposit Account Fraud Survey. 2013. (2013). http://www.aba.com/Products/Surveys/Pages/2013DepositAccount.aspxAccessed: 2016-09-30.Google ScholarGoogle Scholar
  44. Rozita Teymourzadeh, Waidhuba Martin Kizito, Kok Wai Chan, and Mok Vee Hoong. 2013. Smart analytical signature verification for DSP applications. In Systems, Process 8 Control (ICSPC), 2013 IEEE Conference on. IEEE, 301--305.Google ScholarGoogle Scholar
  45. USAtoday. 2017. (2017). http://www.cbsnews.com/news/florida-voters-can-fix-vote-by-mail-ballot-federal-judge-rules/ Accessed: 2017-11-01.Google ScholarGoogle Scholar
  46. He Wang, Ted Tsung-Te Lai, and Romit Roy Choudhury. 2015. Mole: Motion leaks through smartwatch sensors. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 155--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Wikipedia. {n. d.}. Discrete cosine transform. ({n. d.}). https://en.wikipedia.org/wiki/Discretecosinetransform.Google ScholarGoogle Scholar
  48. Chao Xu, Parth H Pathak, and Prasant Mohapatra. 2015. Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, 9--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Berrin Yanikoglu and Alisher Kholmatov. 2009. Online signature verification using fourier descriptors. EURASIP Journal on Advances in Signal Processing 2009 (2009), 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Dit-Yan Yeung, Hong Chang, Yimin Xiong, Susan George, Ramanujan Kashi, Takashi Matsumoto, and Gerhard Rigoll. 2004. SVC2004: First international signature verification competition. In Biometric Authentication. Springer, 16--22.Google ScholarGoogle Scholar
  51. Heng Zhang, Vishal M Patel, Mohammed Fathy, and Rama Chellappa. 2015. Touch gesture-based active user authentication using dictionaries. In Applications of Computer Vision (WACV), 2015 IEEE Winter Conference on. IEEE, 207--214. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Handwritten Signature Verification Using Wrist-Worn Devices

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 3
      September 2018
      1536 pages
      EISSN:2474-9567
      DOI:10.1145/3279953
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 18 September 2018
      • Accepted: 1 September 2018
      • Revised: 1 May 2018
      • Received: 1 November 2017
      Published in imwut Volume 2, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader