Skin Reading: encoding text in a 6-channel haptic display

ABSTRACT

This paper investigates the communication of natural language messages using a wearable haptic display. Our research spans both the design of the haptic display, as well as the methods for communication that use it. First, three wearable configurations are proposed basing on haptic perception fundamentals. To encode symbols, we devise an overlapping spatiotemporal stimulation (OST) method, that distributes stimuli spatially and temporally with a minima gap. An empirical study shows that, compared with spatial stimulation, OST is preferred in terms of recall. Second, we propose an encoding for the entire English alphabet and a training method for letters, words and phrases. A second study investigates communication accuracy. It puts four participants through five sessions, for an overall training time of approximately 5 hours per participant. Results reveal that after one hour of training, participants were able to discern 16 letters, and identify two- and three-letter words. They could discern the full English alphabet (26 letters, 92% accuracy) after approximately three hours of training, and after five hours participants were able to interpret words transmitted at an average duration of 0.6s per word.

References

1. Bliss, J. C., Katcher, M. H., Rogers, C. H., and Shepard, R. P. Optical-to-tactile image conversion for the blind. IEEE Transactions on Man-Machine Systems 11, 1 (March 1970), 58--65.
2. Brewster, S., and Brown, L. M. Tactons: Structured tactile messages for non-visual information display. In Proceedings of the Fifth Conference on Australasian User Interface - Volume 28, AUIC '04 (2004), 15--23.
3. Brown, L. M., and Kaaresoja, T. Feel who's talking: Using tactons for mobile phone alerts. In CHI '06 Extended Abstracts on Human Factors in Computing Systems, CHI EA '06, ACM (2006), 604--609.
4. Cholewiak, R. W., and Collins, A. A. The generation of vibrotactile patterns on a linear array: Influences of body site, time, and presentation mode. Perception & Psychophysics 62, 6, 1220--1235.
5. Cholewiak, R. W., and Collins, A. A. Vibrotactile localization on the arm: Effects of place, space, and age. Perception & Psychophysics 65, 7, 1058--1077.
6. Cholewiak, R. W., and Collins, A. A. Vibrotactile pattern discrimination and communality at several body sites. Perception & Psychophysics 57, 5, 724--737.
7. Cholewiak, R. W., and Craig, J. C. Vibrotactile pattern recognition and discrimination at several body sites. Perception & Psychophysics 35, 6, 503--514.
8. Craig, J. C. Temporal integration of vibrotactile patterns. Perception & Psychophysics 32, 3 (1982), 219--229.
9. de Jesus Oliveira, and Maciel, A. Assessment of tactile languages as navigation aid in 3d environments. In EUROHAPTICS, 2014 (2014).
10. de Jesus Oliveira, and Maciel, A. Introducing the modifier tactile pattern for vibrotactile communication. In EUROHAPTICS, 2014 (2014).
11. Gault, R. H. Progress in experiments on tactual interpretation of oral speech. The Journal of Abnormal Psychology and Social Psychology 19, 2 (1924), 155.
12. Geldard, F. A. Adventures in tactile literacy. American Psychologist 12, 3 (1957), 115--124.
13. Gescheider, G. A., Wright, J. H., and Verrillo, R. T. Information-processing channels in the tactile sensory system: A psychophysical and physiological analysis. Psychology Press, 2010.
14. Gunther, E. Skinscape: A tool for composition in the tactile modality. PhD thesis, Massachusetts Institute of Technology, 2001.
15. Gunther, E., Davenport, G., and O'Modhrain, S. Cutaneous grooves: Composing for the sense of touch. In Proceedings of the 2002 Conference on New Interfaces for Musical Expression, NIME '02, National University of Singapore (2002), 1--6.
16. Hoggan, E. E., Anwar, S., and Brewster, S. A. Mobile Multi-actuator Tactile Displays. 2007.
17. Kern, T. A. Engineering Haptic Devices: A Beginner's Guide for Engineers, 1st ed. Springer Publishing Company, Incorporated, 2009.
18. Kirman, J. H. Tactile perception of computerderived formant patterns from voiced speech. The Journal of the Acoustical Society of America 55, 1 (1974).
19. Lederman, S. J., and Klatzky, R. L. Haptic perception: A tutorial. Attention, Perception, & Psychophysics 71, 7.
20. Lofvenberg, J., and Johansson, R. Regional differences and interindividual variability in sensitivity to vibration in the glabrous skin of the human hand. Brain Research 301, 1 (1984), 65--72.
21. Novich, S. D., and Eagleman, D. M. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput. Experimental Brain Research, 10 (2015).
22. Oakley, I., Kim, Y., Lee, J., and Ryu, J. Determining the feasibility of forearm mounted vibrotactile displays. In 2006 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2006), 27--34.
23. Spence, C., and Ho, C. Tactile and multisensory spatial warning signals for drivers. Haptics, IEEE Transactions on 1, 2 (July 2008), 121--129.
24. Summers, I. R., Whybrow, J. J., Gratton, D. A., Milnes, P., Brown, B. H., and Stevens, J. C. Tactile information transfer: A comparison of two stimulation sites. The Journal of the Acoustical Society of America 118, 4 (2005), 2527--2534.
25. Ternes, D., and MacLean, K. E. Designing large sets of haptic icons with rhythm. In EuroHaptics, M. Ferre, Ed., Springer (Madrid, Spain, 2008), 199--208.
26. Troxel, D. Experiments in tactile and visual reading. Human Factors in Electronics, IEEE Transactions on HFE-8, 4 (Dec 1967), 261--263.
27. White, B. W., Saunders, F. A., Scadden, L., Bach-Y-Rita, P., and Collins, C. C. Seeing with the skin. Perception & Psychophysics 7, 1, 23--27.
28. Xu, C., Israr, A., Poupyrev, I., Bau, O., and Harrison, C. Tactile display for the visually impaired using teslatouch. In CHI '11 Extended Abstracts on Human Factors in Computing Systems, CHI EA '11, ACM (New York, NY, USA, 2011), 317--322.
29. Yannier, N., Israr, A., Lehman, J. F., and Klatzky, R. L. FeelSleeve: Haptic feedback to enhance early reading. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, 2015 (2015), 1015--1024.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices