skip to main content
10.1145/2637248.2637265acmotherconferencesArticle/Chapter ViewAbstractPublication PagesecceConference Proceedingsconference-collections
research-article

The shared priorities measure as a way of assessing team strategic awareness: a bridge between self-assessment and the deep blue sea of field recordings

Authors Info & Claims
Published:01 September 2014Publication History

ABSTRACT

Objective, easy to use, easy to comprehend, high face-validity assessment methods for measuring shared awareness in teams are hard to find. This paper describes an experiment where a new measure called Shared Priorities, which is based on ranking of self-generated strategic items, is tested. Trained teams were compared to non-trained teams in a dynamic problem-solving task in terms of performance and shared awareness. The shared priorities measure was used alongside other, well-documented measures of team awareness based on self-rating. The results show that the Shared Priorities measure correlate with performance and could also distinguish between trained and non-trained teams. However, the Shared Priorities measure did not correlate with the other team measures, suggesting that it captures a different quality of team work than the self-rating measures. Further, the shared priorities measure was found to be easily administered and gained a high user acceptance.

References

  1. Artman, H. and Waern, Y. Creation and Loss of Cognitive Empathy at an Emergency Control Center. In (Ed.) Y. Waern. Cooperative Process Management -- Cognition and Information Technology. Francis & Taylor: London, 1998.Google ScholarGoogle Scholar
  2. Artman, H. Team situation assessment and information distribution. Ergonomics 43, 8, (2000), 1111-1128.Google ScholarGoogle ScholarCross RefCross Ref
  3. Baroutsi, N., Berggren, P., Nählinder, S. and Johansson, B. Training teams to collaborative as cohesive units (Scientific report No. FOI-R--3830--SE), FOI, 2013.Google ScholarGoogle Scholar
  4. Berggren, P. and Johansson, B. Developing an instrument for measuring shared understanding. In Proceedings of the 7th International Conference on Information Systems for Crisis Response and Management: Defining Crisis Management 3.0. Seattle, WA. May 2-5 (2010).Google ScholarGoogle Scholar
  5. Berggren, P., Svensson, J. and Hörberg, U. Mätning av gemensam lägesbild vid ledning på stridsteknisk och taktisk nivå - Studie genomförd på TCCS (Användarrapport No. FOI-R--2647--SE), FOI, 2008.Google ScholarGoogle Scholar
  6. Brannick, M.T., and Prince, C. An Overview of Team Performance Measurement. In (Eds.) M.T. Brannick, E. Salas, and C. Prince, Team performance Assessment and Measurement. Lawrence Erlbaum Associates, London, 1997.Google ScholarGoogle Scholar
  7. Brehmer, B. and Dörner, D. Experiments With Computer-Simulated Microworlds: Escaping Both the Narrow Straits of the Laboratory and the Deep Blue Sea of the Field Study. Computers in Human Behaviour, 9, (1993), 171--184.Google ScholarGoogle ScholarCross RefCross Ref
  8. Brehmer, B. Dynamic decision making: Human control of complex systems. Acta Psychologica. 81, (1992), 211--241.Google ScholarGoogle ScholarCross RefCross Ref
  9. Cook, T.D. and Campbell, D.T. Quasi-experimentation: Design and anlysis issues for field settings. Houghton Mifflin Company, Boston, 1979.Google ScholarGoogle Scholar
  10. Cooke, N.J., Gorman, J.C., Myers, C.W. and Duran, J.L. Interactive team cognition. Cognitive Science 37, 2, (2013), 255--85.Google ScholarGoogle ScholarCross RefCross Ref
  11. Dörner, D., Kreuzig, H.W., Reiter, F. and Stäudel, T. Lohausen. Von Umgang mit Unbestimmheit und Komplexität. Huber, Bern, 1983.Google ScholarGoogle Scholar
  12. Drury, C.G. Methods for direct observation of performance. In (Eds.) J.R. Wilson and E.N. Corlett. Evaluation of Human Work. Taylor and Francis, London, 1992.Google ScholarGoogle Scholar
  13. Endsley, M.R. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors 37, 1, (1995), 32--64.Google ScholarGoogle ScholarCross RefCross Ref
  14. Fullagar, C.J., and Egleston, D.O. Norming and Performing: Using Microworlds to Understand the Relationship Between Team Cohesiveness and Performance. Journal of Applied Social Psychology, 38, 10, (2008), 2574--2593.Google ScholarGoogle ScholarCross RefCross Ref
  15. Gonzalez C., Vanyukov, P. and Martin M.K. The use of microworlds to study dynamic decision making. Computers in Human Behavior 21, (2005), 273--286.Google ScholarGoogle ScholarCross RefCross Ref
  16. Granlund R, Johansson B, and Persson M. C3Fire a Micro-world for Collaboration Training in the ROLF environment. In proceedings to SIMS 2001 the 42nd Conference on Simulation and Modelling, Simulation in Theory and Practice, (2001).Google ScholarGoogle Scholar
  17. Granlund R. Monitoring Distributed Teamwork Training. Linköping Studies in Science and Technology, Linköping University Press, Linköping, 2002.Google ScholarGoogle Scholar
  18. Heath, C.C. and Luff, P. Collaboration and control: Crisis Management and Multimedia Technology in London Underground Line Control Rooms. Computer Supported Cooperative Work (CSCW). An International Journal, vol. 1, nos. 1--2, (1992), 69--94.Google ScholarGoogle Scholar
  19. Helton, W.S., Funke, G.J. and Knott, B.A. Measuring Workload in Collaborative Contexts: Trait Versus State Perspectives. Human Factors, online, (2013), 1--11.Google ScholarGoogle Scholar
  20. Hinsz, V.B. Metacognition and mental models in groups: An illustration with metamemory of group recognition memory. In (Eds.) E. Salas and S. M. Fiore, Team cognition: Understanding the factors that drive process and performance, American Psychological Association, Washington, DC, 2004, 33--58.Google ScholarGoogle Scholar
  21. Hollnagel, E. Cognitive reliability and error analysis method --- CREAM. Elsevier Science, Oxford, 1998.Google ScholarGoogle Scholar
  22. Hollnagel, E., and Woods, D.D. Joint cognitive systems: Foundations of cognitive systems engineering. CRC Press, Boca Raton, FL, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Howie, D.E. and Vincente, K.J. Measures of operator performance in complex, dynamic microworlds: advancing the state of the art, Ergonomics 41, 4, (1998), 485--500.Google ScholarGoogle ScholarCross RefCross Ref
  24. Hutchins, E. and Klausen, T. Distributed Cognition in an Airlane Cockpit. In (Eds.) Y. Engeström and D. Middelton. Cognition and Communication at Work. Cambridge University Press, Cambridge, 1996.Google ScholarGoogle Scholar
  25. Jersild, A.T. and Meigs, M.F. Direct Observation as a Research Method. Review of Educational Research. Methods of Research in Education 9, 5, (1939) 472--482.Google ScholarGoogle Scholar
  26. Jobidon, M.-E., Breton, R., Rousseau, R., and Tremblay, S. Team response to workload transition: The role of team structure. In Proceedings of the 50th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, CA, (2006).Google ScholarGoogle ScholarCross RefCross Ref
  27. Jones, D.G. and Kaber, D.B. Situation awareness measurement and the Situation awareness global assessment technique. In (Eds.) N.A. Stanton, A. Hedge, K. Brookhuis, E. Salas and H. Hendrick, Handbook of Human Factors and Ergonomics methods. CRC Press, London, 2005.Google ScholarGoogle Scholar
  28. Jones, P.E. and Roelofsma, P.H.M.P. The potential for social contextual and group biases in team decision-making: Biases, conditions and psychological mechanisms. Ergonomics, 43, 8, (2000), 1129--1152.Google ScholarGoogle ScholarCross RefCross Ref
  29. Kellermanns, F.W., Walter, J., Lechner, C., and Floyd, S.W. The Lack of Consensus About Strategic Consensus: Advancing Theory and Research. Journal of Management, 31, 5, (2005), 719--737.Google ScholarGoogle ScholarCross RefCross Ref
  30. Kendall, M.G. Rank correlation methods (Forth ed.).: Charles Griffin & Company Ltd, London, 1975.Google ScholarGoogle Scholar
  31. Macmillan, J., Paley, M.J., Entin, E.B. and Entin, E.E. Questionnaires for Distributed Assessment of Team Mutual Awareness. In (Eds.) N.A. Stanton, A. Hedge, K. Brookhuis, E. Salas and H. Hendricks. Handbook of Human Factors Methods, Taylor and Francis, London, 2005.Google ScholarGoogle Scholar
  32. Mathieu, J., Maynard, M.T., Rapp, T. and Gilson, L. Team effectiveness 1997-2007: A review of recent advancements and a glimpse into the future. Journal of Management 34, 3, (2008), 410--476.Google ScholarGoogle ScholarCross RefCross Ref
  33. McGuinnes, B. and Foy, L.A. Subjective measure of SA: the Crew Awareness Rating Scale (CARS). In The Human Performance, Situational Awareness an Automation Conference, (2000).Google ScholarGoogle Scholar
  34. Miles, M.B. and Huberman, A.M. Qualitative Data Analysis: An Expanded Sourcebook. Sage Publishing: London, 1994.Google ScholarGoogle Scholar
  35. Omodei, M.M., Wearing, A.J., McLennan, J., Elliot, G.C. and Clancy, J.M. "More is better?": Problems of Self-Regulation in Naturalistic Decision Making Settings. In (Eds.) B. Brehmer, H. Montgomery, and R. Lipshitz. How Professionals make decisions, Lawrence Erlbaum Associates Inc.: Mahaw, New Jersey, 2004.Google ScholarGoogle Scholar
  36. Orasanu, J. and Salas, E. Team Decision Making in Complex Environments. In (Eds.) Klein, G.A., Orasanu, J., Calderwood, R. and Zsambook, C.E. Decision Making in Action. Ablex Publishing, Norwood, NJ, 1993.Google ScholarGoogle Scholar
  37. Peirce, C. S. Collected Papers of Charles Sanders Peirce, vols. 1--6, 1931--1935, In (Eds.) C. Hartshorne & P. Weiss. Harvard University Press, Cambridge, MA, 1958.Google ScholarGoogle Scholar
  38. Prytz, E., Berggren, P. and Johansson, B.J.E. Performance and shared understanding in mixed C2-systems (Scientific report No. FOI-R--3155--SE), FOI, 2010.Google ScholarGoogle Scholar
  39. Saetrevik, B. and Eid, J. The "Similarity Index" as an Indicator of Shared Mental Models and Situation Awareness in Field Studies. Journal of Cognitive Engineering and Decision Making, (2013), Online first: 1.18.Google ScholarGoogle Scholar
  40. Salas, E. and Cannon-Bowers, J.A. Special issue preface. Journal of Organizational Behavior, 22, (2001), 87--88.Google ScholarGoogle ScholarCross RefCross Ref
  41. Saner, L.D., Bolstad, C.A., Gonzales, C. and Cuevas, H.M. Measuring and Predicting Shared Situation Awareness in Teams. Journal of Cognitive Engineering and Decision Making 3, 3, (2009), 280--308.Google ScholarGoogle ScholarCross RefCross Ref
  42. Tuckman, B.W. Developmental Sequence in Small Groups. Psychological Bulletin 6, 36, (1965), 384--399.Google ScholarGoogle Scholar
  43. Wildman, J.L., Salas, E., and Scott, C.P.R. Measuring Cognition in Teams A Cross-Domain Review. Human Factors: The Journal of the Human Factors and Ergonomics Society, Online, (2013), 1--31.Google ScholarGoogle Scholar

Index Terms

  1. The shared priorities measure as a way of assessing team strategic awareness: a bridge between self-assessment and the deep blue sea of field recordings

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      ECCE '14: Proceedings of the 2014 European Conference on Cognitive Ergonomics
      September 2014
      191 pages
      ISBN:9781450328746
      DOI:10.1145/2637248

      Copyright © 2014 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 September 2014

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed limited

      Acceptance Rates

      Overall Acceptance Rate56of91submissions,62%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader