Modeling management information systems' success: a study in the domain of further education and training

ABSTRACT

Capacity in and enhancement of critical, scarce and intermediate skills are seen as a national priority in South Africa. The Department of Higher Education and Training (DHET) in South Africa has prioritized the success of the Further Education and Training (FET) sector to meet this need for capacity development in critical, scarce and intermediate skills. Management information systems (MIS) are pivotal in the efficient and effective running of FET colleges. Therefore, the evaluation of MIS success is an essential spoke in the wheel of FET college success. The problem is that no MIS success evaluation model for FET colleges could be found. In this paper, we describe the development and testing of an evaluation model and tool for MIS success. Information system's evaluation theory and an analysis of FET policy documents were used to propose an initial success evaluation model and tool (questionnaire) for an educational environment (FET colleges) in South Africa. Using a quantitative approach the tool was applied in a survey at one public FET college to evaluate the success of the MIS deployed at the college. Findings from the survey lead to the refinement of the model which is also articulated in this paper and reflected as the SAFETMIS model. The paper is novel in proposing an IS theory based model and tool which can be used to evaluate MIS success at FET colleges and similar education contexts. The paper should be of interest to researchers in the field of Information Systems success evaluation and also to practitioners and managers in the field of Education.

References

1. Bailey, J. E. and Pearson, S. W. 1983. Development of a Tool for Measuring and Analysing Computer User Satisfaction. Management Science. 29(5), 530--545.
2. Bisschoff, T. C. and Nkoe, M. N. 2005. The merging of Further Education and Training colleges - challenging factors in three provinces of South Africa. South African Journal of Higher Education. 19(3), 203--217.
3. Chow, W. S. 2004. An Exploratory Study of the Success Factors for Extranet Adoption in E-Supply Chain. Journal of Global Information Management. 12(4), 60.
4. Cosser, M., Kraak, A. and Winnaar, L. 2011. Further Education and Training (FET) colleges audit, May-July 2010: National Profile of the FET college sector. Unpublished client report. Human Sciences Research Council: Pretoria.
5. Davis, F. D., Bagozzi, R. P. and Warshaw, P. R. 1989. User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science. 35, 982--1003.
6. DeLone, W. and McLean, E. 1992. Information Systems Success: The Quest for the Dependent Variable. Information Systems Research. 3(1), 36.
7. DeLone, W. H. and McLean, E. R. 2003. The DeLone and McLean Model of Information Systems Success: A Ten-year Update. Journal of Management Information Systems. 19(4), 9--30.
8. Department of Education. 2008. National Plan for Further Education and Training Colleges in South Africa. Department of Education: Pretoria.
9. Department of Higher Education and Training. 2011. Revised Strategic Plan, 2010/11-2014/15, and operational plans for the 2011/12 finacial year: Pretoria.
10. Dishaw, M. T., Strong, D. M. and Bandy, D. B. 2002. Extending the Task-Technology Fit Model with self-efficacy constructs, In Human-Computer Interaction Studies in MIS, Eighth Americas Conference on Information Systems. http://sigs.aisnet.org/SIGHCI/amcis02_minitrack/RIP/Dishaw.pdf, 1021--1027.
11. Doll, W. J. and Torkzadeh, G. 1988. The Measurement of End-User Computing Satisfaction. MIS Quarterly. 12(2), 259--274.
12. Fishbein, M. and Ajzen, I. 1975. Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.
13. Gable, G. G., Sedera, D. and Chan, T. 2008. Reconceptualizing Information System Success: The IS-Impact Measurement Model. Journal of the Association for Information Systems. 9(7), 377--408.
14. Goodhue, D. L. and Thompson, R. L. 1995. Task-Technology Fit and Individual Performance. MIS Quarterly. 19(2), 213--236.
15. Ifinedo, P., Rapp, B., Ifinedo, A. and Sundberg, K. 2010. Relationships among ERP post-implementation success constructs: An analysis at the organizational level. Computers in Human Behavior. 26, 1136--1148.
16. Lange, L. and Luescher, T. M. 2003. A monitoring and evaluation system for South African higher education: conceptual, methodological and practical concerns. South African Journal of Higher Education. 17(3), 82--89.
17. LaPiere, R. T. 1934. Attitudes vs. Actions. Social Forces. 13, 230--237.
18. Mason, R. O. 1978. Measuring Information Output: A Communication Systems Approach. Information and Management. 1(4), 219--234.
19. McGill, T., Hobbs, V. and Klobas, J. 2003. User-developed applications and information systems success: a test of DeLone and McLean's model. Information Resources Management Journal. 16(1), 24--45.
20. Mouton, J. 2001. How to succeed in your Master's & Doctoral Studies. A South African guide and resource book. Van Schaik Publishers, Pretoria.
21. Ong, C.-S., Day, M.-Y. and Hsu, W.-L. 2009. The measurement of user satisfaction with question answering systems. Information & Management. 46, 397--403.
22. Palmius, J. 2007. Criteria for measuring and comparing information systems, In Proceedings of the 30th Information Systems Research Seminar in Scandinavia IRIS 2007: Scandinavia 24.
23. Petter, S., DeLone, W. and McLean, E. R. 2008. Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems. 17, 236--263.
24. Petter, S. and McLean, E. R. 2009. A meta-analytic assessment of the DeLone and McLean IS success model: An examination of IS success at the individual level. Information & Management. 46(3), 159--166.
25. Rai, A., Lang, S. S. and Welker, R. B. 2002. Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis Information Systems Research. 13(1), 50--69.
26. Rosemann, M. and Vessey, I. 2008. Towards improving the relevance of information systems research to practice: the role of applicability checks. MIS Quarterly. 32(1), 1--22.
27. Seddon, P. B. 1997. A Respecification and Extension of the DeLone and McLean Model of IS Success. Information Systems Research. 8(3), 240--253.
28. Seddon, P. B. and Kiew, M. Y. 1996. A Partial Test and Development of DeLone and MacLean's Model of IS Success. Australian Journal of Information Systems. 4(1), 90--109.
29. Sedera, D., Gable, G., G. and Chan, T. 2004. A Factor and Structural Equation analysis of the Enterprise Systems Success Measurement Model, In Proceedings Americas Conference on Information Systems. New York, USA.
30. Shannon, C. E. and Weaver, W. 1949. The Mathematical Theory of Communication. University of Illinois Press, Urbana, IL.
31. Twine, A. and Brown, I. Evaluating web conferencing tool effectiveness. In SAICSIT '11, October 3--5. 2011. Cape Town, South Africa: ACM 978-1-4503-0878-6/11/10.
32. Venkatesh, V. and Bala, H. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences. 39, 273--315.
33. Venkatesh, V. and Davis, F. D. 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies Management Science 46, 186--204.
34. Venkatesh, V., Morris, M. G., Davis, F. D. and Davis, G. B. 2003. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly. 27, 425--478.
35. Visser, M. M. 2011. Towards developing an evaluation tool for business management information systems' success at public Further Education and Training (FET) colleges in South Africa. Master of Science Thesis, Information Systems, University of South Africa (UNISA), Pretoria.
36. Wixom, B. H. and Todd, P. A. 2005. A theoretical integration of user satisfaction and technology acceptance. Information Systems Research. 16(1), 85--102.
37. York University. 2010. Theories used in IS research. {cited 2011 27 May}.

Index Terms

• Human-centered computing

  • Collaborative and social computing

    • Collaborative and social computing design and evaluation methods