Daniela Fogli
Skip Abstract Section
Abstract
In our prior work, we presented a novel approach to the evaluation of quality in use of corporate web sites based on an original quality model (QM-U) and a related methodology (EQ-EVAL). This article focuses on two research questions. The first one aims at investigating whether expected quality obtained through the application of EQ-EVAL methodology by employing a small panel of evaluators is a good approximation of actual quality obtained through experimentation with real users. To answer this research question, a comparative study has been carried out involving 5 evaluators and 50 real users. The second research question aims at demonstrating that the adoption of the EQ-EVAL methodology can provide useful information for web site improvement. Three original indicators, namely coherence, coverage and ranking have been defined to answer this question, and an additional study comparing the assessments of two panels of 5 and 10 evaluators, respectively, has been carried out. The results obtained in both studies are largely positive and provide a rational support for the adoption of the EQ-EVAL methodology.
- M. Q. Abbasi, J. Weng, Y. Wang, I. Wang, I. Rafique, X. Wang, and P. Lew. 2012. Modeling and evaluating user interface aesthetics: Employing ISO 25010 quality standard. In Proceedings 8th International Conference on Quality of Information and Communications Technology (QUATIC’12). 303--306. Google Scholar
Digital Library
- G. A. Adams and G. R. Frost. 2004. The Development of Corporate Web-sites and Implications for Ethical, Social and Environmental Reporting through these Media. Institute of Chartered Accountants of Scotland, Edinburgh, Scotland, UK.Google Scholar
- A. M. Aladwani and P. C. Palvia. 2002. Developing and validating and instrument for measuring user-perceived web quality. Info. Manage. 39, 6, 467--476. Google ScholarDigital Library
- A. Albunquerque and A. D. Belchior. 2002. E-commerce websites: A qualitative evaluation. In Proceedings of the International World Wide Web Conference (WWW’02).Google Scholar
- S. J. Barnes and R. T. Vidgen. 2002. An integrative approach to the assessment of e-commerce quality. J. Electron. Commerce Res. 3, 3, 114--127.Google Scholar
- P. Becker, P. Lew, and L. Olsina. 2012. Specifying process views for a measurement, evaluation, and improvement strategy. Adv. Softw. Eng. 2012, Art. 2. Google ScholarDigital Library
- K. S. Bordens and B. B. Abbott. 2011. Research Design and Methods: A Process Approach. McGraw-Hill, New York, NY.Google Scholar
- G. Büyüközkan, J. Arsenyan, and G. Ertek. 2010. Evaluation of E-Learning Web Sites Using Fuzzy Axiomatic Design Based Approach. Int. J. Comput. Intell. Syst. 3, 1, 24--42.Google Scholar
- D. Campbell and A. C. Beck. 2004. Answering allegations: the use of the corporate website for restorative ethical and social disclosure. Business Ethics: Euro. Rev. 13, 2/3, 100--116.Google Scholar
- A. Fernandez, E. Insfran, and S. Abrahão. 2011. Usability evaluation methods for the web: A systematic mapping study. Info. Softw. Technol. 53, 789--817. Google ScholarDigital Library
- D. Fogli and G. Guida. 2015. A practical approach to the assessment of quality in use of corporate web sites. J. Syst. Softw. 99, 52--65. Google ScholarDigital Library
- A. M. Graziano and M. L. Raulin. 2012. Research Methods: A Process of Inquiry (8th ed.). Pearson.Google Scholar
- M. Herrera, M. A. Moraga, L. I. Caballero, and C. Calero. 2010. Quality in use model for web portals (QiUWeP). In Current Trends in Web Engineering, F. Daniel and F. M Facca (Eds.). LNCS 6385 (91--101). Springer, Berlin. Google ScholarDigital Library
- M. Hertzum and N. E. Jacobsen. 1999. The evaluator effect during first-time use of the cognitive walkthrough technique. In Proceedings Human-Computer Interaction International Conference (HCI’99). Google ScholarDigital Library
- M. Hertzum and N. E. Jacobsen. 2001. The evaluator effect: A chilling fact about usability evaluation methods. Int. J. Human-Comput. Interact. 13, 421--443.Google ScholarCross Ref
- M. Hertzum, N. E. Jacobsen, and R. Molich. 2002. Usability inspection by groups of specialists: Perceived agreement in spite of disparate observations. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI’02). ACM Press, New York. Google ScholarDigital Library
- A. Holzinger. 2005. Usability engineering methods for software developers. Commun. ACM 48, 1, 71--74. Google ScholarDigital Library
- K. Hornbæk and E. Frøkjær. 2008. A Study of the Evaluator Effect in Usability Testing. Human-Comput. Interact. 23, 251--277.Google ScholarCross Ref
- Y. C. Hu. 2014. Fuzzy multiple-criteria decision making in the determination of critical criteria for assessing service quality of travel websites. Expert Syst. Appl. 36, 3, Part 2, 6439--6445. Google ScholarDigital Library
- ISO (International Organization for Standardization). 2011. ISO/IEC 25010:2011--System and Software Engineering--Systems and Software Quality Requirements and Evaluation (SQuaRE)--System and Software Quality Models.Google Scholar
- Y. Lee and K. A. Kozar. 2006. Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach. Decis. Support Syst. 43, 1383--1401. Google ScholarDigital Library
- P. Lew, L. Olsina, and L. Zhang. 2010. Integrating quality, quality in use, actual usability and user experience. In Proceedings of the 6th Central and Eastern European Software Engineering Conference (CEE-SECR’10). 978, 117--123.Google Scholar
- C. T. Liu, T. C. Du, and H. H. Tsai. 2009. A study of the service quality of general portals. Info. Manage. 46, 52--56. Google ScholarDigital Library
- H. F. Lin. 2010. An application of fuzzy AHP for evaluating course website quality. Comput. Educat. 54, 4, 877--888. Google ScholarDigital Library
- I. S. MacKenzie. 2013. Human-Computer Interaction: An Empirical Research Perspective. Morgan Kaufmann Publishers Inc., San Francisco, CA. Google ScholarDigital Library
- G. Malak and H. Sahraoui. 2010. Modeling web quality using a probabilistic approach: An empirical validation. ACM Trans. Web 4, 3, 9:1--9:31. Google ScholarDigital Library
- N. Manouselis and D. Sampson. 2004. Multiple dimensions of user satisfaction as quality criteria for web portals. In Proceedings of the IADIS WWW/Internet Conference. 535--542.Google Scholar
- L. Mich, M. Franch, and L. Gaio. 2003. Evaluating and designing web site quality. IEEE Multimedia, January--March, 34--43. Google ScholarDigital Library
- Jacob Nielsen. 1994. Usability Engineering. Academic Press, San Diego.Google Scholar
- J. Nielsen and R. Molich. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, J. Carrasco Chew and J. Whiteside (Eds.). ACM Press, New York, 249--256. Google ScholarDigital Library
- N. Nwasra, N. Basir, and M. F. Marhusin. 2015. A framework for evaluating QinU based on ISO/IEC 25010 and 25012 Standards. In Proceedings 2015 9th Malaysian Software Engineering Conference. 70--75.Google Scholar
- J. Offutt. 2002. Quality attributes of web software applications. IEEE Software, March--April, 25--32. Google ScholarDigital Library
- L. Olsina, F. Papa, and H. Molina. 2008. How to measure and evaluate web applications in a consistent way. In Web Engineering: Modeling and Implementing Web Applications, G. Rossi, O. Pastor, D. Schwabe, and L. Olsina (Eds.). Springer, Berlin, 385--420.Google Scholar
- L. Olsina and G. Rossi. 2002. Measuring web application quality with WebQEM. IEEE Multimedia, October--December, 20--29. Google ScholarDigital Library
- L. Olsina, P. Lew, A. Dieser, and B. Rivera. 2011. Using web quality models and a strategy for purpose-oriented evaluations. J. Web Eng. 10, 4, 316--352. Google ScholarDigital Library
- T. Orehovački, A. Graníc, and D. Kermek. 2012. Exploring the quality in use of web 2.0 applications: The case of mind mapping services. In Proceedings of the Conference on Current Trends in Web Engineering (ICWE’12). 266--277. Springer, Berlin. Google ScholarDigital Library
- T. Orehovački, A. Graníc, and D. Kermek. 2013. Evaluating the perceived and estimated quality in use of Web 2.0 applications. J. Syst. Softw. 86, 3039--3059. Google ScholarDigital Library
- R. Polillo. 2012. A core quality model for web applications. J. Web Eng. 11, 3, 181--208. Google ScholarDigital Library
- R. Rekik and I. Kallel. 2013. Fuzz-Web: A methodology based on fuzzy logic for assessing web sites. Int. J. Comput. Info. Syst. Industr. Manage. Appl. 5, 126--136.Google Scholar
- R. Rekik, I. Kallel, J. Casillas, and A. M. Alimi. 2016. Using multiple criteria decision making approaches to assess the quality of web sites. Int. J. Comput. Sci. Info. Secur. 14, 747--761.Google Scholar
- D. Sheskin. 2011. Handbook of Parametric and Nonparametric Statistical Procedures (5th ed.). CRC Press, Boca Raton, FL.Google Scholar
- A. Stefani and M. Xenos. 2008. E-commerce system quality assessment using a model based on ISO 9126 and Belief Networks. Softw. Qual. J. 16, 107--129. Google ScholarDigital Library
- A. Stefani and M. Xenos. 2011. Wight-modeling of B2C system quality. Comput. Stand. Interf. 33, 4, 411--421. Google ScholarDigital Library
- D. Straub, M.-C. Boudreau, and D. Gefen. 2004. Validation guidelines for IS positivist research. Commun. Assoc. Info. Syst. 13, 24, 380--427.Google ScholarCross Ref
- C.-C. Sun and G. T. R. Lin. 2009. Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites. Expert Syst. Appl. 36, 9, 11764--11771. Google ScholarDigital Library
- WAI (Web Accessibility Initiative). 2008. Web Content Accessibility Guidelines 2.0. Retrieved from http://www.w3.org/TR/WCAG/.Google Scholar
Index Terms
- Evaluating Quality in Use of Corporate Web Sites: An Empirical Investigation
Recommendations
A practical approach to the assessment of quality in use of corporate web sites
A novel perspective on web site quality based on stakeholders.A comprehensive and structured model for quality in use of corporate web sites.A practical methodology for the assessment of expected quality in use.An evaluation exercise with seven ...
Quality in use model for web portals (QiUWeP)
ICWE'10: Proceedings of the 10th international conference on Current trends in web engineeringWeb Portals are increasingly being used for important tasks, at a work, personal or leisure related issues. Since Web Portals provide with applications, services and information, their levels of quality are an important prerequisite for their success, ...
Assessing Corporate Web Sites: Quality Model and Methodology
Revised Selected Papers of the ICWE 2013 International Workshops on Current Trends in Web Engineering - Volume 8295The paper presents a new quality model for corporate web sites based on three fundamental concepts: ultimate quality, external quality, and internal quality. External quality is defined in detail through a suitable set of characteristics and sub-...
Comments