4 UNIVERSITY OF GHANA VIRTUALISATION OF COURSE EVALUATION PROCESS IN HIGHER EDUCATION INSTITUTIONS: A CASE STUDY OF THE UNIVERSITY OF GHANA BY AUGUSTUS BARNNET ANDERSON (10275226) A THESIS SUBMITTED TO THE DEPARTMENT OF OPERATIONS AND MANAGEMENT INFORMATION SYSTEMS, UNIVERSITY OF GHANA BUSINESS SCHOOL, UNIVERSITY OF GHANA, LEGON IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE AWARD OF AN MPHIL IN MANAGEMENT INFORMATION SYSTEMS DEGREE AUGUST 2015 University of Ghana http://ugspace.ug.edu.gh i DECLARATION I, AUGUSTUS BARNNET ANDERSON, the author of this thesis, do hereby declare that this thesis, with the exception of quotations and references is a result of my own research and has not been presented either in part or completely by anyone for any academic award in this or any other university. I, a student of the University of Ghana Business School, under the Supervision of Dr. John Effah and Prof. Richard Boateng, did the work presented. All references used in the work was fully acknowledged. Augustus Barnnet Anderson …………………………… (10275226) ……………………………. …………………………. Dr. John Effah Prof. Richard Boateng (Supervisor) (Co-Supervisor) University of Ghana http://ugspace.ug.edu.gh ii CERTIFICATION I hereby certify that this thesis was supervised in accordance with procedures laid down by the university. ……………………………. …………………………. Dr. John Effah Date (Supervisor) University of Ghana http://ugspace.ug.edu.gh iii DEDICATION I dedicate this piece of work to the almighty God who gave me knowledge and strength to carry out this work successfully. Also, for his guidance and protection throughout my academic life. I also dedicate this study to my parents, Mr. and Mrs. Anderson, siblings, Mrs. Florence Ndor and in loving memory of my late sister, Christabel Anderson. University of Ghana http://ugspace.ug.edu.gh iv ACKNOWLEDGMENTS My utmost appreciation and thanks goes to God for his grace and mercy, providing me with energy and support to undertake this journey. The product of a graduate degree is never the result of one person’s labour, but rather the amassing of the goodwill, sacrifice and support of people, hence, there are few people I would like to thank, for their inspiration and selfless support. My gratitude and thanks go to my two supervisors, Dr. John Effah and Prof. Richard Boateng. Their scholarly insights and academic rigour helped me to develop a better understanding of the issues involved in this thesis. I am grateful for their wonderful support, patience and kindness over the past years, which have been important to my studies. My deepest appreciation goes to the office of Academic Quality Assurance of the University of Ghana, Legon, for providing me with time and support for my research. I thank everyone in the University of Ghana that helped me in my data collection. I am grateful to my parents; Mr. and Mrs. Anderson and my godfather; Reverend Professor Cephas Omenyo. Also to my sisters; Florence Ndor and the late Christabel Anderson, my sweetheart; Janice Osei-Essah for their invaluable support, encouragement and inspiration in various ways. My special thanks go to all my course-mates Eric, Nii Barnor, Dzifa and Commodore whose comments and encouragement have spurred me on. I say God bless you and thank all of you for your encouragement, guidance, and trust which has contributed to my completion of this study. University of Ghana http://ugspace.ug.edu.gh v TABLE OF CONTENTS DECLARATION............................................................................................................................ i CERTIFICATION ........................................................................................................................ ii DEDICATION.............................................................................................................................. iii ACKNOWLEDGMENTS ........................................................................................................... iv LIST OF TABLES ...................................................................................................................... xii LIST OF FIGURES .................................................................................................................... xii ABSTRACT ................................................................................................................................. xv CHAPTER ONE ........................................................................................................................... 1 INTRODUCTION......................................................................................................................... 1 1.1 Research Background ............................................................................................................ 1 1.2 Research Problem .................................................................................................................. 5 1.3 Research Purpose .................................................................................................................. 7 1.4 Research Questions ............................................................................................................... 8 1.9 Organisation of Research ...................................................................................................... 8 CHAPTER TWO ........................................................................................................................ 11 LITREATURE REVIEW .......................................................................................................... 11 2.1 Introduction ......................................................................................................................... 11 2.2 Process Virtualisation .......................................................................................................... 11 2.3 Information System Development ....................................................................................... 15 University of Ghana http://ugspace.ug.edu.gh vi 2.3.1 Problem Definition................................................................................................... 16 2.3.2 Developmental Phase ............................................................................................... 17 2.3.3 System Use Phase .................................................................................................... 19 2.4 Performance Evaluation ...................................................................................................... 20 2.5 Definition and Conceptualization of Course evaluation ..................................................... 20 2.5.1 Overview of Course evaluation ............................................................................... 23 2.5.2 Modalities of Course evaluation .............................................................................. 24 2.5.3 Traditional Course Evaluation ................................................................................. 25 2.5.4 Web-Based (Online) Course evaluation .................................................................. 26 2.5.5 Hybrid Course Evaluation........................................................................................ 27 2.6 Online versus Paper –based Course Evaluations ................................................................ 27 2.6.1 Accessibility ............................................................................................................. 27 2.6.2 Response rate ........................................................................................................... 28 2.6.3 Revision and frequency............................................................................................ 28 2.6.4 Integrity of feedback ................................................................................................ 28 2.6.5 Anonymity ............................................................................................................... 29 2.6.6 Time spent on providing feedback ........................................................................... 29 2.6.7 Administrative costs................................................................................................. 30 2.7 Gaps in course evaluation research ..................................................................................... 31 2.8 Chapter Summary ................................................................................................................ 34 University of Ghana http://ugspace.ug.edu.gh vii CHAPTER THREE .................................................................................................................... 36 THEORETICAL FOUNDATION ............................................................................................ 36 3.1 Introduction ......................................................................................................................... 36 3.2 Empirical Grounding of the Framework ............................................................................. 36 3.3 Force Field Analysis Model - An Overview ............................................................... 37 3.4 Concepts of the Force Field Analysis Model ...................................................................... 38 3.5 The Force Field Analysis Model in Information Systems Research ................................... 39 3.6 Application of Force Field Analysis Model in this Study ................................................... 40 3.7 Summary ............................................................................................................................. 41 CHAPTER FOUR ....................................................................................................................... 42 METHODOLOGY ..................................................................................................................... 42 4.1 Introduction ......................................................................................................................... 42 4.2 Overview of Research Methodology ................................................................................... 42 4.3 Research Paradigm .............................................................................................................. 43 4.3.1 The Positivist Paradigm ........................................................................................... 44 4.3.2 The Critical Paradigm .............................................................................................. 45 4.3.3 The Interpretive Paradigm ....................................................................................... 46 4.3.4 The Choice of Paradigm for the Study .................................................................... 47 4.2.4 The Philosophical Assumptions ............................................................................... 47 4.4 Research Design and Method .............................................................................................. 49 University of Ghana http://ugspace.ug.edu.gh viii 4.4.1 The Qualitative Research Method ........................................................................... 50 4.4.2 The Quantitative Research Method ......................................................................... 51 4.4.3 The Choice Method for the Study ............................................................................ 51 4.4.4 Choice of Qualitative Interpretive Case Study Research Method ........................... 52 4.5 Selection of Case and Fieldwork ......................................................................................... 54 4.6 Data Collection Method ...................................................................................................... 55 4.6.1 Interviews ................................................................................................................. 55 4.6.2 Documentation and Archival Records ..................................................................... 58 4.6.3 Physical Artefacts .................................................................................................... 58 4.7 Data Analysis ...................................................................................................................... 58 4.8 Ethical Considerations ......................................................................................................... 59 4.9 Summary ............................................................................................................................. 61 CHAPTER FIVE ........................................................................................................................ 62 RESEARCH FINDINGS ............................................................................................................ 62 5.1 Introduction ....................................................................................................................... 62 5.2 Case Description of University of Ghana ........................................................................ 62 5.3 The physical process of Course Evaluation ................................................................... 66 5.4 Decision for Online Course Evaluation System .................................................................. 69 5.5 The Development of Online Course Evaluation System ............................................... 74 5.6 The virtualised process of Course Evaluation ..................................................................... 81 University of Ghana http://ugspace.ug.edu.gh ix 5.7 The Use Phase of the Online Course Evaluation System ............................................... 87 5.8 Summary ............................................................................................................................. 90 CHAPTER SIX ........................................................................................................................... 92 ANALYSIS OF FINDINGS ....................................................................................................... 92 6.1 Introduction ......................................................................................................................... 92 6.2 University of Ghana’s reasons for pursuing the process virtualisation of Course evaluation ................................................................................................................................................... 92 6.2.1 Reduced Processing Time ........................................................................................ 92 6.2.2 Save Valuable Lecture Time .................................................................................... 93 6.2.3 Reduced Operational Costs ...................................................................................... 94 6.2.4 Accessibility ............................................................................................................. 95 6.2.5 Easy modification of Evaluation Questions ............................................................. 95 6.2.6 The ability for students to evaluate at their convenient time ................................... 96 6.3 Enabling factors influencing the decision for Online Course Evaluation System .............. 96 6.3.1 Increased efficiency and effectiveness ..................................................................... 96 6.3.2 Increased End-User Satisfaction .............................................................................. 97 6.3.3 Environmental Forces .............................................................................................. 97 6.4 Factors Constraining the Decision for virtualisation of Course evaluation ........................ 98 6.4.1 Fear of Low Response Rate ..................................................................................... 98 6.4.2 Lack of Perceived Anonymity, authentication and Confidentiality ......................... 99 University of Ghana http://ugspace.ug.edu.gh x 6.4.3 Over-Dependence on Technology ......................................................................... 100 6.5 Development process of the Course Evaluation ................................................................ 101 6.5.1 Enabling Factors influencing the development of the Course evaluation ............. 101 6.5.2 Factors constraining the development of the Course evaluation ........................... 104 6.6 Force Field Analysis Model of Process Course evaluation ............................................... 107 6.7 Summary ........................................................................................................................... 108 CHAPTER SEVEN ................................................................................................................... 109 DISCUSSION OF FINDINGS ................................................................................................. 109 7.1 Introduction ....................................................................................................................... 109 7.2 Reasons Higher Educational Institutions Pursue Virtualisation of Course Evaluation Process ..................................................................................................................................... 109 7.3 Enabling factors influencing the decision to virtualise course evaluation process ........... 113 7.4 Constraining factors influencing the virtualisation of Course Evaluation Process ........... 114 7.4.1 Fear of Low Response Rate ................................................................................... 114 7.4.2 Lack of Perceived Anonymity and Confidentiality ............................................... 115 7.4.3 Over -Dependence on Technology ........................................................................ 116 7.4.4 Organisational issues ............................................................................................. 116 7.5 Factors enabling the development process of the Course Evaluation ............................... 117 7.5.1 Usability ................................................................................................................. 117 7.5.2 Optimizability and Convenience............................................................................ 117 University of Ghana http://ugspace.ug.edu.gh xi 7.6 Constraining Factors influencing the development process of the Course evaluation ...... 118 7.6.1 Lack of Perceived Anonymity ............................................................................... 119 7.6.2 Lack of end-user involvement ............................................................................... 119 7.6.3 Inadequate Sensitization ........................................................................................ 120 7.6.4 Lack of policy on the use of the system ................................................................. 120 7.7 Summary ........................................................................................................................... 121 CHAPTER EIGHT ................................................................................................................... 122 SUMMARY, CONCLUSION AND RECOMMENDATIONS............................................. 122 8.1 Introduction ....................................................................................................................... 122 8.2 Review of Research Questions .......................................................................................... 122 8.3 Contribution to Knowledge ............................................................................................... 125 8.3.2 Offering Rich Insights............................................................................................ 126 8.3.3 Implications for Research ...................................................................................... 126 8.3.4 Implications for Practice ........................................................................................ 127 8.3.5 Implications for policy ........................................................................................... 128 8.4 Limitations of the Research ............................................................................................... 128 8.5 Recommendation for future Research ............................................................................... 129 8.6 Recommendation for Practice at University of Ghana ...................................................... 130 8.7 Conclusion ......................................................................................................................... 130 REFERENCES .......................................................................................................................... 132 University of Ghana http://ugspace.ug.edu.gh xii LIST OF TABLES Table 2.1: Studies on Electronic Learning .................................................................................... 13 Table 2.2: Definitions of Course evaluation in Literature ............................................................ 23 Table 2.3: Gaps in Course evaluation research ............................................................................ 31 Table 3.1: Force Field Analysis Model in Information Systems Research .................................. 39 Table 4.1: Philosophical Assumptions underpinning Information Systems Research ................. 49 Table 4.2: Summary of Interviews Conducted ............................................................................. 57 Table 5.1: Methods of Course evaluation in University of Ghana ............................................... 65 LIST OF FIGURES Figure 2.2: Information Systems Development Life Cycle .......................................................... 17 Figure 2.3: Modalities of Course evaluation ................................................................................. 26 Figure 2.4: Thematic Issues in Course evaluation ........................................................................ 29 Figure 3.1: Force Field Analysis Model ....................................................................................... 38 Figure 4.1: Paradigms in Information Systems Research ............................................................. 44 Figure 5.2: Online Course Evaluation System Interface ............................................................. 82 Figure 5.3: Online Course Evaluation System Interface ............................................................. 83 Figure 5.4: Online Course Evaluation System Interface ............................................................. 84 Figure 5.5: Merge System Interface ............................................................................................. 85 Figure 5.6: Merge System Interface ............................................................................................. 86 Figure 6.1: Kurt Lewin’s Force Field Analysis Model of Process Course evaluation .............. 107 University of Ghana http://ugspace.ug.edu.gh xiii LIST OF ABBREVIATIONS AIS Academic Information Systems AQAP Academic Quality Assurance Policy AQAU Academic Quality Assurance Unit CE Course Evaluation CLES Course and Lecturer Evaluation System CIS Campus Information System DOI Diffusion of Innovation ERP Enterprise Resource Planning GUI Graphic User Interface HIS Higher Education Information Systems IS Information Systems ISD Information Systems Development ICES Instructor and Course Evaluation System ISU Information Systems Use SDLC Systems Development Life Cycle SEQ Student Evaluation Questionnaire SRI Student Ratings of Instruction SET Student Evaluation of teaching University of Ghana http://ugspace.ug.edu.gh xiv TPB Theory of Planned Behaviour TAM Technology Acceptance Model TPE Teacher Performance Evaluation UG University of Ghana UGCS University of Ghana Computing Systems UGCS ADMIN University of Ghana Computing Systems Administrator UTAUT Unified theory of acceptance and use of technology HEIs Higher Educational Institutions PU Percieved Usefulness BI Behavioural Intention to use PEU Percieved Ease of Use University of Ghana http://ugspace.ug.edu.gh xv ABSTRACT The act of student evaluating teaching is a prevailing development in Higher Educational Institutions (HEIs). In most HEIs in Africa, students evaluate courses for both summative and formative purposes, that is, to serve as a feedback mechanism to faculty for instructional improvement. The method of evaluating courses is undergoing a tremendous change due to the rapid development in technology. The purpose of this study is to understand why and how higher educational institutions virtualise their course evaluation process, using University of Ghana as a case study. It further seeks to identify the enabling and constraining forces associated with the virtualisation of the course evaluation process. Information Systems Research in HEIs in developing countries is focused on application in learning and teaching (such as e - learning systems) but silent on application in the evaluation of the teaching delivered. This study tends to address this paucity of knowledge, since evaluation of teaching is critical for the sustainability in HEIs. This study employs the use of Kurt Lewin’s Force Field Analysis Model to delve into the driving or enabling and restraining or constraining forces connected to this move. This research was conducted base on the tenets of interpretive paradigm. The study uses a qualitative research method and an interpretive case study in order to achieve the purpose of this research which is to gain complex and detailed understanding of not only “what” but the “why” of the phenomenon under study. The data was analysed by identifying emergent themes to answer “why” higher educational institutions virtualise their course evaluation process. The Kurt Lewin’s force field analysis model was used to analyse and identify the constraining and enabling forces that influence the virtualisation process. University of Ghana http://ugspace.ug.edu.gh xvi The findings suggest that higher educational institutions virtualise their course evaluation in order to reduce processing time, reduce operational cost, internal environmental influences, anonymity and confidentiality and other organisational issues. These findings from the study contribute to the body of knowledge on virtualisation of course evaluation process by outlining the constraining and enabling forces from a developing country perspective. These findings present practical implications to the case institution that, the systems interface should be redesigned for courses to be classified by colleges. The novelty of this study first stems from the being arguably one of the few empirical studies that has been conducted on the virtualisation of the course evaluation process from a developing countries perspective. In addition, the use of Kurt Lewin’s force field analysis model to study both the constraining and enabling forces associated with the virtualisation of course evaluation process enhances knowledge on the effects of these forces in the implementation of an IS in an educational context. Future studies may consider a quantitative approach to test the generalisability of these findings over a wider population of HEIs or explore the effects of context comparing differences in the implementation in public and private HEIs. Further, as course evaluation systems move on to more accessible technologies like mobile phones, future studies should find out which forces will enable or constrain such a process. University of Ghana http://ugspace.ug.edu.gh 1 CHAPTER ONE INTRODUCTION 1.1 Research Background Evaluation is a strategy used in many organisations and institutions, both public and private institutions of which the educational institutions are not an exemption. According to Lin and Pervan (2001), “evaluation is a process that suggests planning and treatment by providing feedback information and contributing to organisational development.” There is a broad consensus that, the major goal of evaluation should be to influence decision-making or policy formulation through the provision of empirically driven feedback (Trochim, 2006). Evaluation of the performance of lecturers by students started as early as 1915 (Bemile, Jackson & Ofosu, 2014). Alessia and Crow (2012) stated that, the quality of student learning is directly, although not exclusively, related to the quality of teaching. As a result, one of the most promising ways to improve learning is to improve teaching. Several studies have shown that lecturers play an important role in the academic achievement of students and each lecturer’s effectiveness in teaching varies (Adam, 2009; Blair & Kimila, 2014). The core purpose of every educational institution is teaching and this is obvious from most of these institution’s vision and mission statements. These statements emphasize on achieving quality teaching and learning experiences. For example, University of Ghana’s mission statement is, “To develop world-class human resources and capabilities to meet national development needs and global challenges through quality teaching, learning, research and knowledge dissemination” (University of Ghana, 2006). University of Ghana http://ugspace.ug.edu.gh 2 In addition, Kennedy (1997) stated that, of the many expectations that society has about modern universities, the most important is to teach well. In efforts to improve educational outcomes for students and increase accountability for teachers, there is the need to evaluate courses taught by lecturers. The Higher Education Act of 1965 in Ghana defines ‘institution of higher education’ as an “educational institution that admits regular students having a certificate of graduation from a school providing secondary education. Such institutions are legally authorised to provide a program of education beyond secondary education, for which the institution awards a bachelor’s degree” (Higher Education Act of 1965, 2014). Course evaluation serves three (3) main purposes, namely, formative, summative and informative (Spencer and Schmelkin, 2002). Formative purposes are conducted to serve as a feedback mechanism to faculty for instructional improvement. The summative purposes are conducted as an evaluation mechanism for purposes such as tenure and promotion of faculty, and informative purposes are conducted to help students in selecting future courses (Dorit, McClean & Nevo, 2012). Out of the three main purposes, the most important purpose perceived by students and the faculty is the formative or feedback-role (Chen & Hoshower, 2003; Nasser & Barbara, 2002). Sumsion (2011) indicated that over recent decades, higher educational institutions have been under scrutiny from both governments and consumers. A research on student evaluation of teaching by Tak, Marsh and Jones (1988) pointed out that, in 1973 only about 30 percent of colleges and universities were asking students to evaluate professors. Today, almost every university and college allow students to evaluate courses taught at the end of the semester or year (Sumsion, University of Ghana http://ugspace.ug.edu.gh 3 2011). Course evaluations are now the most important measure of a teacher’s teaching ability. This proves that evaluation is a strategy that is highly adopted in the contemporary world. Yet the performances of lecturers are seldom evaluated. The common trend is a teacher being evaluated at most once in an academic semester, usually at the end. The term course evaluation has been used synonymously with other terms in literature. There are almost as many terms used to describe course evaluations as there are articles about them. Some of the commonest terms include “Course and Lecturer Evaluation System” (CLES) (Bemile, Jackson & Ofosu, 2014); “student ratings of instruction”, “Teacher Performance Evaluation”, “Student Evaluation Questionnaire” (SEQ) (Blair & Kimila, 2014); “Student Evaluation of teaching” (SET) (Alauddin & Kifle, 2014) (Pounder, 2007) (Chen & Hoshower, 2003; Dorit, McClean & Nevo, 2012) and “Course Teaching Evaluation” (Azizah et al., 2011). Each of these terms has slightly different connotations depending on whether they emphasize students, courses, or the evaluation process itself. For the purpose of this research, the study uses the term “Course Evaluation System (CES)” (Bemile, Jackson & Ofosu, 2014) because, it covers all aspects of the evaluation process. Course evaluation in the educational industry is undergoing constant changes due to the rapid devolvement of technology and the role it plays in the educational industry (Johnson, 2003). Evaluation process just like any process can be defined as a series of steps to achieve an assessment objective. According to Dwivedi, Wade and Scott (2012), process virtualisation is the transitioning from a physical process to a virtual process. A physical process involves somatic interaction between people or between people and objects. A virtual process is a process in which the physical interaction between the people and/or objects in the physical process has been removed totally or partially (Dwivedi et al., 2012). University of Ghana http://ugspace.ug.edu.gh 4 Hence, it can be argued that the migration of the evaluation process from the physical process to the internet qualifies the phenomenon to be process virtualisation. The internet has created a more effective and an efficient means of collecting data from students online resulting in a course teaching evaluation system. Prior research on online teaching evaluation systems have come to the conclusion that online surveys are more resource-efficient and offer greater convenience, ease of use, and student satisfaction (Donovan, Mader & Shinsky, 2007). They provoke more comments and qualitative responses compared to paper-based surveys and weaker in providing feeling of anonymity to respondents. The commitments of Higher Educational Institutions like University of Ghana to the assurance of the quality of its academic programs are broadly expressed in its mission statement. This has informed the creation of an Academic Quality Assurance Policy, which includes the evaluation of teaching and courses by students (University of Ghana, 2009). The Quality Assurance Unit carries out the evaluation of teaching and courses. In University of Ghana, the unit moved the course evaluation process online in 2012 from its previous paper-based process after several years. Based on the above arguments, there is the need to understand why and how higher educational institutions migrate from paper based to the online course evaluation system in an attempt to virtualise the course evaluation process. There is also the need to explore the implications associated with this migration by delving into the driving and restraining forces of this migration to both students and lecturers alike in using this new system. University of Ghana http://ugspace.ug.edu.gh 5 1.2 Research Problem As earlier stated, one of the dominant routine which forms part of both graduate and undergraduate education is students’ evaluation of the quality of teaching in the university (Mohanty et al. 2005). Studies on course evaluation have been conducted from different perspectives. These perspectives are student and faculty perceptions of course evaluation, the validity and reliability of course evaluation, online versus paper course evaluation, and effects of allowing students access to course evaluation data (Adam, 2009; Ahmad, et al., 2012; Annan, Tratnack, Rubenstein, Metzler -Sawin & Hulton, 2013; Azizah et al., 2011; Dorit, McClean & Nevo, 2012; Donovan, Mader & Shinsky, 2007). Most of the studies concentrate on students’ perception, validity and reliability of course evaluation and effects of allowing students access to course evaluation data. Few studies have been conducted on the online and paper course evaluations (Anderson, Jeff & Eleanora, 2005; Avery, Bryant, Mathios, Kang & Bell, 2006; Bemile, Jackson & Ofosu, 2014; Donovan, Mader & Shinsky, 2007; Nicole, Steven & Erin, 2007; Bemile, Jackson & Ofosu, 2014). However, all the above studies on the online versus paper course evaluation concentrated on comparing the paper- based to the online course evaluation system, in order to identify the importance of the online system over the paper-based system leaving out the virtualisation process of the paper-based to the online system. Furthermore, as earlier stated, most of the studies conducted on course evaluation system have been from different dimensions which includes students’ perception on course evaluations, the validity and reliability of course evaluations, online versus paper course evaluations and effects of allowing students access to course evaluation data (Ahmad et al., 2012; Annan, et al., 2013; Dorit et al., 2012; Donovan et al, 2007). These studies cut across a number of countries, both developed University of Ghana http://ugspace.ug.edu.gh 6 and developing economies alike. But most of the studies done originates from the developed economies like Hong Kong (Tak, Marsh & Jones, 1988); Texas (David, Denise & Robert, 2010); California (David & Matthew, 2010); Indiana (Donovan, Mader & Shinsky, 2006; Johnson, 2003); Canada (Dorit, McClean & Nevo, 2012); United States of America (Timothy, Gregory & Larry, 2013; Adam, 2009); Australia (Alauddin & Kifle, 2014) and many more. Unlike the developed economies, few studies on online course evaluation has been conducted in developing economies like Trinidad and Tobago (Blair & Kimila, 2014) and Ghana (Ansah, 2010). As a result, there is a call for more studies that will probe into this new phenomenon from the developing country’s perspective. The ongoing observations on course evaluation studies present research gaps that inspired this study. These research gaps are summarised as follows:  The culture of education in most developing countries is such that hearing the student voice is a relatively new concept but there is evidence here that students want to be heard (Blair & Kimila, 2014). Nevertheless, other developing countries for which Ghana is not an exemption allow students to evaluate courses taught using the traditional evaluation system (Ansah, 2010). The traditional evaluation system is flawed with the lack of privacy during the evaluation process, accessibility, integrity of feedback, time spent on providing feedback, administrative costs (Nicole et al., 2007; Ardalan et al., 2007; David et al., 2010; Timothy, Gregory & Larry, 2013). Hence, the need to focus on a new form of evaluation process that can combat the flaws of the traditional course evaluation system have been called for. University of Ghana http://ugspace.ug.edu.gh 7  Most of the studies concentrate on students’ perception, validity and reliability of course evaluation and effects of allowing students access to course evaluation data. Few studies have been conducted on the online versus paper course evaluations. However, all the above studies on the online versus paper course evaluations concentrated on comparing the paper-based to the online course evaluation system to identify the importance of the online system over the paper-based system leaving out the why and how the paper-based is virtualised.  Furthermore, as earlier stated, most of the studies conducted on course evaluation span across a number of countries, both developed and developing economies alike, but most of the studies done originates from the developed economies. Unlike the developed economies, few studies on online course evaluation has been conducted in developing economies. Hence, the call for more studies that will probe into this new phenomenon from the developing country’s perspective. This study seeks to address the gaps identified above. 1.3 Research Purpose The purpose of this research is to understand why and how higher educational institutions virtualise their course evaluation process, using University of Ghana as a case study. The study further seeks to explore the implications associated with the virtualisation process by employing University of Ghana http://ugspace.ug.edu.gh 8 the use of Kurt Lewin’s Force Field Analysis Model to delve into the driving or enabling and restraining or constraining forces. 1.4 Research Questions In the context of the background, the research problem and purpose outlined above, this study is conducted to add to the body of knowledge by addressing the following research questions: 1. Why do higher educational institutions virtualise course evaluation processes? 2. How do higher educational institutions virtualise their course evaluation processes? 3. What factors enable or constrain virtualisation of course evaluation processes in higher educational institutions? 1.9 Organisation of Research In order to achieve the purpose of this study, which is to understand the why and how higher educational institutions pursue virtualisation of course evaluation; this research is going to follow a systematic research process. Below is an outline of this study: This is an eight-chapter thesis consisting of an introduction, a literature review presenting the relevant theory, presentation of the research methods, the methodological approaches used, presentation of research findings, analysis and discussion of empirical findings, a summary, conclusions and recommendations. University of Ghana http://ugspace.ug.edu.gh 9 The first chapter of this study, which is the introduction, comprise of; a research background, research problem, research purpose, research questions and the chapter outline of the organisation of the research. The second chapter of the study focuses on an in-depth review of relevant literature on Virtualisation and Performance evaluation. In addition, literature on both paper-based and online course evaluation systems is reviewed to provide an overview of course teaching evaluation by students in a way to helping the researcher to familiarize with the existing body of knowledge on the course teaching evaluation system and a theory will be adopted for this study based on the reviewed literature. In addition, the third chapter presents a detailed description of the research framework of the study. This includes an analysis of the force field analysis model, its constructs from the information systems perspective and the relevance of the constructs to the study. The chapter then delved into how the force field analysis model is applied in this study. Furthermore, chapter four of this thesis probes into the methodological approaches that was employed in carrying out this study. It highlights on the areas below; discussion on the research design and method used. The section also outlines the three most commonly used research paradigms in Information System (IS) research, which are the Interpretive, Positive, and the Critical Paradigm. Followed by a qualitative interpretive case study. In addition, the data collection instrument, data processing, analysis, is discussed in this chapter. University of Ghana http://ugspace.ug.edu.gh 10 The fifth chapter presents the findings of the study in context of the research purpose and question. The chapter presents the background of the University of Ghana. It further outlines both the physical and virtualised processes of the course evaluation system. It presents the reasons for virtualising the course evaluation and the development phase. The sixth chapter analyses the empirical findings of the research in chapter five in relation to the research questions and the force field analysis model to bring out the dominant themes. The first research question was analysed by identifying the emergent themes from the research findings. The force field analysis model is then used to analyse the second and third research questions to identify the constraining and enabling forces. The seventh chapter, which is the discussion of findings, discusses the analysis of findings in chapter five in other to answer the research questions posed in this chapter. These findings are discussed in relation to the literature reviewed in chapter two. Finally, chapter eight of this study summarizes, concludes and makes recommendations for future research based on the limitations of this study. The chapter reviews the research questions and discusses how they have been addressed in this thesis. The chapter also entails a detailed outline of the significance of this study by presenting this research contribution to practice, research and policy. All the references of articles used in this research follow this chapter. University of Ghana http://ugspace.ug.edu.gh 11 CHAPTER TWO LITREATURE REVIEW 2.1 Introduction The preceding chapter discussed the background and the problem under study. It further established the purpose and the questions that underpins this study. Finally, it outlined how this research is organised. This chapter discusses the review of pertinent literature that relates to the topic under study. The review process undertaken in this chapter is divided into four parts. The first part reviews the literature on virtualisation, information systems in higher education, performance evaluation and the definition and conceptualization of course evaluation. The second part of this chapter reviews course evaluation system literature in developing countries, in order to identify the research issues and areas. In the third part, literature on online course evaluation system is reviewed. Finally, this chapter presents the gaps found in literature and argue the need for this study. 2.2 Process Virtualisation The term virtual or virtualisation as used in this study is different from resource virtualisation of network routers (McIlroy & Sventek, 2006), storage virtualisation (Shu, Li & Zheng, 2005), virtualisation of machines in the cloud (Zhang, Li & Zheng, 2013), network virtualisation (Berl, Race, Ishmael & Meer, 2010). It is important to distinguish between how the term virtualisation has been used because it helps untangle contradictions that arise and distinguish between face-to- face processes, hybrid and purely virtual processes (Fiol & O’Connor, 2005). University of Ghana http://ugspace.ug.edu.gh 12 Grzegorz (2012) asserts that the concept of virtualisation is derived from the term virtual, which can be defined as simulated, or carried out by means of a computer or computer network or existing in mind or not real. Virtualisation is described as a process in an enterprise in which various organisational activities and functions are based on, modified by or dependent on a virtual environment (Grzegorz, 2012). According to Overby (2008), process virtualisation is the transitioning from a physical process to a virtual process. A process is considered as a physical process if it involves a physical interaction between people or between people or objects. For instance, in the case of the traditional method of the course evaluation that deals with paper and pencil evaluation in the lecture rooms with students will pass for a physical process. A virtual process is a process in which the physical interaction between people or people and objects are removed (Overby, 2008). The process virtualisation phenomenon is happening in many contexts, including formal education using e-learning (Benta, Bologa & Dzitac, 2014; Wang, 2003; Sun, Tsai, Finger, Chen & Yeh, 2008; Fong, Kwan, Wang, Huang, Ma & Zhang, 2008; Saltz, Hiltz, Turoff, & Passerini, 2007). There has also been virtualisation of marketing known as e-marketing. This is when marketing activities are more and more dependent on computer networks and leads to the reconfiguration of resources and competences of a company (Grzegorz, 2012; Stanfield & Grant, 2003). Furthermore, there has been virtualisation of the business and shopping process, which is also known as e- business and e-commerce respectively (Shin, Ryoo & Kim, 2012). It is evident in literature that, using e-learning as online instructional tools can remove a number of hindrances. Some of these hindrances are associated with the face-to-face lectures providing a University of Ghana http://ugspace.ug.edu.gh 13 forum to address issues through argumentative and collaborative discourse (Karacapilidis & Papadias, 2001). The term e-learning is defined as a type of learning that is enabled or supported by the use of digital tools and content. It typically involves some form of interactivity, which may include online interaction between the learner and their teacher or peers (Asabere & Mends-Brew, 2012). Bates (2008) also defines e-learning as all computer and internet-based activities that support teaching and learning both on–campus and at a distance. Below is a table presenting some studies of electronic learning. The review pinpoints studies that focus on migration of education from the traditional environment to the online environment and how users come to accept it. Table 2.1: Studies on Electronic Learning Research Study Research Focus Underpinning Theory and Framework Research Method Raaij & Schepers (2008) This research focuses on how students accept and use virtual learning environments Technology Acceptance Model (TAM), TAM2 and (UTAUT) Quantitative Hsia et al. (2014) The study explains employee acceptance of e-learning systems Technology Acceptance Model (TAM) Quantitative Masrom (2007) This study examined TAM using student acceptance of e-learning technology Technology Acceptance Model (TAM) Quantitative University of Ghana http://ugspace.ug.edu.gh 14 Park (2009) This research focuses on the verification of the process of how university students adopt and use e-learning. Technology Acceptance Model (TAM) Quantitative Haron, Abbas, & Rahmanb (2012) The study provided insights on the attitude towards the adoption of blended learning. TAM and Mezirow’s Transformational Learning Theory. Quantitative 2.3 Higher Education Information Systems The rapid development of technology has caused major changes at an uneven pace in any growth- oriented industry of which the educational sector is no exemption (Krishnaveni & Meenakumari, 2012). Educational institutions just like any other organisation or institution thrives on information. Information has become an invaluable asset to these institutions since it helps them to gain competitive advantage. Due to this fact, these institutions use of Information systems to efficiently manage this information. Therefore, improving decision making in governance, increasing efficiency in managing the university and reducing operational inefficiency (Sanyal, 1994; Krishnaveni & Meenakumari, 2012). Bisao (2009) found out that, Universities just like any organisation are increasingly embracing information communication technology in educational management due to the premise of the changing requirement for more accountability since universities have become more accountable to its stakeholders. Krishnaveni and Meenakumari (2012) reported that, there are information systems employed to manage three functional areas of information administration, which is essential in managing higher education institution. These functional areas are Student Administration, Staff administration and General administration. Higher educational institutions, to manage each functional area of information administration, adopt several information systems. Some of these systems include the Management of Academic University of Ghana http://ugspace.ug.edu.gh 15 Information Systems (AIS) (Etin, 2013), Campus Information System (Kumar, 2011), Enterprise Resource Planning (ERP) systems (Abugabah & Sanzogni, 2010). The Campus Information System is a unified system that provides a single point of access to all secure administrative systems at higher educational sectors (Kumar, 2011). 2.3 Information System Development As defined by Isaias and Issa (2015) system development consists of the process of creating an information system, with all the variables that it entails. This is usually the system’s ability to be user-friendly, how well it functions and if it meets the needs of the organisation in which it will be integrated. Avison and Fitzgerald (2003) also defined information system development as the way in which information systems are conceived, analysed, designed and implemented”. In Dawson, Burrell, Rahim and Brewster (2010) study, the system development life cycle is also referred to as the application development life cycle. It describes the process of planning, creating, testing and deploying an Information system. This definition shows the existence of a way or ways in which information systems can be developed. Avison and Fitzgerald (2003) argue that, though other approaches have surfaced in recent years, one general approach is much more influential. It has been used intensively for more than 30 years to give shape to the practices of ISD. This general approach, also known as, the “information systems development life cycle (SLDC)”. The systems development life cycle (SDLC) is the process of understanding how an information system (IS) can support business needs by designing a system, building it, and delivering it to users (Dennis, Wixom & Tegarden, 2009). University of Ghana http://ugspace.ug.edu.gh 16 The focus of this model is to outline the development process as a manageable sequence of stages with each containing a number of specific activities, about each other in an organised way. However, the life cycle has been modified in various ways to allow flexibility and over time has become more sophisticated. The information system development life cycle has a set of four fundamental phases: planning, analysis, design, and implementation. Different projects may emphasize different parts of the information system development life cycle or approach Figure 2.2: Information Systems Development Life Cycle 2.3.1 Problem Definition This phase provides the basics for understanding why information systems should be built and deciding how the project team will go about building it. The phase starts with the awareness of the need for a new IS. This originates from the need to provide a solution to a specific problem, to University of Ghana http://ugspace.ug.edu.gh 17 respond to pressure, or to exploit an opportunity (Avgerou & Cornford, 1998). Dennis, Wixom and Tegarden (2009) asserts that at this phase of the Systems Development Life Cycle the system’s business value to the organisation is identified. From this realisation of need, preliminary analysis presents a broad statement of the objectives of the new IS in alignment with the organisation’s business objectives. Feasibility study evaluates project viability and alternative solutions in terms of their technical, legal, economic, organisational and social feasibility which guides management decision-making (Avison & Fitzgerald, 2003). 2.3.2 Developmental Phase For the purpose of this study, the development phase is categorised into the Analysis, Development and the implementation phase. 2.3.2.1 The Analysis Sub-Phase The analysis phase answers the question, who will use the system, what will the system do and where and when it will be used. During this phase, the project team examines the current system, identify improvement opportunities and develops a concept for the new system (Dennis, Wixom & Tegarden, 2009). A strategy is formulated by the project team for the analysis of the current system which is called the ‘as - is - system’, the problems associated with its use. After this, ways to aid in designing a new system called the ‘to -be -system’ is also outlined (Dennis, Wixom & Tegarden, 2009). The next is step a set of requirements of what the new system will do; performance standards, social and technical requirements, systems concepts and models are then developed. These facts are gained by interviewing both management and operational staff, by University of Ghana http://ugspace.ug.edu.gh 18 using questionnaires, by direct observation or experience, and searching documentation and records (Bocij et al., 1999). Finally, the analyses, system concept and models are combined into a document called the system proposal, which is presented to the project sponsor and other key decision makers 2.3.2.2 The Development Sub-Phase The development phase involves transforming the requirements at the analysis phase into a working system. At this stage, it is decided on how the system will operate, in terms of the hardware, software, and network infrastructure; the user interface, forms and reports; and the specific programs, databases, and files that will be needed. However, depending on the decision made at the feasibility stage, the organisation may either decide to acquire the IS solution by purchasing an “off the shelf” package from a vendor, develop it “from scratch” by in-house IS department or software house (bespoke development) or develop it by end users themselves (Bocij et al., 1999). If the solution were to be purchased, a number of possible solutions from vendors would be evaluated in relation to the requirements of the previous stage. If the system is to be developed, a design strategy will be created. This strategy clarifies whether the system will be developed by the company’s own programmers or outsourced to another firm. This leads to the development of the basic architecture design for the system, which describes the hardware, software, and network infrastructure to be used. In most cases, the system will add or change the infrastructure that already exists in the organisation. The interface design specifies how the users will move through the system and the forms and reports that the system will use. The database and file specifications University of Ghana http://ugspace.ug.edu.gh 19 are developed. These define exactly what data will be stored and where they will be stored. The analysis team develops the program design, which defines the programs that needs to be written and exactly what each program will do (Dennis, Wixom & Tegarden, 2009). 2.3.2.3 The Implementation Phase This phase involves technical issues like, transferring data, and handling unexpected errors. Human issues like educating and training of users and in some cases hiring of new personnel are necessary to facilitate a smooth transition from the old to the new system. Training would include installation, usability, security, data management and error debugging techniques (Bocij et al., 1999). This is critical to facilitate the collective and individual knowledge of the functions and responsibilities involved in using the new system. Finally, system review and maintenance occurs after the system has been signed off as suitable for users 2.3.3 System Use Phase The Information System Use (ISU) is an important part of the human behaviours in using a computer in an organisation (Jonghak & James, 2012). This phase is where the end users of the system use the system. After outlining the phases of the development of the course evaluation system, each phase that is, from the problem initiation through the development phase to the system use phase will be analysed using the force field analysis model. At each phase, the restraining and the driving forces will be identified. University of Ghana http://ugspace.ug.edu.gh 20 2.4 Performance Evaluation In order to achieve the final target of organisation, several evaluation processes are adopted to measure the performance of the organisation (Wu et al., 2011; Hardré & Cox, 2009). A performance evaluation that is an assessment model could help organisations to compare past plans and executions of strategies, sketch future strategies, and set up performance targets of employees. Performance appraisal also possesses the function of responding policy and unifying the target of organisations and individuals. Jalaliyoon and Taherdoost (2012) in their research stated that ‘‘performance evaluation is for achieving the entire target”. As university administrators seek to improve resource usage, efficiency analysis has become an important concern in managing performance (Avkiran, 1999). Most studies on performance evaluation in Higher education institutions have focused on how to allocate educational resource inputs more efficiently to improve output performance. Generally, these inputs include human, financial and material resources (Martin, 2006). 2.5 Definition and Conceptualization of Course evaluation The aim of this section is to compare and provide the different definitions and descriptions of course evaluation in literature. The term course evaluation has been used synonymously with other terms in literature. There are almost as many terms used to describe course evaluations as there are articles about them. Some of the commonest terms include “Instructor and Course evaluation system“ (ICES) (Wang, 2003) “student evaluation”, “student ratings of instruction”, “Teacher Performance Evaluation”, “Student Evaluation Questionnaire” (SEQ) (Blair & Kimila, 2014); “Student Evaluation of Teaching” (SET) (Alauddin & Kifle, 2014; Pounder, 2007; Chen & Hoshower, 2003; Dorit, McClean & Nevo, 2012); “Course Teaching Evaluation” (Azizah, et al., University of Ghana http://ugspace.ug.edu.gh 21 2011); Course Evaluation (Anderson, Jeff & Eleanora, 2005; Avery, Bryant, Mathios, Kang & Bell, 2010; Chatvichienchai, 2011) Each of these terms has slightly different connotations, depending on whether they emphasize students, courses, ratings, or evaluation. Wright (2008) has suggested that, the most appropriate term for end-of-teaching summative evaluations used primarily for personnel decisions (and not for teaching development) is “student ratings of instruction” because it accurately reflects how the instrument is used. This study employs the use of the term course evaluation since it is more generic and widely used. There are several definitions of course evaluation in literature, the available definitions vary and it is based on author’s perspective. As a result, each key word within the term is taken and elaborated upon for a deeper understanding before presenting the definitions identified of course evaluation identified in literature. The term evaluation is defined in literature by several authors. Jolliffe et al., 2001 defined it as, judgment on the worth or value of something. Ballantine et al., (1996), define evaluation as being a wider consideration of investments at different times. Lin and Pervan (2001) also define it as being a process that suggests planning and treatment by providing feedback information and contributing to organisational planning. Course evaluation is an important feedback mechanism for instructors and students (Dorit et al., 2012). Ketlhoilwe (2006) defined course evaluation as the systematic investigation of the worth or merit of an educational activity undertaken periodically or on a continuous basis. In addition, it is defined as a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions to evaluate the instruction of a given course. Miyoung and Chiyoung (2013) referred to course evaluation as a method widely used to determine the quality of lectures in most University of Ghana http://ugspace.ug.edu.gh 22 universities. It involves deeper probing into the activities and outcomes of the course. It is also defined as an important means of assessing courses and lecturers and providing formative feedback for future improvement and, as such, their worth should be apparent (Blair & Kimila, 2014). Course evaluations provide a way for students to lodge a complaint, extend a compliment, express appreciation, improve a course, and most importantly, participate in the learning community (Donovan, Mader & Shinsky, 2007). Based on the definitions and explanation of terms, this study adopts the definition given by Ketlhoilwe (2006). This definition is most appropriate for the study since it best describes the process of Course evaluation. Ketlhoilwe (2006) describes it as a systematic process meaning it is characterized by order and planning. Also, it is undertaken on a periodic basis and this is most evident with the course evaluation in University of Ghana, which is undertaken at the end of every semester. Below is a table summarizing some of the definitions of course evaluation as presented in literature Table 2.2: Definitions of Course Evaluation in Literature DEFINITION AUTHOR an important feedback mechanism for instructors and students Dorit, McClean, & Nevo, (2012) The systematic investigation of the worth or merit of an educational activity (e.g. environmental education course), under-taken periodically or on a continuous basis. Ketlhoilwe, (2006) University of Ghana http://ugspace.ug.edu.gh 23 a method widely used to determine the quality of lectures in most universities Miyoung & Chiyoung, (2013) as an important means of assessing courses and lecturers and providing formative feedback for future improvement and, as such, their worth should be apparent Blair & Kimila (2014) provide a way for students to lodge a complaint, extend a compliment, express appreciation, improve a course, and most importantly, participate in the learning community Donovan, Mader & Shinsky, (2007) 2.5.1 Overview of Course evaluation The first course evaluation known as the teacher rating scale was published in 1915 and the first study of students’ evaluation of teacher effectiveness was written in the 1920s (Wachtel, 1998). The primary purpose of which is not to discover new knowledge, as is the case with basic research, but to study the effectiveness with which existing knowledge is used to inform and guide practical action (Ornstein & Hunkins, 1998). As earlier indicated, the importance of course evaluation is categorised into three (Spencer & Schmelkin, 2002). Most often, summative evaluation consists of a pre- observation by using a checklist type instrument with minimal room for narrative, and a post-conference. The instrument used for summative evaluation documents those observable traits and methods that the division considers crucial for continued employment and/or placement on an improvement plan. Formative evaluation points more toward professional development and is not as concerned with employment University of Ghana http://ugspace.ug.edu.gh 24 status as are the summative evaluation tools (Murray, 2009). Teachers and administrators meet to map out a plan and direction for the teachers’ continuing development within the profession. This form of evaluation is usually reserved for experienced teachers who have been afforded tenure. As noted by Beran et al., (2007) and Beran et al. (2005) teaching evaluations are primarily used, by a wide majority, for summative purposes; that is, by administrators to support personnel decisions. Also, formative purposes refer to collecting and interpreting formative feedback to make or support personnel decisions (Hiring, tenure, promotion, and annual review) based in part on a student’s rating of an instructor is teaching effectiveness. Wright (2008) cautions against the use of instruments not specifically designed to provide formative feedback for this purpose, and that separate instruments should be designed to provide summative and formative feedback respectively. The collected data, in particular the qualitative responses, are also used by instructors and teaching support offices to provide formative feedback intended to facilitate improved teaching and course development. Beran, Violato and Kline’s (2007) study, demonstrates that though faculty believe teaching evaluations to be useful in assessing teaching, they rarely employ the results of their own evaluations in a course or professional development decisions. 2.5.2 Modalities of Course evaluation Course evaluation on performance in teaching by students is not a recent phenomenon in the world of education. Course evaluations have undergone major shifts in the last century. Teacher evaluation has changed along with beliefs and values concerning the role of teachers, effective teaching and modes of evaluation, and theories of student learning (Ellett & Teddlie, 2003). The University of Ghana http://ugspace.ug.edu.gh 25 oxford dictionary defines modality as a particular method or procedure. There are three main modalities for course evaluation. These are the traditional, web-based and the hybrid modality. Below is a diagrammatic representation of the modalities of course evaluation. Figure 2.3: Modalities of Course Evaluation 2.5.3 Traditional Course Evaluation The traditional modality of course evaluation is also called the paper-based course evaluation. Several terms have been used in literature to connote the traditional course evaluation. The commonest of these terms is the paper-pencil evaluation (Johnson, 2003). Ardalan et al. (2007) discussed in their research that traditionally, universities have used paper-and-pencil surveys to collect student feedback. Normally, a portion of a regular lecture session is devoted to the distribution, completion and collection of the paper-based surveys. Whether the class sessions in which these surveys are conducted are announced or not, the sample gathered depends on the class attendance for that single session. These printed instruments are typically a mixture of scaled, closed-ended and open-ended questions. The responses to closed-ended questions can be quickly recorded using widely available scanning technology; this set of scaled responses would then be Course evaluation Traditional Web-Based Hybrid University of Ghana http://ugspace.ug.edu.gh 26 processed as quantitative feedback. The open-ended questions, such as ‘Please comment on this course’ and the semi-open-ended questions, such as ‘what was the most important topic covered in the course?’ would elicit written comments from the students. These comments would require manual transcription to provide readability for this handwritten material and, more important, to provide anonymity for the student respondents. This set of comments would then be assembled as qualitative feedback. The processing of this qualitative feedback, requiring transcription, is one of the major disadvantages of this method, because it is expensive, time-consuming and prone to errors (Ardalan et al., 2007). 2.5.4 Web-Based (Online) Course evaluation The rapid development and the increased use of technology in education, especially the internet has led to the development of online administration and reporting of student ratings of teaching. That is, a development of an online course evaluation system. Thorpe (2002) indicated that improvements in information communication technologies have caused the paper-based course evaluation system to move to online formats making an online course evaluation a norm for higher education. An online course evaluation is an electronic questionnaire, which requires a written or selected response or answer to a series of questions to evaluate the instruction of a given course. The online method of evaluations has many advantages over the traditional method. The benefits of having students’ complete faculty evaluations online compared to the traditional paper format include time and cost savings, less susceptibility to influences, and faster reporting of results (Alauddin & Kifle, 2014; Alessia & Crow, 2012; Blair & Kimila, 2014). University of Ghana http://ugspace.ug.edu.gh 27 2.5.5 Hybrid Course Evaluation Although this modality has received little attention based on literature, the few articles that discussed it (Carini, Hayek, Kuh & Ouimet, 2003) indicated that, this form of course evaluation combines the traditional and the web-based methods in evaluating teaching. Thus making each mode of evaluation compensate the other’s flaws. 2.6 Online versus Paper –based Course Evaluations In order to achieve the intended aim of this study, this literature review focused on the online versus paper-based course evaluation theme. Some of the issues identified under this theme included the following; Success factors and response rates (Ardalan et al. 2007; Dorit et al., 2012; Avery et al., 2006; Carini et al., 2003; Donovan et al., 2006). The central idea eminent in these studies are to determine the success factors of an online evaluation system. For instance, Carini et al., (2003) studied college students responses to web and paper surveys: Does mode matter?, examining the responses of 52, 288 college students to a survey administered through the web and paper suggest that mode effects for first year and senior college students tend to be small. Below are the differences between paper-based and the online course evaluation method. 2.6.1 Accessibility A major difference between the paper-based and web-based methods evident in literature is the issue of accessibility. Handwerk, Carson and Blackwell (2000) indicated that unlike the web-based modality, in the paper-based modality, the evaluation questionnaire are administered at the end of the course for the semester. Only the students present for the particular class of the day the University of Ghana http://ugspace.ug.edu.gh 28 questionnaires are administered have the opportunity to evaluate the course. However, in the web- based method, students have the opportunity to complete the evaluations at their convenience as and when they have access to the Web. This gives each student an equal opportunity to evaluate a course even when they are absent from class. 2.6.2 Response rate The second significant difference is the response rate. A major drawback for the web-based modality is the level of response rate. Cummings and Ballatyne (1999) and Hmieleski (2000) indicate lower response rates for the web-based approach compared with the paper-based approach. A major concern with the web-based approach is the possibility of low response rate. Hmieleski (2000) reported in their research out of the 105 responding institutions, 67% indicated a return rate of 70% or more for paper-based surveys and the remaining institutions indicated return rates of 20 to over 90% for web-based surveys. 2.6.3 Revision and frequency The web-based approach offers a distinct advantage when the institution wants the flexibility of using different survey questions for different courses. The online survey instruments can be revised and modified with relative ease (Thorpe, 2002). 2.6.4 Integrity of feedback Under the web-based method, the system authenticates the students using the same authentication mechanisms as those used for university emails or online registrations. The authentication is University of Ghana http://ugspace.ug.edu.gh 29 necessary to ensure that each student provides feedback only once, and those students who are properly registered in that particular section can provide feedback for each course section. The web-based approach guarantees feedback integrity at the same level as the authentication and authorisation mechanisms of the institution’s registration system (Anderson, Jeff, & Eleanora, 2005). 2.6.5 Anonymity Unfortunately, the web-based authentication process for survey access inevitably enables the system to trace students to their feedback and report on information each student has provided, raising student concerns about the lack of anonymity. A perceived lack of anonymity in the use of some email surveys has also been suggested as a reason for low response rates (Moss & Hendry, 2002). 2.6.6 Time spent on providing feedback Most often, instructors administer paper-based surveys at the end of the class time. In some cases, the time allowed to complete the surveys is short, and in some other cases students may be in a hurry to leave the class. Several authors believe that this approach does not allow students to provide thoughtful feedback (Handwerk et al, 2000; Hmieleski, 2000). Also, most students are now accustomed to writing using computers. University of Ghana http://ugspace.ug.edu.gh 30 2.6.7 Administrative costs Several authors support web-based evaluation systems as cost cutter as one of its primitive benefits. The paper-based approach is both labour and cost -intensive process (Annan, Tratnack, Rubenstein, Metzler – Sawin & Hulton, 2013). A typical evaluation cycle begins with the faculty staff placing orders for evaluation forms for every course section. The office responsible for printing the forms sends them to the faculty once the forms are ready. Upon receipt of the forms, the administrative staff in the faculty prepares packets for each course section and delivers the packets to the departments. The lecturers take the questionnaires to their classes and have either student in the class or departmental staff members administer the questionnaires. Once the forms are returned to the department, the numerical and comments sections are separated. The staff sends the scaled responses to the computer centre for scanning and processing. For the qualitative feedback, the staff must either type the comments or send them to an external agency for typing. Upon receipt of the results from the computer centre and the typist, the staff matches the scaled and qualitative results for each course section and delivers the results to the faculty. University of Ghana http://ugspace.ug.edu.gh 31 2.7 Gaps in course evaluation research Table 2.3: Gaps of Course evaluation in Literature Research Study Focus Underpinning Theory and Framework Research Method and Countries Relevant Gaps for Future Research Social Based Theories Blair and Kimila (2014) To determine whether online student evaluation offered a pragmatic alternative to the hardcopy version and whether stakeholders were ready to shift to the online Organisationa l Change Theory Qualitative Trinidad and Tobago Future research needs to ascertain the longer-term reliability and generalisability of this study as the study indicated that students in developing and developed economies would accept the online modality of course evaluation. Technology Based Theories Chen and Hoshower (2003) Chatvichienc- hai (2011) Evaluation of some key factors that motivate students to participate in the teaching evaluation process Students perception on the evaluation of teaching Expectancy Theory Qualitative Ohio Qualitative Indiana Future research needs to solicit input from students on what specifically they see or would like to see as the outcome of an evaluation system. There is the need for more studies to explore faculty’s perceptions on the evaluation of teaching Liegle and McDonald (2005) Lessons learned from moving from a paper- based to online students’ evaluation system Technology Acceptance Model (TAM) Quantitative Atlanta There is the need to find out whether online students’ evaluation has a higher response rate than paper-based evaluation Saadé (2003) The study examined TAM constructs within the context of students’ acceptance of web based assessment of Technology Assessment Model Adapted from TAM Quantitative Canada Future studies should test the factors identified using different methods. University of Ghana http://ugspace.ug.edu.gh 32 education systems for e-learning Beran and Rokosh (2009) Dorit, McClean and Nevo (2012 Faculty’s perspective on the use of course evaluation Critical success factors and advantages offered by online Students’ Evaluations of Teaching from student’s perspective Diffusion Innovation Theory Qualitative Russia Canada Need to understand other stakeholders’ perspective on the use of the course evaluation. Need to test this critical success factors quantitatively to establish their generalisability. Education Based Theories Nikoladis and Dimitriadis (2014) To develop a statistical framework to exploit student evaluations Greek Legislative Framework Quantitative Macedonia Need to test the developed framework to examine the student perceptions on evaluation of teaching performance The process of virtualizing course evaluations has received greater attention in developed economies Texas (David, Denise & Robert, 2010), California (David & Matthew, 2010); Indiana (Donovan, Mader & Shinsky, 2006), Australia (Alauddin & Kifle, 2014) and Canada (Saadé, 2003) than the developing economies like Trinidad and Tobago (Blair and Kimila, 2014) and Ghana (Ansah, 2010). Most of the studies on course evaluation originates from the developed economies. As a result, it is important that more studies should be conducted in the context of developing countries especially in African countries to validate and add to the findings of the existing studies. Most of the studies on online course evaluation have one thing in common; they are all conducted from the nursing, psychology and educational sector. There are few literatures originating from the management information systems perspective. The dominant literature from this field is the University of Ghana http://ugspace.ug.edu.gh 33 studies conducted by Dorit, McClean and Nevo (2012) but their study was on the advantages offered by online Students’ Evaluations of Teaching (SET). They used a Canadian university as a case study to identify critical success factors of online evaluations from students’ point of view. Some of the factors identified as important by the students include anonymity, ease of use and accessibility. This was in line with a study conducted from a developing economies perspective to assess whether students are ready to move to the online modality of course evaluation (Blair & Kimila, 2014). Virtualisation of process in the educational sector for instance e-learning, just a few of the studies in the literature discusses how higher educational institutions virtualise the course evaluation process (Saadé, 2003). Hence, this study will contribute to the virtualisation of course evaluation process. The theoretical lens for these studies tends to cover a number of different theories spanning from Educational based theories, Social based theories to Technology based theories. The above table summarises research undertaken from the paper-based to the online course evaluation system that is relevant to this study’s research purpose. A study by Saadé (2003) modified the Technology Acceptance Model (TAM) into a Technology Assesment Model to examine TAM constructs within the context of students’ acceptance of web-based assessment. The Technology Assesment Model took three variables from TAM, these variables are the Percieved Ease of Use (PEU), Percieved Usefulness(PU) and Behavioural Intention to use (BI). These variables were hypothesised to see whether they have positive effects on behavioural intention to use the evaluation system. The study indicated that in agreement to TAM, results from the research indicates that there is a strong influence of PU on BI whiles PEU was found to have little significant effect on BI. This indicates that, students will use the system if they know and understand the University of Ghana http://ugspace.ug.edu.gh 34 benefits of using the system even if the system is not user-friendly. From the educational background, studies have conducted on course evaluation from several dimensions, one of such studies is by Nikoladis and Dimitriadis (2014) which aimed to develop a statistical framework to exploit students evaluations for both formative and summative purposes based on the Greek Legislative Framework for course evaluation questionnaire in HEIs. The framework developed showed two axis, the first axis indicated a continuous evaluation of one course or more for a lecturer’s performance in consecutive semesters whiles the second axis portrays and instantaneous evaluation of all courses for all lecturers. This framework is to enable institutional decision makers to evaluate effectively the performance of lecturers. The above studies highlight a number of theories that could be of relevance to this research. Despite the contribution of these theories to these studies, the theories do not necessarily focus on the objective of this study, which is to identify the forces that enable and constrain the virtualisation of course evaluation. Hence, the study adopts Kurt Lewin’s force field analysis model from the field of social science to identify the forces that operate when strategic changes are implemented in an organisation. 2.8 Chapter Summary The review of course evaluation literature highlighted a general overview of course evaluation, process virtualisation, information systems in higher education and performance evaluation. The study further discussed the genesis of evaluation. It deliberated on the definition and conceptualization of course evaluation and it modalities. It differentiated between the modalities and finally there was a further discussion on how the literature review was conducted, considering University of Ghana http://ugspace.ug.edu.gh 35 themes and conceptual frameworks. The aim was to find the most researched most researched issues, and most used theoretical approaches in the area, and to suggest areas, which need more research. The next chapter presents an overview of the force-field analysis model with and adopts it as the research framework of the study. University of Ghana http://ugspace.ug.edu.gh 36 CHAPTER THREE THEORETICAL FOUNDATION 3.1 Introduction The previous chapter reviewed literature on the virtualisation of course evaluation that led to the selection and justification of virtualisation of course evaluation as a gap. It also discusses the theoretical approaches of the course evaluation research. In pursuit of finding answers to the research questions in Chapter 1, this chapter delves into the research framework of this study having discussed extensively the literature on virtualisation of course evaluation system. It is “the theory that decides what can be observed” (Einstein 1923). 3.2 Empirical Grounding of the Framework Researchers in carrying out research seek to find the role of theories in their research (Walsham, 1995). Within organisational context, there are three important uses of theories (Eisenhardt, 1989). These uses are an initial guide to research design and data collection; a part of an iterative process of data collection; and analysis and a final product of the research. Furthermore, for researchers to understand issues that practitioners are grappling with and provide solutions there is the need to build a solid theoretical foundation (Levy & Ellis, 2006). According to Levy and Ellis (2006), building a solid theoretical foundation for research helps researchers justify a chosen methodology and enables them to give reasons for using a particular research approach. So, the force field analysis model was chosen as this study’s research framework to inform the researcher’s data collection. And to guide the analysis and discussions on the University of Ghana http://ugspace.ug.edu.gh 37 virtualisation of course evaluation system in a developing country context. The next section discusses the force field analysis model with its various components. 3.3 Force Field Analysis Model - An Overview Kurt Lewin (1890-1947) who was a renowned German social psychologist developed the force field analysis model in 1935 (Swanson & Creed, 2014). He was best known for using force field diagrams and for introducing scientific experimentation to test hypotheses as a pictorial method for describing the forces that operate when strategic changes are implemented in an organisation (Burnes & Cooke, 2013; Cronshaw & McCulloch, 2008). The Model uses force field diagrams to describe the forces that operate when strategic changes are implemented in an organisation. Lewin describes the way ‘enablers’ and ‘constrains’ act as positive or negative forces, pushing a change strategy forward or pulling it back towards the status quo (Swanson & Creed, 2014; Salaheldin, 2003; Burnes & Cooke, 2013; Cronshaw & McCulloch, 2008). The rationale for field theory is “Lewin’s belief that all behaviour arises from the psychological forces in a person’s life space and that behavioural change arises from changes to these forces” (Cronshaw & McCulloch, 2008). Therefore, to be able to understand, predict and begin to change a person’s behaviour, it is important to consider everything about the person. This includes his or her perceptual or psychological environment to construct the person’s life space (Burnes & Cooke, 2013). For Lewin, the purpose of field theory was twofold. First, constructing individual and group force fields allowed him to understand the forces that bring about certain behaviours. Second, it opened the possibility that, by changing some of these forces, individual and group behaviour could be changed (Burnes & Cooke, 2013). University of Ghana http://ugspace.ug.edu.gh 38 3.4 Concepts of the Force Field Analysis Model Lewin described the way ‘enablers’ and ‘constrains’ act as positive or negative forces, pushing a change strategy forward or pulling it back towards the status quo. Lewin did not see this as a type of cost-benefit analysis, with ‘pros and cons’. Rather, he saw the force field analysis as describing the difficulty of the journey that a strategy might face in the course of its implementation. On the other hand, it can suffer from being focused too much on the short or medium term-and it can be incomplete. It needs further challenge and validation of certain other tools that fit well with it (Salaheldin, 2003). According to Lewin, any situation or performance can be viewed as a state of temporary equilibrium. This equilibrium is caused by two sets of opposing forces. These forces are those, which try to bring change: driving, facilitating or positive forces and trying to maintain the status quo: restraining, resisting or negative forces (Swanson & Creed, 2014). Below is a diagram of the force field Analysis Model Figure 3.1: Force Field Analysis Model Restraining forces (against change) Current equilibrium Driving forces (for change) Source: Burnes & Cooke (2013) University of Ghana http://ugspace.ug.edu.gh 39 The equilibrium, or present level of productivity, can be raised or lowered by changes in the relationship between the driving and the restraining forces. Equilibrium is reached when the sum of the driving forces equals the sum of the restraining forces. Restraining forces are forces acting to decrease the driving forces. They are static forces that counteract the driving force and prevent the change from occurring (Payne, 2013). Whiles the driving forces are those forces affecting a situation that is pushing in a particular direction, they tend to initiate a change and keep it going (Burnes & Cooke, 2013). The force field analysis model is normally supported by the Lewin Change management model, which is a three-step model of change consisting of Unfreeze, Change and Refreeze (Bozak, 2003; Bernard, 2004). 3.5 The Force Field Analysis Model in Information Systems Research A few information systems research has used the force field analysis model to carry out research. Below is a table displaying some of this research and the issues they addressed. Table 3.1: Force Field Analysis Model in Information Systems Research Issues addressed Reference Implementation of Nursing Information System; Electronic Documentation System; Bar-Coded Medication Administration (Payne, 2013; Sutherland, 2013; Bozak, 2003; Kaminski, 2011) User Attitudes and Management Information System Use (Robey, 1979) The force field analysis model is still new to information systems research; few research makes use of this model. In information systems research, Kurt Lewin’s force field analysis model is University of Ghana http://ugspace.ug.edu.gh 40 viewed as a user resistance theory (Laumer & Eckhardt, 2012). It appears that many of the information systems research on force field analysis model studies focus on implementation of systems. For instance, Payne (2013) used this model in information systems implementation of nursing information systems. It was indicated that, it offers an approach that can help nurses identify the need for change, navigate through the change process, and achieve a goal or outcome. It is noted that, Lewin’s Force Field Analysis Model can also assist nurses in analysing the change process and in identifying the forces that support or resist the change. For change to occur, the state of equilibrium must be disrupted (Bozak, 2003; Kaminski, 2011). This is done when the driving force is more powerful than the restraining force, or conversely, when the restraining force is weaker than the driving force (Bozak, 2003; Kaminski, 2011; Sassen, 2009). On the other hand, Sutherland (2013) applied Lewin’s Change Management Theory to study Bar- Coded Medication Administration. The purpose of the model is to identify factors that impede change from occurring. The forces that oppose change often called restraining or ‘static forces’ and forces that promote or drive change, referred to as ‘driving forces’. When health care organisations fully understand what behaviours drive or oppose change, then work to strengthen the positive driving forces, change can occur successfully (Bozak, 2003). 3.6 Application of Force Field Analysis Model in this Study The force field analysis model is considered useful in identifying the various forces that act as enablers and constrains and pushing a change strategy or pulling it back towards a status quo (Salaheldin, 2003). It is necessary to use this model since the purpose of the study is to look at enablers and constraints. Furthermore, it has been applied in some information systems research University of Ghana http://ugspace.ug.edu.gh 41 to study implementation of Information Systems (Bozak, 2003). This study is aimed at understanding why and how higher educational institutions virtualise their course evaluation process, using University of Ghana as a case study. The force field is a useful approach to identifying the various enabling and constraining forces that push against the status quo of virtualising the course evaluation process of University of Ghana. 3.7 Summary This chapter discussed research framework used to achieve the purpose of the study. The chapter discussed the force field analysis model, its various concepts, and its application in other information systems studies. Finally, the chapter discussed the limitations of the model in relation to the study. The chapter then discussed the design methodology that will guide the study in answering the “How” part of the research question. The next chapter describes the methodology for conducting this study. University of Ghana http://ugspace.ug.edu.gh 42 CHAPTER FOUR METHODOLOGY 4.1 Introduction Moving forward from the previous chapter, which delved into the research framework of this study and why the force field analysis model with systems design methodology chosen, this chapter explains the methodological considerations and research design adopted in conducting the study on the virtualisation of course evaluation process. The chapter begins with a discussion about the research philosophy or paradigm used. The study continues with an overview of the design employed, then the population for the study and how the sample was drawn out of the population. Another subsection puts the study in context by describing the area where the study was undertaken, with further subsections addressing the instruments used for data collection and finally discussing the data analysis technique used. 4.2 Overview of Research Methodology The research methodology is a way to systematically solve the research problem (Kothari, 2004). It may be understood as a science of studying how research is done scientifically. Research methodology has many dimensions and research methods do constitute a part of the research methodology. The scope of research methodology is wider than that of research methods. Thus, research methodology is not only about the research methods, but also the logic behind the methods used in the context of the research study and why using a particular method or technique and not using others so that research results are capable of being evaluated either by the researcher himself or by others. Creswell and Clark (2010) stated that, the aim of a research methodology is to design University of Ghana http://ugspace.ug.edu.gh 43 and set out a plan to attain the objectives of a research with activities, which includes data collection and analysis in order to answer the research questions. Research methods may be understood as all those methods or techniques that are used for conducting research (Kothari, 2004). 4.3 Research Paradigm The term paradigm has been used interchangeably with worldview, or broadly conceived research methodologies in literature (Creswell, 2009). A paradigm is the identification of the underlying basis that is used to construct scientific investigation; or, a loose collection of logically held together assumptions, concepts and propositions that orientates thinking and research (Krauss, 2005). It is also defined as a set of beliefs, values and techniques, which are shared by members of a scientific community. This belief acts as a guide or map, dictating the kinds of problems scientists should address and the types of explanations that are acceptable (Kuhn, 1970). In a simpler way, it can be defined as the basic belief system or worldview that guides the investigation (Krauss, 2005). From Creswell’s (2009) point of view, there are four main worldviews. These include the post positivism, the constructivism, the advocacy or participatory and the pragmatism worldviews. Although there are several paradigms, over the years three dominant paradigms have evolved in information systems research. These are the positivist paradigm, interpretive or the constructivist paradigm and the critical paradigm (Mingers, 2004; Myers, 1997). University of Ghana http://ugspace.ug.edu.gh 44 Figure 4.1: Paradigms in Information Systems Research Source: Myers (1997) 4.3.1 The Positivist Paradigm The positivism paradigm has an objective reality, which is single and concrete. The researcher is independent from what is being researched. Researchers instrumentally predict or describe reality, that is, social reality is captured using formal propositions, predictions and control (Lee, 1991). It is also postulated that, positivists presume that reality is objectively given and can be described by measurable properties, which are independent of researchers and the instrument they use (Weber, 2004). Reality in a positivist’s research can be known approximately. Hypothesis can be rejected or provisionally confirmed, but not definitively proved. Reality is unaffected by the research process and facts and values are separate (Ritchie, Lewis, Nicholls & Ormston, 2003). Orlikowski and Baroudi (1991) also propose that information systems research is seen as positivist if there is a formal proposition of evidence. Deductive reasoning is used to postulate possible relationships and models before data is collected and reality is unaffected by the research process. Facts and values are separate also, objective value-free inquiry is possible. Qualitative Research Positivist Paradigm Interpretive Paradigm Critical Paradigm Influences/Guides University of Ghana http://ugspace.ug.edu.gh 45 On the word of Orlikowski and Baroudi (1991), there are two (2) main limitations of a positivist paradigm. These are as follows: the pursuit of universal laws has led to a disregard for historical and contextual concerns that form a phenomenon and human participants who take part in research studies. The positivist paradigm seeks to explain and predict external reality as independent of humans, deterministic approaches and it fails to account for political complexities in the social world. 4.3.2 The Critical Paradigm 1. The Critical paradigm researchers assume that social reality is historically constituted and people reproduce that. McAulay, Doherty and Keval (2002) and McGrath (2005) postulated that, critical researchers recognize the ability of people to change their social and economic situations, but contend that this ability is constrained by various forms of social, cultural and political dominations as well as laws and resources limitations. This is in accordance with the studies (Orlikowski & Baroudi, 1991); Klein & Myers, 1999) indicated that critical paradigm researchers normally want change in the status quo and want to help liberate the less fortunate in society from their peculiar circumstances. The main task of critical research is seen as being one of social critique, whereby the restrictive and alienating conditions of the status quo are brought to light. Critical research focuses on the oppositions, conflicts and contradictions in contemporary society, and seeks to be emancipatory (Myers, 1997). Using the assumptions of a paradigm, concerning the methodological assumptions of the critical paradigm, critical paradigm researchers employ the use of ethnographic University of Ghana http://ugspace.ug.edu.gh 46 study to analyse and compare events, both past and present to enable them to identify the influencing forces between the events (Orlikowski & Baroudi, 1999). 4.3.3 The Interpretive Paradigm Interpretive researchers assume the existence of reality or the real world, however knowledge of this reality is subjective and not objective (Weber, 2004). This implies that in interpretivist, objective knowledge is non-existent. According to Myers (1997), interpretivist assumes it is only through social constructions such as conscious language and meanings that are shared that reality can be accessed. Kaplan and Maxwell (1994) noted that positivists focus on the complexity of making sense as situations emerge, thus dependent and independent variables in the study are not defined. Information systems researchers who use the interpretive paradigm focus on understanding the context and how information systems impact and are impacted in the context (Walsham, 2006). Immanuel Kant, in 1781 published the Critique of Pure Reason that there are ways of knowing about the world other than direct observation and that people use these all the time. He proposed that perception relates not only to the senses, but to human interpretations of what the senses tell us. As such, knowledge of the world is based on ‘understanding’, which arises from reflecting on what happens, not just from having had particular experiences. Knowledge is produced by exploring and understanding the social world of the people being studied, focusing on their meanings and interpretations. University of Ghana http://ugspace.ug.edu.gh 47 4.3.4 The Choice of Paradigm for the Study This study employs the use of interpretive paradigm in order to achieve the purpose of this research which is to understand the why and how higher educational institutions pursue virtualisation of course evaluation process. Unlike the critical paradigm and the positivist, the interpretive paradigms goes beyond the observable actions of people in the context of social phenomena and understand the subjective meanings they assign to their actions and thereby interpret and understand the reasons behind those actions (Klein & Myers, 1999). Walsham (1995) indicated that interpretive research in information systems is aimed at researching into phenomenon with the notion to understand the reciprocal relationship between the phenomenon and its context. Drawing from Walsham’s (1995) definition of what interpretive research in information systems does, the phenomenon under research in this study is virtualisation of course evaluation process and the context of the phenomenon is Higher Education Institutions in developing countries’ environment using University of Ghana as a case study. Hence, the relationship between the phenomenon and its context is what this study seeks to understand. 4.2.4 The Philosophical Assumptions In order to understand the differences between these paradigms, a set of dimensions known as the Philosophical Assumptions is used, namely, Ontology, Epistemology, and methodology (Lincoln, Lynham & Guba, 2011). The ontological dimension of a research paradigm looks at the nature of reality and what is there and know about the world (Ritchie, Lewis, Nicholls & Ormston, 2003). It poses the question what University of Ghana http://ugspace.ug.edu.gh 48 is the nature and form of reality. Quantitative researchers embracing the idea of a single reality which is objective whiles qualitative researchers embrace the idea of multiple realities and thus subjective. The second dimension which is epistemology is concerned with the ways of knowing and learning about the world and focuses on issues such as how we can learn about reality and what forms the basis of knowledge (Ritchie, Lewis, Nicholls & Ormston, 2003). They indicated that, epistemology relates to how best knowledge is acquired, whether through an inductive or deductive logic. Inductive logic involves building knowledge from the bottom up through observations of the world, which in turn provides the basis for developing theories or laws and deductive logic is a top-down approach of knowledge. It starts with a theory from which hypotheses are derived and applied to observations about the world. Qualitative researchers posit the researcher is not independent from what is being researched whiles quantitative researchers posit that the researcher is independent from what is being researched. Finally, the methodological dimension of a research paradigm is concerned with the methods involved in data collection and analysis for drawing a valid conclusion during a research project, for example, quantitative, qualitative or mixed (Lincoln et al., 2011). It poses the question what is the process of research (Creswell, 2007). University of Ghana http://ugspace.ug.edu.gh 49 Table 4.1: Philosophical Assumptions underpinning Information Systems Research Philosophical assumptions Interpretive Paradigm Application of interpretive paradigm in the study Ontology Reality is multiple and value laden. Reality is socially constructed The study acknowledges the context as valuable and it shapes the findings of this study Epistemology It assumes that we cannot separate ourselves from what we know. The author of this thesis and the virtualisation course evaluations process are linked such that who I am and how I understand the world is a central part of how I understand others, the world and myself. methodology It relies heavily on naturalistic methods and involves the use of qualitative study. This study adopted both qualitative method and utilize naturalistic methods like interviewing and observation and analysis of existing to study the virtualisation of Course evaluation process in Higher Education Institutions. 4.4 Research Design and Method A research design is the arrangement of conditions for the collection and analysis of data in a manner that aims to combine relevance to the research purpose with economy in procedure (Kothari, 2004). In addition, a research design is viewed as the pattern for collecting, unionizing, and integrating a research data for unearthing research findings (Johnson & Onwuegbuzie, 2004). The research design serves as the conceptual structure within which research is conducted; it constitutes the blueprint for the collection, measurement and analysis of data. There is a need for research design because it facilitates the smooth sailing of the various research operations, thereby University of Ghana http://ugspace.ug.edu.gh 50 making research as efficient as possible, yielding maximal information with minimal expenditure of effort, time and money. Also, research design stands for advance planning of the methods to be adopted for collecting the relevant data and the techniques to be used in their analysis, keeping in view the objective of the research and the availability of staff, time and money. Research method is a way to systematically solve the research problem. It may be understood as a science of studying how research is done scientifically. There are several classifications of research methods, but the most common distinction is between the Qualitative and the Quantitative Research Methods (Myers, 1997; Creswell, 2007; Johnson & Onwuegbuzie, 2004). 4.4.1 The Qualitative Research Method Qualitative research method was originally developed in the social sciences to enable researchers to study and understand cultural phenomena (Myers, 1997). It seeks to develop explanations of social phenomena by helping the researcher understand the world that we live in and why things are the way they are. Denzin and Lincoln (1994) indicated that qualitative research focuses on the interpretation of phenomena in their natural settings and also making sense in terms of the meanings people bring to these settings. Creswell (2009) defined qualitative research as a means for exploring and understanding the meaning individuals or groups ascribe to a social or a human problem. Qualitative research method used in information systems research includes, case study; phenomenology; narrative; grounded theory and ethnography (Creswell, 2007). The qualitative research method involves the use of qualitative data, such as interviews, documents and observation, in order to understand and explain the social phenomenon. According to Mark et al. (2007), qualitative data refer to all non-numeric data or data that have not been quantified and can University of Ghana http://ugspace.ug.edu.gh 51 be a product of all research strategies. To be useful, these data need to be analysed and the meanings understood. 4.4.2 The Quantitative Research Method This method was originally developed in the natural sciences to study natural phenomena (Myers, 1997). Quantitative research method explains phenomena by collecting numerical data that are analysed using mathematically based methods (Aliaga & Gundderson, 2003). It is a means for testing objective theories by examining the relationship between variables, which in turn can be measured typically on instruments, so that numbered data can be analysed using statistical procedures (Creswell, 2009). 4.4.3 The Choice Method for the Study This study employs the use of a qualitative research method for the purpose is to gain a complex, detailed understanding not only what but the why of a phenomenon (Creswell, 2007). It also seeks to know their perspectives and how those influences what is happening. The method of choice is much more appropriate for the study since it can help examine the causal processes and not just simply state what the causal relationships exist. Out of the five qualitative approaches to inquiry which are the grounded theory research, narrative research, ethnographic research, phenomenology research and case study research as proposed by Creswell (2007); this study employs the use of case study. University of Ghana http://ugspace.ug.edu.gh 52 4.4.4 Choice of Qualitative Interpretive Case Study Research Method As earlier indicated the qualitative research method used in information systems research includes case study; phenomenology; narrative; grounded theory and ethnography (Creswell, 2007). This study employs the use of an interpretive case study, which is becoming an essential and dominant research method in the information system knowledge base (Bygstad & Munkvold, 2011). The study uses a single, holistic case study design with embedded units which enable the researcher to understand the case well (Baxter & Jack, 2008). Case study is a strategy for doing research which involves an empirical investigation of a particular contemporary phenomenon within its real life context multiple sources of evidence (Saunders, Lewis & Thornhill, 2009). Case study is “an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident” (Andrade, 2009). Andrade (2009) further indicates that, instead of case study design seeking answers to questions such as “how much” or “how many,” it is useful for answering “how” and “why” questions. This is in concession with Walsham’s (2006) study on interpretive research. Hence, it is plausible to use case study design in researching the virtualisation of course evaluation systems, since it is a type of an information system. The study uses an interpretive case study because interpretivist focuses on close interaction between researcher and participants throughout the case study process, viewing the case members as active participants in the construction of the case narrative and ascribes an active role to the case study informants. In practice, the extent of this involvement is normally confined to the data University of Ghana http://ugspace.ug.edu.gh 53 collection process and discussion of early versions of the case narrative. Case study method enables a researcher to closely examine the data within a specific context by selecting a small geographical area or a very limited number of individuals as the subjects of study (Zaidah, 2007). Zaidah (2007) further indicated that case studies in their true essence explore and investigate contemporary real-life phenomena through detailed contextual analysis of a limited number of events or conditions, and their relationships. There are prominent attributes of this study that necessitate the need to employ an interpretive case study design for this research. These attributes which are the research purpose and questions are such that; the phenomenon under study is new, especially in Ghana, it employs the use of a model, that is the force field analysis model as an initial guide to design and data collection for this research (Walsham, 1995). Also, the study cannot be studied outside the institution but rather within to gain much understanding on the subject. Since case study design is best suited for this study, it is important to appreciate the criticism on this design. One key criticism of case study is the generalisability of its findings (Walsham, 1995; Myers, 1997; Walsham, 2006; Zaidah, 2007). According to Walsham (1995) generalisation from interpretive case studies is defined as explanations of particular phenomena derived from empirical interpretive research in specific information system settings, which may be valuable in the future in other organisations and contexts. Also, case studies are often accused of lacking rigor. Yin (1984) notes that “too many times, the case study investigator has been sloppy, and has allowed equivocal evidence or biased views to influence the direction of the findings and conclusions”. While Byrne and Sahay (2007) posit four types of generalisation (empirical to empirical; empirical to theoretical; theoretical to empirical; and theoretical to theoretical). Walsham (1995) also outlines four types of generalisation. The types of generalisation according to Walsham (1995) University of Ghana http://ugspace.ug.edu.gh 54 are the development of concepts; the drawing of specific implications; contributions of rich insight; and the generation of theory. Thus, in addressing the problem of lack of statistical generalisation of case study method, the generalisation of this study should be based on the theoretical generalisation of drawing specific implications, contribution of insight that the study provides and contribution to the theory (Walsham, 1995). 4.5 Selection of Case and Fieldwork Fieldwork for the study occurred in three phases with the first between December 2014 and January 2015; the second in March 2015; and the third between March and May 2014. The first phase involved the initial selection of cases and data collection. After producing findings from the analysis of collected data, the author went back to the field in March and June 2015 for the second phase to seek clarification of emergent themes and concepts. University of Ghana was selected for the study. The study utilized the purposeful sampling method as guided by Creswell (1998) for the selection of this institution. The following were the considerations for the selection of this institution. The selection process began with the review of the higher education institutions in the country to ensure that at least they have a web presence. After an informal research and using a purposeful sampling technique it was noted that the University of Ghana is the only institution in Ghana that conduct the evaluation of courses online and has been doing that for five years now hence the study deemed it fit to select the University of Ghana as a case study. Formal letter from the researcher’s department in the University of Ghana signed by the Head of Department introducing the researcher as a student researcher was sent to the managers in charge of Academic Quality Assurance Office. This enabled University of Ghana http://ugspace.ug.edu.gh 55 the researcher to gain access to the office after he explained the purpose of the research to the managers who became interested in the study. 4.6 Data Collection Method The process of collecting useful data and information to enable answering the research questions of the study is described as data collection (Creswell & Clark, 2010). This section discusses the data sources and techniques the study used to gather data. There are two main sources of data; these are primary and secondary sources (Boateng, 2014; Hussey & Hussey, 1997). The primary source of data was obtained through face-to-face interviews and observation whiles the secondary source of data for this study included documentation, archival records and physical artefacts (Yin, 1994). Based on the tenets of interpretive research and case study research, this study employs evidence from multiple sources to support the research findings; this strategy is known as triangulation. Triangulation refers to the use of different data collection techniques within one study in order to ensure that the data is telling you what you think they are telling you, this is in accordance with interpretive research (Benbasat et al., 1987; Walsham, 2006; Saunders, Lewis Philip & Thornhill, 2009). 4.6.1 Interviews Saunders et al. (2009) postulated that interviews may be categorized based on the level of formality and structure of the interview, hence there are; structured interviews, semi-structured interviews and unstructured or in-depth interviews. Structured interviews use questionnaires based on a predetermined and ‘standardized’ or identical set of questions and we refer to them as interviewer- administered questionnaires with semi-structured interviews, the researcher will have a list of University of Ghana http://ugspace.ug.edu.gh 56 themes and questions to be covered. Finally, unstructured interviews are informal interviews used to explore in depth a general area in which you are interested hence, it is sometimes referred to as ‘in-depth interviews’ (Saunders et al., 2009). This study employed the use of semi - structured interviews. The stakeholders for the study are the Academic Quality Assurance Unit; University of Ghana Computing Systems; Students and Lecturers. These Stakeholders were purposively sampled. At Academic Quality Assurance, the Director, the Assistant Registrar and the Research Assistant participated in interview sessions providing insight into the Academic Quality Assurance core activities and projects. Also at the University of Ghana Computing Systems, the webmaster who is also the senior programmer in the unit and in charge of designing and building the online platform for the evaluation of courses and lecturers were interviewed. Furthermore, a total number of forty (40) students were interviewed. Out of the forty (40) participants, twenty were graduate students from the University of Ghana Business School, the Linguistics Department, information Studies Department and the Computer Engineering School. The remaining twenty (20) were undergraduate students from both the main and city campuses of the University of Ghana. Finally, a total number of ten (10) lecturers from the School of Languages, Information Studies and the University of Ghana Business School were also interviewed. The duration of the interview was between forty-five minutes to an hour. Upon permission from respondents, the researcher used a voice-recording device to capture all responses, whilst making notes on paper. The paper notes served as cues for follow-up questions not in the interview guide. University of Ghana http://ugspace.ug.edu.gh 57 The author scheduled and conducted a final validation interview with the senior managers after three months. Participants were assured of the privacy of the information they provided. The table below presents a summarized process in which the interview was conducted at the University of Ghana. Table 4.2: Summary of Interviews Conducted Case Units Participants Department Number of Interviews Academic Quality Assurance Unit (AQAU) The Director, Assistant Registrar; Research Assistant Number of participants: 3 Academic Quality Assurance Unit (AQAU) 5 University of Ghana Computing Systems (UGCS) Webmaster and Programmer Number of participants: 1 UGCS ADMIN 3 Students Graduate Students Undergraduate Students Number of participants: 40 Business School Linguistics Information studies Department Computer Engineering 40 Lecturers Lecturers Number of participants: 10 School of Languages Information Studies University of Ghana Business School 10 A semi structured interview strategy was used because it afforded the opportunity to access first- hand information from respondents who were directly involved in the conduct and management University of Ghana http://ugspace.ug.edu.gh 58 of Academic Quality Assurance of University of Ghana. The researcher asked respondents both open-ended and closed-ended questions from a prepared, Interview guide, which was to help tease out some general information about the course evaluation process. 4.6.2 Documentation and Archival Records This method was used because it provided physical evidence of the former paper-based course evaluation used in the university before there was a change. The evidence examined included written material which are formal reports. Some of these reports are University of Ghana Special Reporter (2011) and University of Ghana (2009). These are annual reports on the state of affairs of the institutions. Other documentations affiliated with the business process of AQAU were also used. 4.6.3 Physical Artefacts The study performed artefact examination, that is system testing. The researcher used the online course evaluation system, this was useful in understanding how the system works, and to help verify and better describe the system. Testing was also useful in determining the veracity of what respondents claimed about the course evaluation functionality. 4.7 Data Analysis Data analysis according to Miles and Huberman (1994), is tracing out lawful and stable relationships among social phenomena, based on the regularities and sequences that link these phenomena. This definition is not far from Green et al.’s (2007) which postulates that data analysis University of Ghana http://ugspace.ug.edu.gh 59 is a systematic and essentially a taxonomic process of sorting and classifying the data that has been collected. The researcher, guided by both the research purpose and questions as outlined in Chapter one as well as the Lewin’s force field analysis model, began the analyses of the first research question by reading through all data gathered from all sources several times to identify emergent concepts and themes. As emergent themes and concepts were identified, the researcher went back to the field to gather more data on them. This is in line with interpretive studies where data can be collected and analysed simultaneously and iteratively (Baxter & Jack, 2008). Furthermore, emergent findings of data analysis can influence subsequent data collection and help shape successive data analysis (Walsham, 1995; Orlikowski & Baroudi, 1991). This influenced the researcher’s approach to data collection and analysis where he collected and analysed data simultaneously and iteratively. The second and third research questions were analysed by identifying emerging forces in the data gathered by using the force field analysis model 4.8 Ethical Considerations A number of measures were employed to address ethical issues in conducting this study. This was necessary because interpretive case studies according to Walsham (2006) requires the researcher to be personally involved with the other humans and organisational participants in the research process. The personal involvement of the researcher normally raises ethical concerns and problems, which needs to be addressed. A number of measures were adopted to combat this issue. One of these measures were, before going to the field, the researcher applied for an introductory letter signed by the Head of Department of Operations and Management Information Systems University of Ghana http://ugspace.ug.edu.gh 60 detailing the purpose of the research, the phenomenon of study and the roles and responsibilities of the researcher and the participants. The second document, which is the interview guide, contains an ethical consideration section stating that, the interview is subject to the willingness of the participant to participate and the participant had the liberty to discontinue the interview at any time. Subject to the participant’s consent, the interview was recorded and interviewees were assured that the data collected was for research purposes only. Moreover, Walsham (2006) suggests three sources of ethical dilemmas for information systems interpretive studies as confidentiality and anonymity, working within an organisation, and reporting in the literature. Beyond the information sheet and the consent form, which in this researcher’s case were the introductory letter and the interview guide containing the ethical consideration, Walsham's recommendations were followed to address ethical issues in the study. First, all proper names of people and organisations were anonymized in this thesis. Furthermore, extracts and images captured from the fieldwork were also carefully edited to mask real and suggestive identities of human and organisation participants. In addition, all of the data gathered are properly secured to restrict unauthorised access. The purpose of the study was clearly noted on the participant information sheet. Aside from this, the researcher verbally discussed the purpose with participants before seeking their consent. University of Ghana http://ugspace.ug.edu.gh 61 4.9 Summary This chapter explained the methodological framework that underpins the study and outlined the research method used to answer the research questions posed at the beginning of the study. The various paradigms as well as research methods were discussed. There were also a discussion and justification of why qualitative and interpretative case study was chosen. The chapter also discussed the process of case selection and fieldwork for the study, data collection and analysis. University of Ghana http://ugspace.ug.edu.gh 62 CHAPTER FIVE RESEARCH FINDINGS 5.1 Introduction The previous chapter discussed the methodological issues in this study. It focused on the interpretive paradigm as a fitting research paradigm. It also discussed associated principles of the paradigm, research design, data collection and analysis methods. As pointed out earlier in chapter one (1), the study is about virtualisation of course evaluation in higher education institutions using the University of Ghana as a case study. This chapter provides a brief background of the chosen case, University of Ghana in order to set a foundation for the analysis of the data collected. The chapter concludes by providing a summary of the findings and introducing the next chapter, which is the analysis of the case findings. 5.2 Case Description of University of Ghana University of Ghana was founded in August 1948 as the University College of the Gold Coast on the recommendation of the Asquith Commission on Higher Education in the then British colonies. The Asquith Commission, which was set up in 1943 to examine higher education, recommended establishing university colleges that will be affiliated to the University of London. This system of affiliation meant that, University of London was to superintend over running academic programs and awarding of degrees for the then University College of the Gold Coast (University of Ghana, 2015). The University College of the Gold Coast was founded to provide and promote quality learning and research within the context of a university education. It gained full university status in 1961 University of Ghana http://ugspace.ug.edu.gh 63 and is now one of the seasoned and the largest institutions of higher learning in Ghana. The student population as at September 2014 was thirty-five thousand, six hundred and eighty-three (35,683) which represents a male-female ratio of about 3:2. The breakdown of the population by programs are as follows: there are two thousand five hundred and four (2,504) postgraduate students, with thirty-four thousand three hundred and four (34,304) degree students and five thousand eight hundred and thirty-four (5,304) sub-degrees’ students. The foreign students now stand at one thousand four hundred and nine (1,409). It must be noted that students of the University of Ghana are variously studying at different locations and for that matter at different campuses. These campuses include the Accra City Campus, Korle-Bu Campus and various University of Ghana Learning Centres across the regions. Senior Members engaged in research and teaching number around nine hundred and fifty-one (951) (University of Ghana, 2014). The University aims to “develop a world-class human resource and capabilities to meet national development needs and global challenges through quality teaching, learning, research and knowledge dissemination” (University of Ghana, 2006). The formation of the Academic Quality Assurance Unit (AQAU) was intended to foster excellence and ensure the continuing quality of academic programs offered by the University of Ghana. Prior to setting up of the Academic Quality Assurance Unit (AQAU), the College of Health Sciences of the University had its own Quality Assurance Unit. The Unit managed the standards of academic work in that college. The Assistant Registrar indicated that, The unit was established in May 2005 by the then Vice -Chancellor in line with the university’s objective of positioning itself as a world-class centre of higher University of Ghana http://ugspace.ug.edu.gh 64 learning. The current staff includes the Director, Principal Administrative Assistant, three National Service Personnel and I as the first Assistant Registrar. To guide the activities of the academic quality assurance unit, the Academic Quality Assurance Policy (AQAP) was formed. This, of course, demonstrates that the University of Ghana’s responsibility for awarding its own degrees is being satisfactorily discharged. The aim of the policy is to satisfy the internal and external stakeholders. To ensure that, the whole range of resources, across academic and support areas is of the highest possible quality. In addition, the policy aims to ensure that institutional commitment is not only for quality assurance, but also to the enhancement of the quality of students’ experiences. It is also to provide accessible and adequate infrastructure for dealing with quality assurance at all levels. Finally, to disseminating good practice, including ensuring that all staff are familiar with the University’s quality assurance procedures and mechanisms (Academic Quality Assurance Unit, 2014). The Academic Quality Assurance Unit in achieving its aims and mission is tasked with several responsibilities. This includes offering advice on academic standard, conducting departmental reviews, facilitating quality audits & staff development, performing graduate tracer surveys. These graduate tracer surveys involve organising annual exit surveys of graduating classes, periodically undertaking tracer and employer surveys and, finally conducting students’ evaluation of courses and lecturers. Their responsibility, which is of relevance to the study is student evaluation of courses and lecturers. The policy outlines five main methods through which students can use to evaluate University of Ghana http://ugspace.ug.edu.gh 65 courses and lecturers. This includes the paper questionnaire, electronic questionnaire, staff-student liaison committees, informal feedback, and open meetings with the student body (University of Ghana Special Reporter, 2011). Table 5.1: Methods of Course evaluation in University of Ghana Method of Evaluation Description Paper questionnaire This is a sheet of paper with both Likert scaled and open -ended questions that is normally brought to the lecture rooms during the evaluation period at the end of the semester for students to fill. Electronic questionnaire These are survey questions in a system which is either a web-based system or stand -alone system that can be answered by the student. Staff-student liaison committees This Committee is normally a forum for consultation and reporting between the staff and students of each individual college. Informal feedback These are usually ongoing, in-the moment advice outside of the formal performance review. The paper questionnaire is the traditional-paper based course evaluation system, which are brought to the lecture rooms during the evaluation period at the end of the semester for students to fill. It is a single sheet of paper with both Likert scaled and open-ended questions. In addition, there is the electronic questionnaire, which contains questions in a software system, which is either a web- based system or stand-alone system that can be answered by an individual. An example is the University of Ghana http://ugspace.ug.edu.gh 66 online course evaluation system. The format of the questions are not changed, they contain both Likert scaled and open-ended questions. Another method of evaluation is the staff-student liaison committees; this operates at the Faculty or departmental level. This committee is a forum for consultation and reporting between the staff and students of each individual School. These committees play an important role in the dissemination of information to students, and are an important element in the quality assurance procedures. The committee comprises of several members of staff and students. Finally, there is the informal feedback that is ongoing, in-the moment advice aside the formal performance review. For instance, as and when the lecturer is teaching, if the students do not like the style of teaching they can prompt the lecturer there and then. They do not have to wait until evaluation period before they voice out their grievances (University of Ghana Special Reporter, 2011). Until it was changed to the online Course evaluation system, the most dominant forms of evaluation used in the University of Ghana were the informal feedbacks and the Paper Questionnaire. 5.3 The physical process of Course Evaluation Traditionally, University of Ghana like most Ghanaian universities has used paper-based evaluation method to collect student feedback on both lecturers and courses taught. Before 2013, University of Ghana had used the paper-based course evaluation systems for students to evaluate the courses and lecturers. The course evaluation was conducted at the end of each semester. Portions of the regular lecture session are devoted to the distribution, completion and collection of the paper-based questionnaires. The sample for the evaluation depends on the class attendance for University of Ghana http://ugspace.ug.edu.gh 67 that single session. Students who are not in class during the session miss the opportunity to evaluate. The Assistant Registrar of the Academic Quality Assurance Unit ascertained this he indicated that; “The Paper-Based questionnaires were usually brought to the lecture room, usually during the last lecture of every course, and a portion of the lecture period is dedicated to the filling of the questionnaires. The questionnaires are normally distributed to students by the teaching assistants of the lecturer and students are given a short period of time to fill and submit the questionnaires”. These printed questionnaires are typically a mixture of scaled, closed-ended and open-ended questions. The responses to closed-ended questions can be quickly recorded using widely available scanning technology called the Photo scribe PS900. This set of scaled responses would then be processed as quantitative feedback. The open-ended questions, such as “Please comment on this course” and the semi-open-ended questions, such as “Any comments or suggestions for future improvement?” Would elicit written comments from the students. The Assistant Registrar further indicated, “At times, because of the comments recorded in parts of the questionnaire, we hire the services of extra staff to type the comments and others to scan the forms. This tends to delay the evaluation process, and always has backlogs of unprocessed data from the previous semester”. These comments would require manual transcription to provide readability for this handwritten material and, more importantly, to provide anonymity for the student respondents. This set of University of Ghana http://ugspace.ug.edu.gh 68 comments would then be assembled as qualitative feedback. The Academic Quality Assurance Unit hires the services of extra personnel to manually type these comments. The processing of this qualitative feedback, requiring transcription, is one of the major disadvantages of this method, because it is expensive, time-consuming and usually prone to error. Analysis is conducted on the collected data to determine the performance of the lecturer and how the course was delivered. From the data collected to be valid, an average response rate of at least 60% is required. Where feedback is low, departments give consideration to ways in which the response rate might be improved. Departments monitor the response rate and consider it when evaluating the feedback and developing the action plan. At the end of every evaluation process, a summary of the analysed data and a copy of the unedited subjective/written portion are sent to the lecturer concerned. Copies of the same materials are sent to the lecturer’s Head of Department and Dean. The Director of the Academic Quality Assurance Unit is required to send comments on the analysed data to the Dean concerned. Everything is done under strict confidentiality. Heads of the departments are required to discuss their evaluation report with the teaching staff. Deans are required to talk to teaching staffs whose teaching is found to be below expectation. The Pro-Vice Chancellor (Academic) would serve as discussant in cases where Deans have a teaching load. The University will ask the various departments to comment on the feedback from students, how it is gathered and the usefulness of it to the development and operations of programs through an Annual Program Monitoring Process. The results of evaluations are made available to students along with any comments a lecturer may wish to make in response. Feedback on evaluations is communicated to students and to the lecturer as soon as possible to promote a process of continual improvement. For several years the same questionnaire was used University of Ghana http://ugspace.ug.edu.gh 69 to evaluate courses and the questions, the Unit did not have the flexibility of changing the questions. The Questionnaire captured the following; questions about the course, attendance, delivery, feedback/ assignments, interactions with students and additional comments from the students. 5.4 Decision for Online Course Evaluation System The Academic Quality Assurance policy of the University of Ghana indicates that each course should be reviewed at least once every other year. Hence, the faculty and the departments seek feedback on individual courses as well as lecturers within programs of study for students at the end of each semester or academic year. The policy indicates that both qualitative and quantitative responses are vital for evaluation. The paper-based questionnaire that is a scannable form has both Likert scale questions to give quantitative responses and open-ended question section to elicit qualitative responses. At the end of every semester, these paper-based questionnaires were usually brought to the classroom during the last lecture of every course, where a session of the lecture period is dedicated to the filling of the questionnaires. The questionnaires are normally distributed to students by the teaching assistants of the lecturer and students are given a short period to fill and submit the questionnaires. It was ascertained that the paper-based method of evaluation was somewhat flawed, as the top management of the academic quality assurance unit responsible for overseeing the printing and distribution of these questionnaires indicated the cost of using the paper-based questionnaire as very high. The cost included but not limited to the cost of printing the questionnaire. This is because the printing is not done in-house. It is usually outsourced to DRS Data Services Limited. University of Ghana http://ugspace.ug.edu.gh 70 Most of the time, the printing is delayed and this delays the whole course evaluation process. There are occasions when most courses are not evaluated because of the non-availability of forms. The Assistant Registrar indicated that; “The Cost involve in printing the paper based forms was so high, take one student offering six courses, the person is supposed to fill at least five forms. Take this number and multiply with the total population of the University, which is around thirty-eight thousand (38,000). So each Semester we end up printing approximately one hundred and ninety thousand (190,000) forms every semester, and there are occasions they run out of forms” Also, after they have been printed they are delivered to the Office of the Academic Quality Assurance Unit located in the Jones Quartey building. The cost of distributing them to each faculty throughout the main campus and the city campus was so exorbitant. Hence, the unit devised a plan where the various faculties were called upon to pick up their evaluation forms from their main office when it was ready. However, it was noted that most faculties delayed in going for their forms, resulting in the delays of the evaluation process. In one instance, the faculty received a memo for distribution of the forms a week late since the letter was just dropped in the faculty’s internal mailbox. “Unfortunately, we did not go to the internal mail that week till the following week. Hence, we were a week late. The administrative secretary and I had to rush to the unit’s office only to find out that there were only two thousand forms left for the faculty to serve eight thousand students in the faculty.” This created a lot of inconvenience for both the unit and the faculty, since most of the lecturers in the faculty started asking for the evaluation questionnaire. Furthermore, there was always a huge cost associated with collecting, scanning and storing the University of Ghana http://ugspace.ug.edu.gh 71 paper evaluation forms. Furthermore, the costs of typing students’ responses to open-ended questions and the costs of delivering hardcopy summary reports to faculty were so high. The Assistant Registrar of the Academic Quality Assurance Unit ascertained this. He declared; “Take the case of a student offering six courses. The student is supposed to fill at least five forms. Take this number and multiply it by the total population of the University, which is around thirty-eight-thousand (38,000). Simply put, the unit ends up printing approximately one hundred and ninety thousand (190,000) forms every semester.” Finally, there was high administrative cost in processing this large number of questionnaires. Since they were paper-based, they had to be collected and scanned manually using the Photo Scribe Series PS900 Machine. The Photo Scribe Series PS900 is an image-capturing device that is used to scan or capture data, and conduct validation and key correction of the scanned data. It then exports the scanned and key corrected data into an ASCII or Text format, which is suitable for electronic processing. Moreover, since it was just one machine being used, it took an unusually long time to scan all the questionnaires. On some occasions, the scanner could break down several times and it cost a lot to repair it. This caused various inconveniences and undue delays. Furthermore, there was the pressing issue of time. It always took a long time for all the questionnaires to be scanned, and this led to the late provision of the results. Most of the time, there were backlogs of the un-scanned questionnaires from the previous semester into new semesters. The photo scribe PS 900 scanner could capture only the Likert scale questions. For the University of Ghana http://ugspace.ug.edu.gh 72 open-ended questions, other staff had to be employed overtime to type all the responses, most of the time there were some typographical errors, and the processing time for the evaluation was elongated. On several occasions, deadlines for scanning a semester’s questionnaire were not met. Around this same time, another set of questionnaires would be around to be processed. Hence, there was always a problem of having backlog questionnaires from the previous semester to scan. Moving on, it was revealed that there were many missed opportunities. Students who could not be in a class missed the opportunity to evaluate both lecturer and course. Also, the evaluation for a course is done only once in a semester. This makes the system highly vulnerable to flaws and results in the non-capture of diverse students’ perspective. Both lecturers and students believe that paper-based evaluation prevent absentee students from taking advantage of the platform to present their grievances about courses and the lecturers who handle them. The lecturers agreed that, those missed opportunities could greatly affect them, since they will not know the student grievances and make the appropriate or necessary changes. Also, there is the issue of environmental forces. With the University aiming to be a world-class university, it will be feasible to throw away the traditional way of doing things and adopt contemporary methods, that is more resource efficient. This was ascertained by the Assistant Registrar as he indicated that; “Looking at all the big universities outside, most of them are adopting the online system. So it will be plausible for the University of Ghana to adopt such a system if the University of Ghana really wants to compete with the “outside universities”” University of Ghana http://ugspace.ug.edu.gh 73 Finally, apart from these missed opportunities on the part of the students, there was also an issue of the difficulty in modifying the existing questions for the evaluation. For instance, an interview with some of the faculty members indicated that the evaluation questionnaire did not capture innovation. By so doing, it becomes difficult to assess faculty members on innovative ways of teaching. The paper-based questionnaire is such that it is difficult to be modified. The cost of printing the forms are very high. This made the evaluation questions difficult to be modified. In fact, the same questionnaire has been used for five years now. Even, though it was rare, some students indicated that the paper-based evaluation process is susceptible to faculty influence. This is because in a typical paper-based evaluation process, the faculty member might perform on the day of the evaluations some activity that is designed to elicit a favourable response from students. In some cases, just the presence of the faculty member before or during the evaluation process may either influence or intimidate the students, especially if the student fears that the faculty member may have some way of identifying the student’s response. In view of the above problems, in 2011 there were recommendations from some Heads of department to virtualise the paper-based course evaluation system. The Pioneer of this was the Head of Department of Operation and Management Information Systems. In 2012, decision was made to develop the online course evaluation system. Since the University of Ghana has an internal computing unit, the Assistant Registrar of the Academic Quality Assurance Unit sought the assistance of the webmaster of the University of Ghana Computing Systems (UGCS). The online system was developed in-house by the webmaster and his Unit comprising of a database programmer and a graphic designer. It took four (4) weeks to develop the online course evaluation system. University of Ghana http://ugspace.ug.edu.gh 74 5.5 The Development of Online Course Evaluation System The online course evaluation system was developed “from scratch” by in-house development team from the University of Ghana Computing Systems (UGCS). The major stakeholders in the development of the system where the development team, which comprised of the Webmaster of the UGCS, and a graphic designer and two database administrators. As indicated by the Webmaster in the UGCS, ‘we did not follow the system development lifecycle process through out, it was more of a rapid development process’. As a result, there are no deliverable documentations rather, there was much of oral deliberation with the two main stakeholders. The planning phase started on April, 2011 with the awareness for the need for a new course evaluation System, which originated from the need to provide a solution to the problems the Paper- Based Course Evaluation System was posing. This process was initiated by the Director in Office of the AQAU. The active stakeholders who partook in the problem definition phase for developing the online course evaluation system were the top management of the Academic Quality Assurance Unit spearheaded by the Director of Academic Quality Assurance Unit and the Assistant Registrar. The second stakeholder was the webmaster of the University of Ghana Computing Systems (UGCS). The first meeting was held, and the director briefed the webmaster of what was expected. The webmaster agreed and put together his team to start work on developing the system. The stakeholders did not conduct a thorough investigation of the current system and the requirements and specifications for the new system. It was having discussions with the major stakeholders. Hence, there was no formal analysis of the system with the Academic Quality University of Ghana http://ugspace.ug.edu.gh 75 Assurance Unit, most of the developmental issues and decisions were left to the discretion of the development team. In June 2011, another meeting was held between the Director, Assistant Registrar and Research Assistant of AQAU and the Webmaster. The webmaster, displayed the progress of development, he showed a demo of the proposed interface of the online course evaluation system designed with the help of a graphic designer. Since, the Assistant Registrar made recommendations. Some of these recommendations were the type of font style, font size and the colour scheme to use in order to make the Graphic User Interface user friendly, further communications were held through the use of email, the webmaster indicated that ‘I update the Director of the AQAU through email correspondence on the progress of work.’ In July 2011, a database for the system was created, the webmaster indicated that, the database of the system was developed using MySQL. Since the contents was the information on the paper- based course evaluation system was converted to fields. This made it possible for each entry in the graphic user interface to be placed in its appropriate field. The fields were coded in a way that could easily be extracted and analysed using any standard statistical software. The third meeting was held at the end of July 2011, to display a demo of progress of work. The AQAU was very pleased with the work done. The Unit indicated that they wanted the system to have a longer uptime than its down time. As a university-wide system, the system was supposed to have the capacity to handle a log on of at least three thousand (3000) students per session without breaking down. Also, the system was expected to be designed in a way that, the questions for the evaluations have been already coded. Therefore, as soon as the data is exported it will be ready for analysis. There would University of Ghana http://ugspace.ug.edu.gh 76 be no need for coding and data entry or data scanning. Furthermore, concerning the social requirement of the system, it was expected to be designed in a way that ensures the anonymity of students. This would singularly eliminate the fear and possibility of students becoming victims because of their comments or responses. After, that meeting the project was put on hold for some time since the Director was changed. In the 2012/ 2013 academic semester, the new director called for the completion of the project. The Webmaster integrated the designed GUI and the database and made his database administrators to use an open source software testing system known as the Requirements and Test Management Repository (R.T.M.R) to test the system. The system was implemented in three phases and in each phase the system was rolled out for use, to determine the shortfalls in order to correct it. In the first phase, which is the version 1.0, the system was developed in a way that students had to type their index number in order to gain access to the system to do the evaluation. But the index number did not have any correlation with the information stored in the database. It was realised that this deterred student from evaluating. Also, the interface was not user friendly. The questionnaire covered three sections and each section covered a page. Therefore, after filling the next page, a student clicks on “next” and he or she is moved to the next page, which contains the other sections. It was realised that there were many incomplete questionnaires, as students, mostly left halfway through the questionnaires. Hence, in the following year, which was 2014 academic year, version 1.0 of the system was developed. In the second phase, that is version 1.5 of the system, the index number authentication feature was removed and tokens were introduced. Therefore, each student will have at least five to six randomly generated tokens sent to his or her email to use to long into enter the system to evaluate. University of Ghana http://ugspace.ug.edu.gh 77 The tokens were designed purposely for authenticating the user entering the system. As soon as the user enters the system with the token, the token expires and any information the user enters is stored without any link to the tokens. Hence, in a way, anonymity was ensured. However, students have still have not used the, because of the fear that the tokens can be traced back to them. Also, the tokens were emailed to the University of Ghana email service for students, but it was later realised that only about 40% of the students used the university’s email service. Hence, the remaining 60% did not get their tokens and they could not evaluate. In addition, students were supposed to type the names of their courses and lecturers. This proved to be a problem, since most students did not know their lecturers’ names. In fact, some did not know their course codes. The typing (spelling) of names was a major issue. This brought about inconsistency in the data. The final system that is currently being used, that is the version 2.0 of the online Course evaluation system, is an open system that does not require any form of authentication. Also drop down menus have been created for students to be able to select their campuses, courses and the lecturer teaching those courses. In addition, the questions have been reduced to cover one page. Further modifications are being made using Bootstrap which is an open-source collection of tools for creating websites and web applications. It is also being used to modify the system to make it more compatible to hand-held devices. After the system was developed, there was no organised or predetermined implementation plan. The System was rolled out to the end-users after development. There was no training or education of end users; the Assistant Registrar of the academic quality assurance unit ascertained this. The hiring of new personnel necessary to facilitate a smooth transition from the old to the new system University of Ghana http://ugspace.ug.edu.gh 78 was not done. The webmaster maintained the university as a matter of policy is inclined towards phased implementation. This entails trying a system in certain parts of the school before embarking on a full implementation upon a successful phased implementation. As earlier stated, there was inadequate sensitization of all the stakeholders of the system. The webmaster and the Assistant Registrar arrived at a decision on the need for a new form of evaluation. It was immediately designed and implemented the following semester without any prior training or adequate sensitization of the students and other stakeholders. The announcement of the change was made through emails to the faculty and the link was uploaded on the University of Ghana Homepage for students to use. Regardless of the weakness of the paper-based course evaluation that motivated the virtualisation of the course evaluation process, there were also some issues concerning the virtualisation of the course evaluation that constrain the development process. The greatest fear that hindered the development process was low response rates. The Assistant Registrar who has knowledge of the online course evaluation system believed that the system is always flawed with low response rate. This was a huge setback since good response rates were needed to aid in ensuring the reliability of the data. He stated that; “The reliability of a data becomes questionable in a situation where, for instance, if only two percent of a class of four hundred complete an evaluation. Using that data to evaluate a lecturer makes it questionable." University of Ghana http://ugspace.ug.edu.gh 79 So the issue of lower response rate was iterated in the process of conversion to an online system. From the interview, it appeared that the fear of low response rates was something that was preventing the University of Ghana from adopting an online course evaluation system. It was based on this that the Assistant Registrar who is a member of the top management in the Academic quality assurance unit revealed his fear of students not “actively using the system and hence lowering the response rate.” This was a legitimate concern, since per the university’s policy, only a response rate of 60% can be considered as adequate. Another issue that almost hindered the development of the online course evaluation was confidentiality and anonymity after completing a course evaluation process. There was the need for students to be assured that they cannot be easily identified. Paper-based forms traditionally do not ask for any identifying information. The Assistant Registrar of the Academic Quality Assurance Unit said; “I had a prolonged discussion with the webmaster whether there were some measures to put in place in order to ensure anonymity at the same time to be able to authenticate the student filling the form.” In addition, faculties needed to be assured that only those students enrolled in the class complete the evaluation and that each student completes only one form. Although these requirements are relatively easy to achieve in a classroom setting, they require additional safeguards when using the online platform to conduct the evaluation process. These safeguards put in place for the online platforms require that students’ identification be authenticated and protected hence, it causes the University of Ghana http://ugspace.ug.edu.gh 80 system to be flawed. To achieve this, the assistant Registrar of the AQAU asked the Webmaster of the University of Ghana Computing Systems whether there were some measures to put in place in order to ensure anonymity at the same time to be able to authenticate the student filling the form. The Webmaster suggested tokens could be used to ensure anonymity. He indicated that, “… The best way to solve this was to give each student at least seven (7) randomly generated token linked to his or her faculty.” This will give the students access to, use the token to log into the system. After filling the forms, the student clicks the send button. The responses will then be stored in a different field without the token. This will securely ensure the utter impossibility of tracing the token. Furthermore, another issue that was hindering the development of the online course evaluation process was the fear of reliance on technology. This had the potential of adversely affecting access to an online course evaluation system in several ways. For instance, low levels of computer literacy may cause certain students from partaking in the evaluation process. Infrastructure was also an issue. It came along with the question of adequate internet connectivity for students. It was obvious that students with no easy access to computers will naturally choose not to evaluate. A lecturer indicated that, “he believes the university nowadays is over-relying on technology.” To him, the case of the network infrastructure being down could mean a great problem for the efficiency and effectiveness of the whole evaluation process. Finally, there was the recurring issue of change, since it is not easy to replace a very well established customary practice. There was no elaborate sensitization, both students and lecturers University of Ghana http://ugspace.ug.edu.gh 81 were not aware of the change. The system was implemented without any prior notice to the end users. A lecturer reported that without any prior information and preparation, he received a memo informing him of the university’s decision to switch to an online evaluation system from the next academic semester. This demonstrates that the change was somewhat drastic. Adapting to such a change has been at times difficult since both students and faculty who are the main stakeholders of the evaluation process found themselves to be well acclimatized to the paper-based course evaluation system. 5.6 The virtualised process of Course Evaluation As earlier stated, the course evaluation is conducted in the last week of teaching of each semester. Before the evaluation period, a memo is sent from the AQAU to all faculty members of the University, reminding them of the evaluation of courses and lecturers. Also, some of the memos are sent to students through the University’s electronic mail. During the evaluation period, a link directing students to the evaluation page is put on the main University of Ghana and AQAU homepage. When students click the link, it takes them to a one- page questionnaire item, with four sections. In the first section, the student chooses his or her campus, course code, name of the lecturer (s), academic year, and semester from a dropdown menu. University of Ghana http://ugspace.ug.edu.gh 82 Figure 5.2: Online Course Evaluation System Interface Source: University of Ghana (2015) The student then moves to the second section that is the Course evaluation section, which is a questionnaire item on the course ass indicated in Figure 5.2. University of Ghana http://ugspace.ug.edu.gh 83 Figure 5.3: Online Course Evaluation System Interface Source: University of Ghana (2015) The next section (Figure 5.3) is Lecturer Evaluation, questions are asked about the attendance of the lecture and how the lecturer delivered the course. University of Ghana http://ugspace.ug.edu.gh 84 Figure 5.4: Online Course Evaluation System Interface Source: University of Ghana (2015) The final section is to provide qualitative comments. Students are given the opportunity to add the comments and suggestion for improving the course and its mode of delivery. This is evident as indicated in Figure 5.4. After filling the questionnaire, the student clicks on the submit button at the bottom of the page and his or her form is saved in the database. University of Ghana http://ugspace.ug.edu.gh 85 The academic quality assurance Unit, use the course evaluation system by extracting the data after the evaluation period is over and the raw data is analysed with an internally developed system called Merge in Figure 5.5. The research assistant of the AQAU inputs the raw data into the system. He then input the Name, Course Code, Course Title, Semester and Academic year. After he clicks evaluate and a summarized report is generated as shown in figure 5.6. Figure 5.5: Merge System Interface University of Ghana http://ugspace.ug.edu.gh 86 Figure 5.6: Merge System Interface After analysis of the data, a summary of the analysed data and a copy of the unedited report are sent to the lecturers concerned. Copies of the same materials are sent to the lecturer’s Head of Department and Dean. The Director of the Academic Quality Assurance Unit is required to send comments on the analysed data to the Dean concerned. Everything is done under strict confidentiality. Heads of the departments are required to discuss their evaluation report with the teaching staff. Xxxxxxxxxxxxxxxxxxxx Xxxxxxxxxxxxxxxxxxxx University of Ghana http://ugspace.ug.edu.gh 87 5.7 The Use Phase of the Online Course Evaluation System This section outlines the system use in order to capture the stakeholder perspectives in utilizing the online course evaluation system. The stakeholders for the system usage phase are; Students, Academic Quality Assurance Unit, The Webmaster of the University of Ghana Computing Systems who serves as an administrator for the system and finally lecturers. The fear of low response rate was also ascertained as some of the lecturers interviewed indicated that students are not using the course evaluation system hence leading to low response rates. The Academic Quality Assurance Unit uses this low response rate to grade them. This raises a question on the validity of the results of the evaluation process. One of the lecturers at the College of Education retorted that; “My perception was that online course evaluation will result in lower response rates even before the system was implemented. And this perception became evident as only three (3) students out of a class of four hundred and twenty (420) students evaluated my course last academic semester.” This ascertains one major weakness of the online course evaluation system since the implementation of the system. Also, the problem of authentication also emerged during the interview. The current system is so porous that both students and non-students alike can evaluate any course. University of Ghana http://ugspace.ug.edu.gh 88 The webmaster indicated that, “There is no way of authenticating who uses this system. I believe that a student can evaluate the same course more than ten times, without been detected. This can happen especially when the student does not like the lecturer.” This is because there are no security modules in place or any checks to authenticate the person evaluating the course, eventually resulting in a flawed system. A lecturer was of the considered opinion that the advantages of the online course evaluation notwithstanding; the paper-based evaluation was highly reliable in the sense that a student could only fill one evaluation questionnaire at a time. This the webmaster believed, eliminated the probability of double or duplicate evaluation. Furthermore, the students tend to see the online system to be much better than the traditional paper based course evaluation system. For the online system, they can take their time to do the evaluation in their own convenience with no lecturers available, unlike the paper-based evaluation where either the lecturer or his Teaching Assistant will be available. A student indicated; “… Actually using the online system is a more relaxing way of evaluating. I think it is more effective than the traditional way of evaluation since it offers the opportunity for students to evaluate at their convenient time. A student still gets the opportunity to evaluate even if he or she is not in class during the evaluation period, also it is not intimidating since there is no authority around.” Last but not the least, the interview brought to light that the University has enough infrastructure for the system to run better. University of Ghana http://ugspace.ug.edu.gh 89 The Webmaster of UGCS said, Most of the students on the main University of Ghana Campus in Legon have wireless internet access since the University has invested heavily in building its internet infrastructure in most part of the campus. Hence, most students have internet access and therefore have no problem in partaking in the evaluation process of courses and lecturers. The only problem is that there is not much motivation for students to log online to partake in evaluating courses and lecturers. For instance, a student indicated that, “I saw the link for the evaluation online, but I was not compelled to go and evaluate. I did not see the need to go and evaluate. To sum it, with the paper based, I am compelled to do it, but with the online one, I am not. It is a choice. The lecturers come to class remind us to go and evaluate, but I don’t think most of the students did that.” Finally, it was also found out that some of the students do not see the essence of using the system. To them it is just a waste of time to evaluate courses, especially when they knew exams is around the corner. Moreover, there is not enough motivation to use the system because they do not see changes even if they leave comments. To them, evaluation is a routine activity of the academic semester and is at best optional. A student indicated that, He doubts whether comments and suggestions by students were taken into consideration. To him, there is no point in wasting his time to evaluate courses and lecturers when his comments will in the final analysis not be taken seriously. This student re-echoed a common sentiment of University of Ghana http://ugspace.ug.edu.gh 90 students at the University of Ghana. Of course, not all the students shared the above view. Some students saw evaluation as an obligation. They were of the view that the online course evaluation system is a platform to voice out opinions and suggestions on how to make teaching and learning better. These students even gave suggestions that the university could implement in order for more students to use the system. The student suggestions are explained below in order of frequency. A student suggested that, The university should withhold students’ early access to their grades until they evaluate their courses. Nearly all students who saw evaluation as obligatory supported this approach. Secondly, there were suggestions on the need to create more awareness and education on the use of the course evaluation system. Also, lecturers were advised to take personal interest in the evaluation of their courses and teaching. They were also convinced that the authorities ought to demonstrate that recommendations are being implemented for the overall success of the evaluation system. These students were all in agreement with regards to the usefulness of sending evaluation reminders to students 5.8 Summary This chapter presented the background of the University of Ghana. In addition, both the physical and virtualised process of the course evaluation system were discussed. The chapter then elaborated on University of Ghana’s decision to adopt the online course evaluation system. It also described the development of the online course evaluation system. The next chapter performs a rigorous analysis on the case findings as presented in this chapter, applying the force-field analysis University of Ghana http://ugspace.ug.edu.gh 91 model as discussed in Chapter 3 in order to identify emergent themes and concepts for discussions in Chapter 7. University of Ghana http://ugspace.ug.edu.gh 92 CHAPTER SIX ANALYSIS OF FINDINGS 6.1 Introduction The purpose of this research is to understand why and how higher educational institutions pursue virtualisation of course evaluation process in a developing country context. The previous chapter presented evidence on how University of Ghana developed its online course evaluation system by presenting the case findings. In relation to the research questions outlined in Chapter One, this chapter draws on the Kurt Lewin’s force field analysis model to analyse the research findings of the previous chapter. The analysis concerns are; why do higher educational institutions virtualise course evaluation processes; how do higher educational institutions virtualise their course evaluation processes; what factors enable and constrain the process virtualisation of course evaluation. 6.2 University of Ghana’s reasons for pursuing the process virtualisation of Course evaluation Several reasons influenced the University of Ghana to shift from its traditional paper-based course evaluation process to the online Course evaluation system. Below are some reasons for University of Ghana developing the online course evaluation system. 6.2.1 Reduced Processing Time First, unlike the paper based course evaluation system that took longer to process, approximately four-to five months, the online course evaluation shortens this processing time to just a day’s job. For the online Course evaluation system, there is no need in printing over eighty thousand sets of University of Ghana http://ugspace.ug.edu.gh 93 questionnaire, distributing and scanning after it has been filled. This tends to take away the flaws of backlogs. Furthermore, for the online system, there is no need to hire the services of extra personnel to type in the comments of the students. This tends to prevent typographical mistakes. Since the collected data could be analysed within days, this will substantially reduce the time for lecturers of receive their reports and enable them to quickly act upon the students’ feedback. The time and resources used in processing the data can be diverted to other projects. 6.2.2 Save Valuable Lecture Time Secondly, unlike the online course evaluation system, the paper -based method requires students to fill the evaluation forms during the last week of lecture at the end of the semester. This makes students to lose valuable time where some students just rush through the evaluation by ticking anything. Other students are concerned with getting their last examination tips hence they end up not filling the forms. However, with the introduction of the online Course evaluation system, valuable lecture hours, which were dedicated to the evaluation period are now being eliminated. The online Course evaluation tend to give the students more time to do revision. Since they can do the evaluation in their convenient time outside the lecture rooms. Also, besides freeing up valuable lecture time, online course evaluation provide students with a longer period during which to complete their evaluation compared to filling out forms during lectures where students must do so in a few minutes. This will lead to a ripple effect to enhance both the quantity and quality of their written responses. Students can have sufficient time to think and elicit more thought-provoking comments. University of Ghana http://ugspace.ug.edu.gh 94 6.2.3 Reduced Operational Costs From the research findings, it is ascertained that, the online course evaluation system is very cost- effective. It takes away the cost of printing over one hundred and eighty thousand evaluation forms, each semester as compared to the paper-based course evaluation system. Cost is an important consideration for institutions of higher education. Notwithstanding that, the online Course evaluation system also tend to take away the cost of distributing these evaluation forms throughout the various university of Ghana campuses. All these funds can be diverted to fund other projects in the university. Finally, there is reduced administrative cost unlike the typical paper-based course evaluation process, which is a labour-intensive process. For instance, a typical evaluation cycle begins by the faculty’s staff placing orders for evaluation forms for every course in that semester. The Academic Quality Assurance Unit ensures that, the forms are printed and sent to the faculty. Upon receiving the forms, the faculty’s staff prepares packets for each course section and delivers them to each department. The lecturers take the forms to their various lectures and have other students in the lectures or teaching assistant give the forms. Once the forms are returned to the faculty, they are sent to the academic quality assurance unit where they hire the services of extra personnel in processing the large number of evaluation forms. However, all this is eliminated, in the case where we are using the online system. The funds that would have been channelled into the hiring of extra personnel to process the data, the of cost repairs of scanners if they break down can be diverted to other pressing needs of the unit. University of Ghana http://ugspace.ug.edu.gh 95 6.2.4 Accessibility Also, accessibility, from the research findings can be viewed in two strands. Accessibility on the part of the students and accessibility on the parts of lecturers. Comparing the paper-based course evaluation system to the online system, most students tend to miss the opportunity to evaluate the courses when they are not at lectures during the day of the evaluation. Since the evaluation for a course is done only once in a semester most of the student’s perspective may not be captured. However, this is not so with the online course evaluation system, the students will have a long time, whether at lectures or not to voice out their grievances during the evaluation period. All of the students in the class tend to have equal access to the online course evaluation system. None of the students will miss this opportunity to evaluate because they are absent from class. Furthermore, lecturers can have access to the evaluation report within a short period compared to the paper- based evaluation method and it will aid in informing how they teach the next semester. 6.2.5 Easy modification of Evaluation Questions From the research findings, the academic quality assurance unit has the flexibility in the questionnaire design, which is the ability to easily change the items on the evaluation forms. Unlike the paper based course evaluation system, the questions for the evaluation can easily be modified with no implications. For instance, the current evaluation system did not capture how innovative lecturers were in class. Hence, the academic quality assurance unit has decided to add technological innovation to it. If the university were to be still using the paper-based system, it would have been difficult to modify the system for a new set of evaluation forms to be printed. Also, the field for capturing comments is limited or has fixed length in paper-based forms. However, with the online system this limitation is eliminated. University of Ghana http://ugspace.ug.edu.gh 96 6.2.6 The ability for students to evaluate at their convenient time In addition, students have the flexibility to fill the online evaluation forms at their own pace, that is as and when they want to fill it. The online course evaluation system affords students with flexibility in completing their evaluation forms provided they have access to a computer and the Internet. Enabling students to complete the form at their own convenience increases the likelihood that responding students will have the time needed to consider their rating and write all that they want to say in the student comments section. 6.3 Enabling factors influencing the decision for Online Course Evaluation System This subsection analyses the research findings to identify the factors that influence the decision of higher educational institutions to pursue the virtualisation of course evaluation. The emergent factors can be broadly classified into the following: increased efficiency and effectiveness in processing the evaluation data, Increased End-User Satisfaction and internal Environmental forces. 6.3.1 Increased efficiency and effectiveness Unlike the online course evaluation system, the paper-based system usually creates a lot of administrative workload. Before administering the evaluation forms, a lot of preparatory work has to be done. For instance, printing, packaging, and distributing the evaluation forms. After they have been filled all these materials have to be returned to a central location, which is the academic quality assurance unit for processing. This involves sorting and scanning the evaluation form. This University of Ghana http://ugspace.ug.edu.gh 97 can lead to several human errors. This also prolonged cycle time tends to elongate the time spent on providing feedback. All these issues are eliminated with the online course evaluation system. The only work associated with this system is to input the questions into the online forms and extracting them for analysis. 6.3.2 Increased End-User Satisfaction With the online course evaluation system, students can fill the evaluation forms during their own discretionary time. Hence, they tend to be more satisfied and content with the system, unlike the paper-based Course evaluation where they have to rush through the filling of the forms. Because they need to submit it, some students tend to fill the forms anyhow, while others end up not filling them at all. Also since, the processing time has been reduced, lecturers can get their feedback within a matter of days in order to help inform their decision on how to direct the next semester's lessons, some even tend to change the teaching style, based on the comments the students give. 6.3.3 Environmental Forces Another factor that influenced University of Ghana to virtualise the physical process of its course evaluation is the environment. Environment in this context refers to aggregate of conditions or forces that influence or shape University of Ghana. The environmental force is classified into two (2). The external environment of the University of Ghana refers to its competitive forces. With the notion of developing into a world class university, there is the need to act accordingly since virtualising the Course evaluation process is a contemporary phenomenon most western schools University of Ghana http://ugspace.ug.edu.gh 98 are adopting it. Another internal environmental factor is the growing population of the University of Ghana. The University has experienced a tremendous growth in its population to about forty - two thousand, six hundred and ninety-two (42,692) and looking at this growth rate the paper-based method of course evaluation will not suffice. 6.4 Factors Constraining the Decision for virtualisation of Course evaluation This subsection also analyses the research findings to identify the factors that constrain higher educational institutions from pursuing the process virtualisation of course evaluation. The emerging factors are the fear of low response rates, lack of perceived anonymity and authentication, over dependency on technology and culture of change. 6.4.1 Fear of Low Response Rate As earlier indicated in the previous chapter, irrespective of the benefits of the online course evaluation system, there are some flaws associated with it that can deter higher educational institutions from pursing the virtualisation of the physical evaluation process. A typical factor is the fear for low response rate. Unlike the paper-based course evaluation process where evaluation forms are administered during lectures, the online course evaluation process is usually completed outside of lecture room that is during students’ discretionary time. In the positive light, this will provide students with more time to fill out the evaluation forms, but it also gives them more freedom in their decision of whether or not to complete the evaluation forms. This in a way can render a whole evaluation data for a course either invalid or unreliable. Since as earlier stated in University of Ghana http://ugspace.ug.edu.gh 99 chapter 5, the stipulated valid response rate is 60%, hence in situations where the response rate is less than that, then the evaluation data can be questionable. 6.4.2 Lack of Perceived Anonymity, authentication and Confidentiality The issue of anonymity goes hand in hand with authentication and confidentiality. Anonymity in this context is the ability of end users to access the online course evaluation forms whiles blocking the ability to track or trace their identity. Unfortunately, the online Course evaluation process inevitably enables the system to trace students to their feedback. This raises most student concerns about the lack of anonymity in using the system. Considering the three (3) systems developed, the first system required students to log into the system with their index number. Although, the webmaster indicated that the log in the table is not in any way connected to the response table, the student did not understand that, hence most entertained the fear of being tracing. With the second system, tokens were required to access the system. The randomly generated tokens were emailed to students. This proves that in the end if the administrator wants to trace the user of a particular token, he can trace the person effortlessly. The third system developed does not require any form of authentication. Nevertheless, this shows that any students with access to a computer and an open internet connection can evaluate any course he or she wants without any consequences. This lack of perceived anonymity could either deter end users that is students, from using the system, leading to low response rate or other students will end up giving an insincerely positive comment because they lack confidence in the anonymity of their responses, which can lead to data integrity issues. The ability to authenticate users is inherent with the paper-based course evaluation system. This is because with the paper-based evaluation forms, they are administered in the lecture rooms under supervision. Therefore, it can be assured that, other students who are not offering the course will University of Ghana http://ugspace.ug.edu.gh 100 not be able to access the evaluation form. However, one response per student cannot be guaranteed, especially in larger classes, and confidentiality depends on the honesty of the student sitting close by the person filling the evaluation forms. The paper-based evaluation form is designed in a way that no identifying information is asked. This goes a long run to ensure complete anonymity. 6.4.3 Over-Dependence on Technology Another issue as earlier stated in the previous chapter is the fear of over-reliance or over dependence on technology. The online course evaluation systems unavailability, regardless of the primary cause, can create serious problems for both the end-users and the university. For the purpose of this study, a system is said to be unavailable if users cannot access it, even though it appears to be operational, as when the system interface is working, but the back end database is down, when there is an insufficient hardware to support workers’ needs, or when the system is so slow that work activities cannot be efficiently completed. Such situations can create a real fight at times to get work done, and frustrate busy students when they incorrectly assume the system is entirely functional, must find workarounds. There is also the issue of, a complete system downtime, even though this may be generally rare, it can have serious repercussions for the university. For instance, because evaluation is conducted once in the semester, if the online course evaluation system goes down, it will be difficult to evaluate and may even cause evaluation process not to be done that particular semester. In such cases, there are no contingency plans, since the university cannot immediately print forms, it may cause the evaluation process to be postponed. University of Ghana http://ugspace.ug.edu.gh 101 6.5 Development process of the Course Evaluation Employing use of the Systems Development Life cycle methodology has brought to light that during the problem definition phase of the development process, even though there was an awareness of the need for new mode of evaluation, there was no formal feasibility study on the project to guide the top management decision or show how viable the project is. In addition, there was lack of primary or end user involvement in the process. Hence, their perspective and their expectations on the new online course evaluation system were not solicited. The development team did not document the performance standards, social and technical requirements of the system. The online course evaluation system was designed to be a web-based cross browser compatible system. This means no matter what kind of browser the student was using the online system should be compatible with the student's browser. The system was also developed in a way that provides, the academic quality assurance unit complete administrative control over the question design, and the ability to extract data in an already coded format, thereby eliminating unnecessary human intervention. Even though the development of the system was a success, there were some issues and flaws in the error in the final developed system. Furthermore, the system was not tested, before it was rolled out. It was implemented throughout the university without adequate training and sensitization. However, the system was reviewed as and when it was used and patches were made to the system. This led to the development of three versions of the system. 6.5.1 Enabling Factors influencing the development of the Course evaluation This subsection outlines the factors that enabled the development of the online course evaluation system. The factors include Usability, Optimizability, flexibility and convenience. University of Ghana http://ugspace.ug.edu.gh 102 6.5.1.1 Usability Usability, in the context of this study is a measure of how easily the end -user interacts with the interface of the online course evaluation system. It is the quality of the online system from the point of view of the end users who use it. The online system is more efficient. Efficiency can be viewed from the students’ perspective, where he or she has can easily orient him or herself with the content of the page without facing any problems. The system can also support a large number of students who log onto the system to use. It affords the students the ability to select and deselect answers, type or erase comments on the evaluation forms easily, thereby increasing the end-users’ satisfaction of the online course evaluation system. From the Academic Quality Assurance point of view, usability of the system is enhanced, because with less training they can effortlessly extract the pre-coded data for analysis without encountering any or less errors. 6.5.1.2 Optimizability The online course evaluation systems show a high level of optimization than the paper-based Course evaluation system. Unlike the paper-based, the online system presents a great opportunity to miniaturize the system to fit handheld devices. For instance, the new version of the current online course evaluation system is being modified with bootstrap to make the system compatible with mobile devices. This may lead to increase in response rate, if students realise they can easily do their evaluation on their mobile phones. On the other hand, during the period of evaluation, lecturers can easily ask students to bring out their devices to evaluate the course. University of Ghana http://ugspace.ug.edu.gh 103 6.5.1.3 Flexibility As earlier indicated in the previous section, the online course evaluation system offers the administrator of the system an ability to easily modify questions at relatively no cost. This is not so with the paper-based course evaluation system, since modifying the evaluation question means printing the whole set of evaluation forms again. Also, apart from the ability to modify the questions, the system offers great flexibility to students in terms of use. Until they click the submit button on the form, they can easily modify the choice of answers. It offers them time to provide lengthy and quality feedback. 6.5.1.4 Convenience Finally, the online course evaluation system offers hassle free usage to students. First, they can fill the evaluation forms in their own convenient time. The students can take their time to answer the form at any place or any time provided they have a computer and an internet infrastructure, thereby saving valuable lecture time dedicated to the filling of the paper-based forms. Furthermore, the system offers greater convenience to the academic quality assurance unit as the processing time for the evaluation data is reduced. They can easily extract the evaluation data in a matter of minutes and run the analysis. There are no backlogs when analysing the data. It also offers greater convenience to the lecturers as they can get immediate feedback and act on the feedback earlier if need be. When students start to realise the change based on the feedbacks, they will actively use the system next time. University of Ghana http://ugspace.ug.edu.gh 104 6.5.2 Factors constraining the development of the Course evaluation This subsection outlines the factors that constrains the development of the online Course evaluation system. The factors include lack of anonymity, lack of end-user involvement, inadequate sensitization, and lack of adequate technological infrastructure in the University of Ghana. 6.5.2.1 Lack of Perceived Anonymity Anonymity is a way of safeguarding the privacy of the students who uses the online course evaluation system. It is difficult in designing the system to be anonymous in usage. Because from the first and second versions of the system as earlier stated in the previous chapter, anonymity although it was ensured, the end users found it difficult to believe. In the first version, they had to use their index numbers and in critical cases most students believed, the index number could easily be traced. Notwithstanding, the randomly generated tokens sent to the students' emails could not achieve the purpose of ensuring the anonymity of the end users. Since the students believed that, the tokens can be traced. For the purpose of anonymity, the third version of the system has been left open. Now the issue is how to authenticate the users of the system. Because with the open access to the system, any person, whether student or not can evaluate any course. This factor constrains the development of the system 6.5.2.2 Lack of end-user involvement Another constraining factor is the lack of end user involvement in the development of the system. As earlier indicated in chapter five (5), both students and lecturers were not involved in the University of Ghana http://ugspace.ug.edu.gh 105 designing of the online course evaluation system. Hence, there are some issues, which could have been easily dealt with but existed due to the lack of involvement. For instance, the courses offered by the University of Ghana were all listed from A-Z, and students had to search through all the courses in order to be able to select the particular course they want to evaluate. This deters some students from using the system because they see it as a waste of time. A student indicated that, it would have been easier to use if the courses were classified into the various colleges, so that when the student selects his college, he or she gets access to only course offered by the college to choose from. This rich insight would have been brought to light if end-users were involved in the development process 6.5.2.3 Inadequate Sensitization Since end-users were not aware of the development of the online Course evaluation system. Most students do not even know that there is an online platform for evaluation. Even after the system was developed, there was no adequate sensitization on the use of the system. Some students and lecturers were not aware of the change. As indicated in chapter five (5); A lecturer reported that, “I just received a memo that, from the next semester, the university was going to use an online evaluation system and that was it” 6.5.2.4 Lack of Adequate Technological Infrastructure The basic requirement for meaningful use of any web-based system is to be able to have hassle free access to the internet. The cost of bandwidth seems to be a hindrance to the development of University of Ghana http://ugspace.ug.edu.gh 106 the online course evaluation system. Bandwidth is expensive, and if there is no internet connectivity, it means an evaluation cannot be done. Furthermore, there is the issue of whether students possess the necessary skill to use the system. Even though we are in a learned community, some students find it difficult to even access email account, how much more going online to fill an average of five evaluation forms. This in a way will deter that segment of student from using which brings the response rate down. Also, the university does not have adequate computers or computing centres for all students to use to evaluate. The system was created on the assumption that most students use personal laptops and the percentage that do not can use the school facility. This tends to lead to low response rate if that percentage of students do not get computers to evaluate the course. 6.5.2.5 Lack of Policy on the use of the system Based on the research findings, there is no laid down policy or guideline safeguarding the use of the online course evaluation system. Due to the diversification of University of Ghana campuses, the system has been designed to offer student direct open access without authentication measures. This can open the system up for a breach. It can be a window for hackers to hack the whole university system since anyone can use the system. University of Ghana http://ugspace.ug.edu.gh 107 6.6 Force Field Analysis Model of Process Course evaluation Figure 6.1: Kurt Lewin’s Force Field Analysis Model of Process Course evaluation Current Equilibrium of the Online Course Evaluation System Constraining Forces Influencing the decision for the system Increased efficiency and effectiveness Increased End-User Satisfaction Internal Environmental Forces For the Development Process Usability Optimizability Flexibility Convenience Influencing the decision for the system Fear of low response rate Lack of perceived Anonymity Over-Dependence on Technology For the Development Process Lack of perceived anonymity Lack of end-user involvement Inadequate Sensitization Lack of policy on system use Lack of adequate technological infrastructure Enabling Forces University of Ghana http://ugspace.ug.edu.gh 108 The above figure outlines the various enabling and constraining forces that influences both the decision for the online course evaluation system and the development process of the system. 6.7 Summary The chapter analysed the case study findings in Chapter five (5) in relation to the research questions and to identify emergent themes and concepts. The analysis presented the following as the enabling factors influencing the decision for virtualising the course evaluation. The emergent factors were broadly classified into the following: Increased Processing Time, Save valuable Lecture Time, Reduced Operational Cost, Accessibility, Flexibility, increased efficiency and effectiveness. In addition, the constraining factors influencing the process virtualisation of course evaluation were low response rates, lack of perceived anonymity and authentication, over dependency on technology. Finally, the emerging themes were categorized using the Kurt Lewin’s Force field analysis model. The next chapter presents discussions on the identified themes and concepts in relation to literature. University of Ghana http://ugspace.ug.edu.gh 109 CHAPTER SEVEN DISCUSSION OF FINDINGS 7.1 Introduction The previous chapter analysed the research findings of this study in relation to the research questions in chapter one (1) and the force field analysis model. The analysis was done in order to identify the emergent themes and concepts of the research findings. The current chapter discusses the analysis of the findings in relations to the reviewed literature in order to address the research questions outlined in chapter one of this study. This chapter brings together the literature findings from chapter two (2) and the empirical findings and analysis from chapters five (5) and six (6). The chapter begins by discussing the reasons higher educational institutions pursue process virtualisation of its course evaluation. 7.2 Reasons Higher Educational Institutions Pursue Virtualisation of Course Evaluation Process This subsection discusses the analysis of findings to identify the factors that influence higher educational institutions to pursue the process virtualisation of course evaluation and how they relate to the literature review in chapter two (2). The emergent factors can be broadly classified into the following; Time, Cost, Accessibility and Flexibility. University of Ghana http://ugspace.ug.edu.gh 110 7.2.1 Reduced Cycle Time for Processing The issue of time can be viewed along two strands; the first is the reduction of the processing time of the paper-based course evaluation system. As earlier indicated in chapter six, the processing of the data from the paper-based evaluation system in University of Ghana can take approximately four to-five months, hence feedback of the evaluation is not received in time, also there are always backlogs of unprocessed data from the previous semesters. The cycle time is also prolonged; the cycle time refers to the total time from the beginning to the end of the evaluation process. The cycle time for the paper based course evaluation system at the University of Ghana includes the time to print, distribute, collate, scan the Likert scaled questions, and type the open-ended questions before the data is analysed. Nevertheless, with the introduction of the online course evaluation system, the cycle time for the evaluation is reduced. Again, the evaluation data can be processed within a short period without backlogs. These findings are in line with Handwerk, Carson and Blackwell (2000) and Hmielesk (2000). As Hmieleski (2000) ascertained in their survey that, “Sixty-five (65) percent of the respondents indicated, on average, it took three weeks to two months before teachers received the results of their course ratings”. They further indicated, “an online ratings system can substantially shorten the time to receive ratings reports, thereby enabling teachers to consider and act on student’s feedback in a timely manner”. This was not far from the findings from the study other studies (Avery et al., 2010; David, Denise & Robert, 2010; Dorit, McClean & Nevo, 2012) also ascertain the above fact that the introduction of the online Course evaluation system tends to reduce both the cycle time of the evaluation process, hence increasing productivity. The second issue associated with time, is the ability of the online course evaluation system to save valuable lecture time. As indicated in the chapter six (6) of this study, the paper based course University of Ghana http://ugspace.ug.edu.gh 111 evaluation system requires students to be physically present in order to evaluate. This is in line with the findings of Johnson (2003), David, Denise, and Robert (2010) and Dommeyer, Baum, and Hanna (2002) which postulates that, the online Course evaluation system tend to free up valuable class time since students have the opportunity to do the evaluation outside the lecture rooms. They further indicated that, not only do lecturers perceive the value of this advantage, but most students also value this assertion. 7.2.2 Cost Effectiveness From the analysis of findings, it is ascertained that, the online course evaluation system is very cost effective. There is a reduction in the operational cost. The online course evaluation system tends to take away the cost of printing the paper evaluation forms. It also tends to take away the cost involved in distributing the forms to the various colleges and departments and the cost of repairing scanning machines. In addition, the extra cost involved in paying staff to scan the paper forms are eliminated when it comes to the online system. This is in line with the literature (Bemile, Jackson & Ofosu, 2014); Johnson, 2003); Nikoladis & Dimitriadis, 2014) where empirical findings assert that, online course evaluation system is generally perceived as less expensive than paper- pencil evaluation systems. From their studies, automating the Course evaluation process eliminates the paper costs and reduces personnel costs for processing rating forms. Another study by Alessia and Crow (2012) shows that, conducting course evaluations online course leads to saving 97 percent over the paper-based evaluation method. Bothell and Henderson (2003) have undertaken a rigorous cost study that points out that the overall costs for online systems substantially lower than those for paper-based systems. University of Ghana http://ugspace.ug.edu.gh 112 7.2.3 Accessibility Issues: From the study, it could be realised that the online course evaluation system offers the convenience to students to access the system as and when they want to during the evaluation period. This tends to leverage high and quality response rate, since students can take their time to fill the forms unlike the paper-based evaluation forms that require students to be physically present for the evaluation. Usually, accessibility goes hand in hand with time constrain when it comes to the paper-based system. Students are required to fill the forms in class within a shorter period. In the case of the University of Ghana, the allotted time for the paper-based evaluation is 10-15 minutes, hence student just rush through the filing of the forms at times preventing them from giving quality feedbacks. Furthermore, the online course evaluation system tends to give lecturers access to the evaluation feedback report within a shorter period as compared to the paper-based evaluation (Alauddin & Kifle, 2014). This is in line with reviewed literature in chapter two. 7.2.4 Flexibility and convenience Another reason higher educational institutions may pursue the development of an online course evaluation system is that, it offers the flexibility to easily adapt or change the evaluation questions at a lesser cost compared to the paper-based course evaluation system (Azizah et al., 2011). In the online course evaluation system, HEIs have the flexibility to adapt and personalize the forms as and when is necessary at a lesser cost. They can easily modify questions to elicit feedback according to their individual needs. This perceived flexibility and convenience heavily influences the decision for HEIs to virtualise the paper – based course evaluation system. In addition, with paper - based evaluations conducted in the lecture rooms, students have only one opportunity to University of Ghana http://ugspace.ug.edu.gh 113 provide their opinion that is during the class period the forms are distributed. However, with the online method of evaluation, students have multiple days on which to provide their evaluation. Moreover, during the period of the online evaluation, the online system can be programmed to send reminder notices to those students who have not yet responded to the survey (Anderson et al., 2005; Dommeyer et al., 2002). 7.3 Enabling factors influencing the decision to virtualise course evaluation process 7.3.1 Internal Environmental forces – Increase in Students Population Environmental forces also influence higher educational institutions to virtualise their course evaluation process. This force can be from either the internal or the external environment. The internal environment forces are the growing size of the population in the University of Ghana and some lecturers and Heads of Department who played a key role in pushing for the virtualisation of the course evaluation system. In relations to the external environment, competition can have a significant effect on whether to virtualise the course evaluation process. For instance, university of Ghana aims to be a world-class university; hence, its actions are being influenced or shaped by other world-class universities. This is because most of these universities are adopting technology to shape their business processes (Moss & Hendry, 2002). Moreover, it is feasible because by looking at the internal forces, For instance, considering the high increasing rate of the student population, it would not be feasible to be using the paper-based course evaluation form. This tends to push higher educational institutions to pursue the process virtualisation of the course evaluation system. This finding is new since, the literature has not discussed environmental forces as a reason for virtualising the course evaluation system. University of Ghana http://ugspace.ug.edu.gh 114 7.3.2 Ability to Increase Efficiency and Effectiveness The perceived increased efficiency and effectiveness on the online course evaluation system, tends to be a motivating force for higher educational institutions to pursue the process virtualisation of course evaluation system. For instance, as indicated in the previous chapter, the paper-based evaluation process involves printing, packaging and distributing, collating and scanning the evaluation forms. All these processes require human intervention. This increases the possibilities of high human errors in this process and there are many delays involved in using the paper –based evaluation method. Hence, it is plausible for higher educational institutions to pursue the virtualisation of the physical process (Bothell & Henderson, 2003; Annan et al., 2013). 7.4 Constraining factors influencing the virtualisation of Course Evaluation Process This subsection discusses the analysis of findings to identify the factors that constrain higher educational institutions to pursue the process virtualisation of course evaluation and how they relate to the literature review in chapter two (2). The emergent factors can be broadly classified into the following; fear of low response rate, lack of perceived anonymity and confidentiality, over dependence on technology and organisational issues. 7.4.1 Fear of Low Response Rate A main force that constrains the virtualisation of the course evaluation process is the fear of Low responses. With the paper-based evaluation method, the students in class can provide a considerable number of response rate. However, with the online course evaluation system, because University of Ghana http://ugspace.ug.edu.gh 115 students are completing the forms at their own discretionary time there is a possibility that, they might not do it or they might even forget about the evaluations, since they will be preoccupied with preparations towards exams. This is line with literature where many studies have been conducted on response rate (Cummings, Ballantyne & Fowler, 2001; Dommeyer, Baum & Hanna, 2002; Hmieleski, 2000; Johnson, 2002; Hardy, 2002; McGourty, Scoles, & Thorpe, 2002). It has been proven in the above studies that the online course evaluation system always produces lower response rate than the paper-based system. This may be due to perceived lack of anonymity of responses, lack of compulsion to complete ratings online, student apathy, inconvenience, technical problems, and required time for completing the ratings (Ballantyne, 2000; Dommeyer, Baum & Hanna, 2002). 7.4.2 Lack of Perceived Anonymity and Confidentiality As earlier indicated the perceived lack of perceived anonymity and confidence can deter students from using the system or give unbiased comments, especially with a small number of classes, hence leading to low response rate. This can prevent higher educational institutions from developing the online course evaluation systems. The issue of anonymity goes hand in hand with authentication and confidentiality. This is not off much issue when it comes to the paper-based course evaluations because the paper forms do not require any form of identifications from students. However, as earlier stated, there is no control over the number of forms a student can fill, especially if it is a large class. The perceived lack of anonymity is in line with the literature (Beran & Rokosh, 2009; David, Denise & Robert, 2010). University of Ghana http://ugspace.ug.edu.gh 116 7.4.3 Over -Dependence on Technology As stated in the previous chapter, a system unavailability, regardless of the primary cause, can create serious problems for both the end-users and the university. If the online course evaluation system goes down, it will be difficult to evaluate and may even cause evaluation process not to be carried out for that particular semester. The literature points out that Reliance on technology can adversely affect access to an online course-rating system in several ways. Also, low levels of computer literacy may exclude certain students from submitting their ratings online (Cummings, Ballantyne & Fowler, 2001). Likewise, computer problems can prevent students from submitting their ratings online (Dommeyer, Baum & Hanna, 2002). In addition, students who do not have easy access to computers may decide not to submit their ratings. Using computers in a laboratory may be inconvenient for some students because the computers may be slow (Ravelli, 2000) or because they have to wait in line to get access to the computers. 7.4.4 Organisational issues Proposing a change in how students evaluate the course and lecturers affects almost every unit and every individual in the university (Donovan, Mader & Shinsky, 2007). Lecturer’s resistance to changes in the design of the evaluation questions, the frequency of administering the evaluation forms, and in its medium of administration—from paper to online—is hardly based on faculty affection for the old paper system. Rather, their resistance often seems based more on their preconceived notions about the new system and their lingering doubts about the old system. Since literature shows that, Colleges that intend to replace paper-pencil course ratings with online ratings face the challenge of changing a well-established customary practice. University of Ghana http://ugspace.ug.edu.gh 117 7.5 Factors enabling the development process of the Course Evaluation This subsection outlines the factors the enabled the development of the online course evaluation system in relation to literature. The factors include usability, Optimizability, flexibility and convenience. 7.5.1 Usability The online course evaluation system has a user-friendly interface that users can easily orient themselves with. This makes it easier for students using the system. This is in line with the literature, as Azizah et al. (2011) indicated in their studies that usability measures the quality of experience during the interaction between the user and a product or system. It has also been referred to as a concept identified by research on user, product and environment (Azizah, 2005). ISO 9241-11 defines usability as the effectiveness, efficiency and user satisfaction in order to achieve certain aims in certain environments. A user-friendly interface will increase user satisfaction, which involves perception on ease of use, information organisation, labelling, detail and visualization, content and error correction. 7.5.2 Optimizability and Convenience As indicated in the previous chapter, the online course evaluation systems show a high level of optimization than the paper-based course evaluation system. Unlike the paper-based, the online system presents a great opportunity to miniaturize the system to fit handheld devices. Hence, this tends to offer higher educational institutions the flexibility to manipulate the system easily. This is in support with Chatvichienchai (2011) as the author postulated that using XML technologies to develop course evaluation system offers a greater flexibility and convenience. Because the online University of Ghana http://ugspace.ug.edu.gh 118 course evaluation system offers hassle free usage to students, since students can fill the forms in the own convenient time. Blair and Kimila (2014), also indicated that, unlike the paper-based course evaluation system takes six to seven months to be fully processed, the online evaluation system takes less time, thereby increasing efficiency in processing the data. Because with the online system the evaluation data can be easily extracted and analysis made on the data within a short time. Hence, there are no backlogs when analysing the data and it offers greater convenience to the lecturers as they can get immediate feedback and act on the feedback earlier if need be. They further indicated that, the online course evaluation tends to increase student satisfaction when it comes to using the system. With the online system, students now have the opportunity to take their time to fill the evaluation forms as and when they want during the evaluation period provided they have access to computers and internet infrastructure. 7.6 Constraining Factors influencing the development process of the Course evaluation This subsection outlines the factors that constrains the development of the online course evaluation system. A major factor constraining the development process of the online course evaluation is the fear of low response rate, which can be an effect of the following; lack of perceived anonymity, lack of end-user involvement, and inadequate sensitization of students. University of Ghana http://ugspace.ug.edu.gh 119 7.6.1 Lack of Perceived Anonymity A major issue during the development process was to how to ensure the students of their anonymity in using the system. As indicated by Moss and Hendry (2002), if responses are made anonymous by, for example, using a web-based survey with no password access, then there will be no way to follow up non-responders or to prevent people from completing the evaluation forms multiple times (Miyoung & Chiyoung, 2013). It is a dilemma since if the necessary measures are put in place, it tends to deter students from using the system. As seen in the case of the University of Ghana in chapter five, all the earlier versions of the system that authenticated the users of the system was flawed. Hence, they had to redesign the system to have open access. 7.6.2 Lack of end-user involvement Learning from the case of University of Ghana, it is important to include end users in the development of the online course evaluation system. Because the expectation of end users of the system, that is the students, were not sought during the development process. There were design flaws that could have been easily prevented but those mistakes were made. As indicated in the previous chapter, a student offered a rich insight on how to classify the courses in the system. The student indicated: “It would have been easier to use, if the courses were classified into the various colleges, so that when the student selects his college, he or she gets access to only courses offered by the college to choose from.” University of Ghana http://ugspace.ug.edu.gh 120 This rich insight indicates that if students are involved in the development process, they can help develop a better system and help increase their usage of the system that would increase response rates. 7.6.3 Inadequate Sensitization As stated earlier, end-users play a key role in the success of the use of every newly developed system (Dennis, Wixom & Tegarden, 2009). Since the students were not involved in the development of the online course evaluation system. Some students were not aware of the migration from paper based to the online platform hence it affected the use of the system which led to low response rates. 7.6.4 Lack of policy on the use of the system Another constraining force is the lack of policy on the use of the course evaluation system. As earlier indicated, there is no laid down policy safeguarding the use of the online course evaluation system. Due to the diversification of university of Ghana campuses, the system has been designed to offer students direct open access without ant authentication measures. This can open the system up for a security breach. However, creating those policies comes with a huge cost and takes a long time to implement. As a result, if there is breach in the system it will be difficult to deal with such a situation. University of Ghana http://ugspace.ug.edu.gh 121 7.7 Summary The chapter discussed the analysis of findings in Chapter 6 in relation to the literature reviewed in chapter two (2) to identify emergent themes and concepts. The discussion presented the following as the enabling factors influencing the process virtualisation of course evaluation reduced cycle time, cost effectiveness, accessibility, flexibility and internal environmental forces. Concerning the constraining factors influencing the virtualisation of course evaluation process, the emerging themes from the analysis are fear of low response rates, perceived lack of anonymity and authentication and over dependence on technology. Finally, the chapter further discussed forces that constrain and enable the development process of the online course evaluation in relation to literature. University of Ghana http://ugspace.ug.edu.gh 122 CHAPTER EIGHT SUMMARY, CONCLUSION AND RECOMMENDATIONS 8.1 Introduction The previous chapter discussed the analysis of the empirical findings and addressed the research questions in relation to the literature findings. This concluding chapter presents the summary of the study and interprets its contribution to knowledge, discusses the research implications and offers recommendations as well as presents the research limitations and overall conclusions. This chapter begins by reviewing the research questions as outlined in the introductory chapter and how the study addressed them. The chapter also delves into the study’s contribution to knowledge, offering rich insight and the drawing of specific implication to research, policy and practice. The chapter presents limitations and offers recommendations for future research. Finally, the chapter presents the overall conclusion. 8.2 Review of Research Questions This study began with a research purpose to understand why and how higher educational institutions pursue virtualisation of course evaluation processes using a higher educational institution from a developing country as a case study. The study further sought to explore the implications associated with migrating course evaluation process to online platforms by delving into the enabling and constraining forces associated with this migration by employing the Kurt Lewin’s Force Field Analysis Model. As stated in Chapter one (1), Section 1.2 of this study, the research problem is in three fold: Existing literature points out that very few have been done concerning the virtualisation of course evaluation process (Annan, Tratnack, Rubenstein, Metzler University of Ghana http://ugspace.ug.edu.gh 123 -Sawin & Hulton, 2013). Also, there has been a call for studies from the developing country perspective (Blair & Kimila, 2014; Ansah, 2010). To achieve the research purpose, the study was therefore motivated by three research questions as outlined in the introductory chapter and repeated here as follows: 1. Why do higher educational institutions virtualise course evaluation processes? 2. How do higher educational institutions virtualise their course evaluation processes? 3. What factors enable or constrain virtualisation of course evaluation processes in higher educational institutions? These research questions were addressed by the study as follows:  A literature review of important studies on process virtualisation, course evaluation and performance evaluation since course evaluation falls under performance evaluation in higher educational institutions. The modalities of course evaluations were also reviewed, it was found out that, there are three main modalities. These are the traditional, online and hybrid course evaluation. There was also a review on the differences between paper – based and online modalities to understand why higher educational institutions virtualise their course evaluation. The literature review indicated that, online course evaluation system is much preferred due to issues of accessibility, reduced administrative and operational costs, anonymity issues and time spent on providing feedback. Finally, the research gaps in literature were highlighted.  Chapter three (3) of this study discussed and selected the force field analysis model as the research framework. This theory was chosen in order to analyse the enabling and constraining forces in the virtualisation of course evaluation process in higher educational University of Ghana http://ugspace.ug.edu.gh 124 institutions. The force field analysis model was used to identify both constraining and enabling forces that influences how and why higher educational institutions virtualise their paper – based course evaluation system.  Chapter four (4) of this study discussed the research methodology and outlined the research method, which was a qualitative research method used to answer the research questions posed in the introductory chapter. Since the purpose of this study is to understand, why and how higher educational institutions virtualise their course evaluation system, the study employed the interpretive paradigm as appropriate for this research. The chapter finally discussed the process of case selection, fieldwork for the study and how data was collected and analysed.  Chapter five (5) of this study presented the empirical findings of the research from the case university, the University of Ghana. The chapter preceded with a narrative on the physical and virtualised processes of the course evaluation and reasons why and how University of Ghana pursued process virtualisation of its course evaluation system. It presented the why University of Ghana virtualised its course evaluation process and how it virtualised the process.  Chapter Six (6) analysed the case study findings in Chapter five (5). In relation to the first research question, the findings were analysed and emergent themes and concepts were identified. In answering the second and third research questions, the force field analysis model was used to identify the constraining and the enabling factors that influence the University of Ghana http://ugspace.ug.edu.gh 125 decision and the development process of online course evaluation system. The emergent themes on why university of Ghana virtualise its online course evaluation are; increased processing time, save valuable lecture time, reduced operational cost, accessibility, flexibility, increased efficiency and effectiveness. Some emerging themes from the analysis that constrained the university decision to virtualise the course evaluation process were the fear of low response rates, lack of perceived anonymity and authentication, over dependency on technology and culture of change. Finally, the emerging themes are categorized using the Kurt Lewin’s Force field analysis model.  Chapter Seven (7) discussed the analysis of the findings. Specifically, the chapter addressed the research questions in the light of the literature reviewed in chapter two (2) and the empirical findings in chapter five (5) as well as the resultant analysis in Chapter six (6). The discussion presented the following as the enabling factors influencing the process virtualisation of course evaluation. The emergent forces were reduced cycle time in processing, cost effectiveness, accessibility, flexibility, increased efficiency and effectiveness and internal environmental forces like increasing population of students. Also, the constraining factors influencing the process virtualisation of course evaluation from the analysis were fear of low response rates, perceived lack of anonymity and authentication and over dependence on technology. 8.3 Contribution to Knowledge Walsham (1995) indicated that, there are four ways research can contribute to the body of knowledge in information systems interpretive case study research. These contributions include, the development of concepts, the generation of theory, offering of rich insight, and drawing of University of Ghana http://ugspace.ug.edu.gh 126 specific implications. Based on these assumptions, although this study did not develop new concepts or contribute to theory it offers rich insight, and draws specific implications, which are implications for research, policy and practice. 8.3.2 Offering Rich Insights This study offers a rich insight into the virtualisation of course evaluation in higher educational institutions. The research extends the study on course evaluation in higher educational institutions (Adam, 2009; Avery et. al, 2010; Bemile, Jackson & Ofosu, 2014) by probing into, not just the reason, also how course evaluation processes are virtualised in higher educational institutions. This study pointed out that some of the reasons for higher educational institution virtualising their course evaluation process are reduced cycle time for processing, Cost effectiveness, Accessibility and Flexibility. By employing the use of force field analysis model, the study revealed that a critical force that influenced the development of the online course evaluation systems is internal environment, which is the increasing population of students. This makes it difficult to process the large quantity of paper – based evaluation forms in time. Other forces the study revealed is the lack of end user’s involvement during the development process and inadequate sensitization on the implementation of the system. This inadequate sensitization led to low response rate serving as a great constraining force. 8.3.3 Implications for Research About the significance to research, this study adds to the body of knowledge by bridging the gap in the area of virtualising the process of course evaluation in developing countries like Ghana. University of Ghana http://ugspace.ug.edu.gh 127 Unlike current study that mostly emanate from the Psychology and educational sector, this study goes beyond that by studying the online course evaluation system from Management of Information systems perspective. This study contributes to research be uniquely employing the Kurt Lewin’s Force Field Analysis Model in studying a phenomenon from the Management Information systems perspective. 8.3.4 Implications for Practice Also, with the issue of practice, Richardson (2005) stated that Student evaluation of teaching could be important indicators of what help students to learn, provided they are asked the right questions. This study will guide institutions that would want to virtualise their course evaluation process. Thereby aid in the transitioning of course evaluation from the traditional system to the web-based system. Lecturers can get to know their shortfalls in lecturing a course and how to restructure studies. This is to better deliver to its students within a shorter evaluation cycle time. Unlike the paper-based course evaluation system a good online course evaluation system will have higher response rates and meaningful feedbacks. Furthermore, concerning the issue of practice, the online course evaluation system is mostly used by the students in their first year since they see it as an obligation. The students in the subsequent years tend not to be interested in using the system because they do not see the need to continue doing evaluation if they do not see changes. Hence, higher education institutions need to continuously educate students on the importance and the need to evaluate courses. In addition, the faculty needs to pay keen attention to issues raised by students during the evaluation. University of Ghana http://ugspace.ug.edu.gh 128 8.3.5 Implications for policy Finally, concerning the significance to policy, the results of this study will aid in understanding how and why higher educational institutions migrate from the paper-based to online course evaluation system. It advocates that higher educational institutions formulate policies on the use of online course evaluation system. 8.4 Limitations of the Research Even though the discussions and findings generated may be replicated in other institutions, this study focused on the process involved in the virtualisation of course evaluation system in the a developing country’s higher educational institution identifying the enabling and constraining forces. Since higher educational institutions differ in their process, the virtualisation process of their course evaluation system may also differ. Readers must therefore be careful in generalising the findings of the research since the context might make a difference. This is a project of limited time and resources; the study is being forced to limit the scope of the topics discussed. Some aspects were discarded due to their tedious nature, others because they may be deemed less vital or too time consuming. This study was limited with only one semester of complete course evaluation data. Therefore, it is suggested further studies should include more data to support the findings of this study. The delimitation of this study is the small amount of literature on web and paper based evaluation system in the management information systems sector. University of Ghana http://ugspace.ug.edu.gh 129 8.5 Recommendation for future Research This section of the study elaborates recommendations for future research. The study drew on the use of Kurt Lewin’s force field analysis model to identify the enabling and constraining factors involved in the process virtualisation of the course evaluation system. Prior studies on the phenomenon under study indicated that, few studies have been done on virtualisation of the course evaluation system. Clearly, it is impossible to outline every possible avenue worthy of further investigation. Therefore, the following has been found particularly interesting, relevant and significant for future research. First, this study focused on a single case study with multiple embedded units. Future research should concentrate on a multiple case study. Comparing the virtualisation process of the course evaluation of two higher educational institutions in order to get a broader perspective of the phenomenon under study. Furthermore, other theories can be used in future research to identify other forces that influence why and how higher educational institutions virtualise its course evaluation process and to offer more generalisations to the findings of this study. Also, future studies may consider a quantitative approach to test the generalisability of these findings over a wider population of higher educational institutions or explore the effects of context by comparing differences in the implementation in public and private higher educational institutions. University of Ghana http://ugspace.ug.edu.gh 130 Finally, as course evaluation systems move on to more accessible technologies like mobile phones/smart phones, future studies should concentrate on identifying which forces will enable or constrain such a process. 8.6 Recommendation for Practice at University of Ghana It was noted that, students scroll through all the courses in the University to select their course code. This is difficult for the students; some students leave without finishing the evaluation process. It is recommended that, colleges should classify the courses; in this case, the student just selects his or her college and gets the drop down of all the courses in the college. That will make it easier to find courses rather than listing all the courses offered by the University in a single dropdown menu. Secondly, lecturers should play an active role in sensitizing their students to do the evaluation; this will increase the response rate. 8.7 Conclusion This study began with the aim to understand why higher educational institutions pursue the virtualisation of course evaluation process from a developing country perspective and how they develop online course evaluation system. The study further identified the constraining and enabling forces that influence higher educational institution’s decision to virtualise the course evaluation process and the development process. The study employed a higher educational institution from a developing country that has migrated their paper-based evaluation process to an online platform. Prior research on course evaluation mainly focused on the validity and reliability University of Ghana http://ugspace.ug.edu.gh 131 of course evaluations (Adam, 2009; Ahmad, et al., 2012), Online vs. paper course evaluations and effects of allowing students access to course evaluation data (Annan, Tratnack, Rubenstein, Metzler, Sawin & Hulton, 2013). Few studies, concentrated on the virtualisation of the course evaluation process, leaving that area under - studied. It is in response to limitations in the literature and research questions for this study that, this study drew on the information systems interpretive case study approach and using Kurt Lewin’s force field analysis model to understand why and how higher educational institutions virtualise its course evaluation process. The emergent forces that influence higher educational institution’s decision to virtualise their course evaluation process are reduced cycle time, cost effectiveness, accessibility, flexibility and internal environmental forces that is the increasing number of student population. Out of the above, the enabling factors that influence the virtualisation of course evaluation are internal environmental forces and ability to increase efficiency and effectiveness in processing the analyses of evaluation data. Some constraining factors influencing the process virtualisation of course evaluation are fear of low response rates, perceived lack of anonymity and authentication and over dependence on technology. Concerning the development process, some of the enabling forces identified in the study are usability, Optimizability and convenience. The identified constraining factors pointed out in this study are lack of end – user involvement during the development process, inadequate sensitization of users and the lack of policy on the use of the system. University of Ghana http://ugspace.ug.edu.gh 132 REFERENCES Abugabah, A., & Sanzogni, L. (2010). Enterprise Resource Planning (ERP) system in Higher education: A literature review and implications. International Journal of Social, Management, Economics and Business Engineering, 20 (184), 365-369. Adam, C. C. (2009). Evaluating college students' evaluations of a professor's teaching effectiveness across time and instruction mode (online vs. face-to-face) using a multilevel growth modelling approach. Computers and Education, 53(2), 429-435. Ahmad, K. M., Kamaruddin, A. T., Meor, Z. T., Shahrir, A., Hafizah, H., Dzuraidah, A. W., . . . Nasrul, A. A. (2012). Measurement of Course Evaluation for Lecturers of the Faculty of Engineering and Built Environment. Procedia -Social and Behavioural Sciences, 60, 358- 364. Alauddin, M., & Kifle, T. (2014). Does the student evaluation of teaching instrument really measure instructors’ teaching effectiveness? An econometric analysis of students’ perceptions in economics courses. Economic Analysis and Policy, 8(152), 1-13. Alessia, Z.-Y., & Crow, R. (2012). From Traditional to Non-Traditional: An Adaptive Procedure for Assessing the Instruction Librarian. The Reference Librarian, 206–218. Aliana, M., & Gunderson, B. (2003). Interactive Statistics. NJ, USA: Pearson Publication. Anderson, H. M., Jeff, C., & Eleanora, B. (2005). Online Student Course Evaluations: Review of literature and a pilot study. American Journal of Pharmaceutical Education, 69(1), 34-45 Andrade, A. D. (2009). Interpretive Research Aiming at Theory Building: Adopting and Adapting the Case Study Design. The Qualitative Report, 56(3), 42-60. Annan, S., Tratnack, S., Rubenstein, C., Metzler -Sawin, E., & Hulton, L. (2013). An Integrative Review of Student Evaluations of Teaching: Implications for Evaluation of Nursing Faculty. Journal of Professional Nursing, 29(5), 10-24. Ansah, F. (2010). Designing self-evaluation instruments for academic programmes: lessons and challenges. Evaluation & Research in Education, 23(2), 77-90. University of Ghana http://ugspace.ug.edu.gh 133 Ardalan, A., Ardalan, R., Coppage, S., & Crouch, W. (2007). A comparison of student feedback obtained through paper-based and web-based surveys of faculty teaching. British Journal of Educational Technology, 36(6), 1085-1101. Avery, J. R., Bryant, K. W., Mathios, A., Kang, H., & Bell, D. (2010). Electronic Course Evaluations: Does an online delivery system influence student evaluations? The Journal of Economic Education, 37(1) 21-37. Avery, R. J., Bryant, W. K., Mathios, A., Kang, H., & Bell, D. (2006). “Electronic Course Evaluation; Does an online Delivery System Influence Student Evaluations?". Journal of Economic Education, 34(1), 21-38. Azizah, J., Shahryar, A., Azian, A. L., Thomas, P., Wan, N. A., Narimah, A., Chan, S. L. (2011). Reliability of Student Feedback on the Course Teaching Evaluation System (CTES) and System Usability. Procedia Social and Behavioural Sciences 18 (1), 24–32. Ballantine, J. A., Galliers, R. D., & Stray, S. J. (1996). Information system/technology evaluation practices: evidence from UK organisations. Journal of Information Technology, 96, 129- 141. Bandera, W. M. (2011). A Systematic, Tool-Supported Method for Conducting Literature Reviews in Information Systems. Association for Information Systems Electronic Library, 8(1), 1- 14. Baxter, P., & Jack, S. (2008). Qualitative case study methodology: study design and implementation for novice researchers. The Qualitative Report, (13) 4, 544-559. Bemile, K. R., Jackson, A. Y., & Ofosu, J. B. (2014). Course evaluation by students at Higher Institutions: Review and Challenges: The Case of Methodist University College Ghana. Open Science Repository Computer and Information Sciences, 9(2), 1-13. Banta, D., Bologna, G., & Dzifa, I. (2014). E-learning Platforms in Higher Education. Case Study. Procedia Computer Science, 31, 1170-1176. Beran, T. N., & Rokosh, J. L. (2009). Instructors' perspectives on the utility of student ratings of instruction. Instructional Science, 37(5), 171-184. University of Ghana http://ugspace.ug.edu.gh 134 Berk, R. A. (2012). Top 20 strategies to increase the online Response rate of Student Rating scales. International journal of Technology in Teaching and Learning, 8(2), 98-107. Berk, R. A. (2013). Face-to-Face versus Online Course Evaluation: A "Consumer Guide" to Seven Strategies. Journal of Online Teaching and Learning, 9(1), 140-148. Bernard, B. (2004). Kurt Lewin and complexity theories: back to the future. Journal of Change Management, 14(3), 309-325. Bisao, R. (2009). Evolution of Information Technology in Educational management. Boston: Springer Publications. Blair, E., & Kimila, I. (2014). Student Evaluation questionnaire and the developing world: An examination of the move from hard copy to online modality. Studies in Educational Evaluation 40, 36-42. Boateng, R. (2014). Research Made Easy. Accra: Pearl Richards Foundation. Boateng, R. (2011). Analysing Current Systems. Accra, Greater Accra, Ghana. Bothell, T. W., & Henderson, T. (2003). Do online ratings of instruction make sense? Online student ratings of Instructions, 8, 69 -79. Bozak, M. G. (2003). Using Lewin’s force field analysis in implementing a nursing information system. CIN: Computers, Informatics, Nursing, 11(2), 80-85. Burnes, B., & Cooke, B. (2013). Kurt Lewin’s Field Theory: A Review and Re-evaluation. International Journal of Management Reviews, 8(1), 408-425. Bygstad, B., & Munkvold, E. B. (2011). Exploring the role of informants in interpretive case study research in IS. Journal of Information Technology, 37(1), 32-45. Carini, R. M., Hayek, J. C., Kuh, G. D., & Ouimet, J. A. (2003). College Student responses to web and paper surveys: Does Mode matter? Research in Higher Education, 69(1), 1-19. Chatvichienchai, S. (2011). WCourEva: Web-Based Course Evaluation System using XML Technologies. Procedia Computer Science, 3, 23-28. University of Ghana http://ugspace.ug.edu.gh 135 Chen, Y., & Hoshower, L. (2003). Student evaluation of teaching effectiveness: an assessment of student perception and motivation. Assessment and Evaluation in Higher Education, 28(1), 72-88. Coleman, J., & McKeachie, W. (1981). Effects of Instructor/Course Evaluations on Student Course Selection. Journal of Educational Psychology 73, 224-226. Creswell, J. W. (2009). Research Design; Qualitative, Quantitative and Mixed Method Approaches. United States of America: Sage Publications. Cronshaw, S. F., & McCulloch, N. A. (2008). Reinstating the Lewinian vision: From force field analysis to organisation field assessment. Organisation Development Journal, 10(2), 89- 103. Darke, P., Shanks, G., & Broadbent, M. (1998). Successfully completing case study research: combining rigour, relevance and pragmatism. Information Systems Journal, 8, 273-289. David, A. L., & Matthew, J. K. (2010). Grades, Course Evaluations, and Academic Incentives. Eastern Economic Journal, 30(6), 151–163. David, S. F., Denise, J. D., & Robert, J. C. (2010). Online vs. Paper Evaluations of Faculty: When Less is Just as Good. The Journal of Effective Teaching, 238(1) 42-54. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 11, 319–340. Dawson, M., Burrell, D. N., Rahim, E., & Brewster, S. (2010). Integrating Software Assurance into the Software Development Life Cycle (SDLC). Journal of Information Systems Technology & Planning, 49-53. De Hart, F. (1996). Teacher Evaluation by students. Journal of Education for Librarianship, 33- 38. Debuse, J., Lawley, M., & Shibil, R. (2007). The Implementation of an Automated Assessment Feedback and Quality Assurance System for ICT Courses. Journal of Information Systems Education, 18(4), 491-502. Dennis, A., Wixom, H. B., & Tegarden, D. (2009). System Analysis Design UML Version 2.0 An object Oriented Approach. John Wiley & Sons: United Sates of America. University of Ghana http://ugspace.ug.edu.gh 136 Dommeyer, C. J., Baum, P., Chapman, K. S., & Hanna, R. W. (2002). Attitudes of Business Faculty Towards Two Methods of Collecting Teaching Evaluations: Paper vs. Online. Assessment & Evaluation in Higher Education, 29(5), 455 -462. Donovan, J., Mader, C., & Shinsky, J. (2007). Online vs. Traditional Course Evaluation Formats: Student Perceptions. Journal of Interactive Online Learning, 6(3), 158-180. Donovan, J., Mader, C. E., & Shinsky. A. J. (2006). "Constructive Student Feedback: Online vs. Traditional Course Evaluations. Journal of Interactive Online Learning, 5(3), 283-296. Dorit, N., McClean, R., & Nevo, S. (2012). Harnessing Information Technology to Improve the Process of Students’ Evaluations of Teaching: An Exploration of Students’ Critical Success Factors of Online Evaluations. Journal of Information Systems Education, 21(1), 13-21 Dwivedi, K. Y., Wade, R. M., & Scott, L. S. (2012). Information Systems Theory: Explaining and Predicting Our Digital Society. Integrated Series in Information Systems, 17(2), 107-124. Etin, I. (2013). Management of Academic Information System (AIS) at Higher Education in the City of Bandung. Social and Behavioural Science, 43(4), 628-636. Farag, S., Schwanen, T., Dijst, M., & Faber, J. (2007). Shopping online and/or in-store? A structural equation model of the relationships between e-shopping and in-store shopping. Transportation Research Part A, 41, 125–141. Feldman, K. A. (1978). Course characteristics and college students’ ratings of their teachers and courses: What we know and what we do not. Research in Higher Education 9, 199-242. Garcia-Valls, M., Cucinotta, T., & Lu, C. (2014). Challenges in real-time virtualisation and predictable computing. Journal of Systems Architecture, 12, 726-740. Ghana US Embassy. (2015, April 9). Education of Ghana. Retrieved from ghana.usembassy.gov: http://ghana.usembassy.gov/root/pdfs/wwwfeducationofghana.pdf Green, J., Willis, K., Hughes, E., Small, R., Welch, N., Gibbs, L., & Daly, J. (2007). Generating best evidence from qualitative research: the role of data analysis. Australian and New Zealand Journal of Public Health, 3(37), 545 -550. Grzegorz, M. (2012). Virtualisation of Marketing. Contemporary Management Research, 195- 204. University of Ghana http://ugspace.ug.edu.gh 137 Guba, E., & Lincoln, Y. (1994). Competing paradigms in qualitative research. In K. Denzin, & Y. Lincoln, Handbook of Qualitative Research. California: Sage Publications. Hardré, P., & Cox, M. (2009). Evaluating faculty work: expectations and standards of faculty performance in research universities. Research Papers in Education 24 (4), 383-419. Handwerk, P., Carson, C., & Blackwell, K. (2000). Online vs. Paper and Pencil Surveying of students: a case study. Proceedings of the 40th Annual Forum of the Association of Institutional Research (pp. 49-59). Cincinnati, OH: New Directions for Teaching and Learning, No. 96. Harrington, F. C., & Saxon, G. R. (2005). Online Student Evaluation of teaching for Distance Education: A Perfect match. The Journal of Educators Online, (2) 1, 1-12. Heath, M. N., Lawyer, R. S., & Rasmussen, B. (2014). Web-Based versus Paper -Pencil Course Evaluations. Teaching of Psychology, 34(4), 259-261. Higher Education Act of 1965. (2014, November 5). http://www.nrcyd.ou.edu/etv/defining- institutions. Retrieved from nrcyd.ou.edu: http://www.nrcyd.ou.edu/publication- db/documents/higher-education-act-1965.pdf Hmieleski, K. H. (2000). Barriers to on-line evaluation. Polytechnic Institute Interactive and Distance Education Assessment (IDEA) Laboratory, 34 1-10. Hoffman, K. M. (2003). Online Course Evaluation and Reporting in Higher Education. New Directions for teaching and learning, 96, 25-29. Isaias, P., & Issa, T. (2015). High Level Models and Methodologies for Information Systems. New York: Springer Science and Business Media. Jalaliyoon, N., & Taherdoost, H. (2012). Performance Evaluation of Higher Education; a necessity. Procedia -Social and Behavioural Sciences, 46(1983), 5682-5686. Johnson, T. D. (2003). Online Student Ratings. New Directions For teaching and learning, 3(37), 49-59. Johnson, D. T. (2003). Online Student Ratings: Will Students Respond? New Directions for Teaching and Learning, 96, 49-59. University of Ghana http://ugspace.ug.edu.gh 138 Jolliffe, A., Ritter, J., & Stevens, D. (2001). The Online Learning Handbook. London: Korean Page Limited. Jonghak, S., & James, T. C. (2012). Information Systems Use: Construct Conceptualisation and Scale development. Computers in Human Behaviour, 31(2), 1564-1574. Kaminski, J. (2011). Theory applied to informatics-Lewin’s change theory. CJNI: Canadian Journal of Nursing Informatics, 37, 1-4. Kennedy, P., & Siegfried, J. J. (1997). Class size and Achievement in Introductory economics: Evidence from the TUCE III Data. Economics of Education Review 16(4), 385-394. Ketlhoilwe, M. (2006). Course Evaluation in Adult Education Programmes In support of environmental and sustainability education processes. South Africa: SADC Regional Environmental Education Programme Khorsandi, M., Kobra, A., Ghobadzadeh, M., & Seifei, M. (2012). Online vs. traditional teaching evaluation: a cross-sectional study. Procedia -Social and Behavioural Sciences, 46, 481- 483. Klein, H., & Meyers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67-93. Kothari, C. R. (2004). Research Methodology: Methods and Techniques. New Delhi: New Age International Publishers. Krauss, S. E. (2005). Research paradigms and Meaning Making: A Primer. The Qualitative Report, 758-770. Krishnaveni, R., & Meenakumari, J. (2012). Usage of ICT for information administration in higher education institutions -A study. International Journal of Environmental Science and Development, 1(3), 282-286. Kuhn, T. (1970). The Structure of Scientific Revolutions. Chicago: University of Chicago Press. Kumar, A. B. (2011). Thin Client Web-Based Campus Information Systems for Fiji National University. International Journal of Software Engineering & Applications, 30(6) 13-26. Lamer, S., & Eckhart, A. (2012). Why Do People Reject Technologies: A Review of User Resistance Theories? In Y. K. Dwivedi, L. M. Wade, & S. L. Schneberger, Information University of Ghana http://ugspace.ug.edu.gh 139 Systems Theory; Explaining and Predicting Our Digital Society (pp. 63-86). London: Springer Publications. Lee, A. (1991). Integrating positivist and interpretive approaches to organisational. Organisation Science, 13(1), 342-365. Leong, l. (2004). Improving Students' Interest in Learning: Some positive techniques. Journal of Information Systems Education, 16(2), 129-132. Levy, Y., & Ellis, T. J. (2006). A Systems Approach to Conduct an Effective Literature Review in Support of Information Systems Research. Informing Science Journal, 4, 181-212. Liegle, J. (2005). Lessons Learned From Online vs. Paper-based Computer Information Students' Evaluation System. Information Systems Education Journal, 3(37), 1 - 14. Liegle, o. J., & McDonald, D. S. (2005). Lessons learned from Online vs. Paper-based Computer information student evaluation System. Information Systems Journal, 3(37), 2-14. Lin, C., & Previn, G. (2001). A Review of IS Evaluation and Benefits Management Issues, Problems and processes. USA: Idea group Publishing. Love, A. D., & Kotchen, J. M. (2010). Grades, Course Evaluations and Academic incentives. Eastern Economic Journal, 36(2), 151-163. Marlin, J. W. (1987). Students Perceptions of End of Course Evaluations. The Journal of Higher Education, 58, 704-716. Marsh, H. W. (1987). Students’ evaluations of university teaching: research findings, methodological issues, and directions for future research. International Journal of Education Research, 11, 253-388. Marsh, H. W., & Bailey, M. (1993). Multidimensional students' evaluations of teaching effectiveness: A profile analysis. Journal of Higher education 64, 1-18. McIlroy, R., & Sventek, J. (2006). Resource virtualisation of network routers. Workshop on High Performance Switching and Routing (HPSR), (pp. 7-9). Poznan, Poland. Miles, B. M., & Huberman, A. M. (1994). Qualitative Data Analysis. United States of America: Sage Publications. University of Ghana http://ugspace.ug.edu.gh 140 Mingers, J. (2004). Realising information systems: critical realism as an underpinning. Information and Organisation, 14, 87-103. Miyoung, K., & Chiyoung, C. (2013). Cultural meaning of course evaluation among nursing students and faculties in Korea. Nurse Education Today, 18(2), 639-643. Mohanty, G., Gretes, J., Flowers, C., Algozzine, B., & Spooner, F. (2005). Multi -method of evaluation of instructions in engineering classes. . Journal of Personnel Evaluation 18(2), 139 -151. Moss, J., & Hendry, g. (2002). Use of Electronic Surveys in Course Evaluation. British Journal of Educational Technology, 33(5) 583-592. Myers, D. M. (1997). Qualitative Research in Information Systems. MIS quarterly, 1-10. Nasser, F., & Barbara, F. (2002). Faculty Views of Student Evaluation of College Teaching. Assessment & Evaluation in Higher Education 27, 187-198. Nicole, M. H., Steven, R. L., & Erin, B. R. (2007). Web-Based Versus Paper-and-Pencil Course Evaluations. Teaching of Psychology, 11, 259-261.Nikolass, Y., & Dimitriadis, G. S. (2014). On the student evaluation of University courses and faculty members' teaching performance. European journal of operations Research, 238(1), 199-207. Nikolaidis, Y., & Dimitriadis, S. (2014). On the student evaluation of university courses and faculty members'. European Journal of Operational Research 238, 199 - 207. Norris, J. a. (2005). Investigating Strategies for Increasing Student Response Rates to Online- Delivered Course Evaluations. Quarterly Review of Distance Education 6, 13-29. Noyes, M. j., & Garland, J. k. (2008). Computer vs. Paper -based Tasks: Are they equivalent. Ergonomics, 51(1), 1352-1375. Orlikowski, W., & Baroudi, J. (1991). Studying information technology in organisations: research approaches and assumptions. . Information Systems Research, 12, 1-28. Overby, E. (2008). Process Virtualisation Theory and Impact of Information Technology. Organisation Science, 277-291. Sutherland Phelps, J., & Reynolds, R. (1999). Formative evaluation of Web-based course in meteorology. Computers and Education, 57, 181-193. University of Ghana http://ugspace.ug.edu.gh 141 Pounder, J. S. (2007). Is student evaluation of teaching worthwhile? An analytical framework for answering the question. Quality Assurance in Education, 15(2), 178-191. Punch, F. K. (1995). Introduction to Research Methods in Education. Los Angeles: Sage Publications. Ritchie, J., Lewis, J., Nicholls, M. C., & Ormston, R. (2003). Qualitative Research Practice; A guide for social science students and researchers. Los Angeles: Sage publications Inc. Robey, D. (1979). User Attitudes and Management of Information System use. Academy of Management Journal, 14, 527-238. Saadé, R. G. (2003). Web-Based Educational Information System for Enhanced Learning, EISEL: Student Assessment . Journal of Information Technology Education 2, 267-277 Salaheldin, S. I. (2003). The Implementation of TQM strategy in Egypt: a field-force analysis. The TQM Magazine, 266-274. Sanyal, B. C. (1994). The use of computerized information systems to increase efficiency in University management. Paris: International Institute of Educational Planning. Saunders, M., Lewis Philip, & Thornhill Adrian. (2009). Research Methods for Business Students. England: Pearson Education Limited. Shu, J., Li, B., & Zheng, W. (2005, February). Design and Implementation of a Storage Virtualisation System Based on SCSI Target Simulator in san. Tsinghua science and technology, 10(1), 122-127. Somchai, C. (2011). WCourEva: Web-based course evaluation system using XML technologies. Procedia Computer Science 3, 23–28. Spencer, K. J., & Schmelkin, L. P. (2002). Student perspectives on teaching and its evaluation. Assessment and evaluation in Higher Education, 397-409. Stanley, T. D. (2001). Wheat from Chaff: Meta-Analysis as Quantitative Literature Review. Journal of Economic Perspectives, 131-150. Sumsion, J. (2011). Capacity building in early childhood education research in a regional Australian university. British Journal of Educational Studies 59 (3), 265 -284. University of Ghana http://ugspace.ug.edu.gh 142 Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y., & Yeh, D. (2008). What drives a successful e - learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers and Education, 1183-1202. Sutherland, K. (2013). Applying Lewin’s Change Management Theory to the Implementation of Bar-Coded Medication Administration. Canadian Journal of Nursing informatics, 15, 21- 27. Swanson, D. J., & Creed, A. S. (2014). Sharpening the Focus of Force Field Analysis. Journal of Change Management, 19, 28-47. Timothy, J. F., Gregory, A. J., & Larry, M. P. (2013). The medium is the message: comparing paper -based and web based course evaluation modalities. J. of Acc. Ed., 177-193. Trochim, W. (2006, August). Socialresearchmethods.net. Retrieved from socialresearchmethods.net: http://www.socialresearchmethods.net/kb/interval.htm Trochim, M. K. (1985). Pattern Matching, Validity and Conceptualisation in Program Evaluation Review, 121, 575-604. University of Ghana. (2015, April). Overview. Retrieved from UG.EDU.GH: https://www.ug.edu.gh/about/overview University of Ghana. (2006). Mission statement. Retrieved from University of Ghana: http://www.ug.edu.gh/index1.php?linkid=243&sublinkid=68 University of Ghana. (2009, May). Academic Quality Assurance Policy. Accra, Greater Accra, Ghana. University of Ghana Academic Quality Assurance Unit. (2014). About Statement. Retrieved from About the Academic Quality Assurance Unit: http://www.ug.edu.gh/aqau/about-academic- quality-assurance-unit University of Ghana Special Reporter. (2011). Academic Quality Assurance Policy December 2009. Accra: University of Ghana. Venkatesh, V. M. (2003). User acceptance of information technology: Toward Unified View. MIS Quaterly, 425-478. University of Ghana http://ugspace.ug.edu.gh 143 Vom Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., & Cleven, A. (2009). Reconstructing The Giant: On The Importance Of Rigour In Documenting The Literature Search Process. 1-13. Walsham, G. (2006). Doing interpretive research. European Journal of Information Systems, 320 -330. Walsham, G. (1995). Interpretive case study in IS research: nature and method. European Journal of Information Systems, 6(4), 74-81. Wang, Y.-S. (2003). Assessment of leaner Satisfaction with asynchronous electronic learning systems. Information and Management, 8, 75-86. Wang, Y. -S., Wang, H.-Y., & Shee, Y. D. (2007). Measuring e-learning systems success in an organisational context: Scale development and Validation. Computers in Human Behaviour, 185, 1792-1808. Weber, R. (2004). The Rhetoric of positivism verses interpretivism: a personal view. MIS Quarterly, 3-12. Wilhelm, W. B. (2004). The Relative Influence of Published Teaching Evaluations and Other Instructor Attributes on Course Choice. Journal of Marketing Education, 26, 17-30. Wode, J., & Keiser, J. (2011). Online Course Evaluation Literature Review and Findings. Columbia: Academic Affairs Columbia College Chicago. Word, J., & Magal, R. S. (2009). Essentials of Business Processes and Information Systems. United States of America: John Wiley & Sons Inc. Wright, R. E. (2006). Student Evaluations of faculty: Concerns raised in the literature and possible solutions. College Student Journal, 14, 417-422. Wu, H. Y., Lin, Y. K., & Chang, C. H. (2011). Performance evaluation of extension education centres in universities based on the balanced scorecard. Evaluation and Program Planning 34, 37-50. Yin, R. (1994). Case Study Research, Design Methods, second Edition. Newbury Park: Sage Publications. University of Ghana http://ugspace.ug.edu.gh 144 Zaidah, Z. (2007). Case study as a research method. Journal Kemanusiaan, 8, 1-6. University of Ghana http://ugspace.ug.edu.gh