Number 25 October 1, 2002	A publication of Project Eagle, St. Petersburg College

Best Bets Archives

For a subscription to BEEP, contact the Project Manager: lechnerj@spcollege.edu

Evaluating Quality and Effectiveness of E-Courses

"...to simply focus on the technologies will lead people to continually question the validity of the practice of distance education."
(Rick Shearer, Distance-Educator.com's Daily News, 9/15/02)

Although the definitive word has yet to be written on the best methods for evaluation of e-courses, the subject is under much scrutiny at this time. The links that follow represent just some of the approaches to this difficult but essential part of the e-learning process, identified as the #1 issue in the annual Top Ten Issues in Teaching Online Survey, from TeachingOntheNet.org.

General Information

"Assessment Takes Center Stage in Online Learning." Article by Dan Carnevale in the Chronicle of Higher Education, April 13, 2001. An overview of the efforts of various e-learning institutions to evaluate the quality of their efforts by the use of assorted assessment tests.
"Distributed Education in the 21st Century: Implications for Quality Assurance." Article by Wallace K. Pond, Education America Online, in Online Journal of Distance Learning Administration 5(2), Summer 2002. A look at the new meaning of quality resulting from e-learning.
Evaluation of Online Courses. University of Georgia Web site with links to a number of studies on assessment of the quality of both e-course design and learning outcomes.
"A New Methodology for Evaluation: The Pedagogical Rating of Online Courses." Article by Nishikant Sonwalker in Highlights from Syllabus Magazine, January 2002, that proposes an instrument for evaluating e-courses based on elements of the author's "learning cube" structure.
"Toward an Effective Quality Assurance Model of Web-Based Learning: The Perspective of Academic Staff." Article by Davey Young, Open University of Hong Kong, in Online Journal of Distance Learning Administration 5(2), Summer 2002. An attempt to establish and understand the key factors or elements of quality in the Web-based learning environment.
Who Should Evaluate? A PowerPoint presentation done at the University of Illinois that provides a detailed outline on the who, what and when of effective online course evaluation.

Standards and Criteria for Evaluation

Asynchronous Online Learning Instructor Competencies. From the Learning Peaks Web site, four needed competencies for online faculty: administrative, facilitation, technical, and evaluation.
Criteria for Evaluating Online Courses. A list of suggested criteria for assessment posted on the Southern Regional Education Board's Web site.
"Development of Standards or Criteria for Effective Online Courses." Article in Educational Technology & Society 2(3), 1999, that summarizes comments from a discussion by professionals on the development of criteria or standards for online courses, aimed at higher education. It includes a lengthy bibliography with links to good sources for standards development.
"Evaluating Individual Student Learning: Implications from Four Models of Assessment." Article by Mary Hjelm and Ronald L. Baker in Learning Abstracts 4(3): May 2001. The models examined are graduated scales, foundations of assessment, taking stock, and outcomes.
An Instructional Design Rubric for Secondary Level Web-based Courses. Designed for use in reviewing Web-based courses for secondary students by the Maryland State Department of Education and adapted from work originally done by the Southern Regional Education Board.
Standards for Quality Online Courses. An extensive and detailed collection of standards for judging course quality created by the Michigan Virtual University. Categories include Technology Standards, Usability Standards, Accessibility Standards and Instructional Design Standards.

Faculty Evaluation of Student Learning

"'A' Is for Assessment: Identifying Online Assessment Practices and Perceptions." Article by Mary I. Dereshiwsky in Education at a Distance 15(10), January 2001. A study that identifies key issues for assessment of students' performance in Web-based and Web-enhanced courses.
Assessment Tools for Distance Education. Article by Susan B. Millar, University of Wisconsin, in DESIEN 6(3), March 2001, that explains the evaluation process and links to the Field-tested Learning Assessment Guide (FLAG) Web site, a one stop resource for faculty who want to ascertain how well their strategies are working.
Evaluation for Distance Educators. University of Idaho Engineering Outlook Guide #4. Suggestions for methods of evaluation best suited to the e-course setting.
"How Interactive Are YOUR Distance Courses? A Rubric for Assessing Interaction in Distance Learning." Article by M. D. Roblyer and Leticia Ekhaml, State University of West Georgia, in Online Journal of Distance Learning Administration 3(2), Summer 2000. A detailed assessment tool with four parts for evaluation of a course's level of interaction.
"Methods of Assessing Learning in Distance Education Courses." Article by Jamie Morley in Education at a Distance 13(1), January 2000, that evaluates several synchronous and asynchronous methods for faculty assessment of learning outcomes in e-courses.

Student Evaluation of Faculty Effectiveness

Findings from the California Community College Flashlight Project, 1998-99. Study by Julie Slark, Robert S. Gabriner and Susan S. Obler that investigated student attitudes and found a face-to-face/e-course combination is the most useful and exciting to students. (ERIC Document ED458916 available in full-text through E-Subscribe, abstract only to others.)
Researching Distance Education: Penn State's Online Adult Education Med Degree on the World Campus. Report by Mary Simpson and Eunice Askov on student survey results that showed the progression of comfort in using technology from the start to the end of course. (ERIC Document ED456256 available online in full-text through E-Subscribe, abstract only to others.)
"Web-Based Evaluations Let Instructors Poll Their Students Anonymously." Article by Janice Paskey in The Chronicle of Higher Education, January 9, 2002. It discusses the FAST - Free Assessment Summary Tool, a Canadian Web site that offers e-structors a simple online tool for ongoing assessment of students' impressions of courses and teachers.

Examples of E-Course Evaluations

Assessment of Online Courses and Programs. Web site maintained by Emporia State University with links several examples of evaluation tools developed at the university.
Can Web Courses Replace the Classroom? Lessons from Microeconomics. Paper by Byron W. Brown and Carl Liedholm of Michigan State University that in 2002 evaluated student performance in three modes of instruction. E-course students had the lowest test scores.
"Distance Education: Better, Worse, Or As Good As Traditional Education?" Article by Shelia Tucker, East Carolina University. In Online Journal of Distance Learning Administration 4(4), Winter 2001. Results of a comparison of traditional students and e-learners in a business communications class that showed no significant difference in any category analyzed.
Project Eagle Research Capsules (PERC), St. Petersburg College. Results of Student Survey of Online Instruction, Number 4, January 2002, compares student satisfaction in traditional and e-courses. Trends in E-Student Population, Number 2, June 2001, includes a comparison of student performance in traditional and online statistics courses over 3½ years.
"Stronger Students Benefit from Online Courses, Texas Study Finds." Article by Dan Carnevale in The Chronicle of Higher Education, July 19, 2002, on psychology e-course students who got better grades if they had technology skills when they started, but not more enjoyment.
"Study: Missouri's Ed-Tech Program Is Raising Student Achievement." Article by Cara Branigan in EdNet Distance Education Newsletter, March 14, 2002, that showed elementary school students using technology in classes scored higher in every subject on standardized tests.
"A Survey Compares 2 Canadian MBA Programs, One Online and One Traditional." Article by Janice Paskey in The Chronicle of Higher Education, April 26, 2001, about a survey at the University of Western Ontario that found the online classroom better for communication.

The contents of BEEP were developed under a grant from the U. S. Department of Education (DOE). However, those contents do not necessarily represent the policy of the DOE, and you should not assume endorsement by the Federal Government.