Benchmarking St. Petersburg College:
A Report to Leadership
Project Eagle Evaluation Question #6
How Can St. Petersburg College Parlay Its Good Work and Success with Project Eagle into a National Model?
Submitted by Joyce Burkhart
Coordinator, Research, Evaluation and Dissemination
Project Eagle
June 30, 2002
www.spjc.edu/eagle/research/evaluation/peeq6.htm
Introduction
To formulate an evaluation process of
e-learning practices at St. Petersburg College
(SPC) for Project Eagle, external
evaluator Dr. Gordon "Spud" Van de Water, Education Commission of the States,
suggested that the college first identify critical issues. Dr. James Olliver,
Project Eagle director, with the help of the Project Eagle Action Committee,
formerly known as the Project Eagle Working Group, formulated six questions,
which were then worked into an evaluation plan by
Dr. Van de Water and Joyce Burkhart, Coordinator of Research, Evaluation and
Dissemination.
The decision was made to consider one question per quarter
for the next 18 months, taking the following steps:
- First, examine best e-learning practices related to that question, both nationwide and worldwide, using the Web as the primary source of information. The results of
this external evaluation would be published in an issue of Project Eagle's monthly newsletter, Best
Educational E-Practices (BEEP).
- Next, compile a list of all practices related to the question currently in use at SPC, using a variety of appropriate techniques and strategies.
- Compare the best external practices with those offered at SPC.
- Benchmark SPC and submit a report to the college leadership on the college's performance in the area under consideration.
- Finally, disseminate the results nationally, using the Project Eagle Web site and other forms of information distribution.
This report represents Step 4, incorporating the results of Steps 1-3.
Background
The sixth and final question to be evaluated addressed ways
in which St. Petersburg College (SPC) could transform the eCampus
accomplishments made possible by Project Eagle into a national model.
In January 2002, research was completed to determine the
elements involved in becoming a national model. In February 2002, the results
were published in Best
Educational E-Practices (BEEP), Number 17, "Becoming a National Model for
E-Learning."
Two elements to accomplish this, one directed internally,
the other to the outside world, were identified at that time:
- Evaluating, Assessing and Improving an E-Learning Program
- Informing the Educational Community
From April to mid-June, 2002, SPC's achievements in these two areas were examined.
These steps included the following:
- Determining what is being done internally in terms of evaluation, assessment and dissemination of SPC's e-learning program, by individuals, departments, and the college as a whole.
- Taking an in-depth look at one exemplary subject area, medical education, as a national model in e-learning and reporting the findings in "Online Support for Medical Education: A Best Educational E-Practice," Best Educational E-Practices (BEEP), Issue 20, June 1, 2002.
- Determining the extent to which Project Eagle successes have been disseminated, including a list of external Web sites that regularly link to Best Educational E-Practices (BEEP), in "Like Minds: Web Sites That Link to and from BEEP," Issue 20, May 1, 2002.
In June 2002, the college's activities were compared with
what others considered national models are doing, and this report was
completed. An executive summary appears in Best Educational E-Practices, Issue 22, July 1, 2002.
Results
A. Evaluating, Assessing, and Improving an E-Learning Program
Nationwide/worldwide. There are a number of methods of program
evaluation that have been developed and implemented nationwide. Some of the
most relevant to an evaluation of SPC's e-learning program, particularly as
funded through Project Eagle, have been done by the following institutions and
organizations.
- Basic Guide to
Program Evaluation (The Management Assistance Program for Nonprofits).
Written by Carter McNamara, this extensive guide recommends the following
ingredients for success in assessing a program:
- Planning the evaluation. Consider the purpose, audience, kinds of information needed, sources from which to collect the information, methods of collection, time needed to complete it, and resources available to do the job.
- Choosing the type of
program evaluation. Once the considerations above have been addressed, it will
be fairly easy to determine the type of evaluation: goals-based (are programs
meeting their overall, predetermined objectives?); process-based (understanding
how a program really works, strengths and weaknesses); or outcomes-based (identifying
the benefits to clients).
- Selecting the methods
of collection. The major methods suggested are questionnaires, surveys,
checklists, interviews, documentation review, observation, focus groups and
case studies.
- Determining the level
of evaluation. This can range from reactions and feelings to learning, changes
in skills and improved performance.
- Handling other
elements of evaluation. These include deciding how to report the findings,
assigning the responsibility for the evaluating, and creating the content of
the plan.
- Avoiding pitfalls.
Among other things, evaluators are advised not to seek the perfect evaluation
plan, to work hard to include some interviews, successes and otherwise, and not
to throw away the results once the report is generated.
- Evaluation for Distance Educators (University of Idaho College of Engineering). A somewhat different approach to program evaluation, this university defines the elements of evaluation as follows:
- Types of evaluation.
An evaluation can be either formative, summative, or a combination of both.
Formative evaluations are ongoing, will enable improvement while proceeding,
and identify major gaps or the need for minor adjustments. Summative
evaluations assess overall effectiveness of the finished product, and can be a
springboard for developing a revision plan or a baseline of information for
designing a new plan or program.
- Methods of
evaluation. Data may collected through quantitative and qualitative methods.
The quantitative method involves asking questions that can be statistically
tabulated and analyzed and limiting responses to certain categories. Large
samples are needed for relevant statistical analyses. Drawbacks include the
relatively small populations in some e-learning courses or subject areas, a low
rate of return, and a limited number of possible response options. It does not
lend itself well to formative evaluation because of the difficulty of
collecting data, and often results in "an illusion of precision that may be far
from reality."
Qualitative evaluation is typically more subjective,
involves gathering a wider range and depth of information, is more difficult to
tabulate into neat categories, will be less affected by small class size, and
is more flexible and dynamic and less limited to topics. It can be accomplished
through open-ended questions, participant observation, content analysis and
interviews.
- Content of evaluation. In e-learning assessment, use of technology, class formats, class atmosphere, quantity and quality of interactions with other students and instructor, course content, quality of assignments and tests, support services, student achievement and attitude, and instructor effectiveness can be examined.
- Evaluation tips. Some of the recommendations are to adapt questionnaires already published; draft and revise questions, as necessary; position questions effectively; assure anonymity, especially on summative evaluations; and try to get both positive and negative feedback.
- Planning a Program
Evaluation (University of Wisconsin Extension). The plan for program
assessment at this university, authored by Ellen Taylor-Powell, Sara Steele and
Mohammed Douglah, falls into four parts:
- Focusing the
evaluation. Focusing involves deciding what to evaluate, the purpose of the
evaluation, the users and use of the results, the questions to ask, how to
answer the questions (key indicators), and the resources needed to do the job.
- Collecting the
information. Collection includes determining what will be used in terms of
sources of information, collection methods, and collection procedures.
- Using the
information. Questions to consider are how the data will be analyzed, how it
will be interpreted and by whom, and how the results will be communicated and
shared.
- Managing the
information. The actual implementation of the evaluation involves determining,
the money, time and personnel that will needed.
- Suggested Steps in
Designing a Program Assessment Plan of Student Learning (Portland State University
[OR]). The university offers a series of decisions that need to be made before
beginning a program evaluation:
- Determine program goals and objectives.
- Determine which courses address each of the goals.
Steps c, d, e and f might need to be done simultaneously or in another order.
- Determine time points in which you would like to collect data on student learning.
- Determine the type of data you wouldlike to collect and how it will be evaluated.
- Determine how students will receive feedback.
- Determine who will review the information and how it will be used for program changes.
- Determine where
you will collect the data.
- Ten Steps for Installing
an Effective Assessment Program (Southern Illinois University). A simple
outline is patterned after one done by James L. Ratcliff, Director, National Center
on Postsecondary Teaching, Learning, & Assessment, Penn State:
- State the educational purpose of the assessment program.
- Identify the audiences to which the plan is to communicate.
- Set assessable educational goals.
- Identify the educational experiences and criteria for attaining each goal.
- Select indicators and measures of goal attainment.
- Establish the standards of performance for each measure.
- Interpret and report to audiences.
- Revise criteria, measures, and method.
- Identify strategies for change. Improve.
SPC. Because of the nature of the Project
Eagle grant, goals were set at the outset that have made ongoing evaluation
and improvement easier than that of other institutions. The original work
statement for the grant was very specific in intention and methods:
WORK STATEMENT
Synopsis
Project Eagle is a
multi-year strategic initiative by St.
Petersburg College to build a national model for increasing access to four-year
degrees and workforce training for students attending community colleges.
Access is enhanced by educational opportunities that are increasingly flexible
-- with courses; programs and support services to be delivered at a time and
place and in a way at a pace best suited to the needs of the individual
learner.
SPC intends to:
- develop, implement and evaluate over 160 technology-mediated
(primarily web-based) associate degree and certificate courses as well as
web-based and traditional baccalaureate courses in conjunction with four-year
college and university partners;
- develop, deliver and assess innovative, integrated student and
academic support services for the upper-division, distance, and other
"flexible access" students (e.g., open-entry/open exit); and,
- determine "best practices" for enhancing student
learning via technology-mediated instruction, incorporate the findings into SPC
program development, and disseminate the results of outcomes and evaluation
findings nationally -- all with a focus on creatively leveraging technology for
cost efficient and effective access.
SPC will accomplish these objectives by:
- building the VITAL (Video and Internet Technologies to Advance
Learning) Network -- a team of professionals operating within the College's
Instructional Technology organization to develop the on-line programs and
services;
- creating the College University Center (CUC) hubbed at the
Seminole Campus but with access points (especially via distance technologies)
at other College sites;
- developing the "Electronic College" as
the point of integration and access for all distance courses, and the
"one-stop shop" for information and services for distance students;
- establishing a model training center for flexible delivery of
high technology certificate programs; and,
- providing the infrastructure, equipment and robust support
(e.g., 24/7 help desk) to provide ready access to upper-division and workforces
courses and programs throughout Pinellas County and
beyond."
Additionally,
the grant spells out specific objectives and activities, which have been broken
down into project deliverables
(with a timetable for completion of each) on which Eagle-funded staff report
every month:
Project Objectives and Activities
The specific objectives of Project Eagle are as follows:
-
Determine "best practices" for enhancing student
learning via technology-mediated instruction.
- review "learning style" research (panel of on-line
national experts)
- evaluate use of technology-mediated and at-a-distance programs
matching student success and learning styles
- compile and synthesize information on current leading edge
instructional technologies (panel of on-line national experts)
- develop methodologies for identification and measurement of
skill competencies and learning outcomes (i.e., student progress, success and
satisfaction measured through on-line assessment instruments)
- provide a forum for R&D in technology innovation (panel of
on-line national experts)
- develop, test and revise a quality review process for evaluating
courses offered through the Electronic Campus
- monitor the FIPSE Learning Anytime Anywhere Partnership (LAAP)
grants, with an eye toward providing a different approach to the same set of
issues -- one institution forging multiple partnerships to address the full
spectrum of challenges raised in the LAAP RFP
- research and attempt to define at what "critical mass"
of course and program offerings the institution can cost effectively support
the infrastructure and array of complementary academic and support service
costs
- Develop, deliver, assess and disseminate (D/d/a/d) findings
re: selected four-year and workforce programs and courses in different learning
modalities
- conduct needs assessments on proposed workforce and upper division program options
- D/d/a/d courses for a general transfer A.A. degree in an on-line format
- D/d/a/d business courses in the A.A. or A.S. (preparation forbusiness baccalaureate) track in an on-line format
- identify and address high volume and "bottleneck"
courses and D/d/a/d them if they are not captured in the general or business
tracks
- produce a web-supported ethics telecourse offered as part of a
general education requirements in all A.A. and A.S. programs
- D/d/a/d targeted A.S. degree or certificate programs based on
the needs assessments
- D/d/a/d multiple types of "blended" courses where
traditional "seat-time" courses are modified to supplement some or
most of the traditional elements with "open-entry, open-exit",
just-in- time" and other "distance" learning paradigms
- D/d/a/d systems for distribution video course materials to
desktop computer throughout
the County
- work with university administrators and faculty to identify
courses, appropriate learning modalities, and assessment tools
- work with university faculty and staff on curriculum,
communications and service options
- Develop, deliver, assess and disseminate information
regarding innovative student and academic support services for the upper
division, at-a-distance, and other "flexible access" students (e.g.,
open-entry/open-exit)
- build and/or enhance on-line orientation, registration, advising, testing, counseling and financial aid systems
- expand access to information resources via webliographies and other internet-related links for targeted upper division and workforce courses/programs
- provide 24 hour, seven-day-a-week help line assistance for faculty, students and staff engaged in technology-mediated programs and services
- research and develop a model portfolio assessment process for
evaluating experiential learning
- redefine support systems to establish model alternatives to
traditional semesters, residency, degree requirements, etc.
- Develop tools, templates and techniques for customizing
courses in a reasonable amount of time using pre-existing materials from a
variety of sources
- customize pre-developed materials (e.g., telecourses to include
institutional "wrap arounds" and templates)
- develop model systems and methodologies for course management
(using a product like WebCT)
- provide expanded training for faculty in the use of new learning
technologies
- develop and assess innovative approaches for faculty and staff
from two and four year institutions to communicate and develop and offer
programs with the right balance of interactivity, workload and cost
-
- Create, staff, and equip a College-University Center (CUC), Electronic Campus and the VITAL Network and support college-wide infrastructure needs to address the objectives cited above
- provide temporary space for CUC activities
- equip and staff CUC facilities (specifically in the academic
services area)
- consolidate and integrate all "distance" course
management and information under the umbrella of the Electronic Campus which
will become a national showcase of distance learning modalities
- provide computer and interactive classroom equipment for college
sites to provide access to both workforce and upper division curricula
- secure equipment to streamline consolidated delivery and routing
of interactive video classes and for video on demand to all desktops, including
a production classroom and digital post-production suite
- hire appropriate staff for program development, coordination of
research and evaluation activities, and the student services components of the
project."
During the
course of the project, the Project Eagle Action Committee (PEAC) has met
regularly and reviewed the monthly deliverables reports to assess the progress
being made and to insure that all goals identified in the grant document are
being met.
Additionally, a
number of tools have been developed to aid in completing the kind of formative
evaluation recommended nationally:
- Best Educational
E-Practices (BEEP). One of the full-time
positions created under Project Eagle was that of a coordinator of research,
evaluation and dissemination. One of the coordinator's responsibilities is to
look at best practices nationwide as they relate to the delivery of e-learning.
To that end, a publication, BEEP, is released each month, both in print and
online, that focuses on one aspect of the distance education process and
selects online examples of the best available. Findings are used for assessment
of and change in Eagle-supported activities.
- Project Eagle Evaluation
Plan. Project Eagle has enlisted the services of two external
evaluator, both from the Education Commission of
the States, who have periodically assessed the work being accomplished
under the grant and made any needed recommendations for change. With the help
of the first evaluator, Dr. Gordon "Spud" Van de Water, a plan was devised that
identified six key questions for analysis. (The reports were spaced out over 18
months, and this document addresses the last of the six questions.)
- Project
Eagle Research Capsules (PERC). Published
occasionally, these summaries of current e-learning research, done both at SPC
and nationwide, have provided statistics and data useful in the assessment of
Eagle-funded projects and activities.
- Policies, Procedures, and Forms. As the college's
eCampus and University Partnership have developed, a number of forms have been
created to formalize procedures in these innovative programs. Several of those
forms deal with evaluation of online courses:
- Timeline/Guidelines for Annual Selection of Online Courses for Development.
- E-Course Flexible Access Form and Procedure (Flexible Access/Delivery).
- Online Course Development Check List.
- Student Survey of Online Instruction.
- Plan for the New SPC
Electronic Campus. Shortly after Project Eagle began, the college created the eCampus
to administer all credit e-courses, whether telecourse, teleweb or online.
Ecampus administration devised a comprehensive written plan that has
facilitated continual self-evaluation in order to meet the goals established by
Project Eagle.
- Evaluations by Individual Faculty. Besides the methods of evaluation
devised and implemented by or for Project Eagle, some individual eCampus
instructors have done informal and formal assessments of the progress of
students in their classes. Examples are a review of the online Veterinary
Technology program and one of statistics classes taught by one instructor, as well as the student feedback on blended learning
by a Communications instructor on student reactions to blended learning.
As the fourth
year of Project Eagle begins, the subject of evaluation will be a central one.
Of the many national formulas for evaluation reviewed for this report, the decision
has been made to select and blend elements from the Basic Guide to
Program Evaluation (The
Management Assistance Program for Nonprofits), Evaluation for Distance Educators
(University of Idaho College of Engineering), and Planning a Program
Evaluation (University of Wisconsin Extension). The suggestions offered by
other institutions mentioned earlier in this report will also be incorporated
into the college's plan when and where appropriate.
B. Informing the Educational Community
Nationwide/worldwide. A look at what colleges and universities throughout the
country indicates a number of techniques for disseminating information about
exemplary programs:
- Establishing a program's identity on the Web. There are a variety of quality resources on the Internet
that offer free advice on the design and development of effective Web sites.
All agree on the importance of a Web site in the promotion of products and
services, commercial and otherwise. Some of these sources are PageResource.com, WebMonkey and Yahoo's
Web Site Announcement and Promotion.
Two examples of institutions that
have used the Internet to promote themselves through the creation of excellent
Web sites are the award - winning George
Mason University (VA) Virtual Tour and the multimedia Rio Salado (AZ) College Story.
-
Getting publicity nationally and locally. Publicity can take many forms, from press releases and
advertising in the media to using direct mail and the yellow pages. Some Web
sites that offer ideas and strategies for success are BusinessTown.com.
Marketing and Publicity, How to Obtain Free
Business Publicity, and Your
Video Business Tips, which covers all aspects of publicity and advertising,
not just those for the video industry.
The Institute for Distance and
Distributed Learning, Virginia Tech, for example, has gone a step further and
created a regularly published newsletter, Point
to Point, to highlight e-learning activities at the university.
- Publishing in professional journals. One of the most established methods of getting publicity
for academic institutions is through professional journals, many of which are
now available online as well as in print. An excellent source of online advice
is "How to Publish in
Top Journals" by Kwan Coi, editor of the Review of International
Economics. There is also a list of about 1000 higher
education journals selected by the ERIC Clearinghouse for Higher Education.
An example of such an article, "Big
Ten School in Cyberspace" by Christopher Hons, appeared in T.H.E.
Journal Online, January 2002. It effectively detailed the history and
accomplishments of Penn State's World
Campus.
- Presenting at conferences and meetings. The best way to submit presentations to relevant
conferences and meetings is to stay abreast of upcoming events, particularly
before the proposal deadlines pass. There are three very good sources of
information on such get-togethers:
- Community College Conferences
2001- 2002. A product of the ERIC Clearinghouse for Community Colleges with
special emphasis on conferences relevant to two-year schools.
- Distance Education Clearinghouse Conference
Database. A source that identifies worldwide conferences, seminars,
workshops, and other events of interest to those in e-learning and related
fields.
- T.H.E. Journal Educational
Technology Conferences. An online worldwide list of local and national
educational technology conferences maintained by the publisher of T.H.E.
Journal.
- Hosting events to showcase program
accomplishments. Like getting publicity,
events such as these can be aimed at local as well as national audiences,
either face-to-face or online. Several excellent manuals are available on the
Web; in combination, they cover every aspect of planning and execution:
- Conference Planning Guidelines. An
overview by Read Gilgen, International Association for Language Learning
Technology (ITALL), of what it takes to plan any successful conference.
- The ISACS Handbook for
Professional Development Activities and Conference Planning. A free online
manual that covers every imaginable aspect of conference planning - with forms.
- Online Conferencing:
Lessons Learned. A guide for planning an online conference by Lyndsay Green
for the Canadian Office of Learning Technologies. This agency has a companion
guide for the technological aspects of an online conference called "So, You Want
to Host an Online Conference" by Cathy Boak and Jean Blackburn and another
by Lyndsay Green on moderating online conferences, "Playing Croquet
with Flamingos."
A unique ongoing event for the
last eight years has been the Teaching in the Community
College Online Conference. It is hosted by Kapi'olani Community
College (HI). In addition to posting links to the online programs of all the
previous conferences, the college has also made available advice on planning,
attending, and presenting such events.
SPC. In the past three years, the college has utilized a variety of techniques disseminate information about the
e-learning programs made possible with Project Eagle funds. It will continue to
do so in the final year of the grant. Those completed to date include the
following:
- Web identity. SPC has established an
Instructional Technology (IT) department with more than a dozen talented
professionals, who have created quality instructional and promotional products.
The best starting points to view many of their pages can be found here:
- The eCampus Web site is an
extremely well-designed effort that allows the user to locate virtually all the
information needed about the program.
- The Project Eagle Web site links to any
college effort that has been enhanced with money and/or personnel made possible
by the grant. The site won a Golden Web
Award for 2001-2002.
- The Cyber Advisor Web
site is a unique service offered to SPC's e-learners by the Eagle-supported
full-time virtual counselor the college employs.
- Project Eagle's
monthly online newsletter Best Educational E-Practices
(BEEP) has received worldwide recognition. The publication has subscribers
on every continent except South America and Antarctica. The April 2002 issue, "Like Minds: Web Site That Link
to and from BEEP," offered links to a collection of several dozen external
Web sites, nationwide and worldwide, that regularly link to BEEP.
In addition to the Web pages above, the college has found
several ways to publicize the existence of the information and the programs:
- The use of a search engine listing service has greatly
increased the Web visibility of the eCampus and other Eagle-funded programs.
SPC has contracted with SiteScreamer,
a company that submits selected urls to more than a thousand search engines
each month.
- Being part of
statewide and regional consortia like the Florida Community College
Distance Learning Consortium and the Southern Regional Education Board's
Electronic Campus has helped advertise the successes of SPC's e-learning
programs.
- National and local publicity. Efforts for
disseminating information about Eagle-related programs through the traditional
media (newspapers, radio, TV) have been extensive, but mostly directed to
marketing and enrollment. However, in September 2001, the eCampus sent out more
than 250 media kits to press publications nationwide detailing its structure
and accomplishments.
The results were disappointing,
in some degree because of the events of September 11, 2001 and
the subsequent anthrax threat in the mails. It was decided not to repeat such a
major effort.
However, an eCampus press release
in January 2002 about the college's new sample online course was sent out to
local media contacts and national education contacts. It was mentioned by Distance Educator.com and the
Instructional Telecommunications Council in their online newsletters, as well
as covered in local newspapers like the Seminole Beacon.
Additionally, an article about the
work supported by Eagle appeared last year in Current, the newsletter of
the Florida Association of Community Colleges
(FACC), and Campus instructor Kevin Morgan has been interviewed for Yahoo Internet Life, November
2000, and his work mentioned in the NEA Higher Education Advocate, January 2001.
- Articles in professional journals.
Although there have been a number of contacts and submissions of articles to
professional journals, the results have been disappointing. Nevertheless,
thanks to the efforts of some individual instructors and the eCampus press
release mentioned in #2 above, articles mentioning some Eagle-related success
stories have appeared in the Chronicle of Higher
Education and Syllabus Online,
January 8, 2002 (article no longer online).
- Conference presentations. This has been the college's most successful
form of dissemination of Eagle-supported accomplishments. At least 25 staff
members have made presentations at dozens of meetings and conferences in the United
States and Canada. A Web
page has been created by Project Eagle staff that contains links to all
conference presentations available in a PowerPoint format.
Events have been sponsored by Educause;
Merlot; National
Institute for Staff and Organizational Development (NISOD); Florida Educational Technology Council (FETC); WebCT; Teaching in the
Community College Online; Florida College
English Association; American Mathematical
Association of Two Year Colleges; Fund for the Improvement of
Post-Secondary Education (FIPSE); International
Conference on Teaching and Learning; Florida
Community College Distance Learning Consortium; Florida Higher Education
Consortium; California Virtual Campus; League for Innovation; Syllabus; and others.
- Hosting events. Project Eagle and three major Eagle-supported
programs, eCampus, Instructional Technology
and the University Partnership Center (UPC)
are headquartered at SPC's new Seminole campus.
Designed with technology in mind, both the original building and the new
three-story addition have drawn individuals and groups of visitors from around
the country and provided ample opportunity to showcase Eagle successes.
Organized events that have been held at the campus have
included meetings of the Florida Board of
Education, with Governor Jeb Bush present; a grand opening of the C. W.
"Bill" Young UPC building with Congressman Young in attendance; Florida Community College Distance
Learning Consortium (FCCDLC); Learning Resources and Technology Commissions
of the Florida Association for Community Colleges
(FACC); Academic Systems, and
others.
In addition, a number of regularly scheduled and special
community events have taken place on the campus, as well as an annual open
house held in conjunction with a community fair known as the Seminole Pow Wow.
Each time, campus administrators have used these opportunities to promote the
accomplishments that have taken place.
Review and Recommendations
A. Evaluating, Assessing, and Improving
an E-Learning Program. Research done for this evaluation question indicates that the seeds of
successful evaluation are sown in the planning stages of a program. SPC has had
the advantage of the Project Eagle grant document, which has provided a very
detailed outline for items and activities that require assessment.
Each of the goals set out in that document was isolated and
put down in a list of project deliverables, and individual staff members
assigned to be responsible for their completion. An internal Project Eagle
Action Committee, formerly the Project Eagle Working Group, was formed to
oversee program activities and to review the deliverables' reports on a monthly
basis. Although the deliverables themselves have sometimes been defined too
narrowly, making global assessment difficult, the method has nevertheless been
very successful for keeping the project on track.
In addition, a number of tools have been developed that have
been useful in evaluating the ongoing activities of Eagle and the eCampus:
- Best Educational E-Practices (BEEP). Monthly newsletter used for planning and assessment.
- Project Eagle Evaluation Plan. Series of questions designed to evaluate the college's accomplishments in its e-learning endeavors.
- Project Eagle Research Capsules (PERC). Occasional summaries of e-learning research, internal and external.
- Policies, Procedures, and Forms. A variety of new forms for the eCampus and other Eagle-funded projects that include several useful for evaluation.
- Plan for the New SPC Electronic Campus. Devised for the new eCampus, which was created shortly after receipt of the Eagle grant.
- Evaluations by individual faculty. Formal and informal assessments
have been done by instructors of students in their classes, statistics
and student feedback on blended learning,
for example.
In the fourth year of Project Eagle, the subject of
evaluation will be a central one. Of the many national formulas for evaluation
carefully reviewed, the decision was made to use the Basic Guide to
Program Evaluation (The
Management Assistance Program for Nonprofits), Evaluation for Distance Educators
(University of Idaho College of Engineering), and Planning a Program Evaluation
(University of Wisconsin Extension).
Based on the evidence above, gathering the data necessary
for evaluation of the accomplishments of four years should produce the same
quality results for which Project Eagle is already known.
B. Informing the Educational
Community. The
efforts to disseminate information about the achievements made possible with
Project Eagle support have been vigorous and ongoing. Both Elsa Houtz, the
marketing consultant hired by eCampus, and Joyce Burkhart, the Eagle coordinator
of research, evaluation and dissemination, have helped to elevate the college's
profile in the area of e-learning to one of a national model.
Not surprisingly, the Web has been the most effective medium
for informing others of the work being done at SPC. For example:
- The January 2002 press release on the eCampus' sample online course drew
praise, responses and inquiries via email from more than half a dozen institutions
nationwide, including John Hopkins University.
- The monthly newsletter Best Educational E-Practices
(BEEP) is on the resource list of schools and organizations around the world,
including one in Japan and one in Germany.
- The Web sites for eCampus,
Project Eagle and the SPC Cyber Advisor
have received recognition and even awards nationwide. The Cyber Advisor Web
site was recently featured in a presentation made at the Teaching in the
Community College Online Conference and the eCampus Web site in one related
to student services done at the 2002 Florida Educational
Technology Conference.
In addition to the work of the professionals the college has
hired to undertake the information process, some of the e-learning staff and
faculty have contributed to making the SPC program a model to follow. For
example:
- The online Vet Tech program, under the
leadership of Dr. Guy Hancock, was the first of its kind in the country and
enjoys a national reputation.
- A unique strategy for providing
Web video presentations to deaf students has been developed by instructional
technologists, and has already received national recognition.
- Several eCampus faculty have been
singled out for recognition locally, regionally, nationally and
internationally.
- The college videographer, Dave
Herring, has won national awards for several productions.
Another factor that has provided opportunities for promotion
that might otherwise have not existed is the campus itself. Since it was opened
in 1998, the Seminole campus has drawn
attention as a technology center. Although it provides the same classes and
services offered at most other SPC locations, it has always been the testing
ground for new ideas and technologies.
The physical presence of the eCampus, instructional
technology department, Administrative Information Systems, and the innovative University Partnership Center at Seminole
has brought together many like-minded individuals and groups. They have taken
what they've learned about SPC's innovations back to their institutions and
communities, further enhancing the college's e-learning reputation.
One of the best forms of dissemination has been the presence
and presentations of SPC e-learning personnel at an impressive number of
conferences and meetings throughout the country and beyond. The Project Eagle presentations
Web page shows only those available online in a PowerPoint format, but
gives a feeling for the depth and breadth of work being done at the college.
The area of least success in terms of informing the
educational community has been, probably not surprisingly, the traditional
route of coverage in printed publications, particularly journals. In spite of
repeated and sometimes costly submissions of material, the results have been sparse.
However, based on these experiences and on the advice of Project Eagle external
evaluator Dr. Kay McClenney, the traditional route is probably the least
appropriate for the kind of non-traditional e-learning accomplishments. It is
likely that informational efforts in the final year of the grant should and
will be concentrated on those types that have brought the most positive results
to date.
In conclusion, the efforts evaluating and improving
Eagle-supported activities, as well as those of informing the educational
community of SPC's accomplishments have been varied, thorough and successful to
date. That work will be continued in the next year of Eagle, and will, in fact,
be central to the completion of the goals outlined in the grant.
References
Burkhart, Joyce. "SPJC's Project Eagle Soars." Current
34:2, Spring/Summer 2001, 10.
"Course Web Sites: Are They Worth the Effort?" NEA
Higher Education ADVOCATE, January 2001, 6-7.
"SPC Offers Students an Online Option." Beacon
Publications (Seminole Edition), January 17-23, 2002, 12.
Terry,
Lisa. "The Digital Diploma." YAHOO Internet Life, November
2000, pages not known.
|
.
.
