Student Success - Measuring Critical Thinking Using ePortfolios

ePortfolio systems at SPC

Students will be trained in relevant courses on the practice of keeping Electronic Portfolios (ePortfolios) in which they can keep their critical thinking artifacts and examples of other types of work that may be helpful to them in documenting learning, including the development of critical thinking skills, achievement of specific critical thinking objectives embedded in standard course outlines, and skills relevant to future employment or promotion.

SPC examined the two learning management/document management systems currently in use at SPC for their suitability for hosting ePortfolio artifacts exemplifying students' critical thinking skills: LiveText and ANGEL.

LiveText. LiveText is a document management system used by SPC's College of Education, Dental Hygiene, and other programs to gather evidence of student learning for state reporting and discipline accreditation. It has a well-developed and integrated rubric system that allows multiple scorers and institution-level analysis of scores for designated assignments. SPC will identify appropriate critical thinking assignments in ePortfolios from the disciplines using LiveText for institution-level assessment.

Due to budget cutbacks, it is not possible to staff collegewide-use of LiveText; however, the system may be expanded when required by specific programs to document and report student learning to state or other authorities.

ANGEL. ANGEL is the learning management system used collegewide to provide online content, interaction between and among instructors and students, assignment submission, and grading. A future version will have similar capabilities to LiveText; however, the timing may not support QEP assessment requirements. ANGEL will be set up to collect assignments from faculty collegewide via drop box, which with then be scored manually by committee.

Scoring online versus scoring by committee

As a pilot institution for using the NSF-funded Critical Thinking Assessment Test, SPC conducted a manual scoring session with fifteen faculty members to review and score close to 100 assessments. The scoring committee found the learning gained from collaborative discussion of student work was enormous and its value could not be replicated with online scoring. As a result, the SPC QEP team recommended against collegewide use of an online document management system.

On the other hand, state and accrediting requirements dictate the use of a document management system to collect and report student learning outcomes for certain programs. Critical thinking is one of the learning outcomes for students in those programs. As critical thinking outcomes are being captured in those programs using LiveText, SPC will take advantage of the online scoring system that is part of LiveText.

Exemplary Uses of Electronic Portfolios

Dental Hygiene

As a requirement for graduation, students working towards their Bachelor of Applied Science in Dental Hygiene (BASDH) construct an electronic portfolio during the DEH 4947 - Dental Hygiene Capstone course. Using LiveText, students display samples of course work that verify, exemplify, and validate the achievement of the BASDH Program Outcomes. One of these outcomes includes critical thinking: Provide advanced professional and educational services using appropriate interpersonal, written communication and critical thinking skills required for successful performance and progress in the profession.

Directions for students include:

Student reflection on the critical thinking outcome include:

"This assignment showed me how active learning, an intellectual process which requires interaction with the teacher and other students, can be an engaging and effective substitute for the conventional lecture. Learning is enhanced when students take an active role in what they are learning. The process of preparing and defending an argument for a debate requires an in-depth understanding of the concept and leads to better retrieval of the information."

"Completing this assignment required critical thinking skills that led to me to a more thorough understanding of what is happening on a cellular level when patients exhibit the sometimes subtle changes in tissue color, contour, and consistency. It enhanced my ability to educate patients about periodontal disease and formulate an evidence-based dental hygiene diagnosis and treatment plan that can be tailored to a specific patient's needs."

Paralegal Studies

A component of the A.S. degree in Paralegal Studies is the completion of an electronic portfolio. Students gain access to their own blank portfolio during their first paralegal course, and they are expected to add to their portfolio throughout their semesters in the program. Using an SPC-in-house-developed template, students add samples of assignments they have completed in various courses. Students select particular assignments because they demonstrate one (or more) Major Learning Outcomes (MLOs) which are based upon state-mandated proficiencies for paralegals: Critical Thinking, Organizational, Communication, Legal Research, Legal Writing, Computer, Interview/Investigation, Professional/Ethical, and Law Office Management.

Students access the Portfolio User Guide to learn how to use the portfolio Web site, as well as the guidelines for completing their portfolio. At the conclusion of the program, each portfolio is evaluated by three individuals in the legal field using a rubric. Data acquired from this process are also incorporated into end-of-program assessment.

Students include a rationale for each assignment that explains how that artifact shows that the particular MLO or skill has been acquired. For the Critical Thinking MLO, students are instructed to, "demonstrate the ability to analyze a problem; identify and evaluate alternative solutions; formulate logical solutions to problems; construct logical arguments in support of specific positions; evaluate solutions and arguments; and determine which areas of law are relevant to a particular situation."

Student examples from the Critical Thinking MLO include:

"Discusses and analyzes the affects of a fictional lawyer on the legal profession."

"Assignment was to brief this case and Morgan v. State, 537 So.2d 973 (Fla. 1989) to determine admissibility of hypnotically refreshed testimony."

"Brief this case and Bundy v. State, 471 So.2d 9 (Fla. 1985) to determine admissibility of hypnotically refreshed testimony."