Once the Educational Goals have been defined, the next phase of the process is to gather data. However, it is not necessary to collect data on all courses, and depending on the approach one selects, the length of data collection may range from 2 terms to 2 years.
Approaches to Data Collection
External Review Timeline
Unit Self-Study: Between October and December of the year prior to the unit’s external review, units are to produce a Unit Self-Study which is circulated to External Reviewers before their site visits. Assessment of Educational Goals should be included in the Unit Self Study (see Sections 3.2.a. & b. in Senate Guidelines for External Reviews of Academic Units).
Unit Action Plan: After reviewing the External Review Report, units are expected to produce a Unit Action Plan to address the recommendations from reviewers and to plan for the next cycle.
Progress Report: In year 4 units are expected to produce a Progress Report for internal use and reporting to the SFU administration. As part of this document units are expected to report on their assessment of all Educational Goals.
Data Collection for EG Assessment
In the above graphic the approach presented for data collection and analysis is sequential in nature, proceeding through the phases of definition, collection, analysis and action. It allows almost an equal amount of time for each of the phases and enables units to focus on each phase independently.
However, depending on units’ assessment plan, resources, time allocation, and preference, variations of this approach are equally appropriate.
For instance, units may decide to collect on each Educational Goal individually, by defining, collecting, and analyzing before moving to the next Educational Goal, and repeating this process until all Educational Goals are assessed.
Another variation is to collect on all Educational Goals while also analyzing the data simultaneously.
An assessment plan outlines the process for collecting and analyzing data about the effectiveness of student learning within a program. It includes a statement of educational goals, selected direct and indirect methods of assessment to best evaluate program effectiveness, a plan for collecting and analyzing data, and for implementing change.
Definitions of Elements in the Assessment Plan
Program Level Educational Goal: Identify the knowledge, skills, abilities, etc., that students should be able to demonstrate upon completion of the program. The goals need to be specific and measurable.
Breakdown of Educational Goals: Sometimes it might be helpful to break down Program Level Educational Goals into smaller units that can be operationalized. This may be helpful for finding data in the curriculum which is relevant to the assessment of the Program Level Educational Goals.
Data Source: Programs should identify where in their curriculum (e.g. course number) data is being gathered to assess the specific Educational Goals. N.B.: not all courses need to be assessed. Courses towards the end of a program can provide a measure of student achievement, and first year courses can provide insight on student skills at program onset. Curriculum mapping is useful for identifying which courses Introduce, Emphasize, Reinforce, and Apply the skills and knowledge identified in an Educational Goal.
Direct Assessment: Direct Assessment (also called curriculum-embedded assessment) requires students to demonstrate their knowledge, skills, etc., and faculty members to then assess whether/how well students are achieving/have achieved a program goal. Examples of direct assessment include artistic work products, case studies, exams, juried performances, oral presentations, papers, and portfolios. Examples of direct assessment can be found here.
Indirect Assessment: Indirect Assessment gathers perceptions of whether/how well students are achieving/have achieved a program goal, in the form of reflection questions. Examples of indirect assessment include surveys of alumni, employers, and current students; exit and focus group interviews; enrollment and retention data (e.g. from Institutional Research and Planning); and job placement data. Indirect assessment complements the data collected from direct measures and cannot stand alone as sole measures of student performance. Examples of indirect assessment can be found here.
Years/Semester of Data Collection: Programs should identify when (in which year or semester) the data is being gathered.
Major Findings: Programs should identify the major findings after analyzing the data collected.
Actions That Resulted from Findings: Programs should provide evidence that the findings have been used to further develop and improve student achievement of program level educational goals (i.e., actions that were taken as a result of data collection and analysis). It is also important to state when findings provide evidence that students are successfully achieving a program level educational goal.