Step 3: Planning and Collecting Data

Because students’ success in acquiring the Graduate Attributes is the single strongest indicator of a program’s success, the ability to collect and report on the results of effective, reliable and valid learning assessment is critical.

Your purpose in collecting data is to obtain pertinent information in demonstrating the relative success of your programs and to use for program decision-making. A critical concept to keep in mind regarding assessment and data collection is fit for purpose – purposefully gathering meaningful evidence and information on program effectiveness.

Data, or evidence of student learning, can be categorized in two ways relating to how it was assessed: direct and indirect.

Assessment approaches and tools depend on the context in which evidence is collected. Currently the primary emphasis is on assessing activities inside courses.

For accreditation purposes, your program must provide a range of examples of completed and graded student work as direct evidence of learning. Direct evidence must be related to the indicators defined by the program. It is essential that the assessment tool used aligns with learning objectives.

An example of how targets and thresholds can be incorporated into an assignment is shown below. It shows a rubric used to assess specific indicators. The threshold and target descriptions are highlighted:

There are a myriad of methods available for collecting indirect evidence on student learning and indications for their use including:

Let’s examine some of the methods identified above:

Using multiple sources of data when documenting students’ learning is an effective way of validating initial analyses of a single piece of data.

1. Include opportunities for informal assessment, students’ self-reports of learning, and even unsolicited data from placement supervisors or employers.
   Students’ performance in the classroom can become a data point if teachers adopt a habit of keeping session logs to record ways that students respond to instruction in different learning situations. Data gained through student focus groups or interviews may also provide an alternative perspective on how and what students are learning. Placement supervisors and employers are likely to be able to comment on integrated, authentic, practical learning.

2. Use more than one type of assessment when analyzing data
   When you know what it is that you’re looking for in students’ learning, the more likely you are to want to gather data from different sources to bolster your claims about what they’ve learned. Looking at multiple sources of data on students’ learning is analogous to taking multiple snapshots of a single event – it provides a more complete picture of reality. Using data from multiple assessment sources enables teachers to more accurately determine students’ strengths and identify areas for improvement

3. Value all assessments not just major events
   Any assessment activity that was worth having your students do is worth using as a data point.

4. Use the data gained from assessment to answer questions about authentic learning
   Typically, authentic learning is described as learning that mimics the real-life knowledge and skills that are required in industry. Just as typically, questions about complex learning are unlikely to be fully answered using just one piece of data.

5. Look at data across time intervals
   When looking at data on students’ learning over time, look for patterns and changes in performance and achievement in relation to the demands of the curriculum.