The Importance of The Right Question
Once you have defined your methods for Collecting Data on Students’ Learning, dealt with the process of generating findings. Findings are merely the facts and empirical evidence of your investigation. To be of use in the process of curriculum evaluation, those findings will have to be interpreted so that judgments and then recommendations for improvement might be made.
The quality of the questions asked during the process of program evaluation influence the types of data collected and the interpretations and recommendations made. Use your program’s purpose statement as the basis from which to ask questions about the effectiveness of programming and students’ learning. When the answers to your questions begin to emerge, the next step in the process is identifying recommendations and actions for program improvement.is a platform on which to build additional inquiry.
These primary eight questions are designed to guide you in analyzing and interpreting your data. Each of the eight primary questions is supplemented with additional considerations to guide interpretations, judgements and program recommendations.
The questions in this module will start your process in:
- Identify themes, patterns, associations, gaps, and redundancies in your programs’ curricula
- Interpret trends and discrepancies in data from different sources
In an outcomes-based curriculum such as those now required for engineering schools in Canada it is important to be able to show a direct relationship among the Graduate Attributes, indicators, instructional methods and activities, assessment activities and student performance. That being the case, evaluating the curriculum means asking the question: To what extent does the design of the curriculum accomplish what it was intended to accomplish? Once you’re able to determine the ways in which the curriculum was successful, it’s equally important to follow up with the question of “what went wrong and why?”.
Presented are a series of eight questions that might serve as a starting point in focusing the analysis, interpretations, judgements and recommendations that may arise from your curriculum mapping process. For each of the eight primary questions we have included starter questions to focus attention on things to look for in your findings, other questions to focus interpretations, judgements and recommendation. In each case, the questions we’ve provided are just a start; you are encouraged to refine and supplement the questions we provide with others that represent your own programs’ context and purpose.
Tips To Remember When Interpreting Your Findings
- Interpret evaluation results with the goals of your program in mind.
- Keep your audience in mind when preparing the report. What do they need and want to know?
- Consider the limitations of the evaluation:
- Possible biases
- Validity of results
- Reliability of results
- Are there alternative explanations for your results?
- Have the different data collection methods used to measure your progress shown similar results?
- Are your results similar to what you expected? If not, why do you think they may be different?
Summary
The content of this module is restricted to question-generation. This is so the quality of the questions asked during the process of program evaluation influences the types of data collected and the interpretations and recommendations made. Use your program’s purpose statement as the basis from which to ask questions about the effectiveness of programming and students’ learning.When the answers to your questions begin to emerge, the next step in the process is identifying recommendations and actions for program improvement.
