Why the Big Picture Matters
Implementing a continuous program improvement model is an extensive undertaking, and requires the participation of engineering faculty, course instructors, university administrators and students to be successful.
A review of context and background information is particularly useful to those joining the process, but it is also valuable for those who have been engaged longer to revisit the Big Picture in order to ensure that the larger goals are being held in view. This section provides executive summary level of details on a number of key topics in order to support those seeking such an overview. Links to more in depth information are provided as appropriate, while an extensive offering of relevant supplementary and complimentary content is to be found in the Resources section.
What Are Effective Continuous Program Improvement Practices?
Successfully implementing accepted best practices for outcomes-based assessment not only directly addresses all accreditation criteria set forth by CEAB, but also results in effective continuous program improvement. These best practises help focus and streamline programs, identify areas with innovation potential and provide clear expectations for students that ultimately enhance their learning experience.
The essential characteristics of effective outcomes based assessment, as defined by respected researcher Trudy Banta, can be seen on the right.
Planning
- Involve stakeholders from the outset.
- Begin when need is recognized, and allow sufficient time for development.
- Written plan with clear purposes related to goals that people value.
- Assessment is a vehicle for improvement.
- Bases assessment on clear program outcomes.
Implementation
- Uses multiple measures, maximizing reliability and validity.
- Assesses both processes and outcomes.
- Is undertaken in an environment that is receptive, supporting, and enabling on a continuing basis.
- Involves continuous communication with constituents about activities and findings.
Improving and Sustaining
- Produces credible evidence of learning and organizational effectiveness.
- Ensures assessment data is used continuously to improve programs and services.
- Encompasses expectation that outcomes assessment will be ongoing, not episodic.
- Incorporates ongoing evaluation and improvement of assessment process.
All aspects of Banta's characteristics are considered hallmarks of effective practise in assessment and program improvement. They have been adopted by many programs throughout North America, and can be adopted for use in the context of an undergraduate engineering program as follows:
The 12 graduate attributes that have been set by the CEAB must be demonstrated as being effectively assessed, and demonstrated by program graduates (CEAB Section 3.1). This is done through Indicators that are defined by the faculty or program, mapped to courses within their program, and then implemented and assessed at the course level through Course Learning Outcomes which are set by the instructor, as shown below.
Evidence of student learning is used in support of claims or arguments about improvement and accountability told through stories to persuade a specific audienceNatasha Jankowski, National Institute for Learning Outcomes Assessment
Engineering programs then must use the assessment data for the purposes of continuous program improvement (CEAB Section 3.2). The exact nature of how to utilize assessment data for improvement is left to the individual program to demonstrate, with CEAB requiring that a process be in place to apply the results to further development of the program. Careful consideration should be given to all elements of a program process, such as: sustainability, governance and support.
What Research Is This Based On?
“What and whether students learn successfully is more important than when and how they learn something” William Spady
Outcomes-based assessment and continuous program improvement, while new to Canadian engineering programs, has existed since the late 1980's. It represents a departure from the instructor-focused objectives of the traditional, inputs-based system, instead adopting a student-focused and performance-based description of intended goals and outcomes of the system.
Outcomes-based methods are a key driver in modern curriculum development. They focus on defining what the student should be able to do at the end of a course or program and develop curriculum, learning experiences and environments, enabling students to achieve these goals. These methods have been developed based on educational psychology, teaching and learning research and applied research in higher education. If you would like to learn more, please consult the resources below:
When Did the Shift Begin?
In recognition of the incredibly fast-paced and increasing globalized world that engineers were navigating, an international agreement called The Washington Accord was signed in 1989. It is an agreement between organizations responsible for the accreditation of engineering degree programs. The agreement purpose is to recognize the equivalence of programs within the signatories, ensuring that graduates have met the academic requirements for entry into the practise of engineering. In turn, this vastly increases the global mobility of engineering graduates.
The Washington Accord clarifies its expectations by defining the purpose of engineering education. Please note the words highlighted in the following passage.
"The fundamental purpose of engineering education is to build a knowledge base and attributes to enable the graduate to continue learning and to proceed to formative development that will develop the competencies required for independent practice."
This sets the tone for members of the accord to adopt an outcomes-based approach in their programs that aligns with the definitions of The Washington Accord. It stresses the importance of knowledge, outcomes and a view towards professional practise as well as encouraging the implementation of best practise for the academic preparation of engineers. These standards, expectations and competencies are outlined in a companion framework to the Accord.
Canada, along with CEAB, is a member of the accord and follows these guidelines through Accreditation Units, Graduate Attributes and Capstone Design Experiences. Woven throughout the CEAB accreditation requirements are elements of best or effective practise in outcomes-based assessment as discussed in the previous section.
What Are the Benefits for Canadian Engineering Programs?
If the approach taken to curriculum re-design respects the care, education, development, resources and efforts of engaging in outcomes-based education, the concomitant change in accreditation could transform not only the engineering curriculum, but all those who are engaged in the field of engineering: educators; faculty; students; practitioners; and influence the positive role that engineers can have for the betterment of Canada.
The value of developing explicit learning outcomes and assessment was definitively established through a meta-study conducted by John Hattie, the findings of which are illustrated below:
How Does This Impact Engineering Accreditation?
It Provides a flexible framework for institutions...
The CEAB graduate attribute assessment criteria support the implementation of effective continuous program improvement by aligning themselves with effective practise in outcomes-based assessment through the principles on the left.
... and in practical terms, looks like:
This allows institutions to develop processes that are most effective to the idiosyncrasies of their particular programs and build on their strengths.
How is the Engineering Profession in Canada Regulated?
From Canada’s inception, Section 92 (13) of the Constitution Act, 1867, placed all professions under provincial and territorial jurisdiction. The profession of engineering remains responsible to this day for self-governance, licensing, discipline and enforcement. It is noteworthy that licensing requirements are almost identical across the country. Engineers Canada (EC) is the national organization of all provincial and territorial professional engineering associations.
In 1965, the Canadian Engineering Accreditation Board (CEAB) was established. The CEAB is a committee of Engineers Canada that assesses engineering programs on behalf of the provincial and territorial engineering regulatory bodies. Through the accreditation process, the CEAB identifies programs that meet the academic requirements for licensure as a professional engineer in Canada. Students who graduate from an accredited program are considered to have fulfilled the academic requirements for licensing by Canada’s regulatory bodies.
More information on CEAB from Engineers Canada can be found here.
What is the Actual Engineering Program Accreditation Process?
The purpose of accreditation is to identify to the constituent associations of Engineers Canada those engineering programs whose graduates are academically qualified to begin the process to be licensed as professional engineers in Canada. The process of accreditation emphasizes the quality of the students, the academic and support staff, the curriculum and the educational facilities.Engineers Canada 2015 Questionnaire for Evaluation of an Engineering Program
Accreditation of Canadian undergraduate engineering programs is an on-going process. The process takes approximately 18 months to complete and consists of the following steps:
More information on the accreditation process can be found on the Engineers Canada website.
What Changed in 2007?
While the Canadian Engineering Accreditation Board (CEAB) has made regular adjustments to their accreditation requirements over the years, the changes that were introduced in 2007 and made effective in the fall of 2009 were of particular significance. Minimum accreditation units for programs were increased, and a requirement that two-thirds of engineering design and engineering science curriculum components be taught by registered professional engineers was introduced, neither of which would be considered significant adjustments. The noteworthy modification was the introduction of a graduate attribute based, outcome assessment requirement.
This evolution of accreditation requirements was necessary in order to maintain the international reputation of Canadian engineering degrees by ensuring that our engineering programs are in step with evolving international standards. The changes are intended to support the continuous quality of engineering education.
The new regulations require engineering programs seeking accreditation to develop and implement:
Graduate Attributes
Programs must demonstrate that students possess the 12 Graduate Attributes upon graduation.
Continuous Program Improvement
While the CEAB Questionnaire mandates the specific details in terms of reporting formats, the set forth general terms are clearly defined for both graduate attribute assessment and the process for continuous improvement.
