Evaluate and develop courses - Education quality processes
For employees
Evaluate and develop courses
This page describes how to evaluate and develop a course.
The process of evaluating and developing a course
NTNU’s system for quality assurance of education describes the goals of the quality assurance processes and defines roles, responsibilities, tasks and follow-up processes. This is how you evaluate and develop a course:
The starting point for the plan is the course description, together with evaluation reports and action plans from previous periods when the course was offered.
You can find these in KASPER and in the portal for education quality.
When you plan the quality assurance process for this implementation of the course, relevant factors to consider are:
- What are our goals for this implementation of the course, and what level of ambition is reasonable? Here, you start with the evaluation from the previous implementation period, where development areas and measures have been identified.
- What platform of knowledge do we need to evaluate the effect of the measures, and to evaluate the course in general? How should we gather this knowledge base?
- How should we collect evaluations from students? For more information, see Methods for evaluating courses.
Key data for each course are available in the workspace for the course in KASPER. Data on explanations of grades, appeals, re-sit examinations and more are available in the BEVISST system.
For more links to data sources and knowledge bases, see the summary page Tools and resources.
The quality assurance process has three time horizons: immediate, short-term and long-term.
- In the immediate and short-term quality assurance activities, the key question is how students can best achieve the desired learning outcomes. Some improvements can be made as immediate adaptations, while others are included in the follow-up plan for the next implementation of the course.
- In the long-term quality assurance, the relevance and knowledge platform of the course are assessed in the context of the programmes that include it.
Changes in course descriptions may have consequences for other courses and programmes. Changes must therefore be communicated to the department and chairs of the programme councils concerned.
Gathering a knowledge base
The course coordinator is responsible for ensuring that feedback from relevant parties is included in the evaluation of the course. Feedback from students is in a unique position and must be documented. Other relevant parties may be academic programme directors, other teaching staff, student assistants, host institutions for placements/practical training supervisors, etc.
The course coordinator must
- ensure that arenas for such dialogue are established
- present a basis for reflection, such as previous reference group reports, follow-up plans or results
- document the feedback to provide a basis for the evaluation report and follow-up plan
As a minimum, the course coordinator should have a dialogue with the students
- at the beginning of the semester, to clarify expectations
- during the course, to identify any immediate needs for adjustments
- right at the end, to evaluate the course as a whole. Preferably after the exam, so that the students can also give feedback on the form of assessment.
Examples of other times where evaluation would be natural include during practical training, papers and assignments, fieldwork, etc. This presentation with a video about education quality can be a good starting point for talking to the students about evaluation of the course.
Aspects for evaluation
Relevant aspects for evaluation will largely depend on the nature of the individual course. Key aspects such as the learning environment and learning outcomes have been incorporated into NTNU’s common templates for reference group reports, questionnaires and evaluation reports.
See also
The evaluation is done in KASPER. The following process is intended to ensure continuous development of the course:
1. Evaluate the effect of the development measures (the follow-up plan) from the previous evaluation of courses
- How did it go with the goals and ambitions that we had defined for the implementation of the course?
- What do we conclude about the association between measures and achievement of the goals?
- What other factors might have influenced the results?
- How well did the quality assurance function?
2. Evaluate the implementation of the course and the students’ achievement of learning outcomes
This involves both specific and more general evaluation of
- the individual learning and assessment activities and the way they were implemented
- the associations between learning activities, forms of assessment and learning outcomes
- the actual description of the learning outcomes for the course
Examples of relevant questions related to the discipline:
- How well does the course function in the programmes that include it?
- Are there relevant changes in the research area?
- Are there relevant changes in the field of practice?
- What feedback have the students provided?
- What do the examiner and the exam results tell us?
- What feedback are we getting from practical training supervisors, if applicable?
- Are there changes in the academic staff group that we should keep in mind?
3. Set up a plan for development with actions (follow-up plan) for the next implementation of the course
- What goals and level of ambition should we propose – for the next implementation of the course and in the longer term?
- What actions and measures should we propose – for the next implementation of the course and in the longer term?
You can find resources for this work on these pages:
During the planning, implementation and evaluation of courses, suggestions for small and large changes may well arise.
There is a great deal that you as the course coordinator or course lecturer cannot change once the course description has been approved and published. This is because the programme description and the course descriptions are legally binding.
Substantial changes
Feedback and evaluation from students and academic staff are obtained during the semester and reflected in the course report. This ends up in a proposal for a follow-up plan. You submit the course report and follow-up plan online in KASPER. These must be considered by the head of department and the dean.
Proposals that involve changes to the course description are included in the revision of the programme description. The revision process takes place during the year before the course starts. Here, it is possible to adjust aspects such as learning outcomes, the assessment system, compulsory activities and grading rules. Reasons for the changes must be provided, and the department and programme council chair(s) are important dialogue partners to ensure that the changes that are proposed comply with associated courses and programmes. Major changes in academic content might make it necessary to create a new course. Requirements for the content of courses are described in the programme description guide, which is available in the Guidelines for the quality assurance system.
The course description is entered in the Online Course Planner in the autumn during the programme description work for the next academic year. This is done either by the administration or by you as the course coordinator. The faculties have varying procedures for this.
Minor changes
Feedback and evaluation from students and academic staff may also result in proposals for measures that are within the scope and learning objectives for the course. It will be possible to adjust some of these during the course, while others will be included in the course report as suggestions for the next time that the course is held. It is important to take part in a dialogue with the department and programme council chair about the adjustments you want to make. The course is an integral part of one or more study programmes, and the adjustments may have consequences for the other courses and programmes as a whole.
The syllabus can be adjusted from semester to semester, until the department’s deadline for enrolment. You can also change the learning activities along the way, within the scope of the course description.
Relevant links:
Course descriptions - including those from earlier years
The course coordinator is responsible for planning, coordination, implementation and evaluation of the course.
The aim of the work is to ensure that:
- the learning outcomes for the course are up to date and relevant
- the learning activities in the course help the students to achieve the learning outcomes
- there are links between learning objectives, learning activities and forms of assessment
As part of the quality assurance system for education, the course coordinator must evaluate the course and report to the head of department.
The systematic development work centres on students’ learning outcomes. The feedback from students must be made visible.
Every time a course is held, the course coordinator must conduct a course evaluation. The evaluation must be summarized in a follow-up plan.
- All course evaluations must include a student evaluation, for example based on the reference group method.
- At least every third time the course is held, the course coordinator must obtain feedback from all the students on the course.
Each time the course is completed, the course coordinator must prepare a course report with a follow-up plan and publish this in KASPER. The report is then publicly available in the portal for education quality.
The course coordinator must act on the follow-up plan that has been approved by the head of department.
For courses with practical training, the same methods of evaluation apply as for courses in general.
Quality development of practical training must be based on dialogue between the course coordinator, the academic programme director, students and the field of practice.
Requirements for quality in practical training are defined both in the National Curriculum Regulations for the various programmes of professional study and in the Regulations on supervision of educational quality in higher education (studietilsynsforskriften). The faculties that have programmes of study with practical training must have arrangements to ensure that these requirements are met.
Students’ tasks in quality assurance work are described on a separate web page.
What defines the quality of a course?
The aim is to ensure
- that the description of learning outcomes for the course is up to date and relevant
- that the learning activities in the course help the students to achieve the defined learning outcome
- that there is a connection between the learning outcome description, the learning activities and the forms of assessment
Roles
In the work on evaluation and development of courses, four roles are involved:
Tools and resources
Several tools and resources are available to help you in your work on the course.
Good advice and checklists
Tips on academic and administrative planning are available on this topic page: