Before we move to defining the criteria for measurement for different training programs and methods of assessment at Level III, let us look at the relation between the learning objectives and assessment. This is because to do any evaluation, especially Level III, an understanding of the learning objectives is very essential. This is because the level of evaluation or type of assessment is directly related to the learning objectives of the program.

It should also be noted that definition of the learning objectives will depend on the effectiveness of the needs analysis. Let us assume that the needs analysis was performed well, and the gap was identified and a training delivered. We still won’t be able to measure the effectiveness unless the learning objectives are written accurately. Let us consider this with an example of a training program, say “Presentation Skills.”

Here are some hypothetical learning objectives. Let us see how different levels of evaluation are related to the different learning objectives. We can also indicate which levels of Bloom’s Taxonomy the learning objective is related to.

1. Participants will describe the characteristics of a good presentation
2. Participants will distinguish between poor and effective presentation slides
3. Participants will discuss the importance of non-verbal communication in presentation skills
4. Participants will list the factors that can influence audience engagement
5. Participants will demonstrate non -verbal skills during presentation
6. Participants will engage the audience using different techniques.
7. Participants will be able to handle difficult audience

Before the learning objectives are defined, the instructor should know the NEED for the training. What is the need for conducting training on presentation skills? Is it to improve knowledge of the participants about good training, or is it to help improve their skills in presentation training?

These questions will also help the design of training. For example, if the need is for participants to only know how to prepare effective slides and the characteristics of a good presentation, the method used could be a lecture. The assessment could be a test after the lecture (or before and after) – the level of evaluation is Level II.

Here, because the objectives were defined only at the cognitive level, a Level II assessment is appropriate. Can a Level III be done? Sure it can. But would that be valid – perhaps not. That is because the participants were given knowledge but they did not have a chance to create and demonstrate. So it is not that level III cannot be done. Depending on the individual participant’s capacity to understand and assimilate the lecture, the participant may also create better presentations. But level III may not necessarily be valid.

There may be other instances where a level III may be valid, whereas a level II may not be valid. For example, employees are given training on how to use a fire extinguisher and conditions in which it should be used. They are given training on operating the instrument. But the science and Physics behind the extinguisher is not taught. It would not be fair to evaluate the knowledge about how the extinguisher works, why it works and so on. Here the appropriate level would be level III- in an event of fire, can the participant use the extinguisher effectively.

Coming back to our example of presentation skills, objectives 5 to 7 relate to the psychomotor domain on Bloom’s taxonomy. Whenever objectives relate to this domain, a level III evaluation can be done. Similarly, when objectives relate to the affective domain, Level III is appropriate. That is because the assessment is whether the person can actually do the action, and perform according to the expectation. With respect to the affective domain, it would relate to whether the person is motivated to change his or her behavior at work. The method of assessment would involve evaluating presentation by the participants to assess the quality of the slides, the extent of engagement of audience, use of verbal and non -verbal skills, etc. The assessment could be a mock presentation, or a real incident of presentation with participant feedback

The important thing to note is that evaluation depends on the learning objectives and the domain of learning. The objectives have to relate to certain performance criteria, which would be specific and measurable. For level III training, defining these criteria would help to create evaluation designs and choose assessment methods.

In the next blog we shall consider different criteria for level III evaluations for different types of training programs.