In the last blog, we considered Level II Evaluations, which focus on measuring the gain in knowledge after training. This can be done through pre and post training tests. We can also assess this by comparing the results of participants who have been through training to those who have not been through training on a test. But just gain in knowledge does not guarantee that behavior at work is changing, or performance has improved. So how can this be measured?

Level III evaluation is about changes in behaviors at work, or changes in the way the work is being done. This refers to the “transfer of training” to the workplace. It refers to the transfer of knowledge, skills and attitudes to the workplace. So before we consider how to measure training effectiveness at Level III, let us consider the different factors that influence the transfer of training. Some of the important factors in transfer of training are the following:

Trainee Related Factors:

Were the participants chosen with care? Many times training is delivered for the whole department, or for an entire group, without considering the individual need of the participants. At other times, some participants may not be motivated to attend training. Their personalities, capabilities and skills, knowledge and attitudes can influence the extent to which training is transferred to the job

Training Related Factors:

Was the training appropriate? If the needs analysis is not done with care, the root cause would not be addressed, and therefore the training would not change anything. For example, a client asked for training in “time management” as workers were not reporting on time, and they also took longer breaks. Further questioning proved that the problem was not time management at all, it was the disengagement of workers due to a problem with the supervisor. They were not motivated to work. The morale was low. If training on time management had been conducted based on the inputs of the client, the training would not have solved the problem in any way. The behaviors at work also would not have changed- the workers would still continue to be tardy, and not report on time.

Were the learning objectives written accurately? Even if the needs analysis is done well, the learning objectives need to be written accurately. Not only do learning objectives guide the content of the training, these give the criteria of measuring the effect of training. In the next blog we shall see how learning objectives are related to measurement criteria

Does the training reflect on- the- job realities? Closer the training content is to the job conditions, realities and challenges, better the training transfer. For some training programs this is easy to define or replicate. It can be done using simulations as in technical training, or as in computer based training, one can design training on the system itself. However, for other kinds of training, does the content fit the job environment? In training programs such as interpersonal training, or even process training, the facilitators need to create content that match the job reality closely.

Culture Related Factors:

Even if the instructional designers, content developers, facilitators do their bit in creating an excellent training program, that is customized to the specific needs of the participants, the success of training will to a large extent depend on how much the organization will support the new behaviors/ changes. The culture related factors include:

• Support of line managers/ supervisors, and rewarding the new behaviors
• Support from peers who may not have had the training
• Change in organizational policies, procedures and structures if needed. These are the non- training interventions that may be necessary in order to bring about change in performance

There are many factors that influence these culture related factors, which would need to be explained in detail in a different context.

For any evaluation, the criteria for evaluation have to be defined with care. The definition of the criteria depends on framing the learning objectives. For level III especially this becomes critical, because if the criteria for measuring behavior are not pre-defined with care, level III evaluation can become challenging. In the next blog we shall consider the relation between learning objectives and evaluation. In the blogs that follow we shall look at different criteria and assessment methods for level III.