What has to be measured in Level 1?
I have seen Level 1 feedback taken on just a few criteria like the training facilitator, location, etc. Many times questions on content, material, etc. are grouped together, which leads to ambiguity. Therefore, specific questions related various criteria have to be asked.
The trainer/facilitator: How was the trainer? Was he/she qualified? Did he/she know the subject well? How was her communication style? Did she create a rapport with the audience or customer? What could be done better? The feedback on the facilitator also gives information whether the facilitator was effective for the group of participants. For example, an organization my engage a well-known facilitator for a training. However, the group that is sent for training may not be up to the facilitator’s expectations. This mismatch may negatively affect rapport building and training. On other occasions participants may be more experienced or knowledgeable than the facilitator. And in such an event, unless the facilitator is really skillful, the training can be a waste.
Training content: This relates to whether the training addressed the learning objectives of the training. What was covered in the training? Was the content appropriate for the participants or was the level too high, too simple or just right? What could be added and what could be deleted. Depending on the feedback on questions related to the content, training can be redesigned to include or exclude content for a similar group of participants
The training materials: To what extent were participants satisfied the materials that were used in training? Was the material interesting? What was given out as handouts, what was given as digital media, as references? Was it sufficient? Was it easy to follow? Were there mistakes? Was it effective? Was it appealing? Material that is created in a hurry, without a thought about how it looks, without consistency of font and design, can put off the participants! Materials need to be appealing, so the participants are motivated to read and explore further!
Training pedagogy: This relates to training methods. Participants are easily bored sitting through lectures and presentations on topics. This is especially true of training that runs for half day and more. Each training session has to be specially designed, and throughout the training program different pedagogies need to be used- like case studies, role plays, games and activities. This is true even for technical and domain training. The facilitator or training designers can be as creative as they can get when developing the pedagogy for different sessions. The important thing to remember is that each session has to relate to the session objectives, and the debriefing needs to happen around that.
Training facilities: This is a very important factor that can make or break the training experience for the participants. Facilities do not always have to be high class, or expensive, but they need to address the needs of the particular groups or the training design. It is very important for training coordinators to check the facilities any training. The training facility and pedagogy are related. For example, if a game is involved, do the facilities provide indoor/ outdoor venues to conduct the game? If a discussion group is involved, do the rooms allow for cluster sitting arrangements? If the facilitator is not experienced or creative to adjust to the limitations of the facilities, the training can be ruined. So either the facilities that can accommodate pedagogies have to be selected, or pedagogies have to be modified to suit the existing or available constraints in the facilities
Facilities also include criteria such as restrooms, cafeteria/ lunch rooms, administrative support like printouts, telephones. Factors such as lighting, windows, etc., can have an effect on training. Training facilities need to have phone numbers of important contacts easily reachable, like the administrators, HR managers, emergencies, etc.
Training coordination: This is another critical factor in training. Training is an event. This has to be managed such that everyone has a good experience, and least problems or issues. This experience can change the attitude of participants. The coordination activities begin from announcing the event, helping select participants, sending out confirmations, addressing issues and questions of the prospective participants, Sending written or electronic communications with dates, venues, times, maps of the locations, being a link between the training mangers, designers, facilitators and participants. The coordinator has to get the material ready in whatever format is required, see to it that the training rooms are equipped with the right tools and technologies. The coordinator needs to be there at the venue before the facilitator to set things in place, and would be the last one to leave, seeing that everything is done. The participants also appreciate the coordinator role as they would have interacted mostly with him/her before the training. Supportive coordinators therefore can influence participant readiness for training. And readiness is the mental state with respect to the degree to which the training would be internalized by the participants.
Training administration: Sometimes roles of coordinators and administrators can overlap. However, administrative duties are more supportive of the event, not necessarily involving interaction with the participants. Mostly, administrators give support like photocopying, blocking training rooms and venues, making reservations for travel and stay, and even help out the participants with office related activities, information, etc. The administrators generally work closely with the coordinators and facilitators to see that the event is successful
Training content: This relates to whether the training addressed the learning objectives of the training. What was covered in the training? Was the content appropriate for the participants or was the level too high, too simple or just right? What could be added and what could be deleted. Depending on the feedback on questions related to the content, training can be redesigned to include or exclude content for a similar group of participants
The training materials: To what extent were participants satisfied the materials that were used in training? Was the material interesting? What was given out as handouts, what was given as digital media, as references? Was it sufficient? Was it easy to follow? Were there mistakes? Was it effective? Was it appealing? Material that is created in a hurry, without a thought about how it looks, without consistency of font and design, can put off the participants! Materials need to be appealing, so the participants are motivated to read and explore further!
Training pedagogy: This relates to training methods. Participants are easily bored sitting through lectures and presentations on topics. This is especially true of training that runs for half day and more. Each training session has to be specially designed, and throughout the training program different pedagogies need to be used- like case studies, role plays, games and activities. This is true even for technical and domain training. The facilitator or training designers can be as creative as they can get when developing the pedagogy for different sessions. The important thing to remember is that each session has to relate to the session objectives, and the debriefing needs to happen around that.
Training facilities: This is a very important factor that can make or break the training experience for the participants. Facilities do not always have to be high class, or expensive, but they need to address the needs of the particular groups or the training design. It is very important for training coordinators to check the facilities any training. The training facility and pedagogy are related. For example, if a game is involved, do the facilities provide indoor/ outdoor venues to conduct the game? If a discussion group is involved, do the rooms allow for cluster sitting arrangements? If the facilitator is not experienced or creative to adjust to the limitations of the facilities, the training can be ruined. So either the facilities that can accommodate pedagogies have to be selected, or pedagogies have to be modified to suit the existing or available constraints in the facilities
Facilities also include criteria such as restrooms, cafeteria/ lunch rooms, administrative support like printouts, telephones. Factors such as lighting, windows, etc., can have an effect on training. Training facilities need to have phone numbers of important contacts easily reachable, like the administrators, HR managers, emergencies, etc.
Training coordination: This is another critical factor in training. Training is an event. This has to be managed such that everyone has a good experience, and least problems or issues. This experience can change the attitude of participants. The coordination activities begin from announcing the event, helping select participants, sending out confirmations, addressing issues and questions of the prospective participants, Sending written or electronic communications with dates, venues, times, maps of the locations, being a link between the training mangers, designers, facilitators and participants. The coordinator has to get the material ready in whatever format is required, see to it that the training rooms are equipped with the right tools and technologies. The coordinator needs to be there at the venue before the facilitator to set things in place, and would be the last one to leave, seeing that everything is done. The participants also appreciate the coordinator role as they would have interacted mostly with him/her before the training. Supportive coordinators therefore can influence participant readiness for training. And readiness is the mental state with respect to the degree to which the training would be internalized by the participants.
Training administration: Sometimes roles of coordinators and administrators can overlap. However, administrative duties are more supportive of the event, not necessarily involving interaction with the participants. Mostly, administrators give support like photocopying, blocking training rooms and venues, making reservations for travel and stay, and even help out the participants with office related activities, information, etc. The administrators generally work closely with the coordinators and facilitators to see that the event is successful
Who can give level 1 feedback?
Mostly, level 1 feedback is taken from the participants. However, level 1 feedback has to be taken from all stakeholders.
Participants: Participants give feedback on all the criteria above. This is helpful in improving the contents, materials, design, facilities, coordination, administration, and even changing the facilitators if need be
Facilitators: Without the feedback from other sources, the picture would not be complete. Facilitators’ feedback needs to be sought on the participant levels—this clarifies what the training managers had in mind versus who was sent for the training. Were the learning objectives suitable? Facilitator feedback can also be taken on the overall coordination, facilities and administrative aspects of the training
Coordinators: Feedback from coordinators is important as it would reveal the challenges in arranging the event, and constraints in arranging the events. The feedback would mainly relate to the interaction with the facilitators and administrators, kinds of questions asked by participants, etc. The coordinators’ feedback would help streamline the training event next time round
Facilitators: Without the feedback from other sources, the picture would not be complete. Facilitators’ feedback needs to be sought on the participant levels—this clarifies what the training managers had in mind versus who was sent for the training. Were the learning objectives suitable? Facilitator feedback can also be taken on the overall coordination, facilities and administrative aspects of the training
Coordinators: Feedback from coordinators is important as it would reveal the challenges in arranging the event, and constraints in arranging the events. The feedback would mainly relate to the interaction with the facilitators and administrators, kinds of questions asked by participants, etc. The coordinators’ feedback would help streamline the training event next time round
When to conduct level 1 evaluation and motivating participants:
Often, training feedback forms are distributed at the end of a training couse as a chore, or as an activity that has to be finished in a hurry. Many participants are already in the mental state of leaving the venue. The facilitator may be tired. To avoid taking feedback in a hurry, half hour of time at the end of the training has to be built into the schedule. The facilitator has to prompt the participants that feedback would be taken, it would be used to redesign and improve the program, and that they can honestly give their feedback. Typically, participants should not be required to write their names on the feedback form- this could be optional.
The training coordinator has to give out and collect the feedback forms. The coordinator has to thank the facilitator and the participants, and then explain the importance of the evaluation. The facilitator preferably should not be present in the room when participants are filling out the feedback forms.
Sometimes, level 1 feedback is taken later on as an online exercise or otherwise. However, time lag would cause a poorer rate of return. It also would not give the instantaneous feelings about the event, rather, the respondents would then be reporting from their memory of the event. The time lag may or may not have a big impact on the feedback depending on how well the training was received. However, unless there are systems in the organization in place as a follow up to training, rate of return may be poor.
The facilitator and coordinator can also give their feedback promptly after the training is done.
Who is communicated this feedback?
Level 1 feedback has to be communicated to all the stakeholders in training. This includes the participants, facilitator/s, coordinator, administration. Feedback should also be sent to supervisors, training managers, HR managers and line managers/ supervisors. The communication should be generally sent by the coordinator, along with a contact number or email id for further communication. This way, channels of communication are always kept open.
How is level 1 evaluation designed?
The goal of Level 1 evaluation is to gather data about the training. The data is limited by the limitations of the instrument. This means that the instrument which is used to collect data has to be reliable and valid. How to create a good instrument to collect feedback will be discussed in an upcoming post.
