melissa kustra
Predictive Questions Evaluation
Process
What if we had a way to increase the value of the data we collect with our Level 1 evaluations?
​
Predictive questions are based on Kirkpatrick's 4 levels of training evaluation. The idea is that we can use self-reported level 1 evaluations to predict the other levels of evaluation: learning gain, training transfer and business results by asking questions that have accompanying formulas so that we have real quantitative data. Instead of asking learners- “Did you like the training?” we ask “Before this training, what do you know about the topic?” “After this training, what do you know about the topic?” then apply numbers to survey responses to calculate learning gain score and see how much participants felt their knowledge increased.
While working for my current organization, FCCI Insurance Group, I presented this evaluation method to take level 1 evaluations and collect measurable predictive data for level 2, 3 and 4 evaluations. I was responsible for presenting the evaluation method to my team, and developing various survey questions for data collection. I am continuing to input data into the predictive question formulas and presenting data to
stakeholders.
​
When data is combined and aggregated across webinar sessions and courses, trends and metrics of our training and webinar impact become apparent. This evaluation method is still in progress and is intended to provide more concrete evaluation data going forward to ensure our training is meeting business needs for our organization.




ISLT Competencies:
Evaluation and Research Skills
-
Conduct a formative evaluation of an instructional intervention.
-
Write a proposal for a program evaluation.
-
Apply appropriate qualitative and quantitative data collection methods.
-
Construct valid & reliable data collection tools.
-
Collect, analyze, and summarize data.
-
Manage the evaluation process.
-
Generate evaluation reports and disseminate findings to stakeholders.
-
Provide a rationale for evaluation decisions.