by Theresa L. Bohannan, MPH
Disclaimer: The following is an excerpt and adaption from A Guide to Conducting Effective Training Evaluations: Recommendations, Strategies and Tools for Dependency Court Improvement Programs (Guide), authored by Dr. Sophia Gatowski and Dr. Shirley Dobbin.
Professionals who want to increase their knowledge about specialty topics in their field typically attend conferences, continuing education programs, or other forms of training. Professional development can take resources away from daily activities so it is critical that we know we are spending our time wisely and that trainings are effective. Trainees fill out surveys, either online or in-person, asking what they thought of the food, the venue, the materials, the speaker, etc. However, is that training increasing trainees’ knowledge about a subject matter? Is the training effective at changing attitudes and behaviors? The way to get to these answers is by conducting effective training evaluation. Conducting effective training evaluation is imperative to ensuring people are receiving the intended benefits of the training event.
State Court Improvement Program (CIP) coordinators tasked with judicial trainings related to child abuse and neglect issues may find evaluation a difficult step in planning. The stress on Continuous Quality Improvement (CQI) in recent years has illuminated the need to improve and tailor training. A component of this is careful evaluation before, during, and after training events.
The National Council of Juvenile and Family Court Judges (NCJFCJ), as part of the National Resource Center on Legal and Judicial Issues, a service of the Children’s Bureau, developed A Guide to Conducting Effective Training Evaluations: Recommendations, Strategies and Tools for Dependency Court Improvement Programs. The Guide assists in identifying training needs, developing training methodologies and evaluation tools, and assessing training outcomes. The Guide does not recommend a universal method to evaluation, but rather provides the tools and resources needed for training coordinators to adopt their own evaluation method. It offers approaches for measuring objectives and outcomes. The Guide seeks to help training managers better target, design, and deliver trainings.
Development of the Guide first started with a comprehensive review of effective strategies for adult education and training programs. We evaluated resource materials and interviewed judicial educators. We examined options for training design, mode of delivery, and obtained perspectives on training evaluation generally and specifically for dependency court systems. An advisory committee made content recommendations and several state CIPs piloted the strategies, guidance, and template evaluation tools. Feedback and lessons learned from these pilot sites enhanced the final product.
The main goals of the Guide are to:
- Provide guidance and strategies for the program planning, design, implementation, and evaluation stages of dependency court training efforts;
- Encourage training coordinators to use the strategies to support local, state, and national training agendas; and
- Provide template tools that facilitate an assessment of satisfaction, skill or knowledge acquisition, behavior, practice and attitude change, and training impacts or outcomes.
The Guide consists of six chapters that lead training program managers through the entire evaluation process.
Chapter One, “Training Program Planning and Evaluation”, focuses on the critical program planning tasks and training evaluation basics. In order to tailor training, a first step is to conduct a training needs assessment to discover gaps in professional development. Training needs assessments will help inform future training planning and ensure professionals in the field are receiving up-to-date information that is relevant to their daily work. Having a comprehensive resource available to training coordinators can greatly improve their planning and eventually improve outcomes for trainees.
Chapter Two, “Training Satisfaction and Reaction Measurement”, provides recommendations and strategies for ensuring that the measurement of training participants is useful.
Chapter Three, “Measuring Learning Acquisition”, centers on measuring learning acquisition by explaining the primary learning modes that occur during training and providing methods for evaluating learning.
Chapter Four, “The Assessment of Behavior and Practice Change”, focuses on measuring behavior and practice change and provides guidance of how to assess whether trainees applied their newly acquired skills.
Chapter Five, “The Assessment of Training Outcomes”, reviews approaches to determining the outcomes associated with training programs.
Chapter Six, “Analyzing, Interpreting, and Reporting Training Evaluation Data”, provides ideas for analyzing, reporting and utilizing training evaluation data and covers the analytical techniques needed to understand results.
Associated with each chapter are Tools and Resources that facilitate development of unique evaluation and assessment tools.
This comprehensible resource will assist State CIPs and other training professionals assigned to designing effective dependency court training across the nation. Methods introduced in the Guide will allow training coordinators to go beyond reporting the number of training programs held and participants trained. It will help assess the impacts and outcomes from training events. The Guide is available on the NCJFCJ website, along with a navigation tool to assist users in determining their understanding of training evaluation and where to begin.