Evidencing Compliance Education and Training Effectiveness
Compliance Program Guidance documents issued by the Department of Health and Human Services Office of Inspector General (OIG) call for mandatory ongoing compliance education and training of corporate officers, managers, physicians, and other health care professionals and employees (collectively “covered persons”) as one of the seven critical elements of an effective compliance program. The Department of Justice’s (DOJ’s) Compliance Program Effectiveness Evaluation Guidelines includes over twenty questions related to compliance education and training. Both the OIG and DOJ establish criteria for what information needs to be included in the training curriculum. Key factors they consider in the effectiveness of the training are:
- Conveying the important and defined compliance information for covered persons;
- Verifying all covered persons underwent the training;
- Substantiating participant understanding and retention of the information provided; and
- Evidencing whether the lessons were applied in the workplace.
Evidencing effectiveness of training is a major challenge for compliance officers. The easy part is proving that all covered persons received the training, which can be done with sign-in sheets or other forms of verification. Evidencing understanding of the information provided in the session is most often addressed by quizzes and tests at the end of the session. However, the reliability of these results depends upon the questions being asked and the passing score. Weak or poorly-constructed questions will be of little value and may negatively impact employees’ perception of the training. Well-constructed quizzes and tests can help identify areas that warrant additional focus in future training programs. There are also more fundamental questions as to whether the tests accurately determine understanding and how much effort will be given to applying the lessons in the workplace.
Most organizations fail to consider or address the larger questions:
- What steps are taken to address those employees who do not pass the tests or quizzes?
- How well do covered persons retain understanding of the training lessons after they were provided?
- How meaningful are the lessons to those receiving it?
- How well do they accept the lessons provided and apply it to their work?
If a compliance officer is unable to adequately answer these questions, then the effectiveness of the compliance education and training is in question. Thought and effort should be given to identifying ways to independently providing these answers. One possibility is to retest employees after several weeks or months to see if they retained understanding of the lessons; however, this may not be very practical. Another solution may be found by following suggestions from the OIG Compliance Program Guidance where they note that “Organizations [should be] evaluating all elements of a compliance program through employee surveys, management assessments, and periodic review of benchmarks.”
Compliance program surveys can independently and objectively test employee compliance knowledge, or perceptions and attitudes towards compliance. The former uses a dichotomous (yes-no) format while the latter generally uses a Likert scale (degree of agreement). These are useful methods to examine the extent to which covered persons evidence their compliance program knowledge, understanding, and commitment. They can also be used to obtain quantifiable data on workforce knowledge and perceptions that can be applied to benchmark and monitor progress against past results. Furthermore, they can be used for comparisons to industry peers and national averages. Using validated and tested surveys anchored with a large database of user information may be the most efficient and cost-effective method to not only evidence compliance training effectiveness but the overall effectiveness of the compliance program.
For more information on this topic, contact Richard Kusserow ([email protected]).
