Case Study: Examinations with a Limited Budget

High-Quality Healthcare Specialty Examinations with a Limited Budget

The scenario: The United State’s specialty-focused healthcare system created the need for highly specialized certification examinations. The credentialing authority's budget and timeframe for development, administration and updating of that examination was directly impacted by the fact that only a small number of candidate would take the specialized exam each year. However, budgetary restrictions must be balanced with the fact that the examination is critical to the protection of the public and the practitioners.

The challenge: How does the credentialing authority develop and administer a legally defensible examination, maintain the value of the certification, protect the public and serve its stakeholders when only a small number of testing fees will be collected?

The answer: Because of CASTLE Worldwide's experience with a wide variety of healthcare organizations, we were able to adjust our methods and procedures, while maintaining high psychometric standards, to fit the budget, timeframe, and other parameters of the credentialing authority.

Actual Client Experience

Online item-writing training and secure development: When developing items for healthcare professional examinations, the credentialing authority usually organizes a series of meetings during which a relatively large group of participants will write questions to be included in the certification exam being developed. However, these meetings can be costly to hold in terms of hotel, airfare, and per diem for participants. To avoid these costs CASTLE Worldwide’s trained psychometricians and test developers held a series of online training meetings to instruct participants on how to correctly develop a question that will measure a candidate's knowledge of a certain subject and then review the questions for suitability.

Once the training was complete, participants received “homework” assignments and developed items independently through CASTLE Worldwide’s secure online item development tool, I-DEV. Following development, participants used I-DEV to review, modify, and approve items for use on a certification examination. Throughout the process, a specific psychometrician and test developer were available for over-the-telephone and email consultations, answering questions and offering guidance when necessary.

Stable examination committees: Because the credentialing authority needed the same level of professional input in the development of its examination but could not afford the costs of multiple meetings, CASTLE Worldwide put a new procedure in place: committee members agreed to serve three-year terms on the examination development team. Their terms are staggered, so the committee is always comprised of new members and veterans.

The results: The unique item-writing process has proven a big success for the credentialing authority, and it has continued to update and develop its examinations on a yearly basis with CASTLE Worldwide. Since beginning its relationship with CASTLE Worldwide in 1998, the organization has administered its certification examination to more than 2,500 without legal challenge since its inception.

The conclusion: While many aspects of certification examination development are universal no matter what industry is sponsoring the examination, the healthcare sector faces much more serious consequences if the examinations taken by its practitioners are not fair, accurate, and comprehensive. While increased specialization has challenged many credentialing authorities in terms of developing a high-quality examination that is both affordable and respected, this hurdle can be overcome with the application of psychometric principles by experienced, qualified professionals with a proven track record in the healthcare industry.