Case Study: Moving an Existing Examination from Paper-and-Pencil to Internet-based Test Administration

Moving an Existing Examination from Paper-and-Pencil to Internet-based Test Administration

The scenario: Many organizations are transitioning existing certification program to Internet-based test administration for reasons of flexibility, increased candidate access, and decreased organizational costs. However, the credentialing authority’s stakeholders and candidates are familiar with the paper-and-pencil procedures, timelines, and methodologies.

The challenge: The move to Internet-based testing can be intimidating to candidates, confusing to stakeholders, and time-intensive to the credentialing authority. How should this transition be managed? What information needs to be communicated? What stakeholder groups have input?

The answer: Because of CASTLE Worldwide's experience transitioning organizations from paper-and-pencil to Internet-based test administrations, we were able to provide guidance on the steps necessary to make the transition smooth for candidates, stakeholders, and staff. The key: communication. The credentialing authority and CASTLE Worldwide worked together to create and implement a communication plan outlining the benefits and limitations of Internet-based testing for candidates and other stakeholders. The plan involved different media to fit each audience, including published handbooks and study guides, conferences, websites, press releases, fact sheets, white papers, newsletters, and journals.

Actual Client Experience

Application processing and eligibility review: When making the transition, the credentialing authority had to address the impact of the change on workflow. Moving from four scheduled paper-based administrations per year to on-demand computer-based testing meant that application processing and eligibility review were spread throughout the year rather than in four short windows. As the credentialing authority processed applications and determined eligibility prior to test scheduling, the organization instituted new internal procedures and timelines for review and approval. A shared, real-time accessible candidate database between the organization and CASTLE Worldwide allowed test scheduling to begin immediately following a candidate’s approval to test. The organization and CASTLE Worldwide could immediately check a candidate’s status within the process.

Updating the candidates and stakeholders: Candidates and stakeholders required complete information on new policies, procedures, and fees associated with Internet-based testing. CASTLE Worldwide and the credentialing authority updated all external communications regarding the examination, including handbooks, study guides, websites, and conference mailings to address the new testing process as well as the revised application policies and procedures. Published handbooks and study guides outlined new testing schedules, including a description of the computer-based delivery system, transition timelines, and updated fees. A practice test delivered through CASTLE Worldwide’s website introduced candidates to the new testing environment while generating revenue for the credentialing authority. The practice test mimicked the test-day experience, allowing candidates to relieve some test anxiety.

With the new policies came numerous candidate questions so additional customer service staff were trained and added at both CASTLE Worldwide and the credentialing authority. The cross-trained staff answered questions about the process and assisted candidates in their completion of applications and test schedule activities in order to ensure a smooth transition.

The results: While the first year of Internet-based testing was bumpy for some candidates, candidates and stakeholders appreciated the extra effort taken by the credentialing authority and CASTLE Worldwide to assist with the change. In year two, the customer service staff was reduced as candidates and stakeholders adapted to the new process.

The conclusion: Change is never easy for an organization. However, the credentialing authority was making the change to its test administration method to increase the examination’s access to its candidates and stakeholders. By communicating the changes and the benefits of the changes early and often through various media and methods, the organization managed the transition as well as change can be managed.