Paper-and-Pencil

The tried-and-true testing technique.

For companies, clients, and candidates who prefer paper-and-pencil examinations, our extensive network of testing sites means that we can deliver proctored paper-and-pencil certification and licensure examinations to candidates almost anywhere in the world. Or, if desired, we can administer paper-and-pencil tests in a location of your choice.

Our standard test administration procedures meet or exceed all requirements set forth in the Standards for Educational and Psychological Testing provided by the AERA, APA, and NCME (1999); these procedures are continually evaluated to ensure quality, consistency, and security.

We are so confident of the effectiveness of these measures, CASTLE Worldwide, Inc. actually guarantees security throughout all phases of examination transportation, storage, and disposal of examination materials.

Logistical Expertise

There are a lot of steps, materials, and details involved in paper-and-pencil testing. The good news is that CASTLE Worldwide can expertly handle all of them.

Examination printing. Our examinations are some of the most professional produced by any organization. Experienced editors repeatedly review each examination to ensure proper layout and ease of reading and any grammatical questions are researched thoroughly.

We print examinations in our own onsite printing facility, which includes digital duplicating equipment directly interfaced with our computer networks. Our printing operation, coupled with our substantial editorial experience, gives us total control of all test-related printing from start to finish. The results are not only greater efficiency, but tighter security.

Examination answer sheets. Items developed for multiple-choice examinations are prepared in a format that allows for computer scoring of responses. We then design a computer-scorable answer sheet for the examinations. Each answer sheet is plotted for ease of use; samples are test-scanned on our equipment to ensure accuracy for computer-scanning procedures and reliability of scoring.

Examination shipping. The shipment that goes to the test site administrator includes:

• The appropriate number of examination booklets, answer sheets, and proctor manuals.
• Two #2 pencils per candidate.
• Labels for returning answer sheets.
• Security checklists for the recipient of the examinations.
• A shipping record containing a detailed list of the quantities of the materials.
• Other requested supplies.

Return shipment of materials. CASTLE prepares test administration/shipping instructions as well as shipping labels and containers for each examination. Answer sheets are shipped immediately to our scoring center to expedite scoring.

Upon receipt at CASTLE, test materials are counted and compared to the original shipping inventory as an initial check. Then CASTLE staff members repeat this process as an additional check.

Used answer sheets are then signed in and counted for a third time.

Finally, these numbers are verified using the rosters and sign-in sheets from each test site.

Scoring, score reporting, and statistical analysis. CASTLE uses a proven process to ensure that examinations are properly scored and results are reported accurately to candidates and clients. A thorough statistical analysis of examination results is prepared and we confer with the client to determine the types of reports desired and to ensure that reports are submitted in an appropriate format.

Scanning of answer sheets. We follow several steps to ensure that examinations are properly scored.

Prior to scanning, answer sheets are reviewed individually for accuracy of name and I.D. bubbling and possible scanning difficulties such as sloppy erasures and light bubbling. Any problems detected are immediately recoded.

Examination sheets go through double scanning and scan checking to detect possible errors in the scanning process.

Raw score reports, which show the candidates' answers for each question, are run and reviewed by CASTLE staff on an item-by-item basis. In any case where a candidate appears to have selected two choices, the answer sheet is manually checked for a poor erasure. All cases where candidates appear to omit items are manually checked to see if the candidate's pencil marks were too light. In these cases, manual changes can be made to the scores.

In addition to computer scoring, some candidate sheets are manually scored and compared to the computer results for statistical verification. During all of these procedures, answer sheets are counted and recounted for comparison with rosters and with scanning records to ensure complete accuracy.

After scanning all answer sheets, we box the test booklets and clearly label them for storage in our vault. There they are easily accessible for hand-scoring requests made by candidates while remaining strictly confidential.

Key validation. Immediately after scanning all answer sheets, we produce preliminary item analyses and summary statistics. We then assist our clients in using these data to verify the examination key and make a final determination of the passing point. Once we get the final passing score from the client, we re-score the examination and prepare score reports.

Score reporting and statistical analysis. Within 10 business days of our receipt of the final passing point from the client, CASTLE prepares and mails individual candidate score reports. These reports are printed on carbon mailers for security and are customized to meet the needs of the individual client. Score reports may include the candidate's name, identification number, total and categorical scores, pass/fail status, and a short congratulatory or retest message.

Additional information provided. In addition to candidate score reports, we provide clients with the following:

• Candidate roster.
• "No show" roster.
• Test administration information - a list of the test sites, the number of candidates registered for each site, the number of candidates tested by site, and a description of any irregularities that occurred and their resolution.
• Data analyses - within 30 days of our receipt of the final passing scores, we provide item and summary statistics on each test form. The summary of statistics includes the mode, range of raw scores, average raw score, standard deviation, passing score, standard error of measurement, and reliability indices for the total test. Item statistics include discrimination indices, item difficulty, and distractor effectiveness.
• Companion reports - by applying our statistical software to the candidate database of demographic information, we can also compile a wide range of comparison reports.