What does the NPTE Passing Score Reflect?

Setting a passing score involves the process by which a performance standard is established. The passing score typically reflects the test score corresponding to a desired level of performance and is used for making decisions about what level of performance is high enough for a given purpose. The passing scores established for the national physical therapy licensure examinations (NPTE) reflect the level of performance required to provide minimally safe and competent physical therapy services by physical therapists and physical therapist assistants.

Individuals scoring at or above the passing score have met the performance standard and are eligible for licensure, and individuals scoring below the passing score have failed to meet the performance standard and are not yet eligible for licensure. The term “passing score” is used interchangeably with the terms cut score and performance standard.

Passing Scores are Policy Decisions

There really is no “true” passing score that can perfectly differentiate candidates who are competent to practice physical therapy in a safe and effective manner from those who cannot. The passing score is the product of a deliberative process of the experience and the judgments of people who are qualified to make those judgments. Setting a passing score is a policy decision related to the mission of the organization, in this case, to ensure that the public is protected from practitioners who are not competent. That policy decision must keep in mind that one must make certain the bar is not so high that it prohibits individuals who do have the necessary knowledge and skills from being able to enter the profession. Public protection must weigh the trade off between ensuring there are enough practitioners to meet the needs of the public while also ensuring those practitioners are safe and effective.

Numerous Steps are Taken to Ensure that the Score is Defensible

Establishing a performance standard is critical because of its impact on individuals, and numerous steps must be taken to ensure that the score is defensible. The Standards for Educational Psychological Testing contains criteria for establishing defensible passing scores, which have been endorsed by many organizations as the standard for practice both in terms of defensibility and best practices in our field. Included among the criteria are the qualifications of individuals participating in the setting of performance standards, the procedures by which performance standards are set, and the efforts made to validate the results of the standard setting process on an ongoing basis. All of these criteria combined are intended to ensure that the process is documented and defensible.

Establishing a passing score, including standards for passing the NPTE, involves six steps.

1. Deciding on a method that is appropriate for the test.
2. Selecting qualified participants (individuals in the profession considered well qualified in their practice).
3. Training participants on the standard-setting method.
4. Providing feedback about participants’ judgments in carrying out the method.
5. Determining an appropriate passing score taking into consideration the panel’s recommendations.
6. Gathering validity evidence that is intended to bear on the question, “Do the candidates who are awarded the license or the credential seem to be the kinds of people who actually possess the kinds of characteristics important for practice?”

Choosing a Method

Historically, the Federation has relied on criterion-referenced methods for setting passing scores since the early 1990s, particularly the modified-Angoff method. This criterion-referenced method asks, “What are the levels of knowledge and skill that are required?” and sets the bar at that level. 

The modified-Angoff technique is probably the most widely used method in health professions today, and it probably has the largest research base. Because of its ease of use and large research base, this method also is considered best practice.

Selecting Participants

The next step is to select qualified participants who are representative of the profession. Participants are selected to be representative of the profession in terms of practice setting, specialty, geographic location, race and gender. Participants also are required to be knowledgeable about how the NPTE is developed.

Training Participants

The first task required for the Angoff method is discussion and formulation of a concept of a minimally competent candidate who possesses the minimum clinical knowledge to practice physical therapy safely and effectively.

As part of their training, the standard setting panelists discuss elements of the NPTE Content Outline, focusing on clinical work activities that define levels of performance required of safe and effective performance as distinguished from those that are above minimal competence or those that are expected even of candidates who do not meet the minimal competence standard.

Rating Test Item Difficulty

After the panel is trained in using the modified-Angoff method and has developed the concept of the minimally qualified candidate, panelists are asked to provide a rating of test items—the rating is an estimate of the proportion of minimally competent examinees that would be expected to get that item correct. Items that would be very difficult for most minimally competent candidates, would get a lower rating, corresponding to a lower number of candidates getting the item right. Items that are expected to be more commonly known by minimally competent candidates would get higher ratings. This judgment is made on each test question included on a test form. At the completion of this task, the proportions are summed and averaged across raters. The result is the panelists’ recommended cut score for the exam form.

Feedback

After the initial round of ratings is completed, panelists are guided through a group discussion of their ratings, focusing on items where the panelists disagreed about the proportion of minimally qualified candidates who would answer the question correctly. Throughout this discussion, the panel is referred back to the discussion of the minimally competent candidate. Panelists often have different perspectives on the difficulty of items for minimally competent candidates, so it is important to have a panel that represents a diverse range of perspectives. Through successive rounds of ratings, candidates are also provided with estimates of the historical difficulty of the items and the pass rate for the exam given their provisional cut score recommendations. This process continues until the panel is satisfied with their cut score recommendation.

Recommendation, Adoption, and Implementation

At the completion of the standard setting panel meeting, a report is prepared for the FSBPT Board of Directors detailing the process and recommendations of the meeting, as well as participant feedback on the process. The board reviews these recommendations and sets a performance standard based on the panel’s recommendation, feedback from the panelists, historical cut scores, and the impact on the field. The performance standard is then translated to cut scores on NPTE forms using Item Response Theory, to ensure forms are of equivalent difficulty.

Setting New Passing Scores

The performance standard must be revisited periodically to ensure that it is responsive to changes in practice. FSBPT reviews and resets the performance standard each time a new NPTE Content Outline is introduced. Additionally, the Board of Directors may empanel a new standard setting task force if there is evidence that the performance standard no longer reflects minimal competence.