Lorin Mueller, PhD, David Relling, PT, PhD and Richard Woolf, PT, DPT, CSCS
Beginning with the first testing date in January 2013, the NPTE-PT and NPTE-PTA examinations will be based on new content outlines. These changes are necessary to keep pace with the changing practice requirements for entry-level physical therapists and physical therapist assistants. This article briefly describes the changes to the content specifications for the NPTE examinations, the revision of the passing standard to match the new content outlines, and the changes to the NPTE scaled scores that will be introduced in 2013.
New Content Outlines
The NPTE examinations are based on content outlines that specify the number of items on each examination form that must relate to each topic in a given practice area. These content outlines, also sometimes called “test specifications” or “test blueprints,” are necessary to ensure that each NPTE examination form represents an appropriate combination of topics and is equal to all other forms in difficulty.
The new 2013 content outlines represent a refocusing of the examinations on core physical therapy principles rather than a major change in examination content. Information about the development of the new content outlines, along with a detailed description of the major content areas, can be found on the FSBPT website: https://www.fsbpt.org/ForCandidatesAndLicensees/NPTE/ExamDevelopment/.
The changes that examinees will see in the new content outlines are summarized here. The changes represent the input of thousands of physical therapy practitioners who responded to surveys about current practice and dozens of experts who served on committees to help FSBPT interpret the survey data.
For the NPTE-PT examination, items relating to Physical Therapy Examination of the Metabolic & Endocrine Systems were eliminated from the content outline. Committee members suggested that although these topics were important, the clinical examination activities related to these systems were increasingly performed by other healthcare providers. The content areas of Clinical Application of Foundational Sciences within the various systems were eliminated as separate categories, and appropriate items were reclassified to the remaining content areas in each body system. Similarly, appropriate items relating to Teaching and Learning were reassessed and moved to content areas and body systems areas that were more directly matched to the knowledge they measured. The redistribution of the Teaching and Learning items led to discontinuing this topic area. In addition, some of the items previously classified as relating to Research and Evidence-Based Practice were reallocated to the content areas and body systems that are the focus of the research scenario or data described in the item, and the items remaining in Research and Evidence-Based Practice, are more focused on research methods. A new area in the content outline is Physical Therapy Examination related to the Genitourinary System; one or two items on this topic will be included on each form starting in 2013. Lastly, the content area Safety, Protection, & Professional Roles was split into two areas: Safety & Protection and Professional Responsibilities.
For the NPTE-PTA examination, many of the changes follow a similar theme. The area of Physical Therapy Data Collection of the Metabolic & Endocrine Systems was eliminated from the content outline because the committee members found that these activities were typically performed by other healthcare providers. Appropriate items relating to Teaching and Learning were reclassified into the Interventions content area of corresponding body systems. Items classified under the topic Clinical Application of Foundational Sciences were moved into the appropriate areas of Interventions, Physical Therapy Data Collection, or a new content area called Diseases/Conditions that Impact Effective Treatment. New areas in the content outline include Interventions related to the Gastrointestinal System and Genitourinary System. In 2013, there will be one or two items on each of these topics on each examination form. Lastly, the topic Safety, Protection, & Professional Roles was split into two topics: Safety & Protection and Professional Responsibilities.
Tables 1 and 2 below show the number of items relating to each topic organized by body system. As the tables show, the body systems that are gaining many new items on both the PT and PTA exams are Cardiovascular/Pulmonary & Lymphatic Systems, Musculoskeletal System, and Neuromuscular & Nervous Systems.
New Standards for Minimal Competence
Any time we revise the content outlines, it is necessary to consider the impact of the new content outlines on the standards we used to determine minimal competence. In practice, the standard for minimal competence on the NPTE is the cut score for each examination form. Introduction of a new content outline may make an examination more difficult or easier, relative to the existing standard for minimal competence. At the same time, it is useful to determine whether the changes in practice that influence the new content outlines necessitate a change in how we define the standard for minimal competence. That is, are PTs and PTAs asked to do more or less at entry level than they were when we set the previous standard?
To examine this question, FSBPT hosted expert panels in May and June to help us determine the standard for minimal competence at entry level for PTs and PTAs. These panels were representative of PTs and PTAs in practice with respect to national region, practice setting, area of expertise, and demographics. The panels were approximately equally split between educators, entry-level practitioners, and experienced practitioners who supervise entry-level practitioners. Our goal was to have panels that could have a discussion about the topics on the examination with the widest variety of perspectives represented.
Each panel reviewed a simulated examination form constructed to exact FSBPT standards through a highly structured process. First, the panelists took a half-length version of the examination, as if they were examinees, to remind them of the difficulty of answering questions under exam-like circumstances. Next, we had a discussion of what constitutes minimum competence at entry level, using the content outlines to guide the discussion. Following that discussion, each panel reviewed the full form of the examination used in the simulation, rated each item in terms of the difficulty for entry-level practitioners, and then discussed their ratings. Each panel went through three rounds of rating and discussion. At the end of the process, both panels made slight adjustments to the entry level competence standards for the NPTE. To ensure that we weren’t raising the standard just for the sake of having higher standards, we engaged the panels in a discussion in which they provided specific examples of topics where the current performance on the examination was being elevated.
For the PT examination, the panel noted that the increased prevalence of direct access meant that PTs would need to be more cognizant of factors that might affect treatment. Similarly, societal demographic changes are resulting in an increase in medically complex cases. These two factors also relate to the increasing importance of differential diagnosis for PT practice. The panel also noted that research and evidence-based practice is important, and as research findings become more easily available through electronic access, PTs must be increasingly aware of how to utilize the new information. Lastly, the PT panel noted that insurance reimbursement is driving a lot of change: requiring a higher emphasis on outcomes and documentation.
For the PTA examination, many similar themes emerged. Increasing demand for PT and PTA services have made remote or limited supervision arrangements more common, such as when a PTA is called upon to provide in-home care. It is becoming increasingly important for PTAs to recognize “red flags” or contraindications and respond accordingly by stopping treatment and contacting the appropriate healthcare providers as necessary. The PTA panel also noted the increase in medically complex cases and the need for PTAs to be effective consumers of research. The PTA panel also noted the influence of insurance reimbursement issues in changing the standard for entry-level PTAs, specifically by requiring PTAs to be more familiar with a broader range of medical terminology used to document treatment and outcomes.
The impacts of these changes on the passing standard for the NPTE are estimated to be minimal. In most cases, panelists wanted PT and PTA licensure candidates to perform a little better on a few items. In real terms, we expect the rising standard may impact less than 1% of PT candidates, and less than 3% of PTA candidates. So while the standards are changing, they are not changing drastically.
New Scaling Procedures
With the introduction of the new content outlines and new minimum competence standards, FSBPT also took the opportunity to make one more change to the NPTE examinations. Specifically, we wanted to make some minor refinements to the way we calculate scale scores on the examinations.
Prior to 2013, we used an examinee’s raw score to calculate his or her scale score. The minimum passing score was set to a scale score of 600, and the top score was set to 800. This process meant that scores increased in a straight line, like a simple algebra equation. For all forms, in any given year, this equation was the same or very similar, since we do a lot to make sure that examination cut scores are very close across forms.
However, the processes we used to actually build examination forms is much more complex than a simple linear algebra equation. We use a process known as “Item Response Theory” that provides us with very precise form equating and acknowledges that the difficulty of improving one’s score in subsequent administrations is very different for people who answer very few questions correctly (are guessing), people who answer correctly a number of questions that put them right around the cut score, and people who answer almost all of the questions correctly.
Starting in January 2013, we will begin scaling the NPTE forms in the same way that we construct the examinations. Making this change required a significant change in our scoring software and procedures, but we feel the new scores will be more informative to examinees, especially those who may need feedback on their performance after failing the examination. The advantages of the new scaling procedures are listed below.
• The process of calculating subscale scores for each content area and body system on standard score reports, Performance Feedback Reports, and School Reports will be slightly more accurate.• Some scale scores were almost never used because candidates did not score high enough to obtain them, particularly those above 750. The new process uses the entire range between 600 and 800, making score differentiation better for schools with high-scoring candidates.• We can report the extent to which scores are expected to vary across test forms, which may help examinees who have failed determine how far they were from passing. This information will be available in the new Performance Feedback Reports, available for all test administrations in January 2013 and later.
We were also able to accomplish the rescaling in a way that will make scores maximally comparable for examinees who need scoring information the most: those who failed the examination. Figures 1 and 2 show the comparison of the 2012 scores and 2013 scores scaled on a “standardized knowledge scale” that we use to equate test forms. As the figures show, for examinees scoring below the cut score, total scale scores will remain very close. For examinees scoring above the cut score, scale scores will be somewhat higher in 2013 than they were in 2012.
FSBPT anticipates that some schools may encounter difficulty if they intend to compare scores from 2012 to scores from 2013, especially those schools with a high proportion of students who score well above 600. We will produce a simple score transformation table in Excel to assist schools in making these comparisons. We will post this table on the FSBPT website under the Exam Development section.
Looking Forward
FSBPT is constantly looking at new ways to improve the NPTE examination and the services we offer in conjunction with the examination. Some readers may have already taken advantage of the new Practice Exam & Assessment Tool (PEAT). The new Performance Feedback Reports will be available to candidates beginning with the January 2013 administration. We believe that the new content outlines, new standards, and new scaling procedures all contribute to ensuring that PTs and PTAs entering practice are competent professionals who will provide effective treatment.
Table 1: NPTE-PT Topic Areas |
# Items 2008-2012 |
# Items 2013-2017 |
Cardiovascular/Pulmonary & Lymphatic systems |
23 |
33 |
Musculoskeletal System |
36 |
61 |
Neuromuscular & Nervous Systems |
34 |
50 |
Integumentary System |
14 |
10 |
Metabolic & Endocrine Systems |
8 |
7 |
Physical Therapy Examination (removed in 2013) |
1 |
-- |
Gastrointestinal System |
4 |
3 |
Genitourinary System |
4 |
4 |
Physical Therapy Examination (new in 2013) |
0 |
1 |
System Interactions (Multi-System in 2008-2012) |
16 |
7 |
Equipment & Devices |
10 |
5 |
Therapeutic Modalities |
12 |
7 |
Safety & Protection |
15 |
5 |
Professional Responsibilities |
15 |
4 |
Teaching & Learning (questions redistributed, topic title removed in 2013) |
11 |
-- |
Research & Evidence-Based Practice |
13 |
4 |
Table 2: NPTE-PTA Topic Areas |
# Items 2008-2012 |
# Items 2013-2017 |
Cardiovascular/Pulmonary & Lymphatic systems |
19 |
25 |
Musculoskeletal System |
32 |
39 |
Neuromuscular & Nervous Systems |
30 |
33 |
Integumentary System |
9 |
7 |
Metabolic & Endocrine Systems |
6 |
6 |
Physical Therapy Examination (removed in 2013) |
1 |
-- |
Gastrointestinal System |
2 |
2 |
Diseases/Conditions that Impact Effective Treatment |
-- |
1 |
Interventions (new in 2013) |
-- |
1 |
Genitourinary System |
-- |
2 |
Diseases/Conditions that Impact Effective Treatment (new in 2013) |
-- |
1 |
Interventions (new in 2013) |
-- |
1 |
System Interactions (Multi-System in 2008-2012) |
11 |
5 |
Equipment & Devices |
9 |
10 |
Therapeutic Modalities |
13 |
12 |
Safety & Protection |
12 |
4 |
Professional Responsibilities |
12 |
3 |
Teaching & Learning |
4 |
-- |
Research & Evidence-Based Practice |
3 |
2 |
Figure 1: 2012 and 2013 Scale Score Comparison for the PT Examination
Figure 2: 2012 and 2013 Scale Score Comparison for the PTA Examination
|
|
|
|
|
Lorin Mueller,PhD Managing Director of Assessment, Federation of State Boards of Physical Therapy
Lorin joined the FSBPT’s Assessment Department in November of 2011. Prior to joining the FSBPT, Lorin spent 10 years as Principal Research Scientist at the American Institutes for Research in Washington, D.C. He has contributed his expertise in statistics, research design, and measurement to projects in a wide variety of areas, including high-stakes test development, work disability assessment, K-12 assessment, assessing students with cognitive disabilities, teacher knowledge, teacher performance evaluation, and school climate. He is a nationally-recognized expert in the field of setting standards for occupational assessments and has published or presented in nearly all of the areas he has worked. Lorin received his Ph.D. in Industrial and Organizational Psychology with a specialization in statistics and measurement in 2002 from the University of Houston.
|
|
|
|
|
|
David Relling, PT, PhD, Associate Professor in the Dept. of Physical Therapy at the University of North Dakota in Grand Forks, ND.
He has practiced in a variety of settings including acute care, orthopedics and long term care. Dave began his involvement with the Federation in 2005 as an item writer and has served as a member and then co-chair of the NPTE Exam Development Committee for physical therapists. He was inducted into the Academy of Advanced Item Writers in 2010. Dave is a current member of the FSBPT Board of Directors and the North Dakota Board of Physical Therapy.
|
|
|
|
|
|
Richard D. Woolf, PT, DPT, CSCS, FSBPT Assessment Content Manager
Richard Woolf is the Assessment Content Manager at the Federation of State Boards of Physical Therapy (FSBPT). He joined the FSBPT in 2008. He is also a Certified Strength and Conditioning Specialist with the National Strength and Conditioning Association. He received his Master of Physical Therapy from Northern Arizona University and his Doctorate of Physical Therapy from A.T. Still University -- Arizona School of Health Sciences.
|
|
|
|