The Ultimate Test
BY DANIELLE BOYKIN
Since the first Principles and Practice of Engineering (PE) Exam was administered in 1966 by the National Council of Examiners for Engineering and Surveying, it has served as a critical step on the path to becoming a licensed engineer. Over the course of 50 years, the exams have evolved to meet the distinct needs of various industries and sectors and to ensure that engineers have the experience necessary to protect the public health, safety, and welfare. In order to make this happen, NCEES needs the participation of PE volunteers to serve as subject-matter experts to write and evaluate exam questions and maintain the quality of the exam. This engagement will be even more critical as the council prepares to convert the exam to a computer-based format.
Each year, nearly 25,000 engineers take the PE exam to show that they have the knowledge and skills to become a professional engineer. Ensuring that all licensure candidates have a quality examination is no easy feat. Every six to eight years, each PE exam goes through a rigorous review and updates via a process that takes 12 to 18 months, says NCEES Director of Exam Services Tim Miller, P.E. This process begins with a designated exam development committee assessing the need for new specifications and questions. “They determine the tasks, skills, and knowledge that people with four years of experience and a degree from an accredited program need to know to be able to practice and protect the public health, safety, and welfare,” says the NSPE member.
Finger on the Pulse
NCEES Past President David Whitman, P.E., believes this process is necessary to keep up with changes in the profession. “Historically industries change enough every six to eight years that we need to keep our finger on the pulse of this change,” says Whitman, a Wyoming licensing board member and NSPE member.
Change is inevitable. In 1996, NCEES voted to transform the single FE exam taken by all majors into multiple FE exams based on six primary majors. The PE exams have also morphed over time into exams that are more specific to engineering specialties. The single mechanical engineering PE exam from 1995, says Whitman, doesn’t reflect what mechanical engineers are doing now in 2016. There are now three different mechanical engineering PE exams.
“If we are going to put a PE designation on someone, we want to do that under the current state of that particular industry,” says the University of Wyoming professor. “Once they get their license, they are expected to participate in continuing education to keep up with industry changes.”
The exam review process requires the help of licensed engineers who have a strong knowledge of the industry to participate in a professional activities and knowledge study (PAKS). The study asks participants to rate various tasks, knowledge areas, and skills based on their level of relevance. At least 200 respondents are needed to ensure an accurate picture of current industry practices. NSPE and other societies can help spread the word about the studies, Whitman says. “If we get less than 200 respondents, we may not be building an exam that matches what’s going on in these industries.”
The PAKS is an important part of the process because of a PE’s responsibility to protect the public, says Whitman. “As a board member, I have to be confident that a newly licensed PE is taking the exam under the current state of the industry,” he says.
Once the study is concluded, the committee analyzes the results with a psychometric consultant to gather the top-rated tasks, knowledge areas, and skills to help establish exam specifications and questions. Committee members also determine if there are ratings correlations within different groups of respondents. “For example, are engineers in the Northwest responding differently than groups of engineers in the Southeast,” says Miller. “Are ratings differing among engineers with differing years of experience? We want to be thorough in this process.”
Once the new exam specifications have been assembled and reviewed, the Committee on Examinations for Professional Engineers must give the final approval for the new specifications. This oversight body also focuses on the exams’ long-term quality, which involves pass rates, cut scores, and viability of the questions. Usually there aren’t major changes in the specifications, says Whitman, who serves as chair of the committee. Over the last decade, both the electrical and mechanical exams have become multiple exams. In the electrical area, power is a completely separate exam from computer engineering or electronics. “This reflects the development of the industry into a more concentrated focus,” he says.
Wendy Lick, P.E., recently helped complete the PAKS process for the mechanical engineering exams. The goal was to make sure they keep up with current practices and ensure a smooth transition into the future. “Mechanical engineering is one of the oldest disciplines and isn’t a rapidly changing field like software engineering,” says the chair of the PE Mechanical Exam Development Committee. “We aimed to set up the exam for future success as we move toward computer-based testing.”
In 2012, NCEES voted to transition the PE exams to a computer-based format at the earliest feasible date but no earlier than 2015. The FE exam changed to a computer-based format in January 2014, with seven discipline-specific exams.
Lick adds, “Once a large exam like ours moves to computer-based testing, the computer assembles a different exam for each test taker. We want to make sure there is a good quality control process.”
The Experience Factor
In 2014, NCEES voted to remove from the Model Law the requirement for a licensure candidate to have at least four years of experience prior to taking the PE exam. Arizona, California, Illinois, Kentucky, Nevada, New Mexico, South Carolina, Utah, and Wyoming no longer require four years of experience before a candidate can take the exam.
Even if candidates can take the exam early, says Miller, it is still designed for an engineer who has four years of experience. “What we’ve found so far is that the highest pass rate for first-time test takers are candidates with four to five years of experience,” he says.
Whitman says that the early examination option has increased the number of Wyoming licensure candidates who are taking the exam before having the full four years of experience. Despite the option to take the exam early, candidates are still strongly encouraged to gain three to four years of experience first. “Are there some people that seem to absorb everything quickly? Absolutely; they can manage to pass,” he says. “But the vast majority would struggle in that exam after just one year of experience.”
Whitman adds, “Even if you pass the exam after three years, you can’t get licensed until you have acquired the full four years of experience.”
NSPE recommends that state licensing boards provide flexibility for optional early taking of the PE exam by candidates who have met the educational requirements for licensure and passed the FE exam. The Society also believes, however, that the four years of progressive engineering experience recommended by the Model Law should remain unchanged.
Energy and Dedication
The exam development process runs on the energy and dedication of its volunteers, says Lick, who has served as a volunteer in various capacities since 2001. NCEES makes every effort to ensure that the exam committees consist of a group of licensed engineers diversified by age, gender, ethnicity, geographic location, and years of practice, adds Miller.
Lick believes that her volunteer activities have given her a greater appreciation for the profession and the exam. “Before I took the PE exam, I thought they were just throwing in this extra hurdle to become licensed,” she recalls. “I believed that I had proved myself by getting a degree and gaining work experience. I also had people who could vouch for me.”
Lick’s mindset changed after she volunteered for the PAKS. “I saw the need for the exam because people who had managed to get their degrees and the necessary years of experience were getting questions wrong that they should have gotten right,” she says. “We weren’t asking people to do anything outstanding or be super engineers, but only to demonstrate minimal competence. We really do need this extra layer of protection for the public.”
She adds, “We know that there are people who are not competent, and it’s our responsibility with this exam process to keep them from being in responsible charge of engineering projects.”
Another reason that Lick continues to volunteer: the professional comradery. “We actually get into arguments and deep discussions about technical questions and topics,” she says. “I get a huge kick out being in a room full of people who are not embarrassed about being nerds.”
Do you want to give back to the profession by volunteering your time and expertise for the exam development process? Get more information at http://ncees.org/audience-landing-pages/volunteers.
The PE Exam: Myths and Truths
TRUTH All examinees are graded to the same standard of minimum competence. In looking at the pools of exam takers, the first-time taker pool comprises high-ability, mid-ability, and low-ability examinees. When the exam is given, most of the high-ability examinees, some of the mid-ability group, and very few low-ability examinees pass and leave the examinee pool.
Examinees who fail the exam move to the repeat-taker pool. Unless they significantly change their preparation methods, history has shown that they will fail again. When they take the next exam, some of the mid-ability examinees pass and perhaps a few of the low-ability examinees who made a significant change in their preparation pass. That’s why repeat passing rates are lower.
TRUTH The scoring process negates any differences in difficulty. Examinees are graded against a standard of minimum competence. If the exam is more difficult, the required passing score is lowered. Conversely, if the exam is less difficult, the required passing score is raised. An examinee is neither penalized nor rewarded for the difficulty of the exam.
TRUTH After an exam, subject-matter experts review questions that have statistical anomalies or receive comments from examinees. If a question is found to have two correct answers, all examinees with either answer are marked correct. If a question is deemed to not have a correct answer, all examinees with any answer are marked correct.
TRUTH The passing score does not vary from state to state. Examinees in all jurisdictions have the same required passing score for their discipline for that administration.
TRUTH NCEES develops, publishes, and scores the exams on behalf of the state licensing boards to determine minimum competency. State licensing boards grant licenses to candidates in their jurisdiction based on that jurisdiction’s requirements.
TRUTH The pass score is set at the level of minimum competence. Examinees that exceed the score will pass.
TRUTH Before results were reported as pass-fail, examinees received scaled scores. The passing raw score (different for each discipline and varying from administration to administration based on difficulty) was “set” at 70, and all scores were scaled accordingly. In 2005, NCEES voted to provide only pass-fail results. Somehow, over time, an urban legend developed and the scaled score of 70 erroneously turned into a 70% raw score being required to pass.
TRUTH Exam preparation instructors are not involved in the exam process (an NCEES policy) and don’t know the score required to pass.