Some technicians and contractors feel the certification organization is lifting the image of the industry. Training and continued education are being stressed through NATE, and that has some individuals praising the test.
But on the other side of the debate, past test takers of the NATE exam say that many questions are worthless and do not certify a technician on his or her technical skills. Some even say that questions on the exam have more than one correct answer. And even more interesting and more aggravating for some is that technicians with years of experience in the field cannot pass the exam. The debate over the validity of the exam has caused NATE to step back and reassess the test. Officials at NATE say that they are approaching this in the way any technician would approach
a heating or air conditioning system. From time to time, every system needs regular maintenance.
A Closer Look At The Debate
Harvey Caplan is a technical director for United Supply & Distributing Company. He also teaches hvacr for Anne Arundal Community College in Maryland. With this in mind, Caplan knows what techs should be learning to succeed in the industry. He is also NATE certified and a supporter of what the organization is trying to do.Caplan took his certification as a requirement two years ago for a college he was to start working for. He took the exam with approximately eight other individuals; Caplan says he was the only one in the group to pass.
When this happened, he says he started to hear complaints about the exam. Some complaints had to do with the soft skills questions, while others said that some of the questions had more than one correct answer.
“I saw problems with some of the questions,” said Caplan. “But there weren’t enough problems with the questions that I wouldn’t have passed.”
He also explains that the questions may have had more than one correct answer, but the NATE exam asks for best the answer.
So why does the exam have his support? There are a number of reasons. First, Caplan explains that he is a strong advocate of certification programs, whether it is the Industry Competency Exam (ICE) or NATE.
“Our field has become so technical,” explains Caplan. That is why he feels certification exams such as NATE must continue to keep technicians up to par.
He also says that soft skills questions on the exam are very important. While many technicians have found these questions to be unfair, Caplan believes that they are necessary to raise the overall value of technicians.
“They don’t know how to talk to people,” Caplan said about technicians. “Techs have to learn to talk to customers.”
Mixed Results
For the past three years, the Heating, Refrigeration, and Air Conditioning Contractors of Canada (HRAC) has been considering adopting NATE certification and becoming a nonvoting member of the NATE board of directors.Before the organization made any decisions, a pilot testing session was set up to “test the test.”
HRAC recruited 16 volunteers ranging in hvacr experience from two years to 26 years. The average was 13.4 years.
Also, 13 of the 16 volunteers held refrigeration and air con-ditioning mechanics licenses, four held a Fitter 1 certificate, two were oil burner mechanics, two had sheet metal licenses, and all possessed a gas fitter’s ticket.
According to Martin Luymes, HRAC manager, the volunteers recruited were not a “bunch of rookies.”
All test takers had the option to take the installation or service exam and one specialty exam. After the test, NATE scored the exam and gave the results back HRAC.
For the volunteers taking the test, the results were surprising. Only four individuals out of 16 were able to pass both the core and specialty exams. Two others passed only the core and not the specialty. The average grade on the core exam was 61% and 65% on the specialty exam. A 70% score is required to pass.
Luymes says that according to Rex Boynton, NATE president, the percentage that passed the exam is actually quite good. Luymes said that the technicians were told not to study for the exam ahead of time and the scores were acceptable for not studying.
But after a follow-up with the technicians after the test and after receiving their results, many had mixed feelings. The comments that were made reflected some of the same comments that continue to pop up.
“From the technicians there was concern about the questions being ambiguous or having more than one correct answer,” said Luymes.
Others commented that the exam was much harder than expected, and that the breadth of knowledge one needed in order to pass was not what a typical technician would need to know in the field on a regular basis. Some felt that if they could have studied, they would have done better.
There was also a divided opinion with some of the “ethical” or soft skills questions. Approx-imately half felt they were fair questions while others found them inappropriate.
For the time being, HRAC is keeping the lines of communication open with NATE. No immediate plan of action is in effect to create a merger with NATE.
As for the concerns that some of the HRAC technicians had about the exam, Luymes says that some are justifiable, but they are the same complaints that are heard about any test or exam.
And as for the soft skills questions, Luymes says it is up to NATE to decide to keep these questions if it fits the goal the organization is trying to reach.
“I hope NATE doesn’t just change the exam to make people happy,” said Luymes.
Routine Maintenance
NATE does not have any plans to make drastic changes to the test in order to make technicians happy. But the organization is taking technician concerns seriously.Patrick Murphy, director of technical development for NATE, has been taking on the task of maintaining the exam. But Murphy has not been doing it alone. In the last year, he has recruited over 65 members of the industry to either serve on a technical committee or act as committee advisors. The technical committee looks at the questions and makes changes if necessary, and committee advisors bring input to NATE on how things could be improved or subjects that need to be covered. Each member of the technical committee is also a committee advisor.
When NATE was developed, a technical committee was formed, but Murphy says that it was time to bring the members together and bring in new members.
“We had to bring them all together and sit down with the problems we’re experiencing,” said Murphy.
Within the last two and a half months, Murphy and his technical committee looked at the NATE exams and went through each question. The goal, according to Murphy, was to have a revised NATE exam for the AHR Expo in Atlanta, GA.
Each member of the technical committee separately looked at every question and decided whether it was correct and worded clearly. The members then went over every question together to see if problems were found.
Murphy believes that questions with more than one correct answer or with no correct answer have been cleaned up. The same goes for questions that had vague wording.
Another possible problem NATE officials looked into involved questions that could be answered differently in different parts of the country. For example, technicians in one location may do something differently than technicians in another, but neither is necessarily wrong. “Hopefully we’ve taken these questions out,” said Murphy.
As for the soft skills questions, NATE has no intention to eliminate them. Currently, there are only five of these questions on the 50-question core exam.
Murphy says that NATE tests technicians on everything they need to know in the industry, and technicians need to know more than just technical information.
“It’s more than just turn the wrench and turn the screwdriver,” said Murphy about the industry.
He explains that technicians need to know how to handle sales as well as customers and communicate clearly with them. This also includes having the ability to write a service order.
The revised NATE test was presented at this year’s AHR Expo, and Murphy was able to get feedback from the test takers. The technicians are given comment sheets with their NATE test so they can feel free to make suggestions about certain questions or about the test as a whole. Murphy says that he personally reads every comment sheet.
From looking at the scores for the revised NATE test, Murphy says the scores are a bit higher and the feedback is more promising. A couple of technicians were within one or two questions from being certified.
“Technicians said the test was tough but fair,” said Murphy. “It has to be a tough test. It’s not meant to be a cakewalk.”
But Murphy insists that technicians can past the test. Part of the problem may not necessarily be that technicians do not have the qualifications, but that many technicians may not be good test takers.
“For most of the guys in our industry, the last time they took a test was for their refrigeration certification,” said Murphy. He also stresses that some technicians may not have been good test takers when they were in school.
Murphy also said that technicians can prepare for the test. “Training information is out there,” he said. “ACCA has lesson plans, there is RSES, and NHRAW has a home study course.”
Murphy also suggests that technicians take a refresher course at a local vo-tech school.
Although NATE has attempted to tighten up its exam, there is still work to be done. Murphy says that this maintenance will continue with the technical committee at least twice a year. This will help the organization to keep current on changes in the industry.
“Tests need maintaining,” said Murphy. “It’s an ongoing project and a living document.”
Murphy also says that he is always looking for feedback and suggestions on the exam.
“There may be times when we have made mistakes. I cannot arbitrarily change questions, but I can take it to the technical committee.”
Murphy said he is available to take questions, concerns, and feedback about the NATE exam. He can be reached at 703-610-9033 or at ccl2f212@att.net (e-mail).
Sidebar: ARI Announces December ICE Results
The Air-Conditioning & Refrigeration Institute (ARI) recently announced the passing candidates of the December 2000 Industry Competency Exam (ICE).ICE is an entry-level exam that has been developed, supported, and validated by major industry associations. There are three separate ICE exams: Light Commercial Air Conditioning and Heating, Residential Air Conditioning and Heating, and Commercial Refrigeration.
Participants are eligible to take any three of the exams or one of the air conditioning and heating exams and the commercial refrigeration exam.
Each exam has 100 multiple-choice questions, with 60% and above as passing.
The December exams had 1,090 students and instructors taking part and representing 89 different schools.
Here is the breakdown for the following exams:
Passing rate: 59%
Passing rate: 59%
Passing rate: 56%
For a complete list of passing candidates, visit http://www.achrnews.com/CDA/ArticleInformation/news/news_item/0,1342,22376-2-217,00.html
Publication date: 03/12/2001
