NATE Report Card

We are instructors in a two-year state technical college hvacr program and have been watching the debate rage on about NATE/ACE, ARI’s ICE, and Hvac Excellence. We have noted in particular the comments like “There were too many questions that had more than one correct answer” directed at NATE/ACE. We have heard this from several sources as well asThe News“Feedback” column. As instructors, we are intimately familiar with the problem of writing test questions that yield one clear answer.

I think a perfect example of the criticisms of NATE/ACE and a trap that is very easy for the test writer to fall into was contained in the latest edition of The News dated Jan. 29, which included a set of questions from NATE on “The Fundamentals,” page 50.

Question one asks, “What is the most accurate method for charging an air conditioning system?” The correct answer is “C, the weigh-in procedure.” Good, we agree.

Question four later asks, “Air conditioning systems that use thermostatic expansion valves can use which of the following methods for accurate charging?” The problem here is that there are two correct answers: “B, subcooling” and “C, weigh-in.” The most accurate method is the weigh-in, but the point that the writer may be trying to raise (as indicated by the answer given as correct) is that lacking the manufacturer’s suggested weight, the subcooling method is entirely adequate. Unfortunately, the question doesn’t fit the answer (or, the answer doesn’t fit the question).

I agree in principle with NATE’s goals and encourage any activity to certify and develop the skills of technicians. However, it appears the question database that NATE has developed for its tests is in need of careful editing and coordination, and at present doesn’t reflect the quality that NATE is trying to represent.

Kenneth E. Olsen David E. Maricle Instructors Commercial Industrial Refrigeration/HVAC Technology Bellingham Technical College Bellingham, WA

Nate’s Response

Thanks for the input. And you are right that there are two correct answers. However, these questions do not come from the NATE question data bank. To prevent a compromise of the actual test questions, I wrote the questions that appeared. I missed making sure the material was totally proofed. The News needed the piece right after the Technical Committee completed an extensive review (Jan. 8-12, 2001) of the NATE question data bank. I prepared the information in about three days, and in the future I will need to make sure I cover all my bases. Thanks for the catch.

Anyone who can catch mistakes can be a great help to the overall process of making our technicians better. Please feel free to contact me with any comments that you have to help me make the testing process better. If you would like, please submit questions to me that could be used by the Technical Committee. They will be run through the subject matter question validation process by the committee in August when we meet again. Again, thanks for the catch and the input. Keep in touch.

Pat Murphy Director, Technical Development NATE

Publication date: 02/12/2001

Put Them to the Test

In my present position as a technical director for a large, factory-owned distributor, I find most technicians do not know the basics of this highly technical field. I believe the NATE program attempts to identify good technicians.

A well-rounded technician should have customer service skills as well as technical skills. Therefore, to complain about testing the soft skills in the exam is wrong. Do you want a technician who cannot explain the work he/she is about to perform or has completed? I believe not.

When a technician is on the job, he/she must make many decisions about the installation and service of equipment. Based on a technician’s experience and education, he/she should find the best solutions to problems. I think that NATE questions attempt to test this ability by looking for the best answer to a question.

If certification is awarded only for the purpose of wearing a patch on one’s uniform, then the entire program has no value to the industry. However, if a comprehensive exam covers all that a technician needs to know or do, then we have done a great service for the industry.

Harvey Caplan Savage, MD

Publication date: 02/12/2001