Define the Process
Appraisers must receive clear instructions on how to use the measurement device. According to the College of Science and Mathematics at Kennesaw State University, these instructions are known as the "operational definition." If appraisers are evaluating a new tape measure, instructions could include, for example, statements such as "Hold the tape measure flush with the surface to be measured lining up the beginning of the tape measure to the left edge of the measured object." All appraisers must follow the instructions exactly, every time. Suggested improvements to the instructions can be noted, but the process must be tested "as is."
Complete the Appraisals
Between three to four appraisers should be chosen to participate in the appraisals. All parts to be measured should be numbered and randomly assigned to each operator. The operator will measure each part at least twice. However, the parts will be reassigned randomly, so they will not know what their previous measurement was or even that they have previously measured this part.
Repeatability
The Gage R&R ANOVA method measures the repeatability of a measurement. If the operator gets the same measurement during repeated trials, the Gage is said to have high repeatability. Poor repeatability could be due to the device being inaccurate, the instructions being faulty, the operator not following the instructions, or a number of other factors.
Reproducibility
If several different appraisers get the same result when measuring the same object, the measurement device is said to have high reproducibility. Note that this does not mean the device is accurate, but simply that the same result is produced over time.
Comparison to Norms
Gage studies should include comparison to known, accurate measurement devices or accepted standards. Typically these measurements are conducted by expert appraisers. Comparing the results of the measurement device to known standards will tell you if the new measurement device is a valid.