Surgical waiting lists have led to development of clinical priority access criteria (CPAC) for prioritisation of patients selected for surgery. Although introduced widely into clinical practice in New Zealand CPAC tools have not been validated. Reliability studies were therefore undertaken by the CPAC Evaluation Consortium. Methodology Thirty eight orthopaedic surgeons practising in public hospitals were randomly selected to participate in a prioritisation exercise using computer administered clinical vignettes. Fifty vignettes were developed from the clinical histories of patients selected for total knee arthroplasty (15), carpal tunnel decompression (15) and miscellaneous orthopaedic procedures (20). These were prioritised using each of 3 priority tools producing scores between 0 and 100: visual analogue scale reflecting global clinical opinion (VAS), a generic point scoring system based on points assigned to 5 clinical domains (GOPC), and diagnosis-specific 5 point Likert scale of priority combined with a predetermined table of a range of scores for each diagnosis (ISS). The extent of inter-surgeon variability was striking but significantly less for ISS than GOPC or VAS. This was entirely explained by the complication of a predetermined table. The other two tools were similar except that the inter centile gap was larger for the clinical opinion based tool (VAS). As access to elective surgery is determined by a fixed financial threshold a reliable scoring system will ensure equity of access. This seems to be best achieved by using the Integrated Scoring System.