Currently implemented accuracy metrics in open-source libraries for segmentation by supervised machine learning are typically one-dimensional scores [1]. While extremely relevant to evaluate applicability in clinics, anatomical location of segmentation errors is often neglected.

This study aims to include the three-dimensional (3D) spatial information in the development of a novel framework for segmentation accuracy evaluation and comparison between different methods.

Predicted and ground truth (manually segmented) segmentation masks are meshed into 3D surfaces. A template mesh of the same anatomical structure is then registered to all ground truth 3D surfaces. This ensures all surface points on the ground truth meshes to be in the same anatomically homologous order. Next, point-wise surface deviations between the registered ground truth mesh and the meshed segmentation prediction are calculated and allow for color plotting of point-wise descriptive statistics. Statistical parametric mapping includes point-wise false discovery rate (FDR) adjusted p-values (also referred to as q-values).

The framework reads volumetric image data containing the segmentation masks of both ground truth and segmentation prediction. 3D color plots containing descriptive statistics (mean absolute value, maximal value,…) on point-wise segmentation errors are rendered. As an example, we compared segmentation results of nnUNet [2], UNet++ [3] and UNETR [4] by visualizing the mean absolute error (surface deviation from ground truth) as a color plot on the 3D model of bone and cartilage of the mean distal femur.

A novel framework to evaluate segmentation accuracy is presented. Output includes anatomical information on the segmentation errors, as well as point-wise comparative statistics on different segmentation algorithms. Clearly, this allows for a better informed decision-making process when selecting the best algorithm for a specific clinical application.

Trochlear dysplasia is a specific morphotype of the knee, characterized by but not limited to a specific anatomy of the trochlea. The notch, posterior femur and tibial plateau also seem to be involved. In our study we conducted a semi-automated landmark-based 3D analysis on the distal femur, tibial plateau and patella.

The knee morphology of a study population (n=20), diagnosed with trochlear dysplasia and a history of recurrent patellar dislocation was compared to a gender- and age-matched control group (n=20). The arthro-CT scan-based 3D-models were isotropically scaled and landmark-based reference planes were created for quantification of the morphometry. Statistical analysis was performed to detect shape differences between the femur, tibia and patella as individual bone models (Mann-Whitney U test) and to detect differences in size agreement between femur and tibia (Pearson's correlation test).

The size of the femur did not differ significantly between the two groups, but the maximum size difference (scaling factor) over all cases was 35%. Significant differences were observed in the trochlear dysplasia (TD) versus control group for all conventional parameters. Morphometrical measurements showed also significant differences in the three directions (anteroposterior (AP), mediolateral (ML), proximodistal (PD)) for the distal femur, tibia and patella. Correlation tests between the width of the distal femur and the tibial plateau revealed that TD knees show less agreement between femur and tibia than the control knees; this was observed for the overall width (TD: r=0.172; p=0.494 - control group: r=0.636; p=0.003) and the medial compartment (TD: r=0.164; p=0.516 - control group: r=0.679; p=0.001), but not for the lateral compartment (TD: r=0.512; p=0.029 - control: r=0.683; p=0.001). In both groups the intercondylar eminence width was strongly correlated with the notch width (TD: r=0.791; p=0.001 - control: r=0.643; p=0.002).

The morphology of the trochleodysplastic knee differs significantly from the normal knee by means of an increased ratio of AP/ML width for both femur and tibia, a smaller femoral notch and a lack of correspondence in mediolateral width between the femur and tibia. More specifically, the medial femoral condyle shows no correlation with the medial tibial plateau.