We have developed Web-based software for automatic performance evaluation of multiple image segmentations as a tool for study of lesions related to uterine cervical cancer. The images are from a longitudinal study conducted by the Hormonal and Reproductive Epidemiology Branch of the National Cancer Institute (NCI). The tool uses the STAPLE (Simultaneous Truth and Performance Level Estimation) algorithm, which uses expectation-maximization for estimating multi-observer ‘truth’ from a collection of image segmentations. It computes a probabilistic estimate of the ‘true segmentation’ and performance measures for the individual segmentations. There is a need in the imaging community for such a general-purpose tool. Our Web interface enables users to upload and view server-side images, along with multiple expert segmentations, which may have individual specificities and sensitivities assigned. The server returns to the user a ‘ground truth segmentation map’ and corresponding updated specificities and sensitivities. The software was developed in Java and uses the Java Native Interface (JNI) technology for communication with the STAPLE algorithm in the ITK (http://www.itk.org) package.