UNICORN Banner

Tasks Overview

The UNICORN challenge includes 20 tasks designed to evaluate model performance across both radiology and pathology. Each task has a defined time limit, which sets the maximum allowed runtime for the algorithm Docker container. For vision and vision-language tasks, each case is processed individually, so the time limit applies per case. For language tasks, the algorithm processes all reports in a single run, meaning the time limit applies to the entire dataset.

Note: Additional tasks may be introduced in the future, also as additional (hidden) test data.

ID Task Name Task Type Modality Domain Metric Time limit
T1 Classifying HE prostate biopsies into ISUP scores Classification Vision Pathology quadratic weighted kappa 600s
T2 Classifying lung nodule malignancy in CT Classification Vision Radiology AUC 300s
T3 Predicting the time to biochemical recurrence in HE prostatectomies Regression Vision Pathology censored c-index 1500s
T4 Predicting slide-level tumor proportion score in NSCLC IHC-stained WSI Classification Vision Pathology quadratic weighted kappa 600s
T5 Detecting signet ring cells in HE-stained WSI of gastric cancer Detection Vision Pathology F1 score 300s
T6 Detecting clinically significant cancer in prostate MRI exams Detection Vision Radiology average of AUROC and AP 600s
T7 Detecting lung nodules in thoracic CT Detection Vision Radiology sensitivity 300s
T8 Detecting mitotic figures in breast cancer HE-stained WSIs Detection Vision Pathology F1 score 300s
T9 Segmenting ROIs in breast cancer HE-stained WSIs Segmentation Vision Pathology DICE 300s
T10 Segmenting lesions within ROIs in CT Segmentation Vision Radiology DICE, long- and short-axis errors 600s
T11 Segmenting three anatomical structures in lumbar spine MRI Segmentation Vision Radiology DICE 600s
T12 Predicting histopathology sample origin Classification Language Pathology unweighted kappa 7200s
T13 Classifying pulmonary nodule presence Classification Language Radiology AUC 7200s
T14 Classifying kidney abnormality Classification Language Radiology AUC 7200s
T15 Predicting Hip Kellgren-Lawrence score Classification Language Radiology unweighted kappa 7200s
T16 Classifying colon histopathology diagnosis Classification Language Pathology macro AUC 7200s
T17 Predicting lesion size measurements Regression Language Radiology RSMAPE 7200s
T18 Predicting prostate volume, PSA, and PSA density Regression Language Radiology RSMAPE 7200s
T19 Anonymizing report Named Entity Recognition Language Radiology + Pathology weighted F1 7200s
T20 Generating caption from WSI Generation Vision-Language Pathology BLEU-4, ROUGE-L, METEOR, CIDER 1500 s