Evaluation

Evaluation will be done by comparing the automatically generated results to the results generated by manual annotation of experts.

Evaluation metrics

The primary evaluation metric for the sub-track 1 (DisTEMIST – entities) and sub-track 2 (DISTEMIST – linking) will consist of micro-averaged precision, recall and F1-scores:

The used evaluation scripts together with a Readme file with instructions will be released.