Evaluation Library

The DISTEMIST evaluation script is available on GitHub (beta version, new release coming next week).

This script is distributed as part of the DiSTEMIST track. It is written in Python3 and intended to be run via command line:

$> python main.py -g ../gs-data/gs_ner.tsv -p ../toy-data/toy_data_ner.tsv -s ner
$> python main.py -g ../gs-data/gs_norm.tsv -p ../toy-data/toy_data_norm.tsv -s norm 

They produce the evaluation metrics for the corresponding sub-tracks.