The Cantemist evaluation script is available on GitHub (beta version).

Please, make sure you have the latest version.

These scripts are distributed as part of the CANcer TExt Mining Shared Task (Cantemist). They are written in Python3 and intended to be run via command line:

$> python3 -g ../gs-data/ -p ../toy-data/ -s norm
$> python3 -g ../gs-data/ -p ../toy-data/ -s ner 
$> python3 -g ../gs-data/gs-coding.tsv -p ../toy-data/pred-coding.tsv -c ../valid-codes.tsv -s coding

They produce the evaluation metrics for the corresponding sub-tracks: precision, recall and F-score for Cantemist-NORM and Cantemist-NER; and Mean Average Precision for the sub-track Catemist-CODING.