Submission

Submission instructions

5 submissions per sub-track will be allowed.

You must submit ONE SINGLE ZIP file with the following structure:

  • One subdirectory per subtask in which you are participating. 
  • In addition, in the parent directory, you must add a README.txt file with your contact details and a really short explanation of your system.
  • If you have more than one system, you can include their predictions and we will evaluate them (up to 5 prediction runs).
  • MEDDOPROF-NER and MEDDOPROF-CLASS: 
    • You must include the Brat annotation files (.ANN) with your predictions. 
    • One annotation file per document. 
    • If you have more than one system, create sub-directories inside the MEDDOPROF-NER and MEDDOPROF-CLASS directories. One subdirectory per system. 
    • If you have more than one system, name the subdirectories with numbers and a recognizable name. For example, 1-systemDL and 2-systemlookup
  • MEDDOPROF-NORM:
    • You must include the tab-separated file with your predictions. 
    • One single file with all the predictions.
    • With a .tsv file extension.  
    • If you have more than one system, include one tab-separated file for each system.
    • If you have more than one system, name the tab-separated files with numbers and a more or less recognizable name. For example, 1-systemDL.tsv and 2-systemlookup.tsv

Download here an example submission ZIP file.

Submission method

Submissions will be made via SFTP.

Download here the submission tutorial.

Submission format

  • MEDDOPROF-NER and MEDDOPROF-CLASS:

Brat Format: one ANN file per document. ANN files have the following format:

Figure 1. Example of submission file for MEDDOPROF-NER
  • MEDDOPROF-NORM

A tab-separated file with four columns: filename, mention string, span and code.

Figure 2. Example of submission file for MEDDOPROF-NORM

Schedule

EventDate (UTC)Link
Sample set releaseFebruary 11th, 2021Zenodo
Training set releaseApril 15th, 2021Zenodo
Annotation Guidelines releaseApril 15th, 2021Zenodo
Evaluation Library release May 3rd, 2021GitHub
Test set release (start of evaluation period)June 1st, 2021Zenodo
End of evaluation period (system submissions)June 9th, 2021Submission Page
Test set with GS annotations releasedJune 12th, 2021Zenodo
Working papers submissionJune 21st, 2021
Notification of acceptance (peer-reviews)June 27th, 2021
Camera-ready system descriptionsJuly 4th, 2021
IberLEF @ SEPLN 2021September 2021