Schedule

DateEventLink
January 13, 2020Train set releaseTraining set
January 13, 2020Development set releaseDevelopment set
January 13, 2020Additional sets releaseAdditional corpora
January 20, 2020Evaluation Script releaseEvaluation Script
March 2, 2020Test set release (includes Background set)Test set
April 28-May 1, 2020MIE2020 WorkshopMIE2020 Programme
May 10, 2020End of evaluation period: Participant results submissionsSubmission
May 12, 2020Results notified. Test set with GS annotations releaseTest set GS
July 17, 2020Participants’ working notes papers submittedEasychair submission
August 14, 2020Notification of acceptance participant papersTBA
August 28, 2020Camera ready paper submissionTBA
September 22-25, 2020CLEF 2020CLEF 2020 conference

Submission

Submission instructions (quick guide)

  1. Compile the 5 prediction files of each subtask in a separate ZIP file.
  2. Access https://easychair.org/conferences/?conf=clefehealth2020runs
  3. Select enter as author
  4. Read the instructions.
    Tips:
    – Use your registration team name and the subtask name in the title.
    – Do not include in the same ZIP files from different subtasks.
    – Verify the files have correct format.
    – Include in the ZIP file a text document explaining who you are, your registration mail and registration name.
  5. Fill the form.
  6. Upload one ZIP and click submit.

Click here for a PDF with more detailed instructions.

Submission file format

Prediction files SHOULD NOT have headings (in the pictures below are added to ease interpretability). The first line of the prediction file should be a prediction.

For CodiEsp-D and CodiEsp-P, the submission file is tab-separated with two columns: clinical case and code. Codes must be ordered by rank/confidence, with more relevant codes first. For example:

CodiEsp-D and CodiEsp-P predictions example screenshot.

For CodiEsp-X, the submission file is also tab-separated. In this case, with four columns: clinical case, reference position, code label, code. Codes do not need to be ordered by rank here. For example:

CodiEsp-X Predictions example screenshot.