Submission

Submissions must be made via email to < salvador [dot] limalopez [at] gmail [dot] com >.

All submissions must follow these instructions:

• Up to 5 submissions per team and subtask are allowed.
• You must submit ONE SINGLE ZIP file with the following structure:
  – One subdirectory per subtask in which you are participating.
  – In addition, in the parent directory, you must add a README.txt file that includes: team name, team contact information (name and email), concise description of each of the runs, extra resources used (if any, e.g. dictionaries, augmented data). Submissions that fail to include this information will not be counted towards the task.
  – If you have more than one prediction run, the filename for each one must be ordered (starting with 1) and use a recognizable name. For example, 1-systemDL.tsv and 2-systemBERT.tsv.
• The ZIP file’s name must use the same team name used in registration and a timestamp. Since submissions for Subtask 1 and Subtasks 2 and 3 will be done separately, also include the subtask name. For example, symptemist_team-name_subtask23_20231016.zip.

In addition, submissions for all subtasks must be done using tab-separated files with a .TSV file extension. The .TSV files must include headers (i.e. column names). If you have more than one system, include one tab-separated file for each system.

The columns for each subtask are:

• For Subtask 1 (NER):
  • filename: Name of the file from which the procedure mention has been extracted. (provided by the organizers)
  • label: In our case it will always be SINTOMA. (prediction of your system)
  • start_span: Character number where the detected mention starts. (prediction of your system)
  • end_span: Character number where the detected mention ends. (prediction of your system)
  • text: Mention extracted from text. (prediction of your system)

• For Subtasks 2 and 3 (Entity Linking and Multilingual Experimental):
  • filename: Name of the file from which the procedure mention has been extracted. (provided by the organizers)
  • label: In this case it will always be SINTOMA. (provided by the organizers)
  • start_span: Character number where the detected mention starts. (provided by the organizers)
  • end_span: Character number where the detected mention ends (provided by the organizers).
  • text: Mention extracted from text. (provided by the organizers)
  • code: List of Snomed-CT concept codes linked to the mention. (prediction of your system)