Submission

The evaluation of the competition will be done against a manually annotated data set purposely created for this task.

Submission format

Both subtasks, ToxNER (Subtask 1) and ToxUse (Subtask 2), share the same submission format.

Participating teams must generate a TSV file for each subtask containing the annotations detected in the test documents, following the column structure below:

• File name (filename): Name of the file from which the entity mention has been extracted
• Mention label (label): The category of the entity.
  • For ToxNER (Subtask 1), the entities could be: Tobacco, Alcohol, Drug, Cannabis
  • For ToxUse (Subtask 2) the entities could be: Amount, Type, Frequency, Duration, Method, History
• Starting character offset (off0): Character index where the detected mention starts.
• Ending character offset (off1): Character index where the detected mention ends.
• Mention span (span): Mention extracted from the text (i.e., the text of the entity)

Example (submission.tsv):

Submission Procedure

The submissions must be sent via email to < wesam.alnabki.bsc [at] gmail.com >.

All submissions must follow the instructions below:

1. Submission Limit:
   • Each team may submit up to 5 runs per subtask (ToxNER and/or ToxUse).
2. Submission Format:
   • Submit one single ZIP file containing your runs with the following structure:
     • A subdirectory for each subtask you are participating in (e.g., ToxNER/, ToxUse/).
     • A README.txt file in the root directory including the following:
       • Team name (same as registered)
       • Team contact person (name and email)
       • Concise description of each run
       • Any extra resources used (e.g., dictionaries, external data, pre-trained models)
     ⚠️ Submissions without a README.txt will not be counted towards the task.
3. Naming Runs:
   • If you submit multiple runs for a subtask, name the TSV files clearly and sequentially:
     • Example: 1-baseline.tsv, 2-bert-model.tsv, etc.
4. ZIP File Naming Convention:
   • Your ZIP file must include:
     • Team name (as used in registration)
     • Subtask(s) covered
     • Submission date as a timestamp (YYYYMMDD)
     • Example:
       toxhabits_teamname_toxner_20240528.zip
toxhabits_teamname_toxuse_20240528.zip
5. TSV File Requirements:
   • Submissions must be in tab-separated files (.tsv) with the following:
     • One row per detected mention
     • The required columns: filename, label, off0, off1, span
     • A header row with column names
     • One file per system/run