MultiClinAI Shared Task Homepage

The MultiClinAI Track is organized by the Barcelona Supercomputing Center’s NLP for Biomedical Information Analysis group and promoted by European projects such as DataTools4Heart and AI4HF.

What is MultiClinAI?

MultiClinAI is a shared task focused on the creation of comparable multilingual corpora via annotation projection, as well as the multilingual extraction of clinical concepts.

For more information about the MultiClinAI task, check the Task Info tab, which includes the Motivation, Subtasks, Schedule and Registration, as well as the Evaluation & Submission tab.

To learn more about the MultiClinAI corpora and how they were annotated, check the Data tab.

MultiClinAI will be held as part of the #SMM4H-HeaRD Workshop in the ACL 2026 conference. For more information about them, check the Workshop tab.

MultiClinAI is organized by the Barcelona Supercomputing Center’s NLP for Biomedical Information Analysis group (formerly Text Mining Unit).

Motivation

Named entity recognition (NER) systems are fundamental for clinical natural language processing (NLP), enabling the identification of key clinical concepts—such as diseases, symptoms, medications, and procedures—in medical documents and electronic health records (EHRs). These systems support clinical workflows and decision-making, as well as large-scale health data analysis. However, their development relies on high-quality expert-annotated corpora, which are costly, time-consuming to produce, and typically language-specific. This poses significant challenges in multilingual settings, particularly for low-resource languages (LRLs).

Multinational clinical studies, rare disease research, and multicentric trials require comparable annotation criteria and interoperable information extraction systems across languages. Yet, multilingual clinical corpora annotated under consistent guidelines remain scarce. Recent advances in machine translation, large language models (LLMs), and generative AI provide new opportunities to translate annotated datasets, project annotations across languages, and create comparable multilingual corpora through annotation projection and entity alignment strategies.

In this context, the MultiClinAI (Multilingual Clinical Entity Annotation Projection and Extraction) shared task addresses the creation and evaluation of comparable multilingual clinical resources across seven languages (Czech, English, Spanish, Dutch, Italian, Romanian and Swedish), focusing on three key entity types: diseases, symptoms, and procedures.

By jointly evaluating multilingual extraction performance and annotation projection strategies across seven languages, MultiClinAI establishes a robust benchmarking scenario for multilingual clinical NLP. The shared task encourages the development of generalizable, transferable, and scalable approaches capable of supporting cross-lingual healthcare applications.

Task Overview

MultiClinAI is divided into two independent but complementary subtasks, as described below:

MultiClinNER: This challenge focuses on multilingual clinical named entity recognition across seven languages (Czech, English, Spanish, Dutch, Italian, Romanian and Swedish). Participants must identify and classify mentions of DISEASE, SYMPTOM, and PROCEDURE entities by predicting their exact spans and types. The task follows a standard entity-level evaluation framework and provides a unified benchmark for comparing monolingual, multilingual, and cross-lingual extraction approaches.

MultiClinCorpus: This challenge focuses on the automatic construction of comparable multilingual clinical corpora from a Spanish gold-standard dataset into six target languages (Czech, English, Dutch, Italian, Romanian and Swedish). Participants must automatically generate comparable annotated corpora through cross-lingual transfer methods. The task evaluates how effectively systems can project and align annotations across languages to produce consistent multilingual clinical resources.

Both subtasks are evaluated using standard classification metrics, including precision, recall, and F1-score. An official evaluation script will be provided to ensure transparency and comparability of results.

Participation in MultiClinAI is flexible:

Teams may participate in one or both subtasks.
Teams may submit results for one, several, or all languages.
Covering all languages is not mandatory.

Data

The training data for the subtasks of MultiClinAI is composed of several well-established clinical corpora, namely DisTEMIST, SympTEMIST, MedProcNER, and the extended version of the CardioCCC corpus. It encompasses three clinical entity types: diseases (DISEASE), symptoms & signs (SYMPTOM) and clinical procedures (PROCEDURE), and is available in seven languages: Spanish (es), Czech (cz), Dutch (nl), English (en), Italian (it), Romanian (ro), and Swedish (sv).

Both subtasks rely on the same underlying textual resources. However, the problem definition, modeling objectives, and evaluation procedures differ between MultiClinNER (Multilingual Comparable Clinical Entity Recognition) and MultiClinCorpus (Multilingual Comparable Clinical Corpus Generation).

The training data for each subtask follows a standardised folder structure for each language and entity type, ensuring consistency across tasks and facilitating system development, which is shown below:

MultiClinAI-training_data_v1.1-260225/
 ├── MultiClinNER/
 │    ├── MultiClinNER-es/
 │    │    ├── MultiClinNER-es-train/
 │    │    │    ├── MultiClinNER-es-train-disease/
 │    │    │    │    ├── ann/
 │    │    │    │    │    ├── MultiClinNER-es-train-disease-0001.ann
 │    │    │    │    │    ├── MultiClinNER-es-train-disease-0002.ann
 │    │    │    │    │    ├── ...
 │    │    │    │    ├── txt/
 │    │    │    │    │    ├── MultiClinNER-es-train-disease-0001.txt
 │    │    │    │    │    ├── MultiClinNER-es-train-disease-0002.txt
 │    │    │    │    │    ├── ...
 │    │    │    ├── MultiClinNER-es-train-symptom/
 │    │    │    │    ├── ...
 │    │    │    ├── MultiClinNER-es-train-procedure/
 │    │    │    │    ├── ...
 │    ├── MultiClinNER-cz/
 │    │    ├── MultiClinNER-cz-train/
 │    │    │    ├── MultiClinNER-cz-train-disease/
 │    │    │    │    ├── ann/
 │    │    │    │    │    ├── ...
 │    │    │    │    ├── txt/
 │    │    │    │    │    ├── ...
 │    │    │    ├── MultiClinNER-cz-train-symptom/
 │    │    │    │    ├── ...
 │    │    │    ├── MultiClinNER-cz-train-procedure/
 │    │    │    │    ├── ...
 │    ├── MultiClinNER-{nl,en,it,ro,sv}/ (same as es and cz)
 ├── MultiClinCorpus/ (same as MultiClinNER folder)

Data access is granted upon registration and, where applicable, agreement to the data usage terms. For additional details, please consult the Data tab.

Registration

https://forms.gle/oE9gfaNxFw2f6gyX6

Schedule

Event	Date (Midnight CET)
MultiClinNER subtask training set release	February 6, 2026
MultiClinCorpus subtask training set release	February 6, 2026
MultiClinNER test set release (only texts)	March 18, 2026
MultiClinNER test set prediction submissions	March 25, 2026
MultiClinCorpus test set release (only texts)	March 27, 2026
MultiClinCorpus test set prediction submissions	April 9, 2026
Result / evaluation returned to teams	April 14, 2026
Participant proceedings due	April 24, 2026
Notification of acceptance and participant proceedings reviews	May 15, 2026
Camera-ready papers due	May 25, 2026
ACL Proceedings due (hard deadline)	June 1, 2026
Workshop	July 2–3, 2026

Related resources

At the NLP for Biomedical Information Analysis group (formerly Text Mining Unit), one of our missions is the open publication of datasets to train and benchmark biomedical information extraction, normalization and indexing systems. For that reason, we have released multiple datasets as part of shared tasks over the years. If you are interested in MultiClinAI, you might want to take a look at some of our resources and competitions about:

Clinical content extraction: DisTEMIST (diseases), MedProcNER/ProcTEMIST (clinical procedures), SympTEMIST (signs and findings), CANTEMIST (tumour morphology), CodiEsp (coding to ICD), PharmaCoNER (chemicals and proteins), LivingNER (species and humans), MultiCardioNER (diseases and medications, includes the DrugTEMIST corpus as well as cardiology-specific data)
Socio-demographic / Social Determinants of Health content extraction: MEDDOPLACE (locations and more) MEDDOCAN (sensitive data), MEDDOPROF (occupations), ToxHabits (extraction of substance use-related content)
Information extraction in social media: SocialDisNER (diseases), ProfNER (occupations)
Linguistic aspects: BARR1 and BARR2 (abbreviation resolution)
Machine Translation: ClinSpEn (EN<->ES clinical content translation)
Summarization: MultiClinSUM (multilingual summarization of clinical content)

Contact

Salvador Lima-López, Barcelona Supercomputing Center (BSC), Spain: salvador.limalopez@gmail.com
Fernando Gallego-Donoso, Barcelona Supercomputing Center (BSC), Spain: fgallegodonoso@gmail.com