CLEF 2020

CodiEsp is included in the 2020 Conference and Labs of the Evaluation Forum, this year, an online-only event.

Participants’ working notes will be published in the CEUR-WS proceedings ( Task overview paper will also be published in the CEUR-WS proceedings.

author={Antonio Miranda-Escalada and Aitor Gonzalez-Agirre and Jordi Armengol-Estapé and Martin Krallinger},
title="Overview of automatic clinical coding: annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of {CLEF eHealth} 2020",
booktitle = {{Working Notes of Conference and Labs of the Evaluation (CLEF) Forum}},
series    = {{CEUR} Workshop Proceedings},
year      = {2020},

Since CodiEsp is part of CLEF eHealth lab, lab overview paper will be published in the Springer LNCS proceedings.

author={Lorraine Goeuriot and Hanna Suominen and Liadh Kelly and Antonio Miranda-Escalada and Martin Krallinger and Zhengyang Liu and Gabriella Pasi and Gabriela {Saez Gonzales} and Marco Viviani and Chenchen Xu},
title="Overview of the {CLEF eHealth} Evaluation Lab 2020},
booktitle = {{Experimental IR Meets Multilinguality, Multimodality, and Interaction: Proceedings of the Eleventh International Conference of the CLEF Association (CLEF 2020)
series    = {LNCS Volume number: 12260},
year      = {2020},
editor = {Avi Arampatzis and Evangelos Kanoulas and Theodora Tsikrika and Stefanos Vrochidis and Hideo Joho and Christina Lioma and Carsten Eickhoff and Aurélie Névéol and Linda Cappellato andNicola Ferro},

First Multilingual clinical NLP workshop: MUCLIN (MIE2020/EFMI)

There is an increasing interest in exploiting clinical texts by means of language technologies and text mining approaches. Structured clinical information, in the form of codified clinical records relying on controlled vocabularies such as ICD10 is a key resource for statistical analysis techniques applied to patient data.

Clinical natural language processing (NLP) and AI-based document indexing can result in tools for automatic clinical coding by exploiting directly the unstructured content of EHRs. Such tools are playing an increasing role to generate results that do complement health informatics approaches focusing on translational medicine challenges, by providing relevant diagnostic information extracted from clinical narratives. This implies that text mining generated clinical coding results can provide a rich clinical context for patient health information necessary for other data analysis processes like bioinformatics and OMICS data exploration.

The workshop will include a panel discussion and short flash talks with experts on the role of shared tasks to promote clinical and biomedical NLP.

Some of the proposed topics for the discussion are: generation of shareable data collections for clinical NLP and automatic coding systems, use of AI and deep learning methods applied to clinical text mining, exploitation of unstructured content of EHRs for translational medicine, explainable IA strategies, evaluation metrics and scenarios for automatic clinical coding systems, multilingual clinical coding strategies and shared tasks.

CodiEsp setting and results will be presented at the workshop First Multilingual clinical NLP workshop (MUCLIN) at MIE2020. The original MIE programme can be accesed here.

This year, MIE is virtual thanks to the collaboration of EFMI. Please, register at

This workshop will have two parts:

First, there will be a general overview presentation on shared tasks and the results and setting of a particular shared task, i.e. CodiEsp.

Then, there will be a panel discussion including flash talks by experts on the role of the shared tasks to promote clinical NLP, resources, tools, evaluation methods.

We plan to provide access to the talks in the MUCLIN youtube channel

See EFMI program.

MUCLIN program:

Local TimeTitlePresenterAffiliationTimezoneMore info
1:00 pmOpening RemarksMartin KrallingerBarcelona Supercomputing CenterGMT+2 (Zurich time)Video
1:05 pmCodiEsp: clinical coding shared task in Spanish clinical case reportsAntonio MirandaBarcelona Supercomputing CenterGMT+2 (Zurich time)Video
1:20 pmPanel Session: flash talks
Flash talk 1FLE at CLEF eHealth 2020: Text Mining and Semantic Knowledge for Automated Clinical EncodingNuria García-SantaFujistsuGMT+2 (Zurich time)Video
Flash talk 2Contribution of CLEF eHealth and WMT biomedical translation task to multilingual clinical NLPAurélie NévéolCNRS, FranceGMT+2 (Zurich time)Video
Flash talk 3The importance of shared tasks for biomedical text miningFabio RinaldiUniversity of ZurichGMT+2 (Zurich time)Video
Flash talk 4Multilingual Text MiningFrancisco CoutoUniversity of LisbonGMT+2 (Zurich time)Video
Flash talk 5Lessons learnt on benchmarking at ELIXIRSalvador Capella-GutierrezINB/Elixir, Barcelona Supercomputing CenterGMT+2 (Zurich time)TBA
1:55Closing remarksMartin KrallingerBarcelona Supercomputing CenterGMT+2 (Zurich time)