en | es
Public Aid Program for the Evaluation Campaign
Medical Semantic Indexing in Spanish: BioASQ MESINESP Task
The Plan for Promoting Language Technologies (Plan TL) aims to promote the development of natural language processing, machine translation and conversational systems in Spanish and co-official languages. In the framework of this plan, the Ministry of Economy and Business, through the Encargo made by the State Secretariat for Digital Advancement (SEAD) to the Barcelona Supercomputing Center Consortium – Centro Nacional de Supercomputación (BSC-CNS) to promote activities for specialized technical assistance in the area of biomedicine, among others, we announce the call for shared task awards detailed below.
In accordance with the provisions of articles 17.3.b and 20.8.a of Law 38/2003, of November 17, General of Subsidies, Article 6 of Law 9/2017 of November 8, on Public Sector Contracts, the special rules applicable to the project competitions for the awarding of prizes established therein and the allocation of budgetary resources established for the financial year 2019 through the financing of the formalized management commission under budget appropriation 20.12.467I.640, we detail below the requirements for the call for shared task awards.
Goal and description of the shared task: As part of the BioASQ challenges on biomedical semantic indexing and question answering, the “OTG de Sanidad” of the Plan TL organizes the task: Medical Semantic indexing in Spanish (BioASQ MESINESP Task), the first task on semantic indexing of Spanish medical texts.
The task will be based on the standard process followed by IBECS and LILACS to index journal abstracts in Spanish. The participants will be asked to classify new IBECS/LILACS documents, written in Spanish. The evaluation of the participating systems will be done against a manually annotated data set purposely created for this task.
Registration: To register as a participant, it is requested to fill-in an on-line registration form that can be found at the participant page of the BioASQ Task.
Deadlines for submission: Medical Semantic indexing in Spanish (BioASQ MESINESP Task) will run in Winter 2019-20 (detailed schedule TBA). Participants, after downloading the released test sets, will have to submit results within a limited time window. For further details, please refer to this link.
Evaluation: The evaluation of the participating systems will be done against a manually annotated data set purposely created for this task. The participating systems will be assessed for their performance based on a flat measure: the label-based micro F-measure.
Task organizers: This task has been organized by the OTG de Sanidad of the Plan TL in collaboration with the National Center for Scientific Research Demokritos (Greece) and the Biblioteca Nacional de Ciencias de la Salud of the Instituto de Salud Carlos III. For further details, please refer to Task Organizers.
Scientific committee: Please refer to Scientific committee.
Selection of winners: Medical Semantic indexing in Spanish (BioASQ MESINESP Task) will be scored against on a flat metric: the label-based micro F-measure. The participating teams ranked in the top third places will be awarded a prize. For further details, please refer to this link.
Generation of a public Silver Standard Corpus: Participants will be asked to automatically annotate 1M heterogeneous documents, called the Background set, that will include the 500 documents from the Test set. This will generate a public Spanish Silver Standard Corpus of semantically indexed documents.
The Award for the BioASQ MESINESP Task is 1,500€ to the winner team, 1,000€ to the second team, and 500€ to the third teams
Budget: The total budget for this call is 3,000 euros. The participating team ranked on top ranking will be awarded a prize of 1,500 euros, the second one with a prize of 1,000 euros, and the third one with a prize of 500 euros.
There is an additional prize of 500 euros for any team in the top five of the ranking that publishes its system in GitHub as an open-source component. The required template for the README file can be downloaded from this link.
Download: The official document of this call can be downloaded from this link.
Contact: For further details, please refer to PlanTL-Sanidad@bsc.es.
Barcelona, 10th December 2019