en | es
The Plan for Promoting Language Technologies (Plan TL) aims to promote the development of natural language processing, machine translation and conversational systems in Spanish and co-official languages. In the framework of this plan, the Ministry of Economy and Business, through the Encargo made by the State Secretariat for Digital Advancement (SEAD) to the Barcelona Supercomputing Center Consortium – Centro Nacional de Supercomputación (BSC-CNS) to promote activities for specialized technical assistance in the area of biomedicine, among others, we announce the call for shared task awards detailed below.
In accordance with the provisions of articles 17.3.b and 20.8.a of Law 38/2003, of November 17, General of Subsidies, Article 6 of Law 9/2017 of November 8, on Public Sector Contracts, the special rules applicable to the project competitions for the awarding of prizes established therein and the allocation of budgetary resources established for the financial year 2019 through the financing of the formalized management commission under budget appropriation 20.12.467I.640, we detail below the requirements for the call for shared task awards.
Goal and description of the shared task: As part of the BioASQ challenges on biomedical semantic indexing and question answering, the OTG deScientific committee evaluator. Sanidad of the Plan TL organizes the task: Medical Semantic indexing in Spanish – BioASQ MESINESP Task, the first task on semantic indexing of Spanish medical texts.
Task a Spanish will be based on the standard process followed by IBECS and LILACS to index journal abstracts in Spanish. The participants will be asked to classify new IBECS/LILACS documents, written in Spanish, as they become available on-line, before IBECS/LILACS curators annotate them manually. As manual annotations from IBECS/LILACS become available, they will be used to evaluate the classification performance of participating systems.
Registration: To register as a participant, it is requested to fill-in an on-line registration form that can be found at the participant page of the BioASQ Task.
Deadlines for submission: Medical Semantic indexing in Spanish – BioASQ MESINESP Task will run in Autumn 2019 (detailed schedule TBA). Participants, after downloading the released test sets, will have to submit results within a limited time window. The task will run for two consecutive periods (batches) of 5 weeks each. The first batch will start on October, 2019. For further details, please refer to this link.
Evaluation: In order to measure classification performance, an on-line evaluation system will be maintained. As the true annotations of the articles are not available beforehand, the evaluation procedure will run continuously by providing on-line results. The participating systems will be assessed for their performance based on two measures, one hierarchical and one flat: the Lowest Common Ancestor F-measure (LCA-F) and the label-based micro F-measure, respectively. For further details, please refer to this link.
Task organizers: This task has been organized by the OTG de Sanidad of the Plan TL in collaboration with the National Center for Oncological Research (CNIO), the National Center for Scientific Research Demokritos (Greece) and the Biblioteca Nacional de ciencias de la Salud of the Instituto de Salud Carlos III. For further details, please refer to Task Organizers.
Scientific committee evaluator.
Selection of winners: Medical Semantic indexing in Spanish – BioASQ MESINESP Task has two (2) distinct milestones and will be scored against two (2) metrics: the Lowest Common Ancestor F-measure (LCA-F) and the label-based micro F-measure. So there will be a total of four (2×2=4) rankings. For each one of the milestone, the participating teams ranked on the top two places will be awarded a prize. For further details, please refer to this link.
Budget: The total budget for this call is 3,000 euros. For each one of the milestones, the participating teams ranked on top of any of the rankings (2 for each milestone) will be awarded a prize of 500 euros and the second ones, a prize of 250 euros. The same team may be awarded two prizes in a milestone.
There is an additional prize of 500 euros for any awarded team that publishes its system in GitHub as open source component. The required template for the README file can be downloaded from this link.
Download: The official document of this call can be downloaded from here.
Contact: For further details, please refer to email@example.com
Barcelona, 11th April, 2019