{"id":12,"date":"2022-01-28T16:09:59","date_gmt":"2022-01-28T15:09:59","guid":{"rendered":"https:\/\/temu.bsc.es\/livingner\/?p=12"},"modified":"2022-05-03T11:13:34","modified_gmt":"2022-05-03T10:13:34","slug":"example","status":"publish","type":"post","link":"https:\/\/temu.bsc.es\/livingner\/2022\/01\/28\/example\/","title":{"rendered":"Example"},"content":{"rendered":"\n<p>Evaluation process: (1) you submit your results, (2) we perform the evaluation off-line and (3) return the final scores.<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>LivingNER-Species NER<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>$ cd src\n$ python main.py -g ..\/gs-data\/sample_entities_subtask1.tsv -p ..\/toy-data\/sample_entities_subtask1_MISSING_ONE_FILE.tsv -s ner\nAccording to file headers, you are on subtask ner\nAccording to file headers, you are on subtask ner\n\n-----------------------------------------------------\nClinical case name\t\t\tPrecision\n-----------------------------------------------------\n32032497_ES\t\tnan\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nClinical case name\t\t\tRecall\n-----------------------------------------------------\n32032497_ES\t\t0.0\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nClinical case name\t\t\tF-score\n-----------------------------------------------------\n32032497_ES\t\tnan\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nMicro-average metrics\n-----------------------------------------------------\n\nMicro-average precision = 1.0\n\n\nMicro-average recall = 0.9568\n\n\nMicro-average F-score = 0.9779\n\n..\/toy-data\/sample_entities_subtask1_MISSING_ONE_FILE.tsv|1.0|0.9568|0.9779\n<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\"><li>LivingNER-Species Norm<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>$ cd src\n$ python main.py -g ..\/gs-data\/sample_entities_subtask2.tsv -p ..\/toy-data\/sample_entities_subtask2_predictions.tsv -s norm\nAccording to file headers, you are on subtask norm, GS file\nAccording to file headers, you are on subtask norm, predictions file\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/ann_parsing.py:46: UserWarning: There are duplicated entries in ..\/toy-data\/sample_entities_subtask2_predictions.tsv. Keeping just the first one...\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/ann_parsing.py:59: UserWarning: Lines 1 in ..\/toy-data\/sample_entities_subtask2_predictions.tsv contain unvalid codes. Valid codes are those that appear in ..\/ncbi_codes_unique.tsv. Ignoring lines with valid codes...\n\n-----------------------------------------------------\nClinical case name\t\t\tPrecision\n-----------------------------------------------------\n32032497_ES\t\t0.5\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nClinical case name\t\t\tRecall\n-----------------------------------------------------\n32032497_ES\t\t0.3333\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nClinical case name\t\t\tF-score\n-----------------------------------------------------\n32032497_ES\t\t0.4\n-----------------------------------------------------\ncaso_clinico_medtropical54\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas1\t\t1.0\n-----------------------------------------------------\ncasos_clinicos_infecciosas141\t\t1.0\n-----------------------------------------------------\ncc_onco908\t\t1.0\n-----------------------------------------------------\n\n-----------------------------------------------------\nMicro-average metrics\n-----------------------------------------------------\n\nMicro-average precision = 0.9854\n\n\nMicro-average recall = 0.9712\n\n\nMicro-average F-score = 0.9783\n\n..\/toy-data\/sample_entities_subtask2_predictions.tsv|0.9854|0.9712|0.9783\n<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\"><li>LivingNER-Clinical IMPACT<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>$ cd src\n$ python main.py -g ..\/gs-data\/sample_subtask3.tsv -p ..\/toy-data\/sample_subtask3_predictions.tsv -s app\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/ann_parsing.py:99: UserWarning: Lines 5 in ..\/toy-data\/sample_subtask3_predictions.tsv contain unvalid codes. Valid codes are those that appear in ..\/ncbi_codes_unique.tsv. Ignoring lines with valid codes...\nBasic metrics (not taking into account NCBI codes, just Y\/N assignment)\n-----------------------------------------------------\nPet\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nAnimalInjury\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nFood\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nNosocomial\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/livingner_app.py:90: UserWarning: Precision score automatically set to zero because there are no predicted positives\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/livingner_app.py:104: UserWarning: Global F1 score automatically set to zero for simple metrics to avoid division by zero\n\/home\/antonio\/Documents\/Work\/BSC\/Projects\/micro\/scripts\/livingner-evaluation-library\/src\/livingner_app.py:110: UserWarning: Global F1 score automatically set to zero for complex metrics to avoid division by zero\nPrecision = 0\nRecall = 0.0\nF1score = 0\n-----------------------------------------------------\n\n\n\nComplex metrics (taking into account NCBI codes)\n-----------------------------------------------------\nPet\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nAnimalInjury\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nFood\nPrecision = 1.0\nRecall = 1.0\nF1score = 1.0\n-----------------------------------------------------\nNosocomial\nPrecision = 0\nRecall = 0.0\nF1score = 0\n-----------------------------------------------------<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Evaluation process: (1) you submit your results, (2) we perform the evaluation off-line and (3) return the final scores. LivingNER-Species NER LivingNER-Species Norm LivingNER-Clinical IMPACT<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-12","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/posts\/12","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/comments?post=12"}],"version-history":[{"count":4,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/posts\/12\/revisions"}],"predecessor-version":[{"id":405,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/posts\/12\/revisions\/405"}],"wp:attachment":[{"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/media?parent=12"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/categories?post=12"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/temu.bsc.es\/livingner\/wp-json\/wp\/v2\/tags?post=12"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}