Bibliometric-enhanced Information Retrieval 10th Anniversary Workshop Edition

01/20/2020 ∙ by Guillaume Cabanac, et al. ∙ Association for Computing Machinery Salle du Cap 0

The Bibliometric-enhanced Information Retrieval workshop series (BIR) was launched at ECIR in 2014 <cit.> and it was held at ECIR each year since then. This year we organize the 10th iteration of BIR. The workshop series at ECIR and JCDL/SIGIR tackles issues related to academic search, at the crossroads between Information Retrieval, Natural Language Processing and Bibliometrics. In this overview paper, we summarize the past workshops, present the workshop topics for 2020 and reflect on some future steps for this workshop series.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Motivation and Relevance to ECIR

Searching for scientific information is a long-lived user need. In the early 1960s, Salton was already striving to enhance information retrieval by including clues inferred from bibliographic citations [Salton1963]. The development of citation indexes pioneered by Garfield [Garfield1955] proved determinant for such a research endeavour at the crossroads between the nascent fields of Bibliometrics111Bibliometrics refers to the statistical analysis of the academic literature [Pritchard1969] and plays a key role in scientometrics: the quantitative analysis of science and innovation [LeydesdorffAndMilojevic2015]. and Information Retrieval (IR) — BIR. The pioneers who established these fields in Information Science — such as Salton and Garfield — were followed by scientists who specialised in one of these [WhiteAndMcCain1998], leading to the two loosely connected fields we know of today.

The purpose of the BIR workshop series founded in 2014 is to tighten up the link between IR and Bibliometrics [MayrAndScharnhorst2015a]. We strive to get the ‘retrievalists’ and ‘citationists’ [WhiteAndMcCain1998] active in both academia and the industry together, who are developing search engines and recommender systems such as ArnetMiner, Dimensions, Google Scholar, Microsoft Academic Search, and Semantic Scholar, just to name a few.

These bibliometric-enhanced IR systems must deal with the multifaceted nature of scientific information by searching for or recommending academic papers, patents, venues (i.e., conferences or journals), authors, experts (e.g., peer reviewers), references (to be cited to support an argument), and datasets. The underlying models harness relevance signals from keywords provided by authors, topics extracted from the full-texts, co-authorship networks, citation networks, and various classifications schemes of science.

BIR is a hot topic with growing recognition in the community in recent years: see for instance the Initiative for Open Citations [Shotton2018], the Google Dataset Search [brickley2019], the Indian JNU initiative for indexing the world’s literature in full-text [Pulla2019], the increasing number of retractions [BrainardAndYou2018], and massive studies of self-citations [VanNoorden2019, kacem2019]. We believe that BIR@ECIR is a much needed scientific event for the ‘retrievalists’, ‘citationists’ and others to meet and join forces pushing the knowledge boundaries of IR applied to literature search and recommendation.

2 Summarzing the Past BIR Workshops

The BIR workshop series was launched at ECIR in 2014 [MayrEtAl2014] and it was held at ECIR each year since then. As our workshop lies at the crossroads between IR and NLP, we also ran BIR as a joint workshop called BIRNDL (Bibliometric-enhanced IR and NLP for Digital Libraries) at the JCDL [CabanacEtAl2016] and SIGIR [ChandrasekaranAndMayr2019birndl] conferences. All past workshops had a large number of participants (between 30 and 60), demonstrating the relevance of the workshop’s topics.

In the following, we present an overview of the past BIR workshops and keynotes at BIR (Tab. 12). All pointers to the workshops and proceedings are hosted at Many of the presented workshop papers appeared in extended form in one of our four BIR-related special issues (2015 [MayrAndScharnhorst2015a], 2018 [CabanacEtAl2018a, MayrEtAl2018a], 2019 [atanassova2019]).

3 Workshop Topics

The call for papers for the 2020 workshop (the 10th BIR edition) addressed current research issues regarding 3 aspects of the search/recommendation process:

  1. User needs and behaviour regarding scientific information, such as:

    • Finding relevant papers/authors for a literature review.

    • Measuring the degree of plagiarism in a paper.

    • Identifying expert reviewers for a given submission.

    • Flagging predatory conferences and journals.

    • Information seeking behaviour and HCI in academic search.

  2. Mining the scientific literature, such as:

    • Information extraction, text mining and parsing of scholarly literature.

    • Natural language processing (e.g., citation contexts).

    • Discourse modelling and argument mining.

  3. Academic search/recommendation systems:

    • Modelling the multifaceted nature of scientific information.

    • Building test collections for reproducible BIR.

    • System support for literature search and recommendation.

Year Areaa Title of the keynote presentation Presenter
2015 SCIM In Praise of Interdisciplinary Research through Scientometrics Cabanac [DBLP:conf/ecir/Cabanac15]
2016 IR Bibliometrics in Online Book Discussions:
Lessons for Complex Search Tasks
Koolen [DBLP:conf/ecir/Koolen16]
2016 SCIM Bibliometrics, Information Retrieval and Natural Language Processing: Natural Synergies to Support Digital Library Research Wolfram [DBLP:conf/jcdl/Wolfram16]
2017 IR Real-World Recommender Systems for Academia: The Pain and Gain in Building, Operating, and Researching them Beel [DBLP:conf/ecir/BeelD17]
2017 NLP Do “Future Work” sections have a purpose?
Citation links and entailment for global
scientometric questions
Teufel [DBLP:conf/sigir/Teufel17]
2018 NLP Trends in Gaming Indicators: On Failed Attempts at Deception and their Computerised Detection Labbé [DBLP:conf/ecir/Labbe18]
2018 IR Integrating and Exploiting Public Metadata Sources in a Bibliographic Information System Schenkel [DBLP:conf/ecir/Schenkel18]
2019 NLP Beyond Metadata: the New Challenges in Mining Scientific Papers Atanassova [DBLP:conf/ecir/Atanassova19]
2019 IR Personalized Feed/Query-formulation, Predictive Impact, and Ranking Wade [DBLP:conf/sigir/WadeW19]
2019 NLP Discourse Processing for Text Analysis:
Recent Successes, Current Challenges
Webber [DBLP:conf/sigir/Webber19]
2020 SCIM Metrics and trends in assessing the scientific impact Tsatsaronis
a SCIM: Scientometrics; NLP: Natural Language Processing; IR: Information Retrieval
Table 2: Keynotes at BIR

4 Target Audience

The target audience of the BIR workshops are researchers and practitioners, junior and senior, from Scientometrics as well as Information Retrieval and Natural Language Processing. These could be IR/NLP researchers interested in potential new application areas for their work as well as researchers and practitioners working with, for instance, bibliometric data and interested in how IR/NLP methods can make use of such data.

5 Peer Review Process and Workshop Format

Our peer review process is supported by Easychair. Each submission is assigned to 2 to 3 reviewers, preferably at least one expert in IR and one expert in Bibliometrics or NLP. The accepted papers are either long papers (15-minute talks) or short papers (5-minute talks). Two interactive sessions close the morning and afternoon sessions with posters and demos, allowing attendees to discuss the latest developments in the field and opportunities (e.g., shared tasks such as the CL-SciSumm [JaidkaEtAl2018] at the BIRNDL joint workshop, see Sect. 2). These interactive sessions serve as ice-breakers, sparking interesting discussions that usually continue during lunch and the cocktail party. The sessions are also an opportunity for our speakers to further discuss their work.

6 Next Steps

Research on scholarly document processing has for many years been scattered across multiple venues like ACL, SIGIR, JCDL, CIKM, LREC, NAACL, KDD, and others. Our next strategic step is the First Workshop on Scholarly Document Processing (SDP)222

will be held in November 2020 in conjunction with the 2020 Conference on Empirical Methods in Natural Language Processing. This workshop and initiative will be organized by a diverse group of researchers (organizers from BIR, BIRNDL, Workshop on Mining Scientific Publications/WOSP and Big Scholar) which have expertise in NLP, ML, Text Summarization/Mining, Computational Linguistics, Discourse Processing, IR, and others.

6.0.1 Acknowledgement

We organizers wish to thank all those who contributed to this workshop series: The researchers who contributed papers, the many reviewers who generously offered their time and expertise, and the participants of the BIR and BIRNDL workshops. Since 2016, we maintain the Bibliometric-enhanced-IR Bibliography that collects scientific papers which appear in collaboration with the BIR/BIRNDL organizers.