DeepAI AI Chat
Log In Sign Up

Everything You Always Wanted to Know About TREC RTS* (*But Were Afraid to Ask)

by   Gilles Hubert, et al.

The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used in 2016 and 2017 and found some limitations for the Scenario A of this track. Our main findings point out the weakness of the metrics and give clear recommendations to fairly reuse the collection.


page 1

page 2

page 3

page 4


How well do you know your summarization datasets?

State-of-the-art summarization systems are trained and evaluated on mass...

A customisable pipeline for continuously harvesting socially-minded Twitter users

On social media platforms and Twitter in particular, specific classes of...

Whose Tweets are Surveilled for the Police: An Audit of Social-Media Monitoring Tool via Log Files

Social media monitoring by law enforcement is becoming commonplace, but ...

Unleashing the Power of Hashtags in Tweet Analytics with Distributed Framework on Apache Storm

Twitter is a popular social network platform where users can interact an...

Investigating usability of MSstatsQC software

MSstatsQC [3] is an open-source software that provides longitudinal syst...