An experimental comparison of label selection methods for hierarchical document clusters

05/24/2018
by   Maria Fernanda Moura, et al.
0

The focus of this paper is on the evaluation of sixteen labeling methods for hierarchical document clusters over five datasets. All of the methods are independent from clustering algorithms, applied subsequently to the dendrogram construction and based on probabilistic dependence relations among labels and clusters. To reach a fair comparison as well as a standard benchmark, we rewrote and presented the labeling methods in a similar notation. The experimental results were analyzed through a proposed evaluation methodology based on: (i) data standardization before applying the cluster labeling methods and over the labeling results; (ii) a particular information retrieval process, using the obtained labels and their hierarchical relations to construct the search queries; (iii) evaluation of the retrieval process through precision, recall and F measure; (iv) variance analysis of the retrieval results to better understanding the differences among the labeling methods; and, (v) the emulation of a human judgment through the analysis of a topic observed coherence measure - normalized Pointwise Mutual Information (PMI). Applying the methodology, we are able to highlight the advantages of certain methods: to capture specific information; for a better document hierarchy comprehension at different levels of granularity; and, to capture the most coherent labels through the label selections. Finally, the experimental results demonstrated that the label selection methods which hardly consider hierarchical relations had the best results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2022

Academic information retrieval using citation clusters: In-depth evaluation based on systematic reviews

The field of scientometrics has shown the power of citation-based cluste...
research
05/02/2023

Towards a better labeling process for network security datasets

Most network security datasets do not have comprehensive label assignmen...
research
09/19/2010

Pair-Wise Cluster Analysis

This paper studies the problem of learning clusters which are consistent...
research
08/28/2012

Document Clustering Evaluation: Divergence from a Random Baseline

Divergence from a random baseline is a technique for the evaluation of d...
research
09/20/2017

Constructing a Hierarchical User Interest Structure based on User Profiles

The interests of individual internet users fall into a hierarchical stru...
research
10/05/2018

C-DLSI: An Extended LSI Tailored for Federated Text Retrieval

As the web expands in data volume and in geographical distribution, cent...
research
10/16/2019

HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories

GitHub has become an important platform for code sharing and scientific ...

Please sign up or login with your details

Forgot password? Click here to reset