Log In Sign Up

Learning with Scope, with Application to Information Extraction and Classification

by   David Blei, et al.

In probabilistic approaches to classification and information extraction, one typically builds a statistical model of words under the assumption that future data will exhibit the same regularities as the training data. In many data sets, however, there are scope-limited features whose predictive power is only applicable to a certain subset of the data. For example, in information extraction from web pages, word formatting may be indicative of extraction category in different ways on different web pages. The difficulty with using such features is capturing and exploiting the new regularities encountered in previously unseen data. In this paper, we propose a hierarchical probabilistic model that uses both local/scope-limited features, such as word formatting, and global features, such as word content. The local regularities are modeled as an unobserved random parameter which is drawn once for each local data set. This random parameter is estimated during the inference process and then used to perform classification with both the local and global features--- a procedure which is akin to automatically retuning the classifier to the local regularities on each newly encountered web page. Exact inference is intractable and we present approximations via point estimates and variational methods. Empirical results on large collections of web data demonstrate that this method significantly improves performance from traditional models of global features alone.


page 1

page 2

page 3

page 4


Web Page Content Extraction Based on Multi-feature Fusion

With the rapid development of Internet technology, people have more and ...

Boilerplate Removal using a Neural Sequence Labeling Model

The extraction of main content from web pages is an important task for n...

PLAtE: A Large-scale Dataset for List Page Web Extraction

Recently, neural models have been leveraged to significantly improve the...

Don't read, just look: Main content extraction from web pages using visually apparent features

The extraction of main content provides only primary informative blocks ...

Design of Automatically Adaptable Web Wrappers

Nowadays, the huge amount of information distributed through the Web mot...

Upscaling human activity data: an ecological perspective

In recent years we have witnessed an explosion of data collected for dif...

DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction

Existing permutation-invariant methods can be divided into two categorie...