Penn Treebank Dataset

07/29/2020 ∙ 0

DOWNLOAD Penn Treebank (Word Level)

wget https://data.deepai.org/ptbdataset.zip
Penn Treebank (PTB) dataset, is widely used in machine learning for NLP (Natural Language Processing) research. Word-level PTB does not contain capital letters, numbers, and punctuations, and the vocabulary is capped at 10k unique words, which is relatively small in comparison to most modern datasets which can result in a larger number of out of vocabulary tokens.