We use a Dynamic Bayesian Network (DBN) to build a compact representation of the features relevant to Part-of-Speech (PoS) tagging (Word, Suffix, Prefix, Capitalization, Hyphen, Numeric and Previous Tag). The outcome is a flexible tagger (LegoTag) with state-of-the-art performance (3.6% error on a benchmark corpus). We explore the effect of radically reducing the size of feature vocabularies fo...