Transformer language models have made tremendous strides in natural understanding tasks. However, the complexity of makes it challenging to ascertain how accurately these are tracking world state underlying text. Motivated by this issue, we consider task modeling for game chess. Unlike language, chess notations describe a simple, constrained, and deterministic domain. Moreover, observe that app...