Improvements on Learning Tetris with Cross Entropy

For playing the game of Tetris well, training a controller by the cross-entropy method seems to be a viable way (Szita and Lőrincz, 2006; Thiery and Scherrer, 2009). We consider this method to tune an evaluation-based one-piece controller as suggested by Szita and Lőrincz and we introduce some improvements. In this context, we discuss the influence of the noise, and we perform experiments with several sets of features such as those introduced by Bertsekas and Tsitsiklis (1996), by Dellacherie (Fahey, 2003), and some original features. This approach leads to a controller that outperforms the previous known results. On the original game of Tetris, we show that with probability 0.95 it achieves at least 910, 000 ± 5% lines per game on average. On a simplified version of Tetris considered by most research works, it achieves 35, 000, 000 ± 20% lines per game on average. We used this approach when we took part with the program BCTS in the 2008 Tetris domain Reinforcement Learning Competition and won the competition.