Unchain the Search Space with Hierarchical Differentiable Architecture Search

Differentiable architecture search (DAS) has made great progress in searching for high-performance architectures with reduced computational cost. However, DAS-based methods mainly focus on a repeatable cell structure, which is then stacked sequentially multiple stages to form the networks. This configuration significantly reduces space, and ignores importance of connections between cells. To overcome this limitation, paper, we propose Hierarchical Architecture Search (H-DAS) that performs both at level stage level. Specifically, cell-level space relaxed so networks can learn stage-specific structures. For stage-level search, systematically study stages, including number cells each Based insightful observations, design several rules losses, mange better architectures. Such hierarchical greatly improves performance without introducing expensive Extensive experiments CIFAR10 ImageNet demonstrate effectiveness proposed H-DAS. Moreover, searched be combined structures by existing DAS further boost performance. Code available at: https://github.com/msight-tech/research-HDAS