With the booming of smart home market, intelligent Internet Things (IoT) devices have been increasingly involved in life. To improve user experience homes, some prior works explored how to use machine learning for predicting interactions between users and devices. However, existing solutions inferior User Device Interaction (UDI) prediction accuracy, as they ignore three key factors: routine, i...