Deep neural networks (DNNs) have been widely adopted in many decision-making industrial applications. Their fairness issues, i.e., whether there exist unintended biases the DNN, receive much attention and become critical concerns, which can directly cause negative impacts our daily life potentially undermine of society, especially with their increasing deployment at an unprecedented speed. Rece...