Knowledge Elicitation Using Deep Metric Learning and Psychometric Testing

Knowledge present in a domain is well expressed as relationships between corresponding concepts. For example, zoology, animal species form complex hierarchies; genomics, the different (parts of) molecules are organized groups and subgroups based on their functions; plants, molecules, astronomical objects all taxonomies. Nevertheless, when applying supervised machine learning (ML) such domains, we commonly reduce rich knowledge to fixed set of labels, induce model shows good generalization performance with respect these labels. The main reason for reductionist approach difficulty eliciting from experts. Developing label structure sufficient fidelity providing comprehensive multi-label annotation can be exceedingly labor-intensive many real-world applications. In this paper, provide method efficient hierarchical elicitation (HKE) experts working high-dimensional data images or videos. Our psychometric testing active deep metric learning. developed models embed space where distances semantically meaningful, structure. We empirical evidence series experiments synthetically generated dataset simple shapes, Cifar 10 Fashion-MNIST benchmarks that our indeed successful uncovering structures.