Reflection machines: increasing meaningful human control over Decision Support Systems

Abstract Rapid developments in Artificial Intelligence are leading to an increasing human reliance on machine decision making. Even collaborative efforts with Decision Support Systems (DSSs), where a expert is expected make the final decisions, it can be hard keep actively involved throughout process. DSSs suggest their own solutions and thus invite passive To humans ‘on’ decision-making loop counter overreliance machines, we propose ‘reflection machine’ (RM). This system asks users questions about strategy thereby prompts them evaluate decisions critically. We discuss what forms RMs take present proof-of-concept implementation of RM that produce feedback users’ medical law domains. show prototype requires very little domain knowledge create reasonably intelligent critiquing questions. With this prototype, demonstrate technical feasibility develop hope pave way for future research into effectiveness value.