CCRTS - 2007 1 Title : Identifying the Enemy – Part II : Algorithms versus Human Analysts

To successfully predict the actions of the adversary, identify high-value targets, and develop effective counteractions, the knowledge of the enemy organization, objectives, and the modus operandi are needed. Current approaches to analyze the threat are manual: the intelligence analysts have to deal with huge amounts of data, most of which is irrelevant to the analysis performed. Large information gaps, including missing data, deceptions, and errors, have to be dealt with, and analysts often fill the gaps with their experiences which might not be applicable to the problem they need to solve, thus resulting in decision biases. In addition, people tend to exhibit confirmatory biases when the first seemingly valid hypothesis is selected and further relied upon during the analysis. This issue is compounded by huge amounts of data and complexity of the problem people need to analyze, influencing what data is used and which is filtered out and never studied. All these factors negatively impact the ability of the intelligence team to recognize acting enemy and further results in decreased efficiency of counteractions and unintended consequences. Currently, only a limited set of tools are available to intelligence operators to analyze, correlate and visualize the data. No tools with automated threat prediction and assessment capabilities that can reason from multi-source data and support the decisions about the enemy's command and control organization have been developed. In the past this was due to the inability to bring all data sources together for common analysis. As new tools and data collection techniques become available, the feasibility of new technologies to automate threat prediction is increasing. Problem: This paper is part II of 2-paper submission describing a DARPA-sponsored project to develop and validate the NetSTAR technology for automated threat identification. In this paper, we describe how our identification of adversarial organizations stem from our analysis of command and control (C2) organizations and our analysis of what a model/algorithm must accomplish to identify and describe an adversarial organization. We then summarize the human table-top experimentation and concomitant comparison of the accuracy of adversarial organization discovery obtained by a team of human analysts versus the automated C2 identification process. The threat analysis is based on understanding the decision-making processes in the general C2 organization. While C2 organizations are designed to manage personnel and resources to accomplish the mission requiring their collective skills. However, C2 organizations are not limited to one type of organization and such organizations …