Generating Original Structure in Regulatory Documents

As technology and society continue to evolve, the size of the corpus of government policies and procedures continues to more than keep pace. The U.S. Federal Tax Code today consumes over 2.8 million words or 6000 pages. There are more than 20,000 cross-references both within the code itself and to external regulations. Navigating the sea of information is a daunting task for an IRS expert let alone a well-intentioned tax payer, or policy-maker seeking to eliminate redundancies, internal inconsistencies, or loopholes. Likewise as the new Department of Homeland Security takes shape, the need for managing the 300,000 words and 650 pages of the Immigration and Nationality Act becomes urgent. While tools for tasks such as compliance checking or query answering have long held promise, automated reasoning, however intelligent, needs something to reason upon, a formalized knowledge base of some kind. AI in support of legal reasoning is no exception. It has been an enduring challenge to find ways of obtaining sufficiently structured documents. In other domains people may be the primary targets of knowledge engineering; in the policy realm, much of the requisite knowledge resides in legal and regulatory documents of various sorts. Compromising severely in favor of brevity over accuracy, we may say that there have been three main approaches to extracting formalized knowledge from documents of legal and regulatory interest. (1) Manually symbolize a relevant corpus. The approach here is to pick an appropriate formal representation language and manually (perhaps with computerized support) symbolize the essential information from the relevant collection of documents, typically statutes, administrative rules, and legal cases. (2) Accept minimally structured documents. Under this approach, the documents in the chosen corpus are formalized only in a very weak sense, e.g., by creating inverted files of their terms and limiting inference to what Information Retrieval techniques can produce. (3) Automatically structure a relevant corpus. Under this approach, pattern-finding programs extract structure from documents in a target corpus and create derived documents, such as in XML, which present their structures more transparently. The three approaches (and combinations) have complementary strengths and weaknesses. Manual symbolization affords the best prospect for deep and detailed automated inferencing and information recovery, yet it is the most labor intensive option and presents serious problems of maintenance. Accepting minimally structured documents is the least expensive alternative and the least powerful in terms of potential to support inferencing. Automated structuring lies more or less between the other two approaches. The three may be thought of as defining an operating curve that trades off cost and inferential acuity in an inverse relationship. Are there any ways to shift the operating frontier rather than limiting ourselves to seeking the best compromise point for a given application? There are such ways. Our aim in this project is to explore one such family: change how documents are originally created, write the documents in such a way that, from the initial draft, they have the requisite structure to support automated inferencing. Our objective is to automatically create documents that are equivalent in their expressiveness to documents which are manually coded ex-post as in (1) above. Alternatively, how might we generate semistructured documents that afford more complex reasoning than the automated structuring approaches as in (3) above. This project is therefore about a fourth approach to formalizing the knowledge in legal and regulatory documents: create the documents in a structured form rather than attempting to impose structure later. The intuition is to create these documents using formal sublanguages so that structure emerges as a by-product of the normal, policy-making process. More specifically, in restricted domains of import to systems of law and government, it may be feasible to draft policies and regulations using formal, special-purpose (artificial sub-) languages and vocabularies. We proceed with a brief introduction to artificial sub-languages and then turn to their potential in drafting fully-structured and semistructured documents that support automated reasoning.