Interpretation and the limits of automated legal reasoning

Computers already do several kinds of legal work: word processing, spreadsheets, research, and practice management. The more ambitious prospect is a program that reasons from the law to help resolve disputes. That would go beyond organizing information or finding a document. It would need to decide how the law applies.

The author calls that prospect exciting, then immediately qualifies it. The systems haven't lived up to the early predictions or produced independent expert advice about substantive law. An early disappointment doesn't by itself show that a technology could never improve. So I'm interested in the specific difficulty: what is stopping these systems from doing the job?

The first approach treats the law as a collection of rules. Give the program those rules and the facts of a case, and it works out the consequences. Think of a flowchart: this condition is satisfied, so follow this branch and reach that result. At first, that sounds like something a computer might do well. What's missing?

Interpretation. Before applying a rule, we may need to decide what its words cover. Is a mobile home a house or a vehicle? Does a couple count as married without a formal ceremony? Simply entering the facts doesn't settle those questions. Depending on the legal issue, someone may argue for one characterization while someone else argues for the other.

And the uncertainty isn't always an oversight that better drafting would remove. Legislators may deliberately leave concepts undefined so the law can adapt to circumstances they didn't foresee. That flexibility demands judgment about the world. The author thinks the comprehensive knowledge needed is far beyond the systems described, both at the time of writing and in the foreseeable future.

The next approach uses earlier cases. Instead of relying only on abstract rules, a program stores examples and reaches a conclusion by comparing the new problem with a similar case. That sounds like an attempt to bring more context into the decision. Perhaps examples can help where a bare rule falls short. But can the program judge which similarities actually matter?

The author answers with “Unfortunately.” The designer has already fixed the factors used to compare cases. The program checks whether those factors are present; it doesn't independently discover which features make two cases legally alike. The judgment problem hasn't disappeared. It has moved to the step of deciding which comparisons are relevant.

That's the connection between the two approaches. A rule-based system needs to interpret legal categories. A case-based system needs to identify relevant similarities. A fixed collection of instructions doesn't, by itself, supply the flexible judgment the author thinks both tasks require.

Keep the scope of that criticism clear. It concerns the automated reasoning systems described here. It isn't a claim that spreadsheets are useless, or proof that no computer could ever reason about law. Nor does the author establish that the second approach is vastly better than the first; both encounter unresolved problems.

Our main-point prediction is that these attempts at automated legal reasoning fall short because they cannot adequately make the interpretive judgments the work requires. The important connection is between the failure and its cause, not just the general topic of computers in law.