Law, Probability and Risk - recent issues

A welcome exchange on the scientific status of fingerprinting

Koehler, J. J. — 2008-05-29

Scientific validation of fingerprint evidence under Daubert

Haber, L., Haber, R. N. — 2008-05-29

When a scientific method is used by an expert to reach a conclusion offered in court, the Frye ruling in 1923 and particularly the Daubert ruling in 1993 requires that the method itself has been shown to be valid. When applied to fingerprint methods, valid means accurately distinguishing between pairs of prints made by one and by two donors. Courts have ruled uniformly in more than 40 Daubert hearings since 1999 that fingerprint evidence rests on a valid method, referred to as the Analysis-Comparison-Evaluation-Verification (ACE-V) method. In this article, we discuss the scientific evidence needed to document the validity of ACE-V. We describe examples of experiments that would provide this evidence, and review the available published research. We briefly describe the testimony presented by fingerprint examiners in these hearings, intended to show that ACE-V meets the Daubert criteria for validity. We analyze evidence for the validity of the standards underlying the conclusions made by fingerprint examiners. We conclude that the kinds of experiments that would establish the validity of ACE-V and the standards on which conclusions are based have not been performed. These experiments require a number of prerequisites, which also have yet to be met, so that the ACE-V method currently is both untested and untestable.

Fingerprint examination: towards more transparency

Champod, C. — 2008-05-29

Comment on 'Scientific validation of fingerprint evidence under Daubert'

Cole, S. A. — 2008-05-29

The validity of latent fingerprint identification: confessions of a fingerprinting moderate

Mnookin, J. L. — 2008-05-29

Experiential or scientific expertise

Haber, L., Haber, R. N. — 2008-05-29

Thoughts inspired by Nassim Taleb's 'Fooled by Randomness' and 'The Black Swan'

Gelman, A. — 2008-05-29

Rotationally invariant statistics for examining the evidence from the pores in fingerprints

Parsons, N. R., Smith, J. Q., Thonnes, E., Wang, L, Wilson, R. — 2008-02-22

Recent methodological advances in the processing of DNA evidence have begun to force a closer examination of assertions about the strength of other sorts of evidence. One traditional source of evidence is the fingerprint. Currently a print taken from a suspect is compared against a mark from a crime scene and a match declared using the judgement of an expert based on matching minutiae and the ridge patterns around these. However, such methods have proved difficult to quantify effectively. This has provoked the investigation of even finer features in the print and the mark. One set of such features are the many pores, located along the ridges of the fingerprint. Is it possible to supplement expert judgements associated with a match with a more automatic and quantitative measure of the strength of evidence, based on pore information? The results of this preliminary analysis suggest we can. Our methodology is relatively transparent, using common statistics for two sample comparisons of point patterns. The results discussed here concern the matching of inked prints using grey-level imaging and complement previous studies which tend to focus on the comparison of binarised images.

The role of epidemiology in the law: a toxic tort litigation case

Dominici, F., Kramer, S., Zambelli-Weiner, A. — 2008-02-22

Toxic tort cases provide a natural framework for an in-depth illustration and an application of statistical methods for small-scale studies of putative sources of hazard. In this paper, we describe the aspects of a toxic tort case that focussed on quantifying the strength of evidence concerning the hypothesis that carcinogenic substances emitted from an industry source were associated with a statistically significant higher than expected incidence rate of neuroblastoma in children. We first present the epidemiological analysis carried out by the plaintiffs' experts (Drs P1, P2 and P3). We then summarize the key critiques by the defense experts (Drs D1, D2 and D3) followed by the plaintiff's reply. In the context of toxic torts, the plaintiff must demonstrate that the exposure resulting from the defendants' conduct is more likely than not causally related to the injury. We use this toxic tort case to identify common criticisms of the defense experts that take advantage of the complexity in evaluating causation in toxic torts. In the discussion, we summarize the common defense positions and question whether such questions are scientifically appropriate.

Analysis of sampling issues using Bayesian networks

Biedermann, A., Taroni, F., Bozza, S., Aitken, C. G. G. — 2008-02-22

This paper addresses the implementation of Bayesian sampling methodology in a graphical probability environment, i.e. Bayesian networks (BNs). An architecture of BNs which is able to be used for sampling from small and large consignments is outlined in detail. Through direct interaction with their users, the proposed models provide a framework that is capable of dealing with several distinct sampling issues, such as (i) the calculation of posterior probability distributions for the proportion of ‘positives’ (i.e. discrete units with a characteristic of interest) in a consignment as well as for the number of positives among a consignment's uninspected items, (ii) case preassessment and (iii) likelihood-ratio evaluation. A discussion is included on features of the proposed models that allow one to account for further complications such as competing prior beliefs about the proportion of positives in a consignment and potentially misclassified data (e.g. positive testing results obtained from units that are actually negative).

Case comment: an expert's report criticizing plaintiff's failure to account for multiple comparisons is deemed admissible in EEOC v.Autozone

Gastwirth, J. L. — 2008-02-22

In its opinions in Daubert, Joiner and Kumho Tire, the U.S. Supreme Court assigned trial judges a ‘gate-keeping’ role in assuring that proposed expert testimony is sufficiently reliable that it should be admitted into evidence. A section of the Daubert decision described guidelines to assist trial judges in their evaluation of expert testimony. This comment discusses a careful district court opinion concerning a motion, under the principles of Daubert, to exclude the report of defendant's expert in the sex discrimination case, EEOC v. Autozone. The technical statistical issue dealt with the need for adjusting the observed significance levels or p-values of individual statistical tests when a large number of related tests are conducted. The Equal Employment Opportunity Commission and its expert argued that while such adjustments are used in medical applications of statistics, they are not needed in labour economics. The decision is noteworthy because the judge realized that the principles of statistical inference remain the same regardless of the origin of the data. After describing the issue and reanalysing some of the data, alternative statistical methods that stratify the data into comparable subgroups and combine the results for the separate strata are often more appropriate. They avoid making many separate tests and have higher statistical power to detect discrimination. Since 10 years of hiring and promotion data were analysed, which cover both pre- and post-charge time periods, two possible scenarios are described. The first is the one that apparently was the focus of both experts and the second focuses on the status of a particular female plaintiff for a job as guard in the early period. It is shown that overall the data do not support a claim of class-wide discrimination but might support a claim of sex discrimination in hiring for guard positions in the pre-charge time period.

Expert Evidence and Criminal Justice, by Mike Redmayne

Lucy, D. — 2008-02-22

Genetic Testing and the Criminal Law, by Don Chalmers

Fordham, J. — 2008-02-22

Reliability and Risk: A Bayesian Perspective, by Nozer Singpurwalla

Lee, P. M. — 2008-02-22

Introduction: visualizing evidence and inference in legal settings

Tillers, P. — 2008-01-03

Visualizing the dynamics around the rule evidence interface in legal reasoning

Walker, V. R. — 2008-01-03

This paper presents a visual framework for modelling complex legal reasoning—reasoning that integrates legal rules and policies with expert and non-expert evidence. The framework is based on a many-valued, predicate, default logic. The paper first visualizes the two sides of the rule–evidence interface: rule-based deductions and evidence evaluation. It then explores ways to visualize several dynamics around that interface, including dynamics concerning evidentiary relevance, findings of fact, process decision making about motions, policy-based reasoning about rules and relevant-factor reasoning. The paper then concludes with visualizing dynamics across multiple cases and briefly discusses one pathway by which new legal rules might emerge from the factfinding process. The paper therefore presents a visual working environment for people who litigate or decide actual cases, who study judicial or administrative reasoning or who teach law.

The rationale for RationaleTM

van Gelder, T. — 2008-01-03

Complex reasoning and argumentation are central to legal practice. Software-supported argument mapping may be able to help lawyers reason and argue more effectively. This article describes Rationale^TM, a generic argument mapping software package, and reviews some evidence that using it can help improve reasoning, i.e. make people smarter. It then explores three different explanations for this potential benefit: usability, complementation and semi-formality. First, argument mapping software can be more usable for reasoning activities than traditional methods because it can inherit the wisdom gained through decades of research and experience into usability, can exploit a wider range of representational resources, and is designed specifically to support reasoning activities. Second, such software works by complementing the strengths and weaknesses of our natural or inbuilt cognitive capacities. Third, it helps shift reasoning and argumentation into a semi-formal mode, a kind of ‘sweet spot’ between the laxness of everyday reasoning and the straightjacket of formal logic.

Reasoning and probability

Pollock, J. L. — 2008-01-03

Reasoning justifies its conclusions to varying degrees, depending on the strength of the reasons and the justification of the evidence. It is often supposed that the resulting degrees of justification can be viewed as probabilities, but there are simple reasons why that cannot be right. This paper explores an alternative way of relating probabilities and degrees of justification.

A pluralist approach to argument diagramming

Reed, C., Rowe, G. — 2008-01-03

There is a long history of argument diagramming mechanisms, attesting to the need for, and popularity of visualizations of reasoning patterns in general. Many of these approaches have given rise to software implementations that serve particular audiences with particular needs, leading to a plethora of such tools. Interesting research questions are posed when exploring the ground between these different approaches to diagramming, and these questions become operational challenges in the context of a research programme aimed at developing ‘pluralist’ argument diagramming software—a single tool that supports multiple different theoretical approaches to the analysis of argument. The Araucaria system aims to meet such pluralist goals, by allowing analysis to be conducted in different styles and then providing for translation between them. In this way, a potential is opened up for interchange between different communities.

The inferential arrow: a comment on interdisciplinary conversation

Nance, D. A. — 2008-01-03

Visualization tools and argument schemes: a question of standpoint

Anderson, T. J. — 2008-01-03

Visualizing Carneades argument graphs

Gordon, T. F. — 2008-01-03

Carneades is a computational model of argument based on the state-of-the-art of argumentation theory in philosophy. This article presents a diagramming method for Carneades, similar to Wigmore charts, and illustrates how to map legal evidence using this method. With suitable computer support, in the form of a special purpose argument diagram editor, users need not understand the mathematics of the computational model to make use of its features. Compared to a generic diagram editor, or even special purpose argument diagramming tools based only on informal models of argument, an argument diagramming tool based on the Carneades computational model of argument has the advantage of being able to inform users, in an intuitively comprehensible way, whether or not a claim satisfies a proof standard, given the evidence and other arguments which have been put forward by the parties. The presentation is entirely informal. No prior expertise in argumentation theory, mathematics or computer science is presumed.

Visualization tools, argumentation schemes and expert opinion evidence in law

Walton, D. — 2008-01-03

New models of evidential reasoning have been closely tied in with the development of visualization tools in artificial intelligence, especially automated systems for argument diagramming. Surveying several models and visualization tools recently developed in artificial intelligence, this paper argues that any discussion of visualization methods or tools of this sort should focus on their suitability for visualizing argumentation schemes, including critical questions. The classic scheme, used in this paper to illustrate how schemes need to be a vital part of advancing argumentation technology in tools for evidence visualization in law, is that for argument from expert opinion. The visualization of argumentation schemes is illustrated using a new version of the scheme, which takes into consideration Supreme Court rulings on the admissibility of expert witness testimony.

Visualization tools and argument schemes revisited

Anderson, T. J. — 2008-01-03

Sense-making software for crime investigation: how to combine stories and arguments?

2008-01-03

Sense-making software for crime investigation should be based on a model of reasoning about evidence that is both natural and rationally well-founded. A formal model is proposed that combines artificial intelligence formalisms for abductive inference to the best explanation and for defeasible argumentation. Stories about what might have happened in a case are represented as causal networks and possible hypotheses can be inferred by abductive reasoning. Links between stories and the available evidence are expressed with evidential generalizations that express how observations can be inferred from evidential sources with defeasible argumentation. It is argued that this approach unifies two well-known accounts of reasoning about evidence, namely, anchored narratives theory and new evidence theory. After the reasoning model is defined, a design is presented for sense-making software that allows crime investigators to visualize their thinking about a case in terms of the reasoning model.

Argumentation, stories and generalizations: a comment

Twining, W. — 2008-01-03

The underlying theory of the software program described in ‘Sense-Making Software for Criminal Investigation’ (Bex et al., 2007) complements modified Wigmorean analysis (MWA). Both adopt a qualitative rather than a quantitative approach. MWA is broadly compatible with the kind of logic involved, including abductive inference to the best explanation and the idea of defeasible argumentation. Both approaches are mainly valuable as aids to thinking, especially constructing and evaluating arguments, rather than as methods of presenting them in order to persuade. Both approaches can be applied at different stages of criminal investigation (and more broadly of legal processes), but the specific device of Wigmore charts (one part of MWA) is more useful in hypothesis testing and discarding than in hypothesis formation, which typically requires imaginative reasoning. The Anchored Narratives of Crombag et al. and MWA have similar theoretical assumptions, except that MWA gives a radically different account about the relationship between stories, generalizations and argument. The proposed program has considerable promise, but before it can be of positive practical value in police investigation, more attention needs to be given not only to the obvious dangers of using stories and generalizations in this context but also about what positive guidance can be given to mitigate these dangers. There is, however, an unresolved tension between the simplifying tendencies of formalized computer programming and the tendency of MWA to emphasize the complexities of practical inferential reasoning and argumentation in legal contexts.

Argumentation support software: boxes-and-arrows and beyond

Verheij, B. — 2008-01-03

This paper is about software to support argumentative tasks for lawyers. The focus is on the visualization and evaluation of warranted defeasible arguments. The opportunities provided by ‘boxes-and-arrows’ diagrams of arguments are treated, but the paper also addresses limitations. It is argued that finding natural design, improving usefulness and including content are central research challenges for the field of argumentation support software. It is of highest priority that claims about the extent to which these challenges have been met become better supported by evidence, e.g. by empirical studies.

Can you have too much of a good thing? A comment on Bart Verheij's legal argumentation support software

Schafer, B. — 2008-01-03

Graphical manipulation of evidence in structured arguments

Lowrance, J. D. — 2008-01-03

A semi-automated approach to evidential reasoning uses template-based structured argumentation. Graphical depictions convey lines of reasoning, from evidence through to conclusions.

Comment on Lowrance's 'Graphical manipulation of evidence in structured arguments'

Ashley, K. D. — 2008-01-03

Lowrance's use of template-based structured argumentation and a collaborative tool in which graphical depictions convey lines of reasoning from evidence to conclusions has real potential for supporting legal tasks such as complaint drafting, especially if the tool were extended in various ways.

Assessing the competence and credibility of human sources of intelligence evidence: contributions from law and probability

Schum, D. A., Morris, J. R. — 2008-01-03

These are perilous times in which our security is under continual threat by persons and organizations around the world and at home. Information supplied to us by human sources concerning the capabilities and intentions of these persons and organizations is crucial to our ability to recognize and prevent these threatening actions. We have all seen the many news accounts of our need for more and better HUMINT (human intelligence). But a major issue is the following: when we obtain an item of HUMINT, to what extent can we believe it? Our ability to make these assessments involves determining the competence and credibility of the human source providing this item. These assessments are very difficult and the persons making them need all the help they can get. Fortunately, much assistance comes from the fields of law and probability. This paper provides an account of how we are now exploiting the very rich legacy of experience and scholarship accumulated in the field of law over the past 500 years or so regarding questions to ask about the competence and credibility of witnesses. For quite some time, we have been attempting to bring valuable insights from the field of law concerning various evidential and inferential matters to the attention of persons in the intelligence community. On this occasion, we will describe a computer-assisted system called MACE (method for assessing the credibility of evidence), which we are developing to exploit what we have learned from law regarding assessments of the competence and credibility of sources of HUMINT. This system is designed to assist in addressing the question: to what extent can we believe this particular item of HUMINT that has just been supplied to us by a human source? MACE employs two different, but entirely complementary, probability systems to help us answer this question. Baconian probability methods help us answer the question: how much evidence do we have about this human source and how completely does it answer questions about the source's competence and credibility? Bayesian probability methods allow us to determine how strong is the evidence we do have about this particular human source, and it also provides an assessment of the posterior odds favouring the extent to which we can believe or disbelieve what this source is telling us.

Object-oriented graphical representations of complex patterns of evidence

Hepler, A. B., Dawid, A. P., Leucari, V. — 2008-01-03

We reconsider two graphical aids to handling complex mixed masses of evidence in a legal case: Wigmore charts and Bayesian networks. Our aim is to forge a synthesis of their best features and to develop this further to overcome remaining limitations. One important consideration is the multilayered nature of a complex case, which can involve direct evidence, ancillary evidence, evidence about ancillary evidence, etc. all of a number of different kinds. If all these features are represented in one diagram, the result can be messy and hard to interpret. In addition, there are often recurrent features and patterns of evidence and evidential relations, e.g. credibility processes or match identification (DNA, eyewitness evidence, etc.), that may appear, in identical or similar form, at many different places within the same network, or within several different networks, and it is wasteful to model all these individually. The recently introduced technology of ‘object-oriented BNs’ suggests a way of dealing with these problems. Any network can itself contain instances of other networks, the details of which can be hidden from view until information on their detailed structure is desired. Moreover, generic networks to represent recurrent patterns of evidence can be constructed once and for all and copied or edited for reuse as needed. We describe the potential of this mode of description to simplify the construction and display of complex legal cases. To facilitate our narrative, the celebrated Sacco and Vanzetti murder case is used to illustrate the various methods discussed.

Thinking beyond the shown: implicit inferences in evidence and argument

Feigenson, N., Sherwin, R. K. — 2008-01-03

Visual representations are especially well suited to the construction of implicit meanings. Like advocates in other fields, lawyers use visual displays to prompt cognitive and emotional associations of which viewers may not be aware and which, consequently, they are less likely to evaluate critically. The authors review some of the psychological and rhetorical effects of visual representations in general, discuss several examples of how legal visual displays encourage audiences to draw implicit inferences and argue for the importance of heightened visual literacy in improving our ability to understand the meanings and implications of visual advocacy within the legal system.

Rethinking the role of the image in justice: visual evidence and science in the trial process

Tait, D. — 2008-01-03

Visual displays are increasingly important for presenting scientific evidence in the trial process. This paper engages with some of the arguments of Mnookin, Solomon and Feigenson in examining the challenges and paradoxes of scientific visual displays. The function of the display is ambiguous — is it a tool of logic to clarify arguments or an instrument of persuasion to sway the jury? The role of expert witnesses is also ambiguous — are they authoritative interpreters or teachers or do the displays speak for themselves? The answer to this varies between fingerprint, DNA and other forms of evidence. In a legal environment where disciplinary bodies and judges screen the scientific evidence that juries may see, the role of the jury is undergoing change. On the one hand, the judge rather than the jury may be making decisions about some of the potential facts of the case. On the other hand, the increasingly interactive nature of evidence makes it hard to control the interpretations placed on the information, and may allow juries to pursue their own enquiries. The paper argues for a holistic examination of the sensory environment of the trial including visual displays, but also the performances of the different participants, and the physical and symbolic environment of the courtroom.

Comment on the Cardozo conference on graphic and visual representations of evidence and inference in legal settings

Loui, R. — 2008-01-03