Law, Probability and Risk - recent issues

Seventh International Conference on Forensic Inference and Statistics, The University of Lausanne, Switzerland, August 2008

Aitken, C. G. G. — Thu, 12 Nov 2009 04:59:45 PST

A science of evidence: contributions from law and probability

Schum, D. A. — Thu, 12 Nov 2009 04:59:45 PST

Presented in this paper are arguments that studies of evidence in the fields of law and probability constitute a science of evidence when they are examined in light of five criteria for scientific activity. Three examples of studies from law and probability are presented to justify this argument. Examples from law include a method for classifying recurrent forms and combinations of evidence regardless of their substance or content, studies identifying credibility attributes for different forms of evidence and studies of complex argument construction in which new lines of inquiry and evidence are generated or discovered. Examples from probability include various views among probabilists about what the weight or force of evidence means and how it should be assessed, several examples of how important subtleties or complexities in evidential reasoning can be captured for study and analysis and one example of how alternative theories about the meaning of complex combinations of events can be tested empirically. All evidence has three major credentials: relevance, credibility and inferential force or weight. The law and probability studies cited in this paper concern all these credentials.

Forensics without uniqueness, conclusions without individualization: the new epistemology of forensic identification

Cole, S. A. — Thu, 12 Nov 2009 04:59:45 PST

Among the causes of the current sense that the forensic identification disciplines are ‘under siege’ are conceptual difficulties in these disciplines. Forensic identification disciplines either claim to achieve or strive to achieve conclusions of ‘individualization’, the reduction of the donor pool to a single source. They tend to support such claims by reference to the supposed ‘uniqueness’ of their objects of analysis. Both these notions remain extremely salient among practitioners and courts. And yet, a broad consensus in the forensic literature holds that individualization is unachievable and uniqueness is largely irrelevant to supporting claims of individualization. Focusing on latent print evidence, this article provides a clear articulation of the need to make a clean break from both individualization and uniqueness as forensic concepts. It argues that trace evidence disciplines can live without these concepts, and it explores what defensible conclusions might look like and how they might be supported.

Painting the target around the matching profile: the Texas sharpshooter fallacy in forensic DNA interpretation

Thompson, W. C. — Thu, 12 Nov 2009 04:59:45 PST

Forensic DNA analysts tend to underestimate the frequency of matching profiles (and overestimate likelihood ratios) by shifting the purported criteria for a ‘match’ or ‘inclusion’ after the profile of a suspect becomes known—a process analogous to the well-known Texas sharpshooter fallacy. Using examples from casework, informal and naturalistic experiments, and analysts’ own testimony, this article demonstrates how post hoc target shifting occurs and how it can distort the frequency and likelihood ratio statistics used to characterize DNA matches, making matches appear more probative than they actually are. It concludes by calling for broader adoption of more rigorous analytical procedures, such as sequential unmasking, that can reduce the sharpshooter fallacy by fixing the target before the shots are taken.

How accurate are the power calculations relied on by the SEC in its regulatory deliberations?

Bura, E., Gastwirth, J. L. — Thu, 12 Nov 2009 04:59:45 PST

In two related decisions in Chamber of Commerce of the United States of America v. Securities and Exchange Commission (SEC), the District of Columbia Federal Court of Appeals ruled that the SEC had not fully complied with some provisions of the Administrative Procedures Act when it required that the boards of investment companies managing mutual funds have at least 75% of their membership and the Chairman be independent directors. In preparation for renewed rule making, the Office of Economic Analysis of the SEC prepared a Power Study to respond to an industry-sponsored report claiming that the returns of funds with independent boards and chairs are not superior to funds with boards dominated by management. The Power Study concluded that the available studies on the effectiveness of independent board members do not have sufficient statistical power to detect a meaningful difference in the returns of the two types of funds. This paper demonstrates that the method used by the SEC in their power calculation is not correct, unless a very restrictive condition that rarely occurs in practice holds. When the appropriate power formulas are used, the expected power of studies of the same size as the ones examined by the SEC is actually lower than the corresponding results of the SEC. Thus, the results in the paper actually strengthen the argument that the SEC is advocating. The relevance of both the SEC and the industry studies to the main issue in the case is also questioned in the discussion.

Match likelihood ratio for uncertain genotypes

Perlin, M. W., Kadane, J. B., Cotton, R. W. — Thu, 12 Nov 2009 04:59:45 PST

Genetic data are not necessarily fully informative, leading to uncertainty in an inferred genotype. The posterior genotype probability distribution incorporates the identification information present in the data. To compare uncertain genotypes, we introduce here a match likelihood ratio (MLR), a simple generalization of the likelihood ratio standardly used to understand the import of genetic evidence in forensic applications. The MLR gives the relative probability of a match between questioned evidence and a suspect, with respect to a match between the evidence and a relevant population. Coancestry can be naturally incorporated. We present illustrative examples and provide a detailed analysis and comparison for a two-person DNA mixture. We describe MLR's computation efficiencies when making multiple genotype comparisons and show how MLR was used to explain evidence in court. As statistical computing of forensic DNA inferences becomes more commonplace, the MLR may help in quantifying match identification information.

Workshop to celebrate the 70th birthday of Joseph L. Gastwirth, George Washington University, Washington DC, USA, 1 August 2009

Aitken, C. G. G. — Mon, 16 Nov 2009 06:36:45 PST

Some thoughts at the interface of law and statistics

Aitken, C. G. G. — Mon, 16 Nov 2009 06:36:45 PST

The role of probabilistic and statistical reasoning in two murder trials in England is described. Suggestions are made as to what is required of statisticians who wish to become involved in the judicial process and as to what is required of lawyers who wish to present probabilistic ideas into the courtroom.

Identification, individualization and uniqueness: What's the difference?

Kaye, D. H. — Mon, 16 Nov 2009 06:36:45 PST

Criminalists and many forensic scientists concerned with the identification of trace evidence have distinguished between identification and individualization, but they have not distinguished as precisely between individualization and uniqueness. This paper clarifies these terms and discusses the relationships among identification, individualization and uniqueness in forensic science evidence.

Using the Peters-Belson method in equal employment opportunity personnel evaluations

Sinclair, M. D., Pan, Q. — Mon, 16 Nov 2009 06:36:45 PST

The Peters–Belson method has been used as an alternative approach to standard linear regression analysis to examine potential wage discrimination as it also accounts for any differences in the qualifications of the applicants. To apply the Peters–Belson method, one first conducts a regression analysis on the favoured class and applies the resulting model to the non-favoured class to identify whether this class would have received a different rate of pay if they had been treated the same as their favoured counterparts. Since this method was recently extended to explore disparities in the personnel selections via logistic regression, we will examine the general properties of this method as compared to standard regression analysis. The effects of the demographic mix and size of the applicant pool, the difference in the distribution of the qualifications and the employer's evaluation criteria on the estimated disparity will be examined. Some of the philosophical and legal issues from selected court cases surrounding the use of these two techniques will also be discussed.

Comment on "Using the Peters-Belson method in equal employment opportunity personnel evaluations" by Sinclair and Pan

Graubard, B. I. — Mon, 16 Nov 2009 06:36:45 PST

Response to Dr Graubard

Sinclair, M. D., Pan, Q. — Mon, 16 Nov 2009 06:36:45 PST

Calculating remittiturs

Kadane, J. B. — Mon, 16 Nov 2009 06:36:45 PST

This note reviews several methods proposed for calculating the amount of an appropriate remittitur using a sample of ‘comparable’ cases. It recommends a simple quantile estimate.

Comment on "Calculating remittiturs" by Kadane

Haug, M. G. — Mon, 16 Nov 2009 06:36:45 PST

Response to Professor Haug

Kadane, J. B. — Mon, 16 Nov 2009 06:36:45 PST

Are blacks egregious speeding violators at extraordinary rates in New Jersey?

Kadane, J. B., Lamberth, J. — Mon, 16 Nov 2009 06:36:45 PST

In 1996, a New Jersey Court found that the New Jersey State Police engaged in targeting black motorists on the New Jersey Turnpike (NJT), intensifying the debate around racial profiling. Two recent articles have claimed that the standard of comparison for determining racial profiling was incorrect because either the measurements utilized to make that determination or the standard used in Soto were wrong. The present article concludes that the measures used in the Soto case were valid and reliable. It presents two experiments that show that the suggestion that blacks are stopped at about the correct rate on the NJT because they egregiously violate speed laws much more frequently than do whites is erroneous. The data are consistent with the use of racially informed traffic stops as a pretext for drug searches on the southern end of NJT.

Modelling an omitted factor in employment discrimination cases

Yu, B. — Mon, 16 Nov 2009 06:36:45 PST

Statistical analysis can be used as circumstantial evidence indicating that race, gender, age or other legally prohibited factor (exposure) was a significant cause of an unfavourable employment decision (response). Statistical methods, e.g. Cornfield's inequality, have been used to examine the employer's reason (omitted factor), e.g. requirement of having a professional certificate is able to explain the disparity. Based on a range of plausible values of the associations among omitted variable, exposure and response, the usual sensitivity analysis methods recalculate the p-values of the test of association between exposure and response. However, the current methods treat the sensitivity parameters as fixed and do not account for sampling error. A new method of modelling an omitted factor is proposed for assessing the causal relationship between exposure and response. The association parameters between the omitted factor and the exposure are estimated using extra available data. The proposed method is flexible and can incorporate a wide range of statistical models. The method is illustrated using data from an age discrimination case.

Revisiting the p-value: a comparison of statistical evidence in clinical and legal medical decision making

Zou, K. H., DeTora, L. M., Haker, S. J., Mulkern, R. V. — Mon, 16 Nov 2009 06:36:45 PST

While the use of p-values in evidence-based medicine (EBM) is consistent and well-defined, the application of statistical information in health law varies greatly. A comparative literature review of clinical and legal medical decision making using Medline (PubMed) and LexisNexis reveals large disparities in the mode and frequency with which statistical evidence is used in EBM when compared with health law. We present the historical background of the p-value in the statistical literature and legal case law, including several medical malpractice cases, and then compare the use of the p-value as statistical evidence in health law and clinical research. We suggest that health law may consider the establishment of statistical criteria for appropriate applications.

Formal statistical analysis of the data in disparate impact cases provides sounder inferences than the U. S. government's 'four-fifths' rule: an examination of the statistical evidence in Ricci v. DeStefano

Gastwirth, J. L., Miao, W. — Mon, 16 Nov 2009 06:36:46 PST

Many countries have fair employment laws to protect racial, gender, religious or ethnic minorities from discrimination and courts in the USA can order remedies such as one out of every three new hires should be a member of a protected group after finding an employer discriminated. What steps can an employer undertake to ensure its employment practices do not disadvantage minorities when it does not need to comply with a court order? This issue arose in Ricci v. DeStefano, a ‘reverse discrimination’ case under review by the U.S. Supreme Court. Seventeen Whites and 1 Hispanic who achieved sufficiently high scores qualifying them for promotion to lieutenant or captain of the New Haven Fire Department sued the city because it cancelled the examinations after seeing that no African American could be appointed to an existing vacancy. The City of New Haven justified its action on the basis that both examinations had a disparate impact on African Americans and Hispanics because the ratios of their pass rates to that of Whites were less than 80%, contrary to a ‘rule of thumb’ in the government's Uniform Guidelines. The city did not conduct statistical tests, which are referred to in the guidelines.

The lower courts accepted New Haven's explanation and granted summary judgement to it. A statistical study of the various criteria considered by the city and lower courts in their review of the data demonstrates that nearly 70% of the time a fair non-discriminatory test for either position will fail the government's ‘80% rule’ and at least 60% of the time both fair tests would fail this ‘four-fifths rule’. Since the city created a new criterion after seeing the results, it is difficult to formulate precisely the other ‘rare’ or ‘unusual’ outcomes that would lead to cancellation of the examination. Would New Haven reject a list with no Hispanics or no Whites eligible for an immediate promotion? Would it require that all three groups be represented in the pool eligible for advancement to each position? From the viewpoint of statistical theory, the hypothesis being tested and the definition of pass or selection rates that will be compared should be decided before examining the data. Formal statistical tests on several relevant pass rates show that the lieutenant examination had a disparate impact on minority applicants, but the differences in the pass rates on the captain examination were not close to statistical significance. Furthermore, when the city cancelled both examinations, it only focused on the demographic mix of the high scorers who could receive an immediate promotion and ignored the 2-year life cycle of the list. Neither likely retirements nor job turnover during the 2-year life cycle of the results were considered. If this had been done, the city might have realized that three African Americans were likely to be appointed lieutenants along with two Hispanic captains.

Issues in the use of survival analysis to estimate damages in equal employment cases

Pan, Q., Gastwirth, J. L. — Fri, 17 Apr 2009 07:23:54 PDT

After plaintiffs prevail in employment discrimination cases, courts need to estimate the compensation they deserve. This issue arose after a defendant was found to have discriminated against job applicants older than 40 years. In order to compensate the successful plaintiffs, the court needed to estimate the length of time they would have worked, had they been hired. The job histories of 127 current employees and 43 employees who recently left were available for analysis. The accelerated failure time model, commonly used for the analysis of survival data, provides an estimate of the mean employment duration, adjusting for covariates. Three legal opinions emphasized that damage awards should be based on data concerning employees with similar qualifications as the plaintiffs so an ability model is proposed. A simulation study illustrates the potential underestimation of the job tenures of the deserving plaintiffs by the method adopted in the case. On the other hand, that method, which did not consider ability, can lead to compensating low-ability plaintiffs who would not be hired even in a non-discriminatory environment. Finally, the methods are applied to the motivating example and an appropriate damage award is suggested.

The effect of dependence between observations on the proper interpretation of statistical evidence

Gel, Y. R., Miao, W., Gastwirth, J. L. — Fri, 17 Apr 2009 07:23:54 PDT

In a recent securities law case, the statistical methods used by the regulator in analysing data on daily commissions and hypothetical profits from initial public offerings (IPOs) assumed that the data on consecutive days were independent. Consecutive observations in most business and economic data, however, are positively correlated. While statistical articles demonstrate that this type of dependence affects the distribution of virtually all statistics, including non-parametric and goodness-of-fit tests, the magnitude of the effect may not be fully appreciated. For example, in one comparison of commissions one broker received on days with an IPO to the days when no IPO was issued yielded a statistically significant p-value of 0.02, under the independence assumption. Accounting for serial correlation, the test actually had a non-significant p-value close to 0.09. Other examples of the effect of dependence include jury discrimination cases in locales where grand jurors can serve two consecutive terms as well as cases concerned with environmental pollution where measurements are spatially and temporally correlated. This paper describes the noticeable effect violations of the independence assumption can have on statistical inferences. The methods for correcting some standard non-parametric tests for serial correlation are also discussed.

Liability for negligently increased risk: the repercussions of Barker v. Corus UK (plc)

Miller, C. — Fri, 17 Apr 2009 07:23:54 PDT

The philosophical problem of ‘references classes’ is discussed within the context of probabilistic causation and, more particularly, liability for the negligent increase of the risk of personal injury. The implications of a recent addition to the U.K. case law on asbestos-related disease are then examined.

Forensic identification and criminal justice: forensic science, justice and risk, by Carole McCartney

Roth, A. L., Ungvarsky, E. J. — Fri, 17 Apr 2009 07:23:54 PDT

Clarification and corrections to 'On the attribution of probabilities to the causes of disease' by Peter Cooke and Arianna Cowling (Law, Probability and Risk (2005), 4, 251-256)

Cooke, P. — Fri, 17 Apr 2009 07:23:54 PDT