Law, Probability and Risk - recent issues

Issues in the use of survival analysis to estimate damages in equal employment cases

Pan, Q., Gastwirth, J. L. — 2009-04-17

After plaintiffs prevail in employment discrimination cases, courts need to estimate the compensation they deserve. This issue arose after a defendant was found to have discriminated against job applicants older than 40 years. In order to compensate the successful plaintiffs, the court needed to estimate the length of time they would have worked, had they been hired. The job histories of 127 current employees and 43 employees who recently left were available for analysis. The accelerated failure time model, commonly used for the analysis of survival data, provides an estimate of the mean employment duration, adjusting for covariates. Three legal opinions emphasized that damage awards should be based on data concerning employees with similar qualifications as the plaintiffs so an ability model is proposed. A simulation study illustrates the potential underestimation of the job tenures of the deserving plaintiffs by the method adopted in the case. On the other hand, that method, which did not consider ability, can lead to compensating low-ability plaintiffs who would not be hired even in a non-discriminatory environment. Finally, the methods are applied to the motivating example and an appropriate damage award is suggested.

The effect of dependence between observations on the proper interpretation of statistical evidence

Gel, Y. R., Miao, W., Gastwirth, J. L. — 2009-04-17

In a recent securities law case, the statistical methods used by the regulator in analysing data on daily commissions and hypothetical profits from initial public offerings (IPOs) assumed that the data on consecutive days were independent. Consecutive observations in most business and economic data, however, are positively correlated. While statistical articles demonstrate that this type of dependence affects the distribution of virtually all statistics, including non-parametric and goodness-of-fit tests, the magnitude of the effect may not be fully appreciated. For example, in one comparison of commissions one broker received on days with an IPO to the days when no IPO was issued yielded a statistically significant p-value of 0.02, under the independence assumption. Accounting for serial correlation, the test actually had a non-significant p-value close to 0.09. Other examples of the effect of dependence include jury discrimination cases in locales where grand jurors can serve two consecutive terms as well as cases concerned with environmental pollution where measurements are spatially and temporally correlated. This paper describes the noticeable effect violations of the independence assumption can have on statistical inferences. The methods for correcting some standard non-parametric tests for serial correlation are also discussed.

Liability for negligently increased risk: the repercussions of Barker v. Corus UK (plc)

Miller, C. — 2009-04-17

The philosophical problem of ‘references classes’ is discussed within the context of probabilistic causation and, more particularly, liability for the negligent increase of the risk of personal injury. The implications of a recent addition to the U.K. case law on asbestos-related disease are then examined.

Forensic identification and criminal justice: forensic science, justice and risk, by Carole McCartney

Roth, A. L., Ungvarsky, E. J. — 2009-04-17

Clarification and corrections to 'On the attribution of probabilities to the causes of disease' by Peter Cooke and Arianna Cowling (Law, Probability and Risk (2005), 4, 251-256)

Cooke, P. — 2009-04-17

Case comment--People v. Nelson: a tale of two statistics

Kaye, D. H. — 2008-11-12

Recalibrating deference: comments on Professor Gastwirth's papers on Zuni Public School District 89 v. U.S. Department of Education, and related matters

Rosenblum, M. — 2008-11-12

This paper examines how straightforward application of two relatively simple statistical concepts, the measurement of relative variation or disparity in per-pupil expenditures by school districts and interpretation of the term ‘percentile’ as referring to school districts rather than students, triggered multimillion dollar litigation. As explained in Section 2, different interpretations regarding how to calculate relative disparities based on such percentages resulted in opposite conclusions as to whether funding equalization had been met and, in turn, as to which entity should receive federal funding—the Zuni school district or New Mexico state education officials. Section 3 covers Professor Gastwirth's two papers previously published in this journal, which effectively juxtaposed the lack of statistical expertise in the legislative and administrative processes related to the Federal Impact Aid Act, on one hand, and his own repeated surveys within the academic statistical community regarding the statistical meaning of this statute, on the other hand. Sections 4 and 5 discuss the Supreme Court's 2007 decision affirming the Department of Education's funding formula, how this case reflects the ongoing tension between various Justices concerning the appropriate level of deference that courts should accord to federal agency regulations, the role of expertise (here statistical science) in the public policy arena that includes judicial review of disputes between aid recipients and disbursing agencies and the explicit mechanism through which external expertise on these issues becomes incorporated into the judicial process. Section 6 concludes that Zuni could have been decided differently if the consensus of statistical opinion concerning the term ‘percentile’ was part of the case record and taken into account by the Justices.

Statistical base and background rates: the silent issue not addressed in Massachusetts v. EPA

Taggart, A., Blackmon, W. — 2008-11-12

Global climate change is one of the most serious challenges humans currently face. The US courts have been called upon to adjudicate disputes concerning the proper way to confront this challenge. Most recently, the US Supreme Court faced one such dispute in Massachusetts v. E.P.A. Faced with a question of what to do, and the legal mandate of a federal agency, with a question of natural background rates of warming, the Court decided the issue upon narrow constitutional grounds. In so doing, the authors contend that the Court sidestepped a significant issue of how the US Courts understand and handle important questions of statistical evidence. Repeatedly, disputes are brought to court that are questions to be resolved by determining nothing more that the difference between a base rate and another rate. Yet, the authors’ research has shown this simple principle is addressed rarely, and sometimes not appropriately. The authors call for more education in math and statistics for the legal profession.

A respecification of Hanson's updated Static-99 experience table that controls for the effects of age on sexual recidivism among young offenders

Waggoner, J., Wollert, R., Cramer, E. — 2008-11-12

The original version of Static-99 is widely used for assessing sexual recidivism. It does not, however, account for the negative effect of age on recidivism. Hanson (2006, Sexual Abuse, 18, 343–355) took up this problem by disseminating an updated experience table for Static-99, based on 3425 sex offenders, that was stratified by four rows of risk categories and five columns of age categories. Contrary to expectations, updated Static-99 reported that the highest group-wise recidivism rates accrued to sex offenders in the second youngest category. The explanation for this inconsistency is that the entries in updated Static-99 are misspecified for the youngest offenders because, in effect, Hanson used one scoring system for assigning older offenders to risk groups and another for the classification of younger offenders. Updated Static-99, therefore, needs to be respecified. We applied a Bayesian algorithm to do so. Updated Static-99 holds out so many advantages that we believe it is unethical for evaluators to use original Static-99 unless they can present overwhelming evidence in support of this choice. Other contributions of respecifying updated Static-99 are discussed.

Beyond the individuality of fingerprints: a measure of simulated computer latent print source attribution accuracy

Cole, S. A., Welling, M., Dioso-Villa, R., Carpenter, R. — 2008-09-08

Efforts to harness computer fingerprint databases to perform studies relevant to fingerprint identification have tended to focus on 10-print, rather than latent print, identification or on the inherent individuality of fingerprint images. This paper reports on three experiments that measure the accuracy of a computer fingerprint matcher at identifying the source of simulated latent prints. The first experiment used rolled prints supplied by the National Institute of Standards and Technology (NIST) to simulate latent prints. The second experiment used our own manufactured latent prints. The third experiment used latent prints supplied by NIST. An Automated Fingerprint Identification System (AFIS) was used to simulate the task that a human latent print examiner is typically asked to perform as part of ordinary casework. The AFIS performed this task, for which it was not designed, fairly well. However, there are non-mate images that scored very highly on the AFIS's similarity measure. These images would be susceptible to erroneous conclusions that would be given with a very high degree of confidence. Not surprisingly, the same was also true of the simulated latents which contained less information. We suggest that measuring the accuracy and potential for erroneous conclusions for AFISs might provide a basis for comparison between human examiners and automated systems at performing various identification tasks. Such comparisons might stimulate competition, innovation and improvement in the performance of these tasks.

A forensic approach to the interpretation of blood doping markers

Sottas, P.-E., Robinson, N., Saugy, M., Niggli, O. — 2008-09-08

In the fight against blood doping, the interpretation of the measured levels of blood markers is based on either population-derived reference ranges or the previous test history of the individual under scrutiny. In this report, we demonstrate how an empirical hierarchical Bayesian model can be used to unify both approaches. The aim is to allow anti-doping organizations to bring reliable evidence of blood manipulation in front of a disciplinary panel. Before any tests are performed on an individual, population distributions constitute the priors of a Bayesian network that may depend on heterogeneous factors such as gender, ethnic origin and age. Inferences from the results of a new test are then drawn iteratively. A decision rule can be defined to minimize the expected costs of a decision. Secondly, the same model can be applied to evaluate the evidence of blood doping from a full sequence of individual test results, and not just from a single test result as a function of previous results. We obtained unprecedented sensitivity on a database of 1239 blood samples. Thirdly, if applied to a population of athletes, an extension of the model makes it possible to estimate the prevalence of blood doping for reasonably large populations of athletes. Knowledge of the prevalence allows the decision maker to estimate the prior odds of an athlete being doped. As a consequence, the false-positive fallacy, a form of the prosecutor's fallacy that originates from today multiplication of the number of anti-doping tests, is removed. The joined application of the Bayesian model for (1) the estimation of the prevalence at the population level and (2) the evaluation of the evidence at the individual level will allow anti-doping organizations to prosecute cases for which evidentiary values are derived from indirect blood tests.

Les Demoiselles d'Evanston: on the aesthetics of the Wigmore chart

Hay, B. L. — 2008-09-08

Wigmore's ‘The Problem of Proof’, published in 1913, was a path-breaking attempt to systematize the process of drawing inferences from trial evidence. In this paper, written for a conference on visual approaches to evidence, I look at the Wigmore article in relation to cubist art, which coincidentally made its American debut in New York and Chicago the same spring that the article appeared. The point of the paper is to encourage greater attention to the complex meanings embedded in visual diagrams, meanings overlooked by the prevailing cognitive scientific approaches to the Wigmore method.

The U.S. Supreme Court finds a statute's description of a simple statistical measure of relative disparity 'ambiguous' allowing the Secretary of Education to interpret the formula: Zuni Public School District 89 v. U.S. Department of Education II

Gastwirth, J. L. — 2008-09-08

The degree of deference courts give to the interpretation of a statute made by the government agency administering it arises in a variety of legal contexts. This spring the U.S. Supreme Court decided a case concerning the interpretation of a formula used to determine whether the educational funds available to the school districts of a state are ‘equalized’. A state with an ‘equalized’ school system receives most of the federal money given to school districts in the state to compensate them for the real estate tax revenue lost due to the presence of a large federal facility in the district. The main issue concerned the calculation of the percentiles used to determine the school districts, called local educational agencies (LEAs) as the formula uses the 5th and 95th percentiles. In order for the funding of education in a state to be ‘equalized’, the disparity (D) or ratio of the difference between the per-pupil expenditures of the LEAs at these two percentiles to the fifth percentile needs to be less than or equal to one-fourth. After arranging the LEAs in increasing order of their per-pupil expenditures, the government calculated the percentiles by weighting the LEAs by the number of students and found that New Mexico had an ‘equalized’ system. When the LEA per-pupil expenditure data are not weighted, however, the disparity measure exceeds 0.25. Two impacted LEAs interpreted the law as specifying the unweighted calculation and sued the Department of Education as they would receive substantially more money if the entire federal payment went to the impacted LEAs. The Court found that the statute was ambiguous and that the government's interpretation was permissible. The opinion also gave four other ‘interpretations’, including the one offered by the plaintiffs, that it would also deem permissible. Both the majority and the dissenting opinions are summarized. The results of two informal surveys of statisticians who read and interpreted the formula are reported. Most favoured the ‘unweighted’ calculation and none agreed that the wording of the statute was consistent with all five interpretations the Court would allow. From the opinion and transcript, it appears that the Court might have believed that the government's approach was an ‘approximation’ to what the disparity would be if it were calculated from data on each pupil, had such data been available. Using data from a classic school segregation case, where the disparity between schools with predominantly minority students was 0.70, we formed a ‘state’ with 22 LEAs by randomly aggregating five schools to each state. In 10 000 randomly formed ‘states’, the largest disparity calculated from the LEA-wide data was 0.38, just over one-half the true disparity. Three-fourths of these state systems had disparity measures less than 0.25 and would be deemed ‘equalized’ even though the overall disparity was nearly three times as large. Since using district-wide expenditure data substantially underestimates the value of the disparity calculated on school-wide data, it is likely to underestimate the disparity between students even more. This result applies to the interpretations of both parties. If the purpose of the Federal Impact Aid Act is to ensure that students in schools affected by a federal presence receive an adequate education, Congress needs to modify the formula. Currently, a state with only a few impacted LEAs can be ‘equalized’ even if all of them are at the low end of the distribution and are ‘deleted’ from the current calculation.