Via Ann Althouse, I came across a whining racists screed at “Above the Law” pointing to some really crappy political “research” claiming that law partners find more typos when they know the author of a paper is “African America”. This article is complete crap. Here’s why:
- There are no “p values” anywhere in the article. If you haven’t even bothered to figure out how likely your results were to come about by chance, you have nothing meaningful to say.
- “Of the 41 edits and/or comments on formatting, 11 were for “Caucasian” Thomas Meyer’s memo in comparison to 29 for “African American” Thomas Meyer’s memo.” 11 + 29 = 40, not 41. I’m supposed to trust an article from people who can’t do basic math?
- “There was no significant correlation between a partner’s race/ethnicity and the differentiated patterns of errors found between the two memos. There was also no significant correlation between a partner’s gender and the differentiated patterns of errors found between the two memos. We did find that female partners generally found more errors and wrote longer narratives than the male partners.”
- We undertook this study with the hypothesis that unconscious confirmation bias in a supervising lawyer’s assessment of legal writing would result in a more negative rating if that writing was submitted by an African American lawyer in comparison to the same submission by a Caucasian lawyer. In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities
Each of those sentences in #3 requires a p value, none is given. Were there more female partners reviewing the “African American” than the “white”? Not discussed. That would in and of itself have a significant effect on the scores.
But the most damning part is #4. They established a hypothesis, and then refused to examine the data in any way that might invalidate their hypothesis. They had very small sample sizes, and did not provide any p values to give their results meaning.
The only thing this study actually proves is that anyone who believes it is a credulous fool.