From The Judge Lottery: Within-Office Disparities in Social Security Disability Adjudication, Fiscal Year 2025 by Drew Patterson:
Description
A claimant who appeals a denied Social Security disability claim is assigned, essentially at random, to an administrative law judge (ALJ) within a hearing office. Using the Social Security Administration's public ALJ disposition files for fiscal year 2025, this paper measures how much that assignment matters. Among 1,023 judges who issued at least 100 decisions in a single office (317,462 decisions in total), allowance rates ranged from 8.8 percent to 92.8 percent. The disparity is not primarily geographic: 73 percent of the variance in judge allowance rates lies within hearing offices rather than between them, and in the median office the gap between the most and least generous judge is 32.6 percentage points. A simulation in which every judge in an office decides identical case pools shows that sampling noise can account for only about 5 percent of the observed within-office variance. Judge allowance rates are also highly stable across years (r = 0.93 between FY2025 and FY2026 year-to-date), which rules out one-off docket composition as an explanation. Office-level workload measures explain little of either the level or the spread of allowance rates, and a state-month panel of initial-level determinations from 2015 to 2026 shows only a small association between backlog pressure and allowance rates. The judge draw remains one of the largest observable determinants of whether a disability appeal succeeds, a decade after the agency's quality-review reforms. Includes full analysis code and the exact SSA public data snapshots used. …
16 comments:
Such variance is something that is common knowledge among attorneys/decision writers working in OHO, and I would guess the same is true of claimant representatives.
This phenomenon is not limited to Social Security. Daniel Kahneman, a highly regarded psychologist who won the Nobel Prize in economics, wrote a book about it called “Noise: A Flaw in Human Judgment.” (https://en.wikipedia.org/wiki/Noise:_A_Flaw_in_Human_Judgment ). Disability decisions are cited as one of many examples of flawed, highly variable judgments in that book.
At the risk of inducing an apoplectic reaction, it is worth considering whether AI, used intelligently as a supplemental tool in human decision-making, could partially address the fact that, for many Social Security claimants, their case is largely determined by which judge it is assigned to, regardless of merits.
It’s not just judge variance. VE variance can make a huge difference as well.
Why don’t they ever discuss the variance of district court judges in their reviews of disability cases?
There is going to be variance in any program/situation where people are deciding the credibility of a claimant and the "facts" of the case. Their religious and political views will come into play. Hell, their mood or how they're feeling that day will play at least a small role. However, the problem with SSA is that this variance is MUCH wider than just about anywhere else. If I'm wrong, please give me an example. ALJ's how their own idea of what it means to be disabled and they will rule accordingly. Yes, many of those decisions will contain error after error, but good luck getting the AC and their 8% remand rate to help you out.
Interesting read. Patterson's finding that adjudicators with grant/denial rates that are persistent extreme outliers over the years are probably "applying systematically different de facto standards" of disability is not a surprise. However, it adds value to statistically exclude other likely explanations for the disparity with data, and to document the degree of the problem.
Get rid of the judges and make disability purely a medical determination on whether someone meets a listing. Then, no more complaints about judge decision variance and inaccurate decisions
In my opinion, this is because they have traditionally hired ALJs with no program knowledge or prior SSA experience who think the crux of the job is to decide “credibility” issues. In reality, the job is to decide whether the evidence is consistent with the claimant’s allegations and whether the disability criteria are met. People with program knowledge understand this.
Sadly that would leave many thousands more who are genuinely unable to work bankrupt and homeless.
@11:06. Your comment jibes with my own experience and observations over decades.
So disabled people would have no appeal rights? Maybe the "judges" should be doctors instead of lawyers.
I get the idea of there being a natural variance in allowance rates just like anything. However, I don’t think it’s a coincidence that the increasing political polarization of this country is coincided with a growing feeling that the outcome is very judge dependent
Working as an attorney in a hearing office it’s straightforward to tell the liberal versus conservative personality styles.
The lack of any significant oversight by OHO/SSA of wide variations in pay rates was a contributing factor in the Eric Conn situation. As I recall, the judge involved in that case had a pay rate of over 90%. In some ways Eric Conn was the sucker, since he was bribing the judge to pay cases at approximately the same rate he was paying everybody else anyway. Four ALJ’s with extremely high pay rates and outsized productivity numbers eventually were called to testify before a congressional subcommittee. I think the variation in pay rates did shrink significantly after that. However, as this study indicates, it’s still present and there really isn’t a good reason for it. It gives the lie to the idea that the judges approach cases similarly, in spite of regulations that attempt to get them to do just that.
The implementation of the grids was one attempt to bring some degree of uniformity to Social Security disability adjudication. And it did force judges to pay some cases they otherwise might not have paid. But another not well kept secret is how certain judges figure out what is needed to deny a case and then reverse engineer the RFC with that in mind when their gut tells them someone doesn’t deserve disability benefits. As opposed to following the law and first determining the RFC and letting the chips fall where they may
That would be my general assumption as well, if I hadn't worked directly for two ALJs with very high propensity to pay, and very conservative political views. So maybe political views are a factor, but just one of many.
It wasn't the variation rate that shrunk. The Agency purged the high paying outliers while ignoring the low paying outliers.
31-year retired OHO senior attorney here. During my tenure with the agency there were a large number of judges from across the country and with very diverse backgrounds that came through my office. I probably wrote for more than 50-60 such judges, and knew of many more. I can say that in my experience a significant number of the "high payers" did so more out of laziness than political or personal outlook. While always true, it became even more pronounced with the advent of the efile, which coincided with a geometric increase in the size of the medical evidence. Many of those judges simply did not want to expend the time/effort to go through all that in order to make an informed decision. While there was pressure from the agency to get the cases out, that's not an excuse to abdicate your duty to the claimant and the public fisc. It was, nonetheless, very evident to me that many of these people just wanted to occupy the chair and collect their pay without doing much. Five minute hearings were exceedingly common. I also remember one particular set of "instructions" I received on one occasion, which read: "Pay, deny, dismiss, whatever." I have many similar stories.
My experience as a retired ALJ and former HOCALJ is also consistent with this. The new judges who were not familiar with SSA were at a striking disadvantage that it took years to overcome. And by that time, habits were set.
You don't want that. The listings are very very difficult to meet now, and making the Listings the only determinate will simply encourage them to be made more and more difficult to meet.
Post a Comment