E-Pulling Has E-Failed -- And An Attempt At An Explanation Why Such A Foolish Idea Ever Got Tried

From a recent report by Social Security's Office of Inspector General (OIG) (emphasis added):

Our objectives were to (1) assess the results of the Social Security Administration's (SSA) Electronic File Assembly (ePulling) pilot project and (2) determine whether the assessment procedures were effective in deciding when the Office of Disability Adjudication and Review's (ODAR) hearing offices were ready to implement ePulling. ...

ODAR expects ePulling to increase the efficiency of the EF preparation process and reduce the time it takes to prepare a case for hearing. ODAR estimates that ePulling may result in an annual reduction of 402 work-years and a savings of $16.6 million. ...

From June through December 2008, ePulling was used to prepare the files for 773 cases. ...

The 773 cases prepared using ePulling contained 250,938 pages. For each page, ODAR assessed the ePulling software's accuracy in identifying four categories of information: type of document, document's source, document's beginning date, and document's ending date. ODAR found that multiple corrections were required for the pages processed. In fact, 433,790 corrections were required for the 250,938 pages processed. ...

Since June 2008, the contractor has made five enhancements to the ePulling software. To determine whether the enhancements improved the accuracy of the software, ODAR processed the same 10 cases through each ePulling software enhancement. In addition to the 10 cases repeatedly tested, ODAR selected 5 to 10 new cases to test each enhancement. ... As shown in Table 2, following the January 2009 enhancement, the accuracy rates computed from this limited number of cases in the areas of type of document (65 percent) and source of document (75 percent) show accuracy rates comparable to the rates ODAR calculated for the 773 cases (see Table 1). In the area of dates of document (64 percent), the accuracy rates were slightly better for the 15 to 20 cases as compared to the accuracy rates reported for the 773 cases, which was in the 40-percent range. However, accuracy rates declined between June 2008 and January 2009. ...

We interviewed six employees in the Tupelo, Mississippi, Hearing Office and three employees in the Minneapolis, Minnesota, Hearing Office who prepared cases using ePulling. All nine employees stated that ePulling increased case preparation time when compared to the traditional EF preparation process. ...

We recommend that SSA:

1. Perform a complete assessment of the ePulling pilot project results before expanding the use of ePulling to other hearing offices. The assessment should ensure that ePulling will not adversely affect file preparation time or any other aspect of the hearings process.

2. Consider if historical data can corroborate or improve upon the current 3-hour case preparation time estimate used to assess ePulling's impact on hearing office productivity.

3. Determine whether the ePulling pilot testing should also include cases with more than 300 pages. ...

SSA agreed with our recommendations.

Regular readers may recall that I have been extremely skeptical about e-pulling. I wish I could claim great prescience, but I cannot. I think that almost everyone familiar with pulling exhibits knew from the beginning that e-pulling could not work.

Lisa DeSoto appears to be the person most directly responsible for the e-pulling experiment. She is now gone from Social Security. We may never know whether e-pulling had anything to do with her departure. However, I do not want to blame Ms. DeSoto too much. The underlying problem is that Social Security was being given grossly inadequate resources to perform its mission. Social Security management knew that things were falling apart, but they were under enormous pressure to manage their way out of the problem. They resorted to a variety of "Hail Mary passes" such as e-pulling. All or virtually all of these "Hail Mary passes" failed, wasting money and staff time. These "Hail Mary passes" also misled Congress about the resources the agency needed to get its work done. Why give Social Security more money and personnel when Social Security managers are promising that "Hail Mary passes" are going to save the day? But, of course, Social Security managers were only telling Congress what it wanted to hear.

The real problem was that the Republican controlled Congresses between 1994 and 2006 were just irresponsible. They underfunded Social Security to the point that its managers tried desperate measures. I am sure that the Democratic controlled Congresses since 2006 have made and will continue to make mistakes, but they are not starving Social Security to the point that its managers repeatedly try "Hail Mary passes." Even a Republican like Michael Astrue should be able to appreciate that.