In response to Chris Giles’ allegations in the FT that Thomas Piketty’s book is replete with outright mistakes and dubious methods of blending the raw source data, Piketty has hit back hard. Bloomberg reports:
Piketty, the French economist whose book “Capital in the Twenty-First Century” has transformed the debate on the causes and consequences of disparities in income and wealth, called a Financial Times analysis of his statistics “just ridiculous.” He added in an e-mail to Bloomberg News that “there’s no mistake or error” in his work.
Now this is surely over-the-top. There can be little doubt that Giles uncovered at least some mistakes and errors in Piketty’s book. (It is gated but you can read Giles’ full analysis–complete with screen shots of Piketty’s Excel tables–here.) For example, it is almost certain that Piketty simply mistyped (or otherwise botched) some of the numbers for wealth ownership in Sweden in 1920. Even many of his defenders, when the FT story broke, were offering defenses of the form, “Well sure, it looks like he had a few clerical errors–and who wouldn’t in a book this big?–but nothing undercuts his main thesis.” So at the very least, it’s odd for Piketty to reportedly email Bloomberg that “there’s no mistake or error” in the book’s figures.
Now as far as the substantive issues, in the Bloomberg piece Piketty and at least one of his defenders portrays Giles’ critique as relying on Office for National Statistics (ONS) wealth data. As the Bloomberg piece reports it:
One “serious discrepancy” Giles said he found was in Piketty’s data on the U.K. While Piketty cited a figure showing the top 10 percent of its population held 71 percent of national wealth, a survey by the country’s Office for National Statistics put the figure at 44 percent.
The survey cited by Giles “is based upon self-reported data and is very low quality,” Piketty said in his e-mail. Other economists agreed.
Taken as Gospel
“The FT seems to take that survey as gospel, and I think that’s a mistake,” said Gabriel Zucman, an assistant professor at the London School of Economics whose research focuses on global wealth, inequalities and tax havens. “Anybody involved in this literature knows that survey data can massively underestimate wealth inequality. In this case, that is exactly what is happening.”
Whoa whoa whoa. If you go and carefully read Giles’ FT critique, you’ll see that this focus on the ONS is a complete red herring. First of all, it’s not Giles strolling in and declaring that the ONS is preferable to Piketty’s sources. Specifically, Giles reports that the estate tax data itself (which Piketty thinks is preferable to the survey) comes with the disclaimer: “[The data] is not a suitable data source for estimating total wealth in the UK, or wealth inequality across the whole of the wealth population; the Wealth and Asset survey is more suitable for those purposes”.”
But even putting that aside, Giles gave his readers the option of relying on the ONS data or not; that’s why there are two possible data points (for the 1% and 10% holdings) in Giles’ reconstructed charts. In particular, using the old Inland Revenue Series, you also see a big discrepancy with Piketty’s blue line. Here is Giles’ graph which shows Piketty’s charts in blue, and the various original sources in red:
To help you see the relevant data series, I put a black circle around the “LATEST IRS Top 10%” (the squiggly red line in the top). You can see the huge gulf between that raw data set, and Piketty’s blue line above it. You can see the Inland Revenue Series (IRS) data here; note that the figures for wealth held by the top 10% at the end of the series in the year 2005 is 54%, not the nearly 70% value through which Piketty’s trend line moves in that year. If you want to see an Excel spreadsheet showing all of this data, click here (and select the “UK Raw” tab when you open the Excel file).
If you do download Giles’ Excel file, you’ll see that the ONS survey data–which draws such ire from Piketty and his defender Gabriel Zucman in the Bloomberg story–only has values for 2006, 2008, and 2010. So it should be crystal clear to anyone who actually wants to see if Giles has a point–and went through his work carefully–that Giles’ case doesn’t rest or fall on our opinion of the ONS data. Rather, Piketty’s blue line in the shot above is well above the IRS data for the middle-2000s. The only raw data source Piketty can use to get such a high figure for wealth held by the 10% (at 70.5%) in the year 2010 is to rely on the “HMRC Top 10%” data, but the HMRC report itself proclaims that it is not suitable for such purposes (according to Giles in his FT critique, but I could not personally track down this claim and independently verify it).
Besides focusing on the red herring of the ONS survey data, the other main line of defense for Piketty is to say, “Well shucks, raw data is always spotty and the researcher necessarily has to make some judgment calls. Let’s give Piketty a chance to explain himself before claiming that these adjustments were arbitrary.”
Now in response to that, I want to remind readers that Piketty quite clearly butchered his discussions of both income tax rates and minimum wage hikes. If you haven’t seen those cases before, I encourage you to read them now. They are very important in deciding whether we think Piketty is guilty of systematic distortion of data to fit his desired political narrative. His discussions in both cases are full of basic mistakes on both dates and values, and moreover his errors very conveniently fit into the broader theme of his book. I personally stopped giving Piketty the benefit of the doubt after those two episodes.
But let’s do just one other example. In this blog post, Phillip Magness gives some examples where Piketty hard-codes changes into his Excel cells and makes odd “blending” decisions. But let’s focus on one that is particularly arbitrary and should be hard for his fans to brush off. First look at this screen shot of Piketty’s calculations:
In reference to the above, I’ll quote Magness:
Notice something odd about the first four figures.
They pivot between 6.622% and 6.122%, with the source being 1900…Rather than locate the readily available stats for the previous three decades, he randomly added or removed .5% in alternating fashion. Note that his figures for the UK, Sweden, and France in these same three decades do suffer from missing data points that are obscured (again) by decennial averaging, but they are at least based in sporadic bits and pieces of historical data that he has assembled. But for the US numbers, he simply made them up!
Now admittedly this is a small point on a perfectly noncontroversial chart, done apparently for no other reason than to avoid the labor of obtaining US historical records for a relatively unimportant older figure…But it’s also exactly the type of slipshod data handling that seems to be pervasive in Piketty’s work.
When his data had a gap Piketty did not address it by doing further research to fill the gap. Nor did he openly acknowledge his limitations by omitting the missing numbers and explaining their absence to the reader. Instead he made something up out of thin air, integrated it into his charts and graphs as if it were a part of the same continuous data set derived from the sources he claimed, and hoped that nobody would notice.
There are many oddities in Piketty’s work. We know for a fact that he has made demonstrable mistakes when it comes to tax rates and minimum wage levels, and these were “convenient” mistakes to boot. He almost certainly made transcription errors, as Giles documents. And his trends for UK wealth concentration have large discrepancies with other source data, not just counting the ONS surveys.
In the face of such allegations, for Piketty to call the FT article “just ridiculous” and claim “there’s no mistake or error” in the book is actually further evidence that this guy is a shyster.
It should go without saying that Piketty could be a slippery character, and yet his underlying thesis turns out to be correct. Even so, the way many of Piketty’s fans have shrugged off these criticisms is quite disturbing. To simply ask, “Why would Piketty publish his files if they were full of mistakes?” is hardly a good response, when Giles and others are daily documenting such mistakes.