New York—An academic paper released in February suggests that there was extensive and systematic "ex-post" changes made to Thomson Financial's database of stock analyst recommendations made from 1993 to 2002. As a result of the changes, the stock picks shown in the database would have created annual gains that were 15% to 42% better than the originally recorded recommendations. Given the broad usage of Thomson Financial data in performance measurement from sources such as the Wall Street Journal and Starmine, as well as by academics, this finding has serious implications for measurement of the track records of Wall Street analysts.
Rewriting History
A paper titled "Rewriting History" released February 20, 2007 by professors at NYU, London School of Business and UVA compares two snapshots of Thomson's historical I/B/E/S database of research analyst stock recommendations, taken in 2002 and 2004 but each covering the same time period 1993-2002. The professors found almost 55,000 changes had been made to the database (out of a total of 280,000 observations). The changes were of four types: 1) The non-random removal of 19,904 analyst names from historic recommendations ("anonymized"); 2) the addition of 19,204 new records that were not previously part of the database; 3) the removal of 4,923 records that had been in the data; and 4) alterations to 10,698 historical recommendation levels.
The changes were not random. "For instance, bolder recommendations are more likely to be anonymized, as are recommendations from more senior analysts and Institutional Investor "all-stars." The characteristics of the additions and deletions are similarly unusual. Additions disproportionately consist of holds and sells; indeed, in the case of one prominent brokerage firm, 91.5% of its 234 additions are sells, and these increase the number of sells the firm has on the 2002 tape by a factor of 20. Deletions, on the other hand, disproportionately consist of strong buys, while alterations disproportionately consist of buys and strong buys (which are typically revised down)."
Boosting Performance
The result of the changes is a major improvement to analyst track records. "Analysts whose track records are affected are associated with more favorable career outcomes over the 2003-2005 period than their track records and abilities would otherwise warrant… we find that analysts associated with anonymizations experience a more than 60% increase in the likelihood of subsequently moving from a low-status to a high-status brokerage firm… Similarly, analysts are more likely to be rated the top stock picker in their sectors by the Wall Street Journal, which relies on Thomson Financial data, if some of their recommendations have been dropped or added."
The net result of this was to make many specific trading strategies appear better in retrospect than they actually were. Buying top rated stocks and shorting lowest rated stocks, based on the changed data, now perform 15.9% to 42.4% better on the 2004 revised data than on the 2002 tape.
Thomson: Necessary Maintenance
The paper concludes that using the Thomson database to measure historical performance is seriously impaired. "Collectively, our findings raise serious doubts about the replicability of past, current, and future studies using the I/B/E/S historical recommendations database."
As of February 2007, Thomson Financial's position is that "changes to Recommendation History have occurred as part of necessary processes and maintenance." A more cynical take was given by Barry Ritholtz of Ritholtz Research (who has a short position on Thomson): "A former Thomson executive with knowledge of the I/B/E/S database told me he was skeptical that Thomson's validation procedures could prevent a concerted effort by Wall Street to retouch its track record. The Thomson data are maintained by overworked, inexperienced clerks, said the former executive."
To see the full academic study click here.
Posted at 12:49 pm by Sanford (Sandy) Bragg
Permalink
