Subscribe to Integrity ResearchWatch by Emailor  in an RSS/XML reader
For those of you who don't know about Integrity Research Associates, we publish syndicated research reports; provide an online database of reviews, analysis and ratings on research firms; and offer specialized consulting about the equity research industry for professionals at money management, hedge fund, and broker / dealer firms. You can learn more about our company and our products / services at www.integrity-research.com.
Please feel free to contact us about our company, our products, or our services using the contact information below.
Integrity Research Associates, LLC
1115 Broadway, 12th Floor
New York, NY 10010
Tel: 212-845-9088
Fax: 212-845-9091
E-Mail: info@integrity-research.com
URL: www.integrity-research.com
|
|
|
 |
|
Wednesday, May 23, 2007
There's More to Analysis Than Swingin for the Fence
New York – We have discussed the Wall Street Journal/Thomson Analyst review already this week, but we feel there are a few other points to make. Of interest is the actual set up and measurement of the analysts' returns. We view the process to be somewhat lacking, biased and incomplete.
First, as our Monday blog indicated, the analytical homerun hitters are generally rewarded by the way performance is calculated. If the analysts makes one call that adds 200% over the period, his performance would be stellar, but we question whether it may have been simply lucky. With a greater amount of data, we could assess the probability of the calls being lucky.
Second, the analysis has no bogey. In other words, there is no way to know if the winner in a particular industry beat his own industry or not. Of course, chances are that he/she did, but this should be explicit rather than implicit via comparing the analysts' returns with the returns of a passive buy and hold portfolio of the analyst's stock over the same time frame.
Third, raw performance does not account for the analyst's ability allocate stocks into the correct category. This can be estimated by looking at the percentage of stocks that the analyst got in the right bucket (batting average). For example, the analysts had and average return of 30%, but had only 3 of 30 stocks he called Buys that actually increased in value. He would then have a but batting average of 10%.
Fourth, the analysts are allowed to have the announcement effect by virtue of getting the previous day's closing price when they make recommendation changes. This means that the analysts of the Bulge bracket firms can benefit from announcing the sell and watching it move south in response to their own call. Also the analyst can game the system by changing recommendations during the next day of trading to take advantage of any news that may have hit the markets that day and booking the change in price from the previous day's close as part of their own performance.
Fifth, there is no analysis of the amount risk the analyst has taken on board to achieve the returns posted. This risk can be take the form of market risk and default risk. There are well documented ways to test for the risk/return of a portfolio, including the Sharpe Ratio and the Information Ratio. Additionally, one could calculate the semi-variance of the portfolio – i.e. the risk of the stock falling, given a buy recommendation, or the risk of a stock rising, given a sell recommendation.
The obvious benefits of the Wall Street Journal /Thomson analyst assessment is its longevity and the consistency with which it has been calculated over the years. However, the performance could be augmented by adding some of the missing elements mentioned above.
Posted at 11:26 am by Thomas Hutchinson
 | Posted by NameScott Drysdale @ 06/05/2007 01:58 PM PDT | | None of the available services measure sell side research on a basis comparable to a portfolio manager. That includes Starmine.
For example, assume a fund manager buys stock XYZ at $30. For the next two months, he owns the stock and finally sells it at $45. Not bad. Well, what's his performance? Up 50%? Well, to the client who invested in the fund the day the stock was trading at $45, their price was $45 and their return was zero. In other words, the performance of a call is a function of the price of the stock during the entire time it was "rated" buy. An analyst who recommends a stock at $30 and keeps it rated a buy up to $60 and continues to keep it a buy down to $45 is a negative 25%. One might also say it is 50% minus 25%, or, 25%. But, in the world of a money manager, it is never 50%.
The issue then of measuring performance of sell side analysts w/o considring the realities of investing and economics is a bit like saying a car goes from zero to sixty in zero seconds flat. It's absurd. At a minimum recommendations must be average returns, and better yet, returns weighted on a price per day/days traded basis. All the rest of the measures are just some of the reasons PMs really couldn't care less about the "performance" of sell side calls. PMs are not impressed by stocks that are rated buy and relatively outperform their benchmark for 2 weeks and then underperform for 10 weeks regardless of the nominal returns from beginning to end of the holding period. | |
| Posted by Scott @ 05/24/2007 10:11 AM PDT | | As someone who is intimately familiar with the WSJ's annual analyst review (in a former life, I was involved with it), I can assure you that "the consistency with which it has been calculated over the years" is a mirage. Two different firms using totally different data sets and methodologies have conducted the review over the years.
I would be surprised if any past results could be replicated today. In fact, I would bet good money that 3 months from now this year's results could not be replicated using then existing historical data (i.e., 3 months from now, re-create the data set from original source data used for this year's review and then re-calculate the results). I would also be willing to extend that bet to other vendors' platforms (since, I believe, the underlying data comes from the same sources). This annual exercise has advertising value for the WSJ and Thomson, for sure, but probably not much else, in my opinion. | |
| Posted by David Lichtblau @ 05/23/2007 03:17 PM PDT | | These flaws are well known, which is why the sell side puts so little weight on the WSJ results.
StarMine has been measuring analyst performance using the most rigorous, fair, and objective performance metrics since 1999. That's why so many of the world's publishing analysts actually have their StarMine scores used in their reviews and compensation plans.
StarMine's metrics include the appropriate bogey; they include the ability to sort stocks correctly; they ensure a minimum batting average; and they score analyst performance only starting AFTER the call was made (the most inexcusable flaw in the WSJ methodology.) Analysts can't score highly on StarMine metrics through the luck of being assigned the hottest industry or by moving the stocks with their own calls.
Separately, StarMine also quantitatively measures the accuracy of earnings estimates and not just the results of buy/sell recommendations.
StarMine's 6th annual public tally of the top analyst performances are available at http://www.starmine.com/index.phtml?page_set=press_room&sub_page_set=analyst_awards | |
|
|
|
