Integrity Researchwatch has moved to
http://www.integrity-research.com/cms/researchwatch
Please update your bookmarks.

New York - One of the more interesting aspects of research recommendation accuracy measurement is the concept of stock sorting ability (internal consistency). The main concept here is whether the firm is able allocate the stocks it covers into the correct buckets. To do this, the recommendations are allocated to three categories; Buy, Sell and Hold. The performance data used as the basis of this blog is from Investars.

There have been a number of the metrics put forward to track this ability. The simplest is to create an index. The firm will receive25 points if the firms Buys outperform its Holds, 25 points if its Holds outperform its Sell and 50 points if the Buys outperform the Sells. This is a reasonable way in which to assess the overall ability of the research, but is less useful in choosing among research firms. Because this metric has no scalability, lumps many firms into each category. We have worked to measure stock sorting ability in a more granular manner.

A further complication is from which portfolio does most of the return come, and how many stocks created this return. As for the latter, Investars adjusts for the number of stocks issue by comparing research providers with like coverage. The universe is split into 0-99, 100 to 499 and over 500 stocks under coverage. This is a perfectly acceptable practice, except that it limits the number of firms in each group. Integrity has kept all firms in the same pool and looks to adjust in other ways.

To estimate the sorting ability of a research provider, Integrity calculates the actual differences between the Buy and Sell, The Buy and Hold and the hold and Sell and then sorts each of these and assigns the rank to the research firm. The sum of all of the ranks is then calculated and this series is turned into a index to determine which firms have the best record (lowest score). The top ten firms by this measure are tabulated below.

Research Provider	B-S Rank	B-H Rank	H-S Rank	Index
Taglich Brothers	4	9	9	100.0
Nollenberger	6	6	18	97.7
Susquehana	3	21	7	97.4
FutureAlpha	8	14	20	94.3
Zacks Investment Res.	11	25	21	90.0
Ativo Research	19	30	28	84.4
Ford Equity Research	20	28	29	84.4
Merrill Lynch	22	24	38	82.4
Longbow Research	2	80	3	82.2
Pacific Crest	21	15	50	81.9

 
The above firms can be said to consistently have sorted the stocks in their coverage universe very well over the year.

One of the issues with this approach is what were the returns of the various portfolios. This will uncover the relative contributions of the Buy, Hold and Sell portfolios. For example, looking at the Buy minus Sell portfolios, Taglich Brothers achieved a return of 26.1 basis points per day over the time frame. The composition of this was a 9.9 bps return for the Buy portfolio and a -16.2 bps return from the sell portfolio (hence B-S = 28.1). Taglich Brothers got a better return from their Sells than they did from their Buys. This was also the case for second place Nollenberger, but Nollenberger got a 26.6 bps boost from its Sell portfolio and a 5.1% loss from its Buys. This is by no means an isolated event. In 2007, the average RP received 85% of its total return from the Sell portfolio and 15% from its Buy portfolio.

Buy, Hold, Sell and Buy - Sell Returns for the Top Ten Above
Year Ended December 31, 2007

 

Research Provider	Buy	Hold	Sell	Buy-Sell
Taglich Brothers	9.94	-0.02	-16.14	26.08
Nollenberger	-5.14	-16.97	-26.55	21.41
Susquehana	-7.82	-13.90	-35.86	28.04
FutureAlpha	-0.16	-7.61	-16.96	16.80
Zacks Investment Res.	3.83	-1.63	-10.81	14.64
Ativo Research	6.25	1.33	-4.60	10.85
Ford Equity Research	2.47	-2.61	-8.37	10.84
Merrill Lynch	4.13	-1.43	-6.04	10.17
Longbow Research	-7.26	-8.89	-37.14	29.88
Pacific Crest	4.14	-2.9	-6.20	10.34

A glance at the above table and it is apparent that even though these research providers exceeled at stock sorting ability (as measured), there were, in some cases, other issues that detracted from their overall appeal. Given the blindness of many of specific metrics, the analysis has no one way to assess the performance of a research provider, let alone the value-added that it may provide.

Integrity is currently developing a combine metric that will isolate the various characteristics that are favorable into an overall performance of research recommendations. This is similar to the Dupont Ratio in financial analysis, which decomposes overall earnings into constituent parts. The index should include:

• Overall Returns
• Returns of the Buy, Sell, Hold portfolios
• Stock Sorting ability
• Batting averages
• Coverage and number of recommendations
• Average holding period of recommendations
  

If we succeed developing the Integrity Performance Index, we will be rolling it out next quarter.

Note: The data for this report comes from Investars Light (www.investars.com). The analysis is that of Integrity Research Associates.