Hi Guys,
I posted to especially congratulate Charles on his superior research that appears in the June edition of MFO’s Commentary section.
His study is first-rate and compares favorably with any that academia could originate. If this were a Hollywood production, he would be nominated, and likely win an Oscar. This in no way is meant to denigrate Dave’s always excellent monthly summary. Their output is consistently insightful and useful.
And remember that they don’t earn a salary for these extensive efforts. But they do earn our heartfelt thanks.
Charles, you handle and manipulate this huge mutual fund database with extraordinary skills and finesse. I especially like your chosen tasks; your objectives are predictably practically oriented.
Based on these recent findings, only a fool would select a mutual fund from the lower performance ranks that you identified. Sure results are timeframe dependent, but I suspect fund migration from one grouping to another will be modest. Earlier studies suggest that poor funds remain poor funds over time and often disappear from the scene.
Your dedicated work ethic is reflected in your monthly contributions. Thomas Jefferson observed: “I’m a great believer in luck, and I find the harder I work the more I have of it”. Your commitment to these projects will indeed make you a luckier and a more prosperous investor. Me too.
A portion of my portfolio will always be invested in actively managed funds. I’ll be sure to select from those that topped your Alpha listing.
In an uncertain and changing world, your work tilts the investor’s odds of prudent fund selection a little more in his/her favor. Thank you once again, but please, please continue the march.
My Best Regards to all, but especially to Charles.
Comments
My first reaction on reading the statistical analysis in the June commentary was a big sigh. Just like logical fallacies, there are statistical fallacies to be avoided. I had decided to post later suggesting that Charles consult a statistician to validate the inferences made from such calculations. Then I read this and see how dangerous and irresponsible such material can be. Many wear the mantel of science, only a few practice it.
I do not know Charles' background but I have been getting increasingly concerned with the focus on computing numbers in the tools while losing the meaning of the numbers and in particular the validity of numbers crunched.
My horror was in seeing the computations comparing average of averages to other averages and swimming in a pool of averages to suggest invalid and incorrect inferences.
No, science isn't this. But certain part of academics is a social club where mutual back slapping has taken over the search for knowledge and understanding. Finance has its fair share.
If I may borrow from @MJG's mindless drivel "No fool would make a decision on selecting or not selecting a specific mutual fund based on this analysis".
This doesn't mean that the table is entirely meaningless. It is not surprising to see, some expected divisions. However, when potentially faced with false positives and false negatives, such analysis is unreliable for any action plan.
A statistician will tell you the shortcomings of this computation and the potential incorrect inferences it can lead to. But a simple step in use of any statistical use is a sanity check of the results to see whether it leads to anomalies and false positives and negatives which might suggest a problem with data or its inference.
Typically, this is done with observing if any of the results look out of place and if the measurement or analysis has a problem.
One thing that caught my eye immediately was the inclusion of Sextant in the list of Bottom Quartile. Some people may know that this is a well respected fund family with some very well managed funds that manage very conservatively even if it means lagging at certain periods. Sanity check would mean looking at an explanation of how a famy like this lands up in the bottom quartile. Not necessarily that the data is wrong but rather if the computation is being affected by something in the formulation that biases for or against the family that makes inferences unreliable at best.
Tracing through the computation to explain such an anomaly (which should be a responsibility of anyone using a computation to make such inferences), one can see serious problems with this computations where what seems like reasonable approximations at each step compound over multiple steps to make the outcome problematic.
Sextant has 6 funds (don't know if the funds under Amana are counted here).
Statistically, each fund being below category without even looking at the reason will pull the average down in such a small sample by almost 17%. On the other hand, a family with 20 funds can get away with 3 "stinkers" without being so affected.
This ranking for that reason alone is statistically nonsensical because of the small sample space in many families that can help or hurt the family.
Now, digging a little bit deeper, what does lagging a category mean? To statistical significance? A fund being 0.01% below category can pull the family down by say 10% in a small sample space and a fund being 0.01% above may pull the family up by 10%.
More statistical noise in the ranking which implies a family which may have destroyed many portfolios may fare higher than a family that has lagged behind the category for strategic reasons but not in any statistically significant manner.
The above is an example of how two reasonable approximations can compound to less validity.
Sextant just happens to be a family that seems to suffer from this kind of computations. SSIFX, for example is a very good conservatively managed fund and if analysis like this results in people like @MJG making recommendations not to consider the family unless one were a fool, such a write up may not be libelous in legal sense but is definitely irresponsible.
Looking at this analysis, one can see a number of problems with use of averages and confusion between correlation and causality especially when multiple parameters are involved.
- Sextant has very low ER on most of its funds, while Robeco has relatively high ERs on their funds. Making inferences from the average over such data and worse associating causality is bad science. Just because one can compute an average doesn't mean comparing two such averages has any statistical meaning.
- A few of the families in the laggards are those that provide leveraged ETFs. The category for this class is not well defined in M* and the average over them has a wide variance with anomalies where a 3X fund can fare better than the 2X or unleveraged depending on the market conditions. Given that these are trading instruments and not long term holdings, results over a long time period has no valid statistical interpretation whether to choose them or not.
- Different existence periods can affect computation. Were the funds that have not been around for a full market cycle in their current form (say RiverPark funds) considered? If so, then families with potentially large number of fair weather funds will rank much higher while families with conservative management will rank lower.
- Has the difference from category been normalized for AUM or magnitude of delta for category?
Etc.
Not implying that every study has to be exhaustive or perfect but the serious problem particularly in economics or finance is making inferences beyond that is valid from the limitations of the study or computation. This is a huge problem.
Such things are fine as entertainment or for pop usage. But if this site wants to be taken seriously in the industry and be responsible about it, it has to be much more diligent in not leaving it open for broad and mindless inferences like the one from @MJG above. I don't think he really understands science or academic endeavors based on what he shows with his understanding of what he comments on rather than what he keeps claiming and painting himself with regarding academics and science. Like those false patriots that keep doing dumb things while wrapping themselves in the flag.
My apologies for ending my participation on this sour note but the hubris is a bit too much to take.
This is nothing personal against David or Charles, but I hope that they will take the criticism for what it is, an attempt to increase knowledge and information by pointing out invalid inferences and perhaps make them think a bit about the implication of their work on a fund family like Sextant and their ethical responsibilities in this matter.
I would also strongly recommend some consultation with a good statiatician to state clearly what would be good and bad inferences that can be drawn from the analytical tools made available here. Otherwise, these tools may do more harm than good.
I have come to know many good people here even if we have not necessarily agreed with each other and I will miss them greatly.
I bid you all adieu.
Regards,
Ted
So Long It's Been Good To Know You: The Weavers:
Happy Trails,
MarkM
Thanks man for the kind words.
You are tough to please. So, very glad you liked this one.
I too shake my head trying to understand how folks can invest with some of names in the bottom tier of the fund family study.
I actually found several funds in these families drew down about 90% (eg., BOGSX, ROGSX, and POGSX have MAXDDs of -92.1, -91.9, and -88.0%).
Ha! Bet that is not something you will see touted on their web pages.
Kind of makes sense for families that charge lower ER and impose no loads or 12b-1 fees to tend to out-perform.
Does this really surprise anyone?
Hope all is well.
@cman.
Bummer. Sorry you did not like the piece. But more sorry if it caused you to move on. Really? Come on man.
I actually attended a high school that did not give grades. The school objected to distilling complex behaviors and attendant performance down to rankings.
A little like you?
The piece specified assumptions up-front along with several disclaimers, like sample size, potential categorization issues, different eval periods, etc. Made point to publish count and age info, so readers could weigh accordingly.
It also pointed out that just because a family ranked top tier, did not mean it's a shoe-in (eg., Marsico). Think your Sextant example does same for bottom tier.
Break, break.
Maybe like Roy Weitz said with the Three Alarm rankings...it's not an automatic call, but a signal to look further into and see why.
Good?
Honestly, I believe the comparative data like those published in this study do not get discussed much. Same with max drawdown metrics.
For example, recently I saw a fund eliminate its investor share class, merging it into its institutional share class, which has a lower ER. At first I thought, one for shareholders, right? But then noticed the investor class was also the oldest share class, which had pretty scary performance early on for its category. The institutional class was younger, and so the scary performance was gone. Just like that. Kind of swept under the rug.
I find exposing instances like these basic to MFO's charter.
Given the assumptions and recognizing the potential short-comings of the disclaimers stated in the piece, I actually find the implications of the fund family study quite satisfactory. Both the over- and under-performers.
Sure, Dodge & Cox has only five funds (now six)...but they all beat. Worth noting? Yes.
Sure, RiverPark is a younger family, but all their funds are off to good start.
I personally have not had much luck at AQR and do not like their rather anti-shareholder communication policy, but seven of its eight funds beat their category averages since inception. (Ha! I must have had the exception.)
The data on Oakmark and Artisan are breathtaking, no? I did not realize until this study. (David I'm sure did, but not me...I'm slow.)
And would anybody really want to invest in the biggest laggards noted unless they understood the under-performance...and potential fit in their own portfolios?
For example, these rankings are based on absolute return. Perhaps the fund family is more concerned with risk-adjusted return. And, if that is case, AoK.
I've always seen MFO as a place to get information about fund investing not commonly available elsewhere. I think this piece is in that spirit.
Listen, if you or anyone else emails or posts on the board any legit mistakes with the piece, I will correct. And, if you have evidence that higher fees, or more prolific loads and 12b-1 fees help fund families over-perform, I will also correct.
Otherwise, I stand-by the implications of this study. (I do indeed have the medians, sigmas, and T-Stats on all published metrics...let me know if you want them.)
I tried to include a lot of info beyond the rankings as well, which I find fascinating and trust other MFO readers do also...expense ratio, loads, AUM, market share, share class, 12b-1 fees, etc.
Certainly appreciate your contributions and feedback on the board, and do hope you will continue to participate.
Thanks for your contribution to the commentary and for your post here (as well as all your other posts). You are a model of thoughtfulness and gentlemanly behaviour.
I too hope cman does not go! His contributions have been extraordinarily helpful, and the debate he encourages, even here, is productive and interesting.
I came to praise Charles, not to criticize him. Apparently, not everyone agrees with my enthusiasm for his study. That’s okay since that’s one of the purposes of the MFO Discussion section. I was surprised by the vituperous emotional outburst directed at me. I intended an innocuous, friendly post.
I merely wanted to acknowledge Charles’ June statistically-based study. I believe it is a nice, honest, and useful analysis. He carefully defined his universe, discussed his database sources, and outlined his method. It is a fair way to measure a fund’s performance relative to an appropriate benchmark. The individual fund evaluations were assembled to establish a fund family overall performance. Ranking breakpoints are clearly stated. Various statistical measures are provided.
The analysis offers a point of departure when exploring fund opportunities. Certainly other fund sorting criteria should be exercised when making a final mutual fund choice. Charles has no axes to grind and allowed the analysis chips to fall where they may.
Apparently some of the chips landed on Cman’s shoulder and they irritated him. I did not contribute whatsoever to Charles’ work, yet I became an embedded part of Cman’s diatribe. I suppose some of my earlier postings raised the hair on the back of his neck. Is not the MFO Discussion section tarnished by unsubstantiated, ad hominem name-calling?
I am forced to reply to these spurious, derogatory characterizations. I must defend to counter this smear and insult campaign.
That’s too bad since those rantings detract from his assessment that challenges Charles’ work. Cman’s invective tirades focused against Charles’ study are misguided. Charles is well aware of the shortcomings of any statistical analysis. So am I.
I wish I could say that Cman’s gripes about me are also misguided, but I can’t. He doesn’t detail my perceived shortfalls that so excite his attack. Please tell me if you can, what specifically troubles you guys about this latest post. I will listen.
The mention of averages seemed to trigger Cman’s emotional outcry. Charles recognizes that there is much more to portfolio construction and unit selection than simple averages; it’s only a single parameter in a multi-dimensional criteria array. However, since averages are the bread-and-butter of any statistical examination, quoting an average is not a bad way to start the evaluation process.
We are all saddled with behavioral biases and my assertion includes Cman. He constantly attacks the logic of those he disagrees with. He distrusts academic studies. He frequently argues that statistics are overused or their interpretation is flawed. These shortcomings do occur, but I still trust academics over fund managers and solid statistical analysis over gut instincts. Anything can be abused, including the toasters in our kitchens.
From my own perspective, I always thought of Cman’s submittals as being highly nuanced. He often introduces outlier outcome possibilities as if they were dominant players for the decision process. He is definitely the two-handed economist that Harry Truman identified as “on the one hand……, but on the other hand…….” advisors.
Cman is an intelligent man with considerable investment knowledge and experience. Overall, I honor and respect his judgments and assessments. But, just like all MFOers, he too has made posting errors. I wish he were more tolerant and more respectful of those having disparate market interpretations. For every buyer there is a seller. At least one of these participants will be price wrong, although their motivation for trading might not be price sensitive.
Unlike Cman, I will continue to post my “mindless drivel”. Thank you for your attention. If you don’t agree with my comments, please let me know why. I will defend my position if I disagree with your comments, or I will acknowledge the merits of your arguments if warranted. That’s fair. I learn daily in the investment world and adjust my thinking accordingly. Investment opinions are fragile, transient judgments that are always subject to the changing dynamics of the marketplace.
Cman came late to the MFO party. If he leaves, that’s his choice and I respect it. Although his submittals tend to be elliptical according to another MFO participant, he is (was) a positive addition to the MFO membership.
Best Wishes.
On cman, I dido expatsp's comment. His contributions have been intelligent and well spelled out. But in this case, who really cares. I think there might be too much importance put on being right all the time. Give your 2 cents and let others decide what to do with the information.
On Junsksers comment on Investor. I also really liked Investor’s straight forward logical comments and I miss him too. But I don't believe he left this forum out of discouragement. He often said he had to break his addiction to the computer and spend more family time. I'm guessing that's what he did.
Well, that's my 2 cents... but who cares?
At the same time, the MFO community ought to be big enough to accommodate not only different investing styles but also various levels of sophistication in the various investment-related disciplines. I think it should be fun to come here and exchange our own thoughts and perceptions - even though imperfect at times. Gets back to what Mike said about "being right all the time."
Others may have different perceptions of what the site should be. Nothing wrong with that. Seems that David has kind of left it open for members to decide (collectively) on how best to use the forum. The inclusion of a separate "off topic" board category suggests to me some tolerance on his part for a wide variety of uses and levels of acumen.
Thanks to Charles for the write-up. Can only imagine the many hours of hard work that went into it. Had trouble downloading it (and other parts of the Commentary) in its entirety on the first day. But, your research findings looked intriguing and will get back to it and give it the attention it deserves. Thanks again. Regards hank.
>> One thing that caught my eye immediately
and his calls elsewhere for careful sample selection and delimiting and caveating and common sense. It is all well and good to say quintile ranking is but a call for investigation, or something, but Weitz's comment was similarly disingenuous and is not the way busy investors use tables. I'm often perplexed by Charles's conclusions.
I was the one who asked cman to be less elliptical, and he became so. I hope someone takes up his causes at least in part.
It certainly was too strong to say that "only a fool would choose a mutual fund from the lower performance ranks". Cman mentioned SSIFX as a good fund. I think it probably is, but that leads to a whole discussion as to what is or is not a "good fund". Statistics are only going to take you so far. I think Charles knows that (of course he knows it in his head, but I think he knows it in his heart, too). If somebody presents an argument that gets misused, it's not the presenter's fault. Let the reader beware, so to speak.
One of the reasons I visit this site is learn what others are doing and how they see the markets and which mutual funds they might be using for investment purposes. Not all of us are going to see these views in the same color but we should all have harmony with each other with respect to their views whether we agree with them or not. Cman expressed himself in a way that did not disrupt harmony.
Let’s not forget why we all post … and, that is to help one another to become a better investor by sharing our views knowing there is more than one pathway to climb to the top of the investment mountain. Again, from my perspective, Cman did this. It is for sure he did not share my views on portfolio management and certain strategies that I posted that I have employed in the past to take advantage of varying market currents. Hey ... They worked for me and I was willing to share them. And, I still plan to.
Just know this … Old_Skeet plans to keep on keeping-on.
Anyway, Cman … I am saddened to learn you are leaving and I sincerely wish you the very best. Get some tough skin and venture on back. There is indeed a place for you at MFO ... and, the porch light will be left on.
Old_Skeet
Vert,
To what point? I might bet you do not believe this yourself about, say, vaccination, winning athletes, or climate change. Also, what's the alternative --- faith-based investing?
I'm no biologist but my understanding is that vaccinations inject weakened versions of viruses into a body with the intent of stimulating the production of anti-bodies which can destroy the weakened virus. In future, should that virus attack the body, these now existent anti-bodies will be able to destroy them before they gain so much strength that they become more powerful than the anti-bodies produced to defend the host. Conceptually the process seems pretty well understood. Whether it works or not in any specific case will be indicated by statistical studies, i.e. empirically, of course. Some skepticism will be warranted as regards whether these statistical studies have been done properly. Alternative explanations of the results are unknown to me, but if they are offered they should be considered as well.
I don't know exactly what "winning athlete" studies refer to. My guess is that a lot of the inferences made from them would deserve to be treated very skeptically.
As for climate science, I'm still no scientist but my understanding, such as it is, suggests that the underlying processes are not nearly so well understood as they are with vaccinations.
I should emphasize that in all cases the inferences made should be treated at least as skeptically as the studies themselves.
What's the alternative to treating statistical arguments skeptically? Well, treating them non-skeptically, that is, accepting any statistical argument that you come across with no further thought.
Your alternative of 'nonskeptically' is a false dichotomy. Sorry to have reacted and posted as I did to your overstatement; it was not important. But many of us are plenty skeptical; what you wrote simply seemed to me to be vastly overstated wackiness. I treat all sorts of statistical arguments without any skepticism whatsoever, solid arguments, and was suggesting that, like everyone, you do too. Does not mean I'm not a skeptical person, not at all, and I am not readily misled. My point was about judgment.
Regards,
Ted
@MJG -- you had engineering training?