Hi Guys,
The December 14 issue of the Wall Street Journal (WSJ) has a review by Burton Malkiel on a book titled “Models Behaving Badly” by Emanuel Derman. Malkiel liked the book.
Mr. Derman was trained as a physicist and attempted to deploy a physicist’s analytical modeling methods and experimental techniques to the financial industry while working the halls of Goldman-Sachs. Overall he suffered a humbling experience.
Malkiel quotes from Derman’s book: “In physics you’re playing against God and he doesn’t change His laws very often. In finance, you’re playing against God’s creatures.” And God’s creatures use “their ephemeral opinions” to value assets.
In the complex and chaotic marketplace, human behavioral instincts destroy carefully crafted statistical models.
The review referenced an earlier article written by Derman and Paul Wilmott with the intriguing title of “The Financial Modeler’s Manifesto”. I have extracted The Modelers' Hippocratic Oath closing section from that article and have provided it below for your scrutiny.
The Modelers' Hippocratic Oath
~ I will remember that I didn't make the world, and it doesn't satisfy my equations.
~ Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
~ I will never sacrifice reality for elegance without explaining why I have done so.
~ Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
~ I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
That’s it. Here is a Link to the entire Manifesto:
http://www.wilmott.com/blogs/eman/index.cfm/2009/1/8/The-Financial-Modelers-ManifestoAll investors would benefit if market wizards, gurus, and modelers endorsed the oath. Good luck on that New Year’s wish. They do not.
Derman and Wilmott published their Manifesto in January, 2009. It is a fair and quick 2-page read. Enjoy it. Nothing much has changed with regard to the financial behavior of the market’s elite wizard cadre since its publication date. I guess not many of the financial clan signed the Oath. Too bad.
Best Regards.
Comments
Thank you for the note and link.
These words, "In the complex and chaotic marketplace, human behavioral instincts destroy carefully crafted statistical models."; brings to mind the "chicken and the egg". Was it the behavior that flawed the models; or were the models flawed from the behavior to begin, and never fully accurate. I will suppose pure math models to themselves, are fine; but not necessarily when placed into investment models.
Regards,
Catch
Hi Catch,
Thanks for your note.
You focus on my comment that “In the complex and chaotic marketplace, human behavioral instincts destroy carefully crafted statistical models.” Allow me to expand on this thought.
All folks mirror and apply their training and life experiences. That’s exactly what Emanuel Derman did when he went to work for Goldman-Sachs. He was a physicist by training and previous employment. So he applied a physicists perspective and skill set when confronted by investment issues and decisions.
Derman used his learned scientific philosophy and methodology when addressing the financial world. But that world is populated by agents (investors) who do not react consistently to a given set of stimulants.
That’s what Burton Malkiel meant when he said that “In physics you’re playing against God and he doesn’t change His laws very often. In finance, you’re playing against God’s creatures. And God’s creatures use their ephemeral opinions to value assets.”
In the Hard scientific disciplines reactions to stimulants are reliably reproducible and therefore predictable; in the Soft disciplines that are most often governed by undisciplined human behavior, reactions are not reliably repeatable. The modeling problem is a horse of a different color with an added human dimension.
Of course, another separate issue is that of modeling fidelity. In the real world, all models are simplifications of complex phenomenon and/or, in the investment business, human interactions. Even when representing physical or chemical interactions, the models are simplifications. Often they reflect complex events in a linear manner just so that the problem can be mathematically tractable. The hope is that the simplifications, while eliminating some lesser factors, still retain the dominant influences so that the primary interactions are correctly captured.
The human factor adds messy, unpredictable complexity; and evolving economies and changing marketplace interpretations destroy any overarching model even if it perfectly fits the historical data base. The marketplace is not an invariant physical world ruled by Gods laws and rules. Regardless of its sophistication, analytical market models are doomed to fail because of market dynamics and human behavioral shortcomings.
Models are always imperfect approximations of the real world. So they must be challenged by real world data for verification. The first test is to correlate the proposed model against the available historical data sets; the ultimate and acid test is to deploy the model to reliably project future performance with high precision. In the financial universe, no entity has successfully accomplished this Hercules-like task.
If you have a scientific or an engineering background you already know all this stuff; it is a lesson learned in your earliest professional training sessions. I apologize if that is the case.
Some modeling, even if it is incomplete and perhaps even superficial, is constructive since it provides some guidance. But that guidance must be tempered with a healthy doze of skepticism that recognizes the limitations of the analytical models, and with a plentiful serving of common sense that is honed by extensive investing experience.
Malcolm Gladwell has publicized the 10,000-hour rule; that rule stipulates that 10,000 hours of dedicated study and practice are needed to establish expertise in any demanding occupation. I suspect investment acumen falls into that general rubric.
Thanks for your thoughtful contributions. And I want to particularly thank you for tolerating my ranting on this subject, especially since you are likely very familiar with every observation and claim I just made.
Best Wishes for a Merry Christmas and/or a Happy Holidays.
"Two Colorado State University climatologists, who have independently been tracking and predicting the severity of hurricanes for nearly 30 years, are abandoning their long-range forecasting efforts.
Despite advances in supercomputers, hurricane-hunter aircraft, satellites, weather buoys and more, long-term prediction simply isn't feasible today. But short-term forecasts get better every year.
"We have suspended issuing quantitative forecasts at this extended-range lead time, since they have not proved skillful over the last 20 years," Philip J. Klotzbach and William M. Gray wrote in their annual December report intended to predict the severity of the upcoming year's Atlantic hurricane season.
"From a computation perspective, there's enough chaos in weather that [we] don't know if you'll ever be able to predict it months in advance."
MJG and Catch:
In investing, we are often seeking superior "manager skill" and its potential for out-performance . . . except for those who suggest that there is no possibility of ever creating a model that might outperform the ultimate wisdom of the total market.
Bad models can mislead us, as you point out well, Catch.
And in spite of the evidence that all models are eventually replaced by better ones, you've really gotten to the heart of why they are still so useful, MJG. (If Einstein were alive, I'm guessing that he'd be happy that the gnomes living underground at CERN in Geneva are gathering evidence to refute part of his grand theory.) All the more reason for us to be humble and doubtful about investment trends and their gurus, as your Modeler's Oath suggests.
Thanks for this dialogue.
You noted...."In investing, we are often seeking superior "manager skill" and its potential for out-performance . . . except for those who suggest that there is no possibility of ever creating a model that might outperform the ultimate wisdom of the total market."
Sadly, I do not get enough time to pursue more time with the charts; as I do feel their are valid parts of charting, and especially details yet unknown to me.
As to superior managers/fund types. They do exist of course; and we know and have seen some of the cylces within such funds that sometimes cause us to scratch our heads.
For this house, our current "model" is to have more than one fund of a given sector, knowing that some funds are going to miss the target for whatever reasons.
Even to what one may view as a list of TIPS bond funds that one could presume would not vary in performance much from one another over a 12 month span; may have a significant variance. These funds may hold short, medium or longer duration bonds and also may have other special tools to be used with the bounds of the prospectus.
Our method could be considered a blend of funds within a similar sector; knowing they may all perform similarly over 12 months, but also knowing 1 in 5 may have a problem; as well as 1 of the 5 may have superior performance; with our end performance result being a blended, but acceptable return on the investment in the sector.
I consider such a method to be entirely reasonable within a sector type, be it equity, bonds or other. If one has high regard for the forward performance of a sector; do the homework and buy 5 funds with the monies vs the money only in one fund; if such choices are available.
Ok, just a few late night thoughts. The 6am awaking bell will arrive too soon...away I am.
Take care,
Catch
Thanks for your note referencing weather forecasting and Chaos Theory.
I surely am NOT an expert on Chaos Theory and do not understand much of the work and findings in that difficult arena. But I do appreciate its goal to unravel the mysteries of nonlinear, complex systems. Our financial marketplace is a complex system that has both periods of reasonable stability and other periods of seemingly unstable chaos. Progress has been disappointingly slow in application of chaos findings to an understanding of financial markets.
From your post, it's interesting to learn that a couple of university climatologists have recently abandoned their quest to model long-term Atlantic hurricane forecasting. One of the pioneers in Chaos Theory development is Edward Lorenz, a research meteorologist. In the 1960s using highly simplified analytical models on his computer he accidently discovered that the final predictions were extremely sensitive to minor changes in the initial input starting conditions. This discovery promoted additional research in the chaos discipline.
It also led Lorenz to the conclusion that long range weather forecasting was a futile effort. With the addition of countless ground sensors and orbital scans, short term weather predictions have become much more accurate, but long term forecasting is still beyond the scope of current science. I guess Klotzbach and Gray are either slow learners or perhaps dedicated to advancing their professional specialty.
Best Wishes.
Hi Catch,
In your response to Ginko, I noticed that you are studying charting techniques.
Please be cautious when exploring this attractive, but often dangerous dragon. It is a commonly used tool that has many loyal believers, but few verifiable success stories to vindicate that loyalty.
Anecdotally, I speak from considerable experience. I lost a decade of investment opportunity centered in the 1960s using old school pencil and graph paper charting methods exclusively when making equity buy/sell decisions. My instruction manual for those decisions was the classic Edwards and Magee tome “Technical Analysis of Stock Trends”. I still retain a well-worn copy and infrequently refer to it.
I used Edwards and Magee to identify my entry and exit points without regard to fundamental analysis. I experienced some successes, but many failures. I observed that after many trades, the stocks I purchased delivered poorer returns then those that I sold. At best, results were a neutral mixed bag when buy-and-hold would have delivered superior performance. I abandoned my commitment to charting as a tool for decision making in the 1970s.
I must admit that given today’s computer power and multicolor graphing capability, charting has snappy sex appeal. The issue is, does it deliver excess returns? I personally doubt it, but you surely are free to explore that unlikely probability.
Why do I say “unlikely probability”? I have never seen a verifiable study that demonstrates its robustness.
I suppose we should start with a definition of a person who basis his investment decisions on chart patterns, a chartist. From Frank Reilly’s textbook “Investments”, a chartist is “a technical analyst who uses graphic presentations of stock prices to predict future stock prices.”
Many factors come to mind that operate to defeat charting as a reliable market timing tool. Markets are dynamic; so events that triggered one reaction are not likely to repeat the same reaction when it is next exposed. Most market patterns are an illusion; similar patterns are easily generated using random coin flipping in simulations that expert technicians can not identify as randomly produced computer outputs. Human behavioral responses to a given stimulant are unpredictable; both institutional and individual behavioral responses are coupled to economic factors and emotional status. Graphs summarize history in an organized way and do not predict future market moving events.
As Charles Ellis observed “The evidence on investment managers' success with market timing is impressive – and overwhelmingly negative”.
In a more metaphoric manner, Leonardo de Vinci noted that “He who wishes to be rich in a day will be hanged in a year”.
I guess what I’m saying is that the marketplace rewards patience and prudence. Charting has merit as a visual tool to summarize data for quick review and assessment. It will not likely generate excessive profits as a predictive device. Charting is like fool’s gold that glitters until exposed to the tarnishing effects of the real world.
I wasted a decade chasing the heady promise of charting without success. I recommend you cut your wasted time to a minimum by limiting your commitment to the false lure of charting as a guide to technical decision-making.
Please don’t base your final decision on this matter on my inputs alone. Take time to explore and understand any shortcomings that charting might offer. Behavioral science studies demonstrate that we often only consider facts that reinforce our decisions, and exclude facts that are counter to our viewpoint. Don’t fall victim to that confirmation bias.
Good Luck.