Trading Systems Research & Development
A look at optimization
To the new systems developer one of the most exciting things to play with is optimization. Optimization is using the power of the computer to examine every possible sequence of parameters and rules to find those that have worked out best in the past. With enough computer crunching power its possible to find systems that perfectly “predicted” the past. We can run number crunching PC's on automated routines and have them analyze billions of bits of data while we are sleeping! Many traders do this long enough and eventually "discoverer" the holy grail of trading systems. They jump into the markets with their new super predictive algorithms only to find they fall apart in real trading!
“What happened?” they ask themselves. The answer is that what they created was likely a system that was a statistical coincidence (known as a "curve fit"). Curve fitting is where a system has been optimized to a unique set of historical data. The problem is that the markets will behave much differently in the future than the past, therefore, a “perfect” trading system in the past could be useless in the future. For example, your computer finds the perfect dates in the past to have bought and then sold the market. Obviously this data does not mean anything in the future. . This is a simple example but most curve fits are some complex form of this basic concept.
Lets look at another flawed example. Assume we wanted to optimize a set of nickels that were most likely to land on heads. What we could do is flip a million nickels and only select those that landed on heads. Then, we can take those remaining nickels and flip them again, once again only choosing those that land on heads. We could repeat this process over and over again each time only choosing those nickels that land on heads. At this point we might conclude that we had narrowed down our nickels to only a small handful that were “optimized” to land on heads. We could then go out and make large bets with those nickels putting all our money on heads. We would quickly make a fortune right? WRONG!
Unfortunately we would very quickly lose our money. These nickels were not optimized for heads; they always did and always will have 50/50 odds. What might have confused some is that they thought they had found a predictable set of nickels when in fact they had just found a statistical coincidence!
Because there is so much data and so much computing power available, these kinds of errors find there way into trading systems all of the time. One of the worst offenders of such flawed optimized systems can be neural networks. When developing a system its imperative that optimizing is avoided as much as possible. You need to find NON curve-fit robust systems. There can be a place for certain types of optimizing, but it must be handled correctly.
All of our systems are designed in a robust manner that we feel avoids the optimization pitfalls of so many other systems.
Feel free to email or contact us with any questions or comments on this subject. dhoffman@traderstech.net
© Copyright 2001 - 2003 STS Inc. All Rights Reserved
Risk Disclosures
COMMODITY TRADING involves high risks and you can lose a significant amount of money. Commodity trading is not suitable for many investors. Any performance results listed in all marketing materials represents simulated computer results over past historical data, and not the results of an actual account. All opinions expressed anywhere on this website are only opinions of the author. The information contained here was gathered from sources deemed reliable, however, no claim is made as to its accuracy or content. Different testing platforms can produce slightly different results. Our systems are only recommended for well capitalized and experienced futures traders.
CFTC REQUIRED RISK DISCLOSURE
HYPOTHETICAL PERFORMANCE RESULTS HAVE MANY INHERENT LIMITATIONS, SOME OF WHICH ARE DESCRIBED BELOW. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN. IN FACT, THERE ARE FREQUENTLY SHARP DIFFERENCES BETWEEN HYPOTHETICAL PERFORMANCE RESULTS AND THE ACTUAL RESULTS SUBSEQUENTLY ACHIEVED BY ANY PARTICULAR TRADING PROGRAM.
ONE OF THE LIMITATIONS OF HYPOTHETICAL PERFORMANCE RESULTS IS THAT THEY ARE GENERALLY PREPARED WITH THE BENEFIT OF HINDSIGHT. IN ADDITION, HYPOTHETICAL TRADING DOES NOT INVOLVE FINANCIAL RISK, AND NO HYPOTHETICAL TRADING RECORD CAN COMPLETELY ACCOUNT FOR THE IMPACT OF FINANCIAL RISK IN ACTUAL TRADING. FOR EXAMPLE, THE ABILITY TO WITHSTAND LOSSES OR TO ADHERE TO A PARTICULAR TRADING PROGRAM IN SPITE OF TRADING LOSSES ARE MATERIAL POINTS WHICH CAN ALSO ADVERSELY AFFECT ACTUAL TRADING RESULTS. THERE ARE NUMEROUS OTHER FACTORS RELATED TO THE MARKETS IN GENERAL OR TO THE IMPLEMENTATION OF ANY SPECIFIC TRADING PROGRAM WHICH CANNOT BE FULLY ACCOUNTED FOR IN THE PREPARATION OF HYPOTHETICAL PERFORMANCE RESULTS AND ALL OF WHICH CAN ADVERSELY AFFECT ACTUAL TRADING RESULTS.