If anyone should be writing about this subject it is my co-founder Vladimir Krouglov, as he worked formally on this very problem before joining PsyQuation. Perhaps he can share with us how he approached what I am about to share with you.
The above image highlights a problem that salesman have had for more than a couple of centuries. Its not just salesman affected but a host of other professions. In mathematics and computer science this is known as a “constrained optimization” problem. It is where you are trying to find the single best solution to a set of variables given rules and a score keeping measure.
This problem only found its way into the mathematics community in the 1930’s with Karl Menger. Hassler Whitney posed the problem in a 1934 talk at Princeton where mathematician Merrill Flood became its latest champion. Mathematician Julia Robinson is the person who in 1949 brought the problem known as the Travelling Salesman Problem into print for the first time.
For decades people have worked on this problem and they have all failed to make any breakthrough. In the mid 1960’s Cobham-Edmonds came up with a definition whether a problem is solvable or not. In short if it is solvable it is called “tractable” if it isn’t it is called “intractable”. Coming back to our salesman problem it seems that this is one of those intractable problems.
So what the hell is Berman carrying on about so late in the year with all these math problems. Don’t worry I feel your pain, so let me share why I think this can potentially help you as a trader.
Let’s start by recapping what every systematic trader is trying to do, that is you are all working on a strategy that makes money by cracking the market problem. If you think the travelling salesman problem is complex then you can bet this one is pretty darn intractable too.
Do not despair when Michael is near. Computer scientists have an approach to problem solving that may not give the ultimate solution but by simply relaxing some of the constraints of the problem you can get to a solution that is often pretty close to the real solution (whatever that may be, remember we don’t know it because its intractable but we can still quantify how close we are ((more or less)) to the real answer).
So the message I have for you model developers who are backtesting the living daylights out of your over-fitted models. Accept that you are not going to come up with a perfect solution and relax some of your constraints so that you have a model that is robust and usable.
You will hear from me one more time this year. Have a great festive week for all those celebrating.