rectangleThe world's leading A.I. powered trading network
PsyQuation
PlatformPremiumCompanyContact Us

PsyQuation Score™ Overview

Leave a comment

Introduction

The PsyQuation definition of a trader is someone who buys and sells financial instruments for speculative gain and is typically someone who trades at least once a week. The financial instruments we cover in our definition are: FX derivatives, Stocks, CFD’s, Futures and Options. A 2015 research report estimates the size of the global trading community at more than 80 million active traders with a large part of the recent growth coming from advancements in technology (especially mobile), easy access to leverage and the abundance of provocative advertising selling the dream of financial freedom.

However, the reality is not all traders can be successful and its therefore incumbent on the trader to ask themselves a fundamental question, “am I a good trader, or said more formally do I have skill?” This might seem like a simple question, after all isn’t it easy to establish this fact, all you need to do is look at your trading account and see if there are any profits?

It turns out answering this question is a lot more complicated. For a performance metric to be accepted as a good measure of a trader’s skill it needs to provide insights to future profitability, that is before the fact. The industry is littered with countless “fortune tellers” claiming a trader to be a star; however, these proclamations are often laced with a fair dose of hindsight bias, that is proclaimed after the fact.

The Nobel Memorial Prize for Economic Sciences was awarded to Professor William Sharpe in 1990 for his contribution to measuring risk-adjusted performance. The Sharpe Ratio as it has become known is still regarded today as the gold standard for measuring risk-adjusted performance. However, as with most generalised mathematical solutions, the information derived lacks sufficient detail to accurately classify a trader on the battlefield in real time with skill.

For the past 5yrs the PsyQuation founders have dedicated their careers to solving how to quantify and identify trading talent. The first productive steps on this journey took place in 2012 with the creation of a scoring algorithm, that enjoyed a strong following until its sale, using daily time series of returns to score and rank traders. The next important step was developing an improvement on this score by including all the trade data to more accurately assess a trader’s given level of skill. We acknowledge that no matter how good our efforts may be this will always be a work in progress, and that there are no definite answers to the question of “does a trader have skill?” Rather success for PsyQuation is determined along a probability spectrum with an acceptable level of confidence. What this means is that our Score’s out of sample back-testing results need to outperform all other known performance metrics.

Join us on a journey as we explore the PsyQuation Score and describe why we believe it is the best-known measure of trader skill on the market, and how knowing your level of skill as a trader will help you make more profits.

Philosophical Approach to the PsyQuation Score™ Construction

Every model needs an overriding philosophy and the PsyQuation Score™ is no different. If trading FX, Futures and Options is a zero-sum game then on average nobody wins trading. However, we know from personal experience and from the media in general that there are traders who do repeat their success for lengthy periods of time.

If a trader consistently harvests profits from their trading in a market that is structurally designed to return, on average, zero $; there must be an X factor present, we call that factor skill.

This is our starting point, because without skill there can be no confidence in one’s ability to forecast future success. Skill serves as the foundational bedrock of the PsyQuation Score™ providing valuable information to the trader and allocator. However, skill on its own is not sufficient to inform you of future trading success. For example, a highly skilled greedy trader who pushes his strategy too aggressively to maximize profits with risky position sizing will probably not be that successful.

We therefore add two factors to the score to take these realities into consideration. We include a risk component that is used to adjust a trader’s score by the appropriateness of the risk being applied and we do the same with behaviour, measuring the degree of bad behaviour identified in the trader’s log of trades.

At this point the score is starting to take shape and provide a firm foundation for scoring a trader’s probability of future success. However, we need a factor that glues the 3 factors: skill, risk and behaviour together and provide the score a mechanism for removing luck out of the equation. The fourth and final factor in our score is history.

As we dive deeper into the 4 factors (skill, risk, behaviour, history) comprising the PsyQuation Score™ we hope you will start to realise that the only way to provide a useful performance metric to the trading community is by taking a multi-factor approach. There is more!

To provide deeper knowledge one needs to look at ways of increasing the information from which signals can be mined. To do so requires expertise in extracting information from noise. PsyQuation processes its score from data in 15 min time slices using proprietary data-mining techniques to harvest the higher moments captured in more granular time slices.

PsyQuation is a platform for identifying and developing trading talent. We believe having deeper insights about the level of skill and a trader’s score relative to the PsyQuation ecosystem of more than 17,500 traders provides valuable information for a trader to develop and achieve the goal all traders strive for, that is to make more profits.

Skill Coefficient

We define skill as the ability to consistently produce positive PnL (after transaction costs), to strip out the effect of different $ size accounts we consider the distribution of profit and loss relative to $1 invested. We define this as “trade return” and measure it in the base currency of the account, from this point for consistency and comparison purposes we convert the base currency to US dollars. This distribution spans all the trades in a trading account including the mark-to-market PnL of open positions.

We then estimate the probability that this empirical distribution has a positive mean value. This can be described more formally as the mean of a probability distribution as defined as a simple average, 1/N (sum of all trade returns). However, this is simply a point estimate and we need to satisfy the confidence interval for the prob(mean>0) only if this is high enough do we say the mean is positive.

The problem with the distribution that we are working with is it typically contain outliers, trades of abnormal size, behavioral biases and other anomalies that make it difficult to establish clearly if there is “skill”. This is where PsyQuation applies some “secret sauce” to extract valuable information from the noise.

Risk Component

The definition of the risk component of the score follows closely the formulas for capital requirements that are used in Basel III and the Dodd-Frank regulatory environments. Recall, that VaR (value-at-risk) is a statistical risk measure which measures a 5% probable loss in a day. As a rule of thumb VaR is a minimal loss that happens once every 20 days on average. In mathematical terms VaR is a value of a 5% quantile of return distribution. There are two main definitions of VaR used in the literature: VaR of portfolio and historical VaR. The difference is the return distribution for computing the historical case is the distribution of the daily returns of the actual account, while in the portfolio case it is the daily return distribution of the current portfolio (open positions) in the account.

Portfolio VaR is far more computationally difficult to compute but it is a much better reflection of the risk during the day under consideration. It is particularly useful for detection of Martingale and Dollar Cost Averaging (DCA) trading strategies, option writers and other fancy order management strategies that are characterized by smooth up-trending equity curves followed by bankruptcy.

For the computation of the score we compute two VaR’s: current VaR of the portfolio and the value of the average portfolio VaR over a certain period of time. Then, the minimum of these two values is taken (minimal value = largest loss). This is done to make sure that if something unexpected happens to the risk of the currently open portfolio it will be instantly reflected in the risk score.

Behavioural Component

This component of the score looks at the frequency of bad alerts and compare this frequency with the typical frequency of bad alerts in our database and calibrates the score accordingly.

Historic Component

The PsyQuation score based on only the three components mentioned above will not distinguish between well established traders who have been trading for a lengthy period and relatively new traders whose strategy produces many trades. To compensate for this drawback we add a fourth component to the score to which we call – history. While we know that past returns are no guarantee of future performance, knowing that a trader has survived for a certain amount of time and in different market conditions as well as the way the trader made profits relative to risk is quantitatively and qualitatively important. History plays an even more important role in the FX markers where the mean lifespan of an account is around 6 months.

To compute the historical component we use an Omega ratio. Omega ratio is a risk-adjusted performance measure that was introduced by Keating & Shadwick in 2002. It is defined as follows:

where E means expected value and r are daily returns. This formula measures the ratio between the typical positive return over the typical negative return.

Based on our analysis the Omega ratio is superior to other risk-adjusted performance measures in out of sample performance. In particular, future performance is positively correlated with Omega.

We use the Omega ratio calibrated on our database of 17,500 plus live accounts to produce the historical component of the score.

Composition of Components

We do not disclose the weightings of the different components of the score. We use a combination of empirical research and qualitative experience.

Data & Testing Methodology

To test the PsyQuation Score™ we have split the data into three subsets: train, validation and test. Random shuffle was used for this split which destroyed potential temporal or cross broker correlations. Each of the subsets has an input component: part of the account’s history that is used to compute the value of the score and output component which is another part of account’s history that is used to compute the out of sample performance of this account.

For example, assume that train set contains the set of accounts {A, B, C} and the test set contains accounts {D, E, F}. Let’s assume for instance that account D from the test set has a timespan from “01-01-2014” until “01-01-2016”. The input for account D would consist for its history from “01-01-2014” until “01-01-2015” and the output for account D would consist of its history from “01-01-2015” until “01-01-2016”. The model is trained on the set of accounts {A, B, C}. As a result we get a function “score”. We compute the value of this “score” function on the input part of each of the accounts {D, E, F} getting {score(D), score(E), score(F)}.

All accounts from the test set were sorted in the sequence of their increasing scores and we used the value of the out of sample “trade return” (defined above) as the fitness measure. The test set was split into “Skilled” and “Unskilled” groups and the difference between the means of their trade returns was used as a measure of how good the scoring model is.

Results

The first interesting part of the results is how we clearly see the peak for mean trade returns for “skilled” traders (green) shifted to the right of the zero axis and more to the right of the red “unskilled” grouped traders. From this you can see quite clearly how our model for defining “skill” is effective at predicting out of sample trade returns, i.e. before the fact.

However, this is only part of the story as we want to see the full PsyQuation Score™ (all 4 components) outperform all the traditional risk-adjusted performance metrics in a back-test environment. Let’s get to it and describe the back test methodology for producing our robust testing results and see how the PsyQuation performs – I promise you won’t be disappointed.

First we create a filter for an account to be included in our back test. The account needs to have (1) more than U$10,000 equity, (2) more than 6 months track record and (3) to have traded at least once in the last month. We now have our qualifying subset. From here we do the following to create our different series for measurement, we choose the top 25% based on their PsyQuation Score™, Sharpe Ratio, Omega Ratio and then we create 5 random portfolio’s from the qualifying subset. We then apply a risk parity model each month and make allocations based on a desired level of volatility, if an account loses more than 50% of its account equity the account is closed immediately. This process is stepped through time and produces our back test results. The BLUE line is the PsyQuation Score top 25%.

From these results you can see clearly how the PsyQuation Score™ produces results that outperform on an absolute basis with the highest Annualised Performance but it also produces a much higher risk adjusted performance as measured by the Sharpe Ratio.

Conclusion

The Psyquation Score™ is a performance filter that is able to to predict whether a trader is likely to produce future risk adjusted performance better than the well known industry standard performance metrics such as Sharpe Ratio and Omega Ratio. This is not a holy grail, past performance is no guarantee of future success. PsyQuation works in a probabilistic world and is best able to maintain outperformance by working with sample sizes of skill that allows the PsyQuation tools to do their magic. This has been a very technical post so we will leave it here and encourage you to fire as many questions as you like. We also encourage feedback so please ask us why your particular score is what it is if you have a strong feeling that it is not representing what you believe to be “your truth”.

Finally there is no one size fits all model solution. What we have attempted to do is design a model that fits as many sizes as possible. This is PsyQuation Score™ V2 – there will no doubt be many more versions in the months and years to come as we become better at our craft. While the whole team has contributed to this effort, the person responsible for this project is my co-founder Vladimir Krouglov we would like to use this forum to publicly applaud him for this mammoth effort that has caused him sleepless nights for the last 6 months, the results have been well worth it.

Mindfulness Why a Trader Should Use PsyQuation