Data science has created a new capacity for powerful analysis by traders that, so far, few have taken advantage of. While quants have been doing data science for awhile now, traders have been largely relying on those quants to execute and have not widely made the shift to developing data science skills themselves. But why?
In the extremely competitive world of finance, it is the traders that are closest to the market. And in fact, compared to quants who are often working from a much more theoretical point of view, traders are a much better candidate for incorporating data science into their work.
Unfortunately, for the most part, they haven’t yet. This is a shame because the advantage of diving in to data science is a much better understanding of underlying models. Again, in such a competitive industry, one cannot thrive simply by blindly relying on information (models) handed over from a quant. It’s time for traders to get into the data themselves.
Before making the leap into my current position as lead data scientist at Dataiku, I was a trader at Schneider in the UK. Now more than ever, traders needed to find new competitive edges in an industry that has become entirely dominated by technology — machine learning (ML) and, ultimately, artificial intelligence (AI) have a huge opportunity to provide that edge.
While this may seem scary to the unprepared trader, the good news is that it is never too late to begin the transition towards more data science-oriented strategies. And frankly, the general aversion toward ML and AI isn’t unique to traders — employees across industries who don’t fully understand how they work (and how they can work together with these technologies) often aren’t willing to take the plunge.
What most people (traders or not) think of when they hear "AI" (which is quite different than Enterprise AI!)
But by understanding the ways in which data science in trading analytics and risk management are transforming the industry, and taking a few simple steps to shift your mindset, anyone can begin to leverage tools like ML and algorithmic trading.
Why ML in Trading?
ML is able to use real-time data from unstructured and structured sources to find underlying patterns and trends that might otherwise have been hidden. While high-frequency, algorithmically-determined trading has always been around, traders were often limited by what they could do in Excel.
Data science offers a way for traders to incorporate new and meaningful sources of data at scale and in real time. And the automation of these processes allow for even further improvements in reusability and productivity.
Data science offers a way for traders to incorporate new and meaningful sources of data at scale and in real time.
Changes in the market are notoriously hard to source because they can come from anywhere — data science techniques alleviates this by allowing traders to combine data from a huge variety of sources and examining massive amounts of data on past market activity. Aside from the combining of data from various sources, one of the largest benefits of data science in trading practices is reproducibility. Quickly and reliably, information can be distilled that would be impossible for even the most diligent of traders to find before.
All of this brings up a very important question, which are the regulatory and security aspects. If using the right tools, combined with good data governance practices, data science should be a very white-box process. All activity on a data project is logged, and it’s clear what data sources are being used for which data projects, how the data is being manipulated, etc. Therefore, the use of data science should be very manageable when it comes to things like audits or regulations.
Making the Transition to a Data Scientist
The typical profile of the data scientists is often well aligned with the profile of a trader — and this is likely why the two fit together so well.
Some skills that are generally common to both industries include:
- An understanding of how to manage risk;
- An ability to recognize patterns in large datasets;
- A knowledge of how to build relationships with stakeholders; and
- An understanding of financial math, including asset pricing theory, probability and statistics.
Some nice-to-have skills to work on developing for data science that will aid the transition:
- A coding language will be a plus
- Data science tools that will provide some structure
Learning a coding language (like R or Python) will make the transition to data science even easier (+ more powerful)
Data science is an extremely dynamic field, so the best policy for any data science oriented trader is simply to stay up-to-date with the cutting edge of the field.
Clearly developing skills in data science will have long-lasting impacts on the career of every trader. While automated trading has almost reached the point of ubiquity, ML and AI have the potential to completely change what it means to be a trader.