Insights
Solutions
About Us
Careers
Investor Relations
Artificial Intelligence Sweeps Hedge Funds
March 2019
By Peter Salvage
Artificial Intelligence, or “AI”, has featured heavily in industry innovation headlines for some time. Yet for all the excitement and promise, the uptake in the hedge fund industry has been limited – until recently.
Hedge funds’ use of AI is accelerating and reshaping the industry, particularly in investing, cost models and recruitment. Managers also face challenges to explain new AI-based approaches to investors. Given the strategies are the byproduct of super computers crunching billions of data points and learning how to adjust to markets in real-time, explaining how returns are generated is pushing the boundaries of human comprehension.
Why Now?
In September 2018, BarclayHedge's Hedge Fund Sentiment Survey found that over half of hedge fund respondents (56%) used AI to inform investment decisions – nearly triple the 20% reported a year earlier. Around two-thirds of those using AI were doing so to generate trading ideas and optimize portfolios. Over a quarter were using it to automate trade execution, according to the survey.
The early results are promising. For example, the Eurekahedge AI Hedge Fund Index1 slightly outperformed the flagship Eurekahedge Hedge Fund Index in both 2017 and 2018. Moreover, the Eurekahedge Hedge Fund Index decreased by 4% in the fourth quarter of 2018, while the Eurekahedge AI Hedge Fund Index was flat for the period.
Several technical advances have driven AI adoption. New, vast ‘big data’ sets are now available from satellite imagery, the internet of things, global capital flows, point of sale systems, and social media. More data can now be generated in one day than during the entire 1990s. A large hedge fund heavily utilizing AI is likely to have dedicated experts devoted to evaluating and procuring new sets of data. With raw computing power continuing to advance, graphics processing units (GPUs) and customized hardware now solve problems in hours instead of weeks – a necessity given the ongoing rapid growth in data. Finally, with cloud computing now widespread and deployment costs falling, barriers to entry for machine learning are tumbling.
How Hedge Funds Use AI
A number of hedge funds are using AI to analyze masses of data, predict corrections in supply and demand imbalances, and forecast market movements for tactical asset allocation. This has the potential to assist a CIO’s team to combine different strategies and tailor allocations.
Use of AI is playing out across a wide spectrum of investment managers from pure AI-driven specialists, to large quant-driven shops, to traditional fundamental investors looking for an edge. A growing number of firms across the spectrum are also turning to AI to improve efficiency in their operations, accounting and investor relations functions.
Indeed, a class of AI pure play hedge funds has emerged in recent years that are based entirely on machine learning and AI algorithms. Examples include Aidiyia Holdings, Cerebellum Capital, Taaffeite Capital Management and Numerai. Numerai, a recognized AI hedge fund, is pushing the boundaries of the hedge fund business model. The firm uncovers investment strategies by hosting competitions among external AI experts, mathematicians and data scientists. Recently, Numerai expanded its business model by making elements of its platform available to the rest of financial community with its product Erasure, which is a decentralized prediction marketplace using blockchain technology.
Dwarfing the upstart AI pure plays are the large quant funds that are household names in the hedge fund industry such as Man AHL, Two Sigma, Citadel, Bridgewater and D.E. Shaw. For years, players like these have used computer-driven models to uncover new trading strategies and identify themes, factors and trading signals. Human “quants” will then feed these factors and signals into trading systems. With markets continually changing and shifting, these pre-AI models often need frequent monitoring and reprogramming by the quants. AI models are different because while initially crafted by humans, they are able to adapt to changing market circumstances on their own with far less human supervision and intervention. Quant managers have developed algorithms that gather and fine tune data, then autonomously change the investment course when a new pattern is identified.
Efficiency Plays
Hedge fund managers and their service providers are also using AI to optimize middle and back office operations. As teams move away from managing work through spreadsheets and towards digital and cloud enterprise resource planning (ERP) solutions, AI can provide an edge. Clearly not all fund processes can be completely automated, but AI can speed reconciliation, reduce errors and ultimately reduce costs.
Software and service providers to the hedge fund space are using AI in this area to help their hedge fund clients operate more efficiently and accurately. For example, BNY Mellon’s hedge fund middle office and administration services are using an artificial intelligence and machine learning platform to analyze historical trade break data and predict with high probability the root cause of current trade breaks. In an industry that still suffers from manually intensive reconciliation challenges, this use of AI has the potential to significantly reduce costs and speed up the NAV production process.
The Talent Bottleneck
Few doubt the impact AI will have, but the immediate impact could be delayed due to a scarcity of talent. Although estimates vary, it is clear that the number of people with high level education and skills in AI is only a few thousand. In practice, financial firms have had to recruit from tech players like Google and Facebook to obtain AI talent. The side benefit to bringing in talent from global tech firms is the cascading of new ideas into the financial sector.
The scarcity of talent is now colliding with a realisation that AI is mission critical to hedge funds both in keeping pace with traditional rivals and tech-savvy new entrants. The appreciation of this has ushered in major new investments in academic programs and training capacity to attract millennials and address the problem of talent scarcity.
Investing and Partnering
MIT, for example, recently announced one of the most ambitious steps yet with the creation of the $1bn Stephen A. Schwarzman College of Computing. It comes as no surprise that funding originates from the CEO of Blackstone, one the world’s largest alternative investment managers. It underscores the fact that the alternative investment sector needs to increase the talent pool, in part because so many top graduates are being pulled away from finance by the flourishing tech sector.
Some of the largest industry players are employing non-conventional partnerships and methods for gaining an AI edge on the talent front. Man Group has partnered with Oxford University to create The Oxford-Man Institute of Quantitative Finance. Man’s engineers, statisticians, and coders share facilities and collaborate with academics and researchers to study how algorithms, AI, and related advances can be applied to finance.
Another example is Two Sigma which is reported to hire more technologists than traditional portfolio managers. Like Man, Two Sigma is looking for an advantage by partnering with elite academia, in this case Cornell University. To recruit staff, Two Sigma uses an AI programming challenge in the form of its own game called ‘Halite®’. The game tests applicants’ ability to control a bot using the programming language of their choice.
Talent Retention
Understanding the need for talent and investing in its creation is vital. Yet the clear imperative is to understand how investment managers need to position themselves to attract the highly skilled AI specialists of tomorrow. What should hedge fund firms do to attract and retain talent?
Free snacks may help, but more important is to stress the fiduciary responsibilities of this potential career and emphasize that millennials will have an abundance of opportunities to make a difference. This implies trusting graduates with genuine responsibility for real issues involved with pension fund management, portfolio construction and investment idea generation. The role of human creativity is key. The big winners will be those firms that integrate AI with human talent. Machine analysis of data is already a necessity. Getting the most from AI requires empowering motivated and curious individuals who are encouraged to ask profound and creative questions of it.
A New Acronym - XAI
One of the new challenges facing the use of AI in hedge funds is the ability of human programmers to keep up with the speed and sophistication of their own creations. Bloomberg profiled this effect in its Sept 2017 report “The Massive Hedge Fund Betting on AI”. It tells the story of a large hedge fund with a new AI-based trading strategy that ran for months with very positive test results. If it had been a traditional quant strategy, it would have been quickly rolled out to investors. In this case, it had to be kept away from investors and run on separate servers until the creators fully understood how it worked. While pure performance is attractive, most investment management firms and their investors want to be able to fully explain how results are generated before they run with real money.
Indeed, a new acronym – XAI or Explainable Artificial Intelligence – has cropped up to describe the challenge of understanding how and why AI is generating a specific set of results. XAI isn’t a concern if the AI is being used to help choose the next film you want to watch on Netflix. However, if AI is being applied to trade large pension fund investments then clearly XAI is essential. The immediate challenge is to give humans a way to make sense of what computers are doing and be capable of explaining exactly how alpha is being generated.
Getting hedge fund AI programmers to embrace XAI to explain results is a good first step even though how AI works will remain opaque to fund outsiders. Within this explanation is a firm’s proprietary intellectual capital, a new form of ‘black box’. Understandably, firms will go to great lengths to keep this information confidential. Although hedge funds’ use of AI is accelerating and the number of use cases keep expanding, the specifics of how AI and machine learning contributes to fund performance is likely to remain largely a secret.
1 The Eurekahedge AI Hedge Fund Index (Bloomberg Ticker - EHFI817) is an equally weighted index of 14 constituent funds. The index is designed to provide a broad measure of the performance of underlying hedge fund managers who utilize artificial intelligence and machine learning theory in their trading processes. The index is base weighted at 100 at December 2010, does not contain duplicate funds and is denominated in local currencies. For more information on the index methodology, please click here.
BNY Mellon is the corporate brand of The Bank of New York Mellon Corporation and may be used as a generic term to reference the corporation as a whole and/or its various subsidiaries generally. Products and services may be provided under various brand names and in various countries by subsidiaries, affiliates, and joint ventures of The Bank of New York Mellon Corporation where authorized and regulated as required within each jurisdiction. The material contained in this document, which may be considered advertising, is for general information and reference purposes only and is not intended to provide or construed as legal, tax, accounting, investment, financial or other professional advice on any matter, and is not to be used as such.
BNY Mellon assumes no direct or consequential liability for any errors in or reliance upon this material. This material may not be reproduced or disseminated in any form without the prior written permission of BNY Mellon. Trademarks, logos and other intellectual property marks belong to their respective owners.
This document, and the statements contained herein, are not an offer or solicitation to buy or sell any products (including financial products) or services or to participate in any particular strategy mentioned and should not be construed as such.
The views expressed within this article are those of the author(s) only and not those of BNY Mellon or any of its subsidiaries or affiliates.
© 2019 The Bank of New York Mellon Corporation. All rights reserved.