Insights
Solutions
About Us
Careers
Investor Relations
Artificial Intelligence Sweeps Hedge Funds
March 2019
By Peter Salvage
Artificial Intelligence, or “AI”, has featured heavily in industry innovation headlines for some time. Yet for all the excitement and promise, the uptake in the hedge fund industry has been limited – until recently.
Hedge funds’ use of AI is accelerating and reshaping the industry, particularly in investing, cost models and recruitment. Managers also face challenges to explain new AI-based approaches to investors. Given the strategies are the byproduct of super computers crunching billions of data points and learning how to adjust to markets in real-time, explaining how returns are generated is pushing the boundaries of human comprehension.
Why Now?
In September 2018, BarclayHedge's Hedge Fund Sentiment Survey found that over half of hedge fund respondents (56%) used AI to inform investment decisions – nearly triple the 20% reported a year earlier. Around two-thirds of those using AI were doing so to generate trading ideas and optimize portfolios. Over a quarter were using it to automate trade execution, according to the survey.
The early results are promising. For example, the Eurekahedge AI Hedge Fund Index¹ slightly outperformed the flagship Eurekahedge Hedge Fund Index in both 2017 and 2018. Moreover, the Eurekahedge Hedge Fund Index decreased by 4% in the fourth quarter of 2018, while the Eurekahedge AI Hedge Fund Index was flat for the period.
Several technical advances have driven AI adoption. New, vast ‘big data’ sets are now available from satellite imagery, the internet of things, global capital flows, point of sale systems, and social media. More data can now be generated in one day than during the entire 1990s. A large hedge fund heavily utilizing AI is likely to have dedicated experts devoted to evaluating and procuring new sets of data. With raw computing power continuing to advance, graphics processing units (GPUs) and customized hardware now solve problems in hours instead of weeks – a necessity given the ongoing rapid growth in data. Finally, with cloud computing now widespread and deployment costs falling, barriers to entry for machine learning are tumbling.
How Hedge Funds Use AI
A number of hedge funds are using AI to analyze masses of data, predict corrections in supply and demand imbalances, and forecast market movements for tactical asset allocation. This has the potential to assist a CIO’s team to combine different strategies and tailor allocations.
Use of AI is playing out across a wide spectrum of investment managers from pure AI-driven specialists, to large quant-driven shops, to traditional fundamental investors looking for an edge. A growing number of firms across the spectrum are also turning to AI to improve efficiency in their operations, accounting and investor relations functions.
Indeed, a class of AI pure play hedge funds has emerged in recent years that are based entirely on machine learning and AI algorithms. Examples include Aidiyia Holdings, Cerebellum Capital, Taaffeite Capital Management and Numerai. Numerai, a recognized AI hedge fund, is pushing the boundaries of the hedge fund business model. The firm uncovers investment strategies by hosting competitions among external AI experts, mathematicians and data scientists. Recently, Numerai expanded its business model by making elements of its platform available to the rest of financial community with its product Erasure, which is a decentralized prediction marketplace using blockchain technology.
Dwarfing the upstart AI pure plays are the large quant funds that are household names in the hedge fund industry such as Man AHL, Two Sigma, Citadel, Bridgewater and D.E. Shaw. For years, players like these have used computer-driven models to uncover new trading strategies and identify themes, factors and trading signals. Human “quants” will then feed these factors and signals into trading systems. With markets continually changing and shifting, these pre-AI models often need frequent monitoring and reprogramming by the quants. AI models are different because while initially crafted by humans, they are able to adapt to changing market circumstances on their own with far less human supervision and intervention. Quant managers have developed algorithms that gather and fine tune data, then autonomously change the investment course when a new pattern is identified.
Efficiency Plays
Hedge fund managers and their service providers are also using AI to optimize middle and back office operations. As teams move away from managing work through spreadsheets and towards digital and cloud enterprise resource planning (ERP) solutions, AI can provide an edge. Clearly not all fund processes can be completely automated, but AI can speed reconciliation, reduce errors and ultimately reduce costs.
Software and service providers to the hedge fund space are using AI in this area to help their hedge fund clients operate more efficiently and accurately. For example, BNY Mellon’s hedge fund middle office and administration services are using an artificial intelligence and machine learning platform to analyze historical trade break data and predict with high probability the root cause of current trade breaks. In an industry that still suffers from manually intensive reconciliation challenges, this use of AI has the potential to significantly reduce costs and speed up the NAV production process.
The Talent Bottleneck
Few doubt the impact AI will have, but the immediate impact could be delayed due to a scarcity of talent. Although estimates vary, it is clear that the number of people with high level education and skills in AI is only a few thousand. In practice, financial firms have had to recruit from tech players like Google and Facebook to obtain AI talent. The side benefit to bringing in talent from global tech firms is the cascading of new ideas into the financial sector.
The scarcity of talent is now colliding with a realisation that AI is mission critical to hedge funds both in keeping pace with traditional rivals and tech-savvy new entrants. The appreciation of this has ushered in major new investments in academic programs and training capacity to attract millennials and address the problem of talent scarcity.
Investing and Partnering
MIT, for example, recently announced one of the most ambitious steps yet with the creation of the $1bn Stephen A. Schwarzman College of Computing. It comes as no surprise that funding originates from the CEO of Blackstone, one the world’s largest alternative investment managers. It underscores the fact that the alternative investment sector needs to increase the talent pool, in part because so many top graduates are being pulled away from finance by the flourishing tech sector.
Some of the largest industry players are employing non-conventional partnerships and methods for gaining an AI edge on the talent front. Man Group has partnered with Oxford University to create The Oxford-Man Institute of Quantitative Finance. Man’s engineers, statisticians, and coders share facilities and collaborate with academics and researchers to study how algorithms, AI, and related advances can be applied to finance.
Another example is Two Sigma which is reported to hire more technologists than traditional portfolio managers. Like Man, Two Sigma is looking for an advantage by partnering with elite academia, in this case Cornell University. To recruit staff, Two Sigma uses an AI programming challenge in the form of its own game called ‘Halite®’. The game tests applicants’ ability to control a bot using the programming language of their choice.
Talent Retention
Understanding the need for talent and investing in its creation is vital. Yet the clear imperative is to understand how investment managers need to position themselves to attract the highly skilled AI specialists of tomorrow. What should hedge fund firms do to attract and retain talent?
Free snacks may help, but more important is to stress the fiduciary responsibilities of this potential career and emphasize that millennials will have an abundance of opportunities to make a difference. This implies trusting graduates with genuine responsibility for real issues involved with pension fund management, portfolio construction and investment idea generation. The role of human creativity is key. The big winners will be those firms that integrate AI with human talent. Machine analysis of data is already a necessity. Getting the most from AI requires empowering motivated and curious individuals who are encouraged to ask profound and creative questions of it.
A New Acronym - XAI
One of the new challenges facing the use of AI in hedge funds is the ability of human programmers to keep up with the speed and sophistication of their own creations. Bloomberg profiled this effect in its Sept 2017 report “The Massive Hedge Fund Betting on AI”. It tells the story of a large hedge fund with a new AI-based trading strategy that ran for months with very positive test results. If it had been a traditional quant strategy, it would have been quickly rolled out to investors. In this case, it had to be kept away from investors and run on separate servers until the creators fully understood how it worked. While pure performance is attractive, most investment management firms and their investors want to be able to fully explain how results are generated before they run with real money.
Indeed, a new acronym – XAI or Explainable Artificial Intelligence – has cropped up to describe the challenge of understanding how and why AI is generating a specific set of results. XAI isn’t a concern if the AI is being used to help choose the next film you want to watch on Netflix. However, if AI is being applied to trade large pension fund investments then clearly XAI is essential. The immediate challenge is to give humans a way to make sense of what computers are doing and be capable of explaining exactly how alpha is being generated.
Getting hedge fund AI programmers to embrace XAI to explain results is a good first step even though how AI works will remain opaque to fund outsiders. Within this explanation is a firm’s proprietary intellectual capital, a new form of ‘black box’. Understandably, firms will go to great lengths to keep this information confidential. Although hedge funds’ use of AI is accelerating and the number of use cases keep expanding, the specifics of how AI and machine learning contributes to fund performance is likely to remain largely a secret.
1 The Eurekahedge AI Hedge Fund Index (Bloomberg Ticker - EHFI817) is an equally weighted index of 14 constituent funds. The index is designed to provide a broad measure of the performance of underlying hedge fund managers who utilize artificial intelligence and machine learning theory in their trading processes. The index is base weighted at 100 at December 2010, does not contain duplicate funds and is denominated in local currencies. For more information on the index methodology, please click here.
BNY Mellon is the corporate brand of The Bank of New York Mellon Corporation and may be used as a generic term to reference the corporation as a whole and/or its various subsidiaries generally. This material and any products and services may be issued or provided under various brand names in various countries by duly authorised and regulated subsidiaries, affiliates, and joint ventures of BNY Mellon, which may include any of the following. The Bank of New York Mellon, at 240 Greenwich Street, NY, NY 10286 USA, a banking corporation organised pursuant to the laws of the State of New York, and operating in England through its branch at One Canada Square, London E14 5AL, registered in England and Wales with numbers FC005522 and BR000818. The Bank of New York Mellon is supervised and regulated by the New York State Department of Financial Services and the US Federal Reserve and authorised by the Prudential Regulation Authority. The Bank of New York Mellon, London Branch is subject to regulation by the Financial Conduct Authority and limited regulation by the Prudential Regulation Authority. Details about the extent of our regulation by the Prudential Regulation Authority are available from us on request. The Bank of New York Mellon SA/NV, a Belgian public limited liability company, with company number 0806.743.159, whose registered office is at 46 Rue Montoyerstraat, B-1000 Brussels, authorised and regulated as a significant credit institution by the European Central Bank (ECB), under the prudential supervision of the National Bank of Belgium (NBB) and under the supervision of the Belgian Financial Services and Markets Authority (FSMA) for conduct of business rules, a subsidiary of The Bank of New York Mellon, and operating in England through its branch at 160 Queen Victoria Street, London EC4V 4LA, registered in England and Wales with numbers FC029379 and BR014361. The Bank of New York Mellon SA/NV (London Branch) is authorised by the ECB and subject to limited regulation by the Financial Conduct Authority and the Prudential Regulation Authority. Details about the extent of our regulation by the Financial Conduct Authority and Prudential Regulation Authority are available from us on request. The Bank of New York Mellon SA/NV, operating in Ireland through its branch at Riverside 2, Sir John Rogerson’s Quay, Grand Canal Dock, Dublin 2, D02 KV60, Ireland, trading as The Bank of New York Mellon SA/NV, Dublin Branch, which is authorized by the ECB, regulated by the Central Bank of Ireland for conduct of business rules and registered with the Companies Registration Office in Ireland No. 907126 & with VAT No. IE 9578054E. If this material is distributed in or from, the Dubai International Financial Centre (DIFC), it is communicated by The Bank of New York Mellon, DIFC Branch, (the “DIFC Branch”) on behalf of BNY Mellon (as defined above). This material is intended for Professional Clients and Market Counterparties only and no other person should act upon it. The DIFC Branch is regulated by the DFSA and is located at DIFC, The Exchange Building 5 North, Level 6, Room 601, P.O. Box 506723, Dubai, UAE. BNY Mellon also includes The Bank of New York Mellon which has various subsidiaries, affiliates, branches and representative offices in the Asia-Pacific Region which are subject to regulation by the relevant local regulator in that jurisdiction. Details about the extent of our regulation and applicable regulators in the Asia-Pacific Region are available from us on request. Not all products and services are offered in all countries.
The material contained in this document, which may be considered advertising, is for general information and reference purposes only and is not intended to provide legal, tax, accounting, investment, financial or other professional advice on any matter, and is not to be used as such. The contents may not be comprehensive or up-to-date, and BNY Mellon will not be responsible for updating any information contained within this document. If distributed in the UK or EMEA, this document is a financial promotion. This document and the statements contained herein, are not an offer or solicitation to buy or sell any products (including financial products) or services or to participate in any particular strategy mentioned and should not be construed as such. This document is not intended for distribution to, or use by, any person or entity in any jurisdiction or country in which such distribution or use would be contrary to local law or regulation. Similarly, this document may not be distributed or used for the purpose of offers or solicitations in any jurisdiction or in any circumstances in which such offers or solicitations are unlawful or not authorised, or where there would be, by virtue of such distribution, new or additional registration requirements. Persons into whose possession this document comes are required to inform themselves about and to observe any restrictions that apply to the distribution of this document in their jurisdiction. The information contained in this document is for use by wholesale clients only and is not to be relied upon by retail clients. Trademarks, service marks and logos belong to their respective owners.
BNY Mellon assumes no liability whatsoever for any action taken in reliance on the information contained in this material, or for direct or indirect damages or losses resulting from use of this material, its content, or services. Any unauthorised use of material contained herein is at the user’s own risk. Reproduction, distribution, republication and retransmission of material contained herein is prohibited without the prior consent of BNY Mellon.
© 2019 The Bank of New York Mellon Corporation. All rights reserved.