The giant leap in natural language processing and massive computing
You’ll see that OpenAI’s GPT-3 and Google’s T5 have far surpassed previous models in terms of parameters, but it’s more than that. Most models are trained to complete specific tasks, such as the “go” playing machine AlphaGO that outperformed the human world champion at the game of go, but can’t play tic-tac-toe or checkers (despite these games being much simpler). Open-AI’s GPT-3, by contrast, can do many different tasks with no additional training or fine-tuning.8
Newer language models can perform business applications, such as answering questions via digital assistants, summarizing data, translation, and performing sentiment analysis, more accurately than ever before.
While there is still more work to be done, these advancements serve as a glimpse for what could be possible tomorrow. As quantum computing advances and models continue to become more sophisticated, the AGI many dream of could become a reality.
1 One zettabyte is approximately equal to one thousand exabytes or one billion terabytes.
2 Coughlin, Tom. “175 Zettabytes By 2025.” Forbes, Forbes Magazine, 29 Nov. 2018, www.forbes.com/sites/tomcoughlin/2018/11/27/175-zettabytes-by-2025/?sh=5d98dd665459.
3 Rogers, Adam. “Council Post: The 80% Blind Spot: Are You Ignoring Unstructured Organizational Data?” Forbes, Forbes Magazine, 29 Jan. 2019, www.forbes.com/sites/forbestechcouncil/2019/01/29/the-80-blind-spot-are-you-ignoring-unstructured-organizational-data/.
4 Gutierrez, Daniel. “How Financial Institutions Can Deal with Unstructured Data Overload.” InsideBIGDATA, 16 Mar. 2021, insidebigdata.com/2021/03/15/how-financial-institutions-can-deal-with-unstructured-data-overload/.
5 Heath, Nick. “What Is Artificial General Intelligence?” ZDNet, ZDNet, 22 Aug. 2018, www.zdnet.com/article/what-is-artificial-general-intelligence/.
6 Yao, Mariya. “10 Leading Language Models For NLP In 2021.” TOPBOTS, 21 May 2021, www.topbots.com/leading-nlp-language-models-2020/.
7 Wiggers, Kyle. “Google Trained a Trillion-Parameter AI Language Model.” VentureBeat, VentureBeat, 14 Jan. 2021, venturebeat.com/2021/01/12/google-trained-a-trillion-parameter-ai-language-model/.
8 Lauret, Julien. “GPT-3: The First Artificial General Intelligence?” Medium, Towards Data Science, 25 July 2020, towardsdatascience.com/gpt-3-the-first-artificial-general-intelligence-b8d9b38557a1.
9 “Towards Interpreting Solidity Smart Contract: An Automatic and Practical Realization.” IEEE Transactions on Services Computing, vol. 9, no. 1, 2016, pp. 1–11., doi:10.1109/tsc.2015.2512393.
10 Heaven, Will Douglas. “OpenAI's New Language Generator GPT-3 Is Shockingly Good-and Completely Mindless.” MIT Technology Review, MIT Technology Review, 10 Dec. 2020, www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/.
11 Li, Chuan. “OpenAI's GPT-3 Language Model: A Technical Overview.” Lambda Blog, Lambda Blog, 11 Sept. 2020, lambdalabs.com/blog/demystifying-gpt-3/.
12 Crunchbase has a listing of 1,275 active companies who self-describe as being in the NLP sector as of 5/3/2021.
BNY Mellon is the corporate brand of The Bank of New York Mellon Corporation and may be used to reference the corporation as a whole and/or its various subsidiaries generally. This material does not constitute a recommendation by BNY Mellon of any kind. The information herein is not intended to provide tax, legal, investment, accounting, financial or other professional advice on any matter, and should not be used or relied upon as such. The views expressed within this material are those of the contributors and not necessarily those of BNY Mellon. BNY Mellon has not independently verified the information contained in this material and makes no representation as to the accuracy, completeness, timeliness, merchantability or fitness for a specific purpose of the information provided in this material. BNY Mellon assumes no direct or consequential liability for any errors in or reliance upon this material.
BNY Mellon will not be responsible for updating any information contained within this material and opinions and information contained herein are subject to change without notice.
BNY Mellon assumes no direct or consequential liability for any errors in or reliance upon this material. This material may not be reproduced or disseminated in any form without the prior written permission of BNY Mellon. Trademarks, logos and other intellectual property marks belong to their respective owners.
© 2021 The Bank of New York Mellon Corporation. All rights reserved.