In a time where data is the world’s most valuable resource, financial market participants are spending more time searching for an edge using alternative data sources. In the world of sports, technology is becoming increasingly sophisticated bringing to light seemingly limitless data sets which can be used to produce valuable predictive analytics using AI.
Professional sports are placing major emphasis on advancing technology, but how can financial services firms trading in global markets learn from sports use of analytics? In the NFL, for example, who recently held their Super Bowl and have begun their free agency period this week, tracking the performance of athletes and sports teams is nothing new.
Growth of AI Analytics
From quarterback ratings and yards per attempt to stadium ticket sales, the importance of sports analytics is vast and traverses across the industry – and now there are numerous ways that AI, machine learning, and data science are transforming the world of sports, including scouting, recruiting, training, athlete health, and advertising.
Similarly, AI and data analytics have become more influential across financial markets in recent years – investment banks and asset managers will need to make use of data through AI systems that will produce meaningful predictive analytics. From managing risk to discovering alpha, financial tools are enabling access to more data and data of a greater complexity.
Yet, what is needed is intelligent financial data management. Unstructured data has long been seen as expensive to import and difficult to digest and provide value, but as alternative data budgets grow, firms will start taking initiatives with alternative data in order to unlock unforeseen advantages.
Proactive error detection
So, what benefits can these tools bring? One is proactive error detection. This occurs before the usual validation rules even kick-in and can be key for improving performance. Processing data from a variety of feeds and data vendors poses significant operational overhead, because the data needs to be independently validated, by a defined set of rules, and then cleansed as necessary.
The problem is that, more often than not, validation processes are set up retroactively after problems have already occurred. And with new data feeds, it’s difficult for firms to keep up, having clearly defined rules for what constitutes usable data.
But now, machine learning routines can detect outliers proactively by using previously validated data of the same specific type, meaning they can detect errors early without the overhead of writing additional validation rules.
In the NFL, this is like the defense being able to accurately predict the play that the offense is going to run, and either run the original play-call or call an audible (like exception management).
Data mapping
Once a data set has been identified as relevant and affordable, the next step is to on-board the feed quickly, link the data elements with related data elsewhere in the data infrastructure, and then analyse. But all your different data now needs to be put into a standardized structure and it takes a huge amount of time to consolidate hundreds of feeds and thousands of fields for analysis and mapping. It’s a significant investment for any sized firm.
In sports analytics this is the equivalent of trying to identify the best deployment of a new player. How well can this player execute certain plays? Does the player perform better in certain situations? Is the player better with specific players around them? AI in sports management is used to overlay the new player’s in-game statistics with the plays that occurred in recent games played by his or her newly signed team. Quickly getting a new player to deliver results is a pleaser for fans and investors alike. Substitute fans for portfolio managers and the analogy crosses easily to new data sources in financial services.
Natural Language Processing (NLP) matching techniques provide a solution to relating and aligning new data sets in a firm. These tools can use a series of methods, including a configurable combination of ML and statistical techniques, to create data profiling which are then used to suggest mappings. As such, the data can be applied more quickly and effectively. This is proving to be very effective in helping analysts to reduce the time and money spent on decoding data.
Accessing unstructured data through AI analytics
Over the last several years there has been an explosion of data available to help buy side firms, both to improve operations and investment decisions. End users, such as market analysts, are demanding direct access to load and analyze this data to support their research and to make timely decisions.
However, data from new sources, including web scraping, is increasingly coming in semi-structured or unstructured forms, which makes it more difficult to organize and analyze. As such, new sets of tools are enabling self-service data access while still maintaining good data governance. This allows end users to load data directly into a sandbox area, register the data for governance purposes, do an initial cleanse of the data and run analytics against it.
In addition, users can perform common NLP functions against unstructured data, including entity extraction and resolution, sentiment analysis and PDF extraction. This allows users to extract company names from financial reports and news stories and link them to the company information as well as determine if the content of that story is positive or negative for their investment position.
Sports franchises and sponsors have been analyzing social media sentiment about teams and individual players for a number of years. Making meaningful decisions based on social sentiment is complicated, but less challenging when you have time to arrive at a decision. Using social sentiment for near-time investment decisions requires that analysts have immediate access to the data. However, usage of that data must still be managed under the governance policies of a firm. Having a sandbox or staging environment provides the best of both worlds and allows AI tools to be used on data that has at least met basic quality standards.
Overall, while it’s uncertain where AI and data will take the sports industry, it’s clear that the potential is there for financial firms to benefit from alternative and unstructured data. From risk mitigation to market predictions, the potential applications of intelligent automation in finance are significant and it will undoubtedly shake up the market, providing opportunities and a jump in productivity and returns.