AI and ML in Volatility Forecasting: Trends and Future Directions

By Muhammad OsamaMay 9 2024Reviewed by Susha Cheriyedath, M.Sc.

In a review article recently published in the journal International Review of Financial Analysis, researchers systematically explored existing studies using artificial intelligence (AI) and machine learning (ML) techniques to forecast realized volatility and implied volatility indices.

They addressed four main research questions: (i) Do ML and AI methods offer superior forecasts compared to traditional econometric models? (ii) What is the extent of the application of explainable AI (XAI)? (iii) What are the primary drivers and challenges of volatility forecasting? and (iv) What are the potential areas for further research?

Background 

Volatility represents the fluctuation and uncertainty of asset returns, inferred from sources like high-frequency intraday returns, option prices, or survey data. Accurate forecasting of realized volatility and implied volatility is crucial for risk management, portfolio optimization, option pricing, and policymaking. Realized volatility, derived from squared intraday returns, offers a precise measure of asset-integrated variance, while implied volatility, derived from option prices, indicates the market's anticipation of future volatility.

AI and ML methods excel at handling complex, nonlinear, and high-dimensional problems without strict assumptions on data generation processes. Their popularity has surged in finance and economics due to the availability of big data, enhanced computing power, and advanced statistical software. Moreover, they can integrate exogenous variables like macroeconomic indicators, financial metrics, technical signals, or measures of uncertainty to enhance forecasting accuracy.

About the Research

In this review, the authors conducted a thorough search across four major scientific literature databases, utilizing keywords related to volatility forecasting, AI, and ML. They applied inclusion and exclusion criteria, considering factors like peer-review status, language, publication date, and relevance, resulting in a final sample of 32 papers published between 2014 to 2021. Additionally, they employed snowball sampling techniques to identify additional relevant papers from the selected articles' reference lists.

From each paper, the researchers extracted and synthesized key information, including details such as the target asset, data source, and frequency, model type and specification, evaluation method and metric, as well as the primary findings and contributions.

Furthermore, they analyzed quantitative metrics such as journal quality, citation scores, impact factors, and rankings. Descriptive statistics and graphical representations were utilized to present various aspects of the literature, covering academic publishing channels, the markets, and datasets under investigation, the models and evaluation techniques employed, as well as the forecasting horizons and frequencies utilized across studies.

Significance of AI/ML in Volatility Prediction

The authors demonstrated that AI and ML methods hold significant promise for volatility forecasting, often providing results comparable to or superior to traditional econometric models. Their analysis revealed several prevailing trends and patterns:

• The US equity market received the most extensive attention, followed closely by the oil and Bitcoin markets. However, they emphasized the need for further exploration of other markets and regions to diversify portfolio strategies.
• Incorporating exogenous data frequently enhanced performance, especially for longer forecasting horizons. They highlighted the importance of selecting exogenous variables based on economic theory and empirical evidence, alongside considerations of data quality and availability, recommending hybrid models combining ML and econometric approaches.
• Neural networks, particularly those with memory capabilities like long short-term memory (LSTM) and gated recurrent unit (GRU), demonstrated superior performance in capturing complex and nonlinear volatility dynamics. However, they noted challenges such as high computational costs, overfitting susceptibility, and limited interpretability.
• Tree-based models, including random forest and boosting methods, proved effective in managing multicollinearity and nonlinearity, providing insights into variable importance. However, they exhibited drawbacks such as instability, sensitivity to outliers, and limitations in extrapolation.
• Less prevalent models like support vector machines, penalized regression, and reservoir computing showed promising results but with trade-offs depending on the problem and data characteristics. They suggested further research to compare and integrate diverse models and explore novel methodologies.

Applications

The review discussed that accurate volatility forecasting offers numerous benefits in finance and economics:

• Helping investors assess potential losses and gains, enabling calculations of value at risk (VaR), expected shortfall (ES), or conditional value at risk (CVaR) measures, ultimately improving risk management strategies.
• Aiding in optimizing portfolio performance and diversification. By estimating asset returns and covariances, investors can leverage strategies like mean-variance optimization for a more balanced portfolio.
• Enhancing the accuracy of option pricing and hedging for option traders. Estimating implied volatility allows for a more informed application of option pricing models like Black-Scholes.
• Improving market monitoring and informing policy decisions. By estimating market sentiment and assessing policy impacts, policymakers can make more informed choices regarding monetary and fiscal policy.

Conclusion

The researchers summarized that AI and ML methods for volatility forecasting proved highly effective and promising, offering advantages over traditional econometric models. However, they acknowledged limitations such as data quality and availability, model complexity and interpretability, and model robustness and generalization.

The authors also suggested potential areas for further research, including (1) the use of XAI to analyze and support empirical results, providing more transparency and accountability for model decisions and predictions; (2) the use of probabilistic ML to quantify uncertainty in volatility forecasts and provide more reliable and informative predictions, and (3) the exploration of alternative data sources, such as textual, image, or audio data, to enrich information sets and capture latent factors affecting volatility.