GPT-4 Beats Analysts at Earnings Forecast
User avatar
Curated by
Perplexity Team
1 min read
1 month ago
A groundbreaking study by researchers at the University of Chicago has revealed that OpenAI's GPT-4 can outperform human analysts in predicting future earnings based on financial statement analysis. This discovery has significant implications for the future of financial analysis and decision-making, as it demonstrates the potential for large language models to augment and streamline the work of financial professionals.

GPT-4 Prediction Accuracy
  • GPT-4 achieved a prediction accuracy of 60.35% in determining the direction of future earnings, surpassing the 52.71% accuracy of human analysts. The model also outperformed analysts in terms of F1-score, which balances precision and recall, with GPT-4 scoring 60.90% compared to 54.48% for human analysts.
  • The study utilized anonymized financial data from the Compustat database, spanning from 1968 to 2021, and compared GPT-4's performance to human analysts' predictions sourced from the IBES database. By removing company names and dates from the standardized financial statements provided to GPT-4, the researchers ensured a fair comparison between the model and human analysts. favicon favicon favicon
3 sources

Comparison with ANNs

  • GPT-4's performance was commensurate with advanced machine learning models, such as artificial neural networks (ANNs), specifically designed for earnings predictions. In certain aspects, GPT-4 even outperformed these specialized models, demonstrating its robustness in financial analysis.
  • The researchers compared GPT-4 with state-of-the-art ANNs and found that the language model generated performance on par with these narrowly specialized applications. This finding highlights the potential for general-purpose AI to rival or surpass purpose-built models in complex analytical tasks. favicon favicon favicon
3 sources

Chain of Thought Methodology
  • The researchers employed a "Chain of Thought" (CoT) reasoning approach with GPT-4, mimicking the analytical steps a human analyst would take. This method involved identifying changes in financial statements, computing key financial ratios, and synthesizing this information to predict earnings trends.
  • The CoT prompts played a pivotal role in helping GPT-4 identify trends, compute financial ratios, and synthesize information akin to a human analyst. This approach allowed GPT-4 to analyze and generate accurate predictions even when given raw financial data devoid of context. favicon favicon favicon
3 sources

Skepticism and Challenges

  • Despite the promising results, some skepticism remains. Critics have questioned the validity of comparing GPT-4's performance with that of human analysts and specialized ANNs, pointing out potential differences in the complexity of tasks and the models used for comparison.
  • The study acknowledges the difficulty in pinpointing exactly how and why GPT-4 performs well, highlighting the challenges in understanding the inner workings of large language models. As AI researcher Matt Holden noted, it is unlikely that GPT-4 could select stocks that outperform broader indexes like the S&P 500. favicon favicon
2 sources
what are some of the challenges in implementing gpt-4 in financial analysis
how can gpt-4's performance in financial analysis be improved
what are some of the limitations of using ai in financial analysis
Keep Reading