An interesting article from the Chook in todays AFR - "ChatGPT can make human-like judgments, pick good stocks, researchers say"

Unsure what this eventually means exactly for our collective futures ... :)

For those not inside the paywall ...

Expect to see more fund managers and brokers using AI to analyse information. ChatGPT could be a game-changer, according to some academics researching how it can analyse results.

The human versus machine debate is set to flare up on a trading floor near you – and probably sooner than you think.

Although machines have won the trading execution and quantitative strategies battles (most trades on the ASX happen every day without human interaction), the new battleground will be old-fashioned, bottom-up stock-picking – the stuff fund managers first heard about at university and crafted by spending a few years in a sell-side sweatshop.

Last reporting season, we heard anecdotes about domestic fund managers mucking around with earnings transcripts and AI – using large language models to pick through earnings call transcripts to pick up on trends or assess management’s language to get a broader read on the market. Some of the big institutional equities houses are also using it in their quant teams.

But it could go a lot deeper than mapping out hot words or trends. It could do exactly what sell-side analysts do – earnings revisions, price targets, buy/sell/hold recommendations – and perhaps even better.

A bunch of University of Sydney academics have spent the past year testing whether a trained ChatGPT-4 model could analyse companies’ earnings results and use information from them as well as an expert analyst.

The early results, based on a sample of the 200 biggest stocks in the S&P 500 and tested over a 10-year period, were surprisingly strong; generative AI could replicate the human analyst and produce a portfolio that outperformed the latter by 40 to 80 basis points a month.

It was encouraging enough that the academics – PhD student Jason Ming, lecturer Hamish Malloch and professor Joakim Westerholm – have published an early paper to plant their stake in the ground.

AI is a hot topic, including in the academic research world. They stress it is early days – the study was not peer-reviewed, for example, or broad enough. But it was more than enough to keep going with the research.

Macquarie’s Australian equities analysts picked the paper up in their periodical scan of academic research. “The results show that ChatGPT’s insights are consistent with analyst behaviour and outperform traditional analyst metrics, allowing for the creation of portfolios that earn abnormal returns,” they said.

Perhaps the most surprising thing, according to the academics, was it showed ChatGPT could read between the lines, which is what earnings calls analysing companies more generally are often about.

The calls are full of lingo and acronyms – “could you give us a bit of colour on this”, or “how should we think about that”. They can be part commentary, part inquiry – the best analysts go beyond trying to work out next half’s earnings numbers and question management’s overarching strategy, capital allocation, or both.

ChatGPT could decipher it, work out what an analyst was asking and, most importantly, why.

“I was really impressed; I didn’t think it would do it,” Malloch says.

Ming, doing the research for his PhD, says it shows ChatGPT could “think like an expert, like an artist”.

“That was our main effort,” he says. Ming started the research in February last year, just ahead of ChatGPT-4’s launch in March.

Now, we’re not suggesting all analysts will be replaced by AI as soon as academics like this can stand up to a more detailed study – far from it.

But it shows where the finance industry is likely headed; much of the grunt work could be automated, freeing up human analysts to work their contacts, dig up new sources of unpublished information (something ChatGPT will never be able to do), and try to add more value for their clients.

Will it mean a smaller sell-side industry? Probably. But it is headed that way anyway.

For their research, the academics trained ChatGPT on 400 earnings calls held by the 100 largest US-listed stocks in 2014, to come up with analyst motivations for questions they asked – was it to learn more about a market expansion, new product or M&A strategy, for example.

They then tested the model on nearly 10 years of earnings calls by the top 200 stocks in the US, and had ChatGPT think like an analyst and come up with a score to assess each result.

Not to be sneezed at

By sorting stocks based on those scores, they put together a portfolio that outperformed humans by 40 to 80 basis points a month, which is not to be sneezed at.

The results show earnings calls are valuable sources of information – which is no surprise – and AI models have the capacity to uncover information that later drives stock prices. That last bit is the main bit; investing always comes back to returns.

The ChatGPT model should be replicable and consistent across all earnings calls – even on those days when there are four REITs or three energy companies reporting on the same day, and the human analyst is straddled across multiple calls, annual reports and financial models. It was also consistent across sectors, the academics said.

While they stress it is early days, the sample of 200 large-cap US stocks over 10 years showed the theory has legs. It just needs more rigorous testing – and review by peers.

Does it mean AI will replace human analysts in buy- and sell-side teams? The academics say ChatGPT can make human-like judgments and is proficient in identifying stocks, but there is also “enduring value of human expertise” in stock evaluation. A hybrid future, perhaps.

But it’s a fertile ground worth watching. We’ve seen sell-side research teams change dramatically in the past two decades as average trade parcels shrank, trading commissions decreased and banks cut back on costs.

Those pressures have led to a smaller, arguably younger cohort of sell-side analysts across the market. The irony is there has probably never been more quantitative and passive money using those analysts’ forecasts in their own computer models and investment decisions. Analysts’ earnings upgrades and downgrades still move markets, even if active fund managers often see through them.

At the end of the day, managing retirement savings or investment portfolios all comes down to trust. And who do you trust more – a large language model like ChatGPT-4, or the old-world equity analyst?