Do new AI systems like chatgpt use human annotated data?

Yes, new AI systems like ChatGPT use human-annotated data to train and improve their performance. This data is used to "teach" the AI model how to understand and respond to different inputs, and to help it learn the nuances of human language. This process is called "supervised learning", and it is one of the most common ways to train AI models.

Is it likely to become less necessary over time to use this data from humans?

It is possible that the need for human-annotated data may decrease over time as AI systems continue to improve. With advancements in unsupervised learning and self-supervised learning techniques, AI models may be able to learn from large amounts of unannotated data, which could reduce the need for human annotation.

However, it's important to note that even with these advancements, human-annotated data will continue to play an important role in training AI systems, as it can be used to verify and validate the results of unsupervised learning methods. Additionally, training on human-annotated data can also help to reduce bias and ensure that the AI system is fair and ethical.

So, the need for human annotated data may decrease over time but it will still be an important part of the training process for AI systems.

Does this data need to be constantly refreshed or can the same datasets be reused?

The need to refresh the training data depends on the specific use case of the AI system and how frequently the real-world data changes. In some cases, the same dataset can be reused multiple times to train different AI models or to fine-tune existing models. However, in other cases, the data may need to be constantly refreshed to ensure that the AI system stays up to date with the latest information.

For example, in natural language processing (NLP) tasks, such as language translation or text generation, a dataset of text can be used to train multiple models over time. However, as new words, phrases, and idioms emerge, the dataset will need to be updated to ensure that the AI system can understand and respond to them.

In other applications, such as computer vision, the training data needs to be constantly updated as the real world changes, for example, to add new classes of objects or to reflect changes in the way objects look.

In summary, it depends on the specific use case and how dynamic the real-world data is. In some cases, the same dataset can be reused multiple times, while in other cases, the data needs to be constantly refreshed to ensure that the AI system stays up-to-date.

Not a terrible result for Appen but never good to:

• Miss the lower end of the guidance range
• Drop near term guidance and start talking about investing for “the long term”

Share price has been punished and rightly so. 

Couple of highlights:

• QoQ growth of China segment (I maintain that if this was a stand alone business on the ASX it would probably be worth as much as APX as a whole – This would be also be stupid but probably true)
• 50mil in the bank no debt

Bit of commentary on competitors and I wonder if this is a more plausible narrative for the decrease in revenue growth and pressure on the business model that we are seeing.

I still think that the price provides a margin of safety if you believe in the future on human annotated data. Assuming underlying cashflow the business is spitting out of ~$50mil USD (this number is hard to tease out and you may come up with a different one) I still get a return of ~14% moving forwards. 

Assuming roughly a 10% return to be an acceptable number gets me to a fair value of $10.50 – I will update my valuation to reflect this.

From the Q4 facebook earnings release:

Our planned capital expenditures are primarily driven by investments in data centers, servers, network infrastructure, and office facilities. As we discussed previously, this range reflects a significant increase in our artificial intelligence and machine learning investments, which will support a number of areas across our Family of Apps. 

Something to ponder as it drops today inline with Facebook.

A great summary of Appen and the investing thesis. By @galiciaInvestor on twitter.

Bit of scuttlebutt too signing up to be an annotator. 

Key to the discussion about Appen is whether or not large scale crowd annotation is required for the ongoing advancement of AI. Being the largest player in this space Appen will have scale advantages meaning It’s likely to be an effective competitor if the first statement is true. If you think that’s false probably Appen is a non-starter.

Certainly, you have to wonder whether less is needed when you look at the commentary surrounding the space. Lowered costs of AI training, automation and self labelling. Certainly these things exist. Do they displace crowd annotation or perhaps just push the margins the wrong direction?

My personal take and I could be very wrong is that human curation of large data sets still has a place. Listening to Tesla engineers describe the utter confusion of the AI on arriving at some routine roadworks or labelling unique situation of people and pet running down a highway is that really challenging final few percent of improvement to get to a FSD product. Recent exodus of senior staff from Waymo and delayed rollout schedules speak to the scale of the issue. One of the solutions is likely to be more data and some of that in my opinion will need to be human annotated data.

I think takeover is very unlikely unless there is a random SPAC that is looking for a home as Appen’s diversity of big tech companies as customers means that any one of them is unlikely to see it as a synergistic relationship. I think this is a negative.

Data is the new oil, Appen is a refinery it is a heck of a tailwind to have. Valuation is getting cheaper and cheaper if you believe they can grow moving forwards.

Assuming lower end of EBITDA guidance (~$109mil AUD) that places it on around a 12x Price to EBITDA multiple. Not bad at all if you believe there is some growth ahead. I would guess that fair value would be closer to 20x (but that’s just like my opinion man).

Recent reporting was a mixed bag, flat revenue, growthy bits of the business, better pipeline and a negative narrative regarding the ongoing value proposition of Appen which as I’ve said I’m not convinced is valid.

All that said I certainly don’t feel there is a rush to buy currently. 12 month momentum is extremely negative and sentiment is poor. I think it’s likely Appen falls more in price (I don’t make the rules). I actually welcome this and will have to have a really good think about what constitutes an attractive point for myself (Probably around $8.5 so it probably won’t quite get here). I do continue to hold a small position (getting smaller all the time in fact) with a longer term horizon.

Appen shares have fallen roughly 75% (crazy huh!) from their peak in August 2020. It is a massive amount and this once market darling is today’s dog.

In 2020 the utilisation of Appen’s services plateaued significantly by most of their major customers, currency headwinds have been felt harshly as well.

Momentum and sentiment are very negative. An investor presentation today for example which reiterated the companies messaging about the headwinds the company is currently facing lead to a 20% drop I suspect the market was taking the no news is bad news approach.

Investors in Appen at this point must all be asking themselves “Am I wrong?”

Do large technology companies actually not require Appen’s data sets?

Are they able to produce them themselves or find cheaper alternatives?

Will margins be compressed due to competition?

Are annotated data sets just a commodity?

Is Appen just a sweat shop for data, easy to reproduce or is there more of a moat in the business?

Maybe AI is more hype than substance and still many years away from being a meaningful part of the economy?

Maybe the AUD will go much higher due to a commodity super cycle?

Management seem to be less and less transparent can I trust their guidance and explanations?

Welcome to uncertainty friend.

Uncertainty is good for a fundamental investor. People don’t like to be uncertain. Uncertainty leads to loss of confidence and loss of confidence leads to selling. Selling leads to depressed prices, negative sentiment and momentum. Falling prices mean a share might drop below its intrinsic value.

I think Appen may represent good value and I think that the more the price falls the less of these questions I have to ask or be able to answer.

I suspect momentum will likely take Appen lower yet. My suggestion, work out your conviction now, look out at the years, rather than the months ahead. There just might be gold in them there hills (or maybe not).

Appen has grown into a fairly complex company over the last few years and the results take a bit of time to wade through, digest and contextualise. This is exacerbated by the extremities presented in 2020 making extrapolation of results in that year nigh impossible.

Some points are raised that are worth considering…

Appen have said that online advertising spend softened in 2020. Are we supposed to believe this given the record shift to online and a US election thrown in? While some are rightfully sceptical I think that it is possible as uncertainty leads companies to preserve cash and even if customers were spending more on advertising online APX is several steps removed from this direct spend and it is entirely possible that their customers delayed or reduced R&D and non-essential spending in their AI projects that require these data sets.

Currency had a wild year with the AUD swinging between mid 50 cents to mid 70 cents USD. Appen have given projections with the rough average of the exchange rate last year to allow them to “contextualise” the numbers. While that sounds reasonable unfortunately the AUD has rocketed up to almost 80 cents now so is going to be a significant headwind unless there is a reversal back down. Of course it could even move higher. That said the important thing is being aware of this. I try to construct my portfolio in a way to diversify across large currency movements. If you are like me and earn AUD it can also be worth remembering that although this hurts Appen your global buying power is rising so I don’t hate this kind of exposure. I certainly wasn’t complaining when the FX provided a free kick in the other direction.

When you look at the revenue by customer of the five largest 1 had good growth two reduced spending and two stayed roughly the same. Something to watch.

China growth is currently +60% quarter on quarter from a low base while it won’t continue at this rate it implies it could have revenue from $15 – 90million in just two years time (probably lower end). While speculative and currently immaterial if this was a separate listed entity on the ASX it would probably trade for $200+ mil valuation.

New customer acquisition is accelerating as they learn how to sell to smaller entities. This will help to mitigate the significant customer concentration risk.

Competition still not a major issue with essentially a competitive duopoly in place between Appen and Lionbridge.

No concerns raised invalidate my thesis for Appen and my expectation is medium to strong sustainable growth for at least the next five years. No view on the AUD.

I would recommend listening to the call to get some extra explanation of the presented numbers from the company that can be found here.

In a world where data is the new oil Appen is the leading refinery. Raw unannotated data is about as useful as crude in your family car.

- Content relevance currently requires constant human annotated data to stay relevant (repeat business)
- Upside to language services due to big tech selling into a global market
- Computer vision + text and speech recognition requires lots of high quality annotated data as does machine learning 
- 8/10 top global companies using Appen - What other company has this spread of top quality companies on its books? Also use is still increasing rather than decreasing.
- Substantial free cash flow leading to acquisitions and revenue diversification

Risks:
- High customer concentration (loss of one major would be substantial impairment)
- Big tech automating data collection through machine learning making Appen's services redundant (unlikely to happen quickly)
- Acquisition integration issues
- Competition (Lionsbridge)

- AUD strength
 

Appen released its full year results today and there was a lot to like. Highlights for me include the following:

Core business revenue of 536 million a 47% increase on FY18. They further break this down into: Speech and image 67.7M (+32%) Content relevance 430M (+37%).

Figure 8 ARR of 33.7M (56% CAGR over last 4 years) – Acquisition now performing strongly and expected to positively contribute to EBITDA from second half 2020 but possibly even in current half.

Underlying EBITDA growth of 42% to $101M excludes figure 8 contribution.

EBITDA margin slightly lower from 19.6% to 18.8%. On call stated one large customer had agreed to fixed pricing for next five years which management were happy with stating that it relieved margin pressure although said they would be lower in first half due to investment in sales and marketing before returning to mid to high teens in second half. Interesting according to CEO business efficiencies will be reinvested in the business rather than allowing significant margin expansion.

Very positive commentary regarding growth avenues with two US facilities opening up allowing work with more sensitive US data. China office sounds well set up but low expectations being set for growth there. US government work and LIDAR 3d data set annotation also mentioned as areas for new contracts (lookout Pointerra).

Regarding corona virus seemingly little expected impact so far...

The Company’s full year underlying EBITDA for the year ending Dec 31st 2020 is expected to be in the range $125M - $130M (at A$1 = US$0.70, Feb-Dec 2020)

Appen is aquiring Figure-eight for US$175 million or 5.7x FY18 revenue plus a further US$60 - $125 million depending on performance in March 2020.

Is this a good thing?

Well at first glance I was concerned, figure-eight is a loss making company with negative 17mil in EBITDA in thier most recent results. This obviously instantly takes a swipe at the bottom line whist simutaneously diluting my shares by ~10%. Not ideal. So why are they doing it? Management has a good history of value creation and the past three aquisitions and mergers have been well executed and value accreditative. 

So what is figure-eight? 

A company that apparently has a machine learning platform (that puts a human in a loop or something) which magically transforms unstructured text, image, audio, and video data into customized high-quality training data (impression is sort of what appen does but more autonomous)

At a glance

- Revenue 42mil (51% CAGR last 4 years)

- ARR 27mil (68% CAGR last 4 years)

- 17mil negative EBITDA

- They expect cashflow breakeven in 2020 (Before synergies)

- Basically it seems figure-eight have a lot of annotation tools that appen want PLUS customers that appen want PLUS the ever loved recurring revenue model with consumer facing software (SaaS) which apparently attracts smaller customers as well

- Figure-eight currently outsource for thier annotation crowds so thats a pretty obvious synergy for them

So... I wouldn't panic about the short term hit to bottom line. I think this looks like a good idea for long term value creation. Also given managments track record and skin in the game I am inclined to give them a bit of slack on this one and assume this is a good idea.

Finally Appen's current valuation is really dependant on its growth rate and:

A high growth company + a high growth company in a high growth sector a slow growth company it does not make.