Ipsum
Google’s NotebookLM tool has been built as a note taking tool with an AI large language model at its core.
It’s an experimental project but really interesting for combing through lots of documents and putting ideas together. The focus at the moment seems to be on helping writers and academics but I wondered if I could use it as a tool for researching public companies.
The basic interface allows you to create a notebook and upload a number of sources, such as PDFs, Google drive documents, images. You can also paste a web URL and it will pull in the content from that page.
Each source automatically gets an AI generated summary, which is handy. The real focus is the chat prompt at the bottom of the screen. This is like many AI powered chat bots but with NotebookLM the content of your sources are used for generating responses. Answers include references to which sources provided the relevant information to answer the query which is really useful. Hover your mouse over the reference and you can quickly review the references source and confirm the AI’s response is accurate.
I tried it out by creating a notebook and uploading a few PDF’s related to Maas Group Holdings (ASX: MGH) which has been on my research list for a while.
So far I’m impressed. The responses don’t come back as quickly as OpenAI’s ChatGPT but I appreciate having the source listed.
When I asked a question about information that wasn’t available the AI responded correctly, and didn’t attempt to make up an answer.
I the uploaded the FY23 accounts and the it returned the correct answer.
The Notebook guide panel has some buttons with prebuilt prompts, such as study guide and table of contents. Not sure how useful these will be but the Timeline one looks interesting. It creates a bullet point of key details in chronological order. It’s not very good at deciding what is important, and doesn’t include the source references like other responses, but I like the idea of being able to get a quick chronological overview. Something to look into further.
I’m looking forward to playing with this more. I’ve love to load in earnings transcripts and related data sources and see what I can come up with.
Ipsum
@mikebrisy It's a great point. The quality of responses can vary widely and the responses won't necessarily flag if there is any confusion.
I thought your example was great, and decided to try it myself but using an alternative process.
I found the presentation on YouTube (I assumed it was this one) and extracted the audio. I then ran it through Whisper (OpenAI's audio transcription tool) to generate a transcript. The transcript seemed reasonable but one limitation is that it doesn't break format the text according to speaker. It's just sentence after sentence of what was spoken.
This transcript was better than Youtube's, at least for this segment of the recording:
And our capital expenditures are up a couple hundred thousand dollars, which have been intact.
I uploaded this transcript to NotebookML and tried it out with "Summarise capital expenditure". The response was:
Stealth Group Holdings' capital expenditures are up to $200,000.1 The company has been disciplined in this approach as profitability is increasing.1
This is... kind of better? But still not great. The response has used a fairly specific figure, while the presenter used more vague terminology. The response has also confused "up a" with "up to".
Now in NotebookML's defence I appreciate how quick it is to verify the source of a response. Just hover over the reference and you can scan the source document. Here I can see the $200,000 figure wasn't used verbatim:
But overall I think my quick experiment supports what you're saying: quality of output still varies widely. I still won't put a lot of trust in AI response without verifying. But I think these tools can be useful as a way to find where to start looking for certain ideas or topics.
Anyway, appreciate your reply and sharing your experience!