Best AI for large documents?

[assuredmirror]

Are you using any AI tools that can handle large documents like ebooks? I think it requires RAG.
So you would input the PDF ebook or ebooks and the AI would answer questions exclusively from the content within (not from it's general knowledge).

 

[lazra]

grok is really good at this, it can be done with cloud uploads

 

[assuredmirror]

grok is really good at this, it can be done with cloud uploads

Last time I tried it did not good a good job, but I will try again. Thanks.

 

[ClaudeKing]

Notebook lm will be worth a look. I have uploaded several at once for it to use

 

[Dilana]

You can use Deepseek, i have used it with large documents, but yoy maybe need to adjust the ebook to make max 170 pages as from my expreince thats the max it can read, also to make sure uplaod the ebook and ask it what is the content of the last page on that ebook, and go look for his answer and your ebook to see at what apge he can read from your document, if still need to read more pages, you can edite your book to make eatch two pages in one page.

 

[ClaudeKing]

Also if you are using pdfs put them through a pdf shrinker first

 

[SMMRocket]

I've used Deepseek a few times to edit large files and long texts, and it worked quite well.

 

[medialounge]

I've used Deepseek a few times to edit large files and long texts, and it worked quite well.

Giving a plus 1 to deepseek
I had similar experiences as yours

 

[Saitama Agency LLC]

Are you using any AI tools that can handle large documents like ebooks? I think it requires RAG.
So you would input the PDF ebook or ebooks and the AI would answer questions exclusively from the content within (not from it's general knowledge).

ChatGPT works very well. For large datasets, I often use ChatGPT for processing, and it delivers quite stable results.

 

[sportsmen]

Are you using any AI tools that can handle large documents like ebooks? I think it requires RAG.
So you would input the PDF ebook or ebooks and the AI would answer questions exclusively from the content within (not from it's general knowledge).

There is no AI to "read" more than ~64 Kb at the moment. To process the ebook you need to spend arount 3 x [word number] tokens - that's totally out of reach for any existing AI.

 

[code API]

I develop that by integrating an advanced model in OpenAI API with ChromaDB for one of my clients' projects. We then indexed the PDF, Docx, and CSV files (about 1.2 Gigabytes total) into the AI memory.

The initial cloud storage for the data was AWS S3 bucket from which I indexed those data via a python script on the destination VPS. You had to tell the script what to index, how to sort and retrieve the data when needed. It was super fast, and faster than calling the OpenAI API at avery single request of data.

This is a solid RAG system. Took some time to build and a bit more time to get those data into the AI memory from S3 to ChromaDB but I thank myself for investing the time in learning and doing that, saving us hundreds of dollars monthly in API calls.

I have built five more RAG systems this same way for clients and that always beat the cost of API every single time. You may want to also retrieve the data via webhook and have it sent to your external server or client's server or simply request your indexed data with a simple AI prompt.

 

[bulkcloudaccs]

The best solution for large documents is RAG-based AI such as ChatGPT (file upload + only-doc mode), Claude (very large context), or self-hosted LlamaIndex / Haystack which respond only to input content from PDF/eBook.

 

[G33K]

Try GLM model, it has superior context window.

 

[flackz]

Google Gemini Pro has a context window of 1 million tokens.

 

[adwaitk007]

Tried notebook LM? It's built for document research.

 

[alga]

How about Claude? It can work with PDFs and handle what you need. It’s probably one of the best AI tools available right now. The free plan is limited, but it’s worth paying for.

 

[Starblazer]

Claude can process huge amount of data. I've used free version create output into datasets. I think you can use it to analyse ebooks too. You may need a pro version for larger files but it's definitely better than Chatgpt and Gemini for data processing.

 

[liomarver]

I just love NotebookLM, use it almost for everything

 

[yourhighness]

Are you using any AI tools that can handle large documents like ebooks? I think it requires RAG.
So you would input the PDF ebook or ebooks and the AI would answer questions exclusively from the content within (not from it's general knowledge).

I would say GPT and Claude works the best! I have been using them for a while.
You can give a try to Perplexity and Notion AI as well

 

[alienacademy]

I think Google's NotebookLM is best for this kind of work.
You can even put lengthy YouTube videos as a source and get your answers.
I even use it for writing video scripts.