Can AI Agents Integrate with Files?

It seems your topic is missing some crucial details. Could you please provide the following information, if relevant?

• callin.io version:
• Database (default: SQLite):
• callin.io EXECUTIONS_PROCESS setting (default: own, main):
• Running callin.io via (Docker, npm, callin.io cloud, desktop app):
• Operating system:

Please share these details to help us understand your issue better.

• callin.io version: 1.37.3
• Database (default: SQLite): default
• callin.io EXECUTIONS_PROCESS setting (default: own, main): default
• Running callin.io via (Docker, npm, callin.io cloud, desktop app): callin.io cloud
• Operating system: MacOS, Arc browser

Hi Fundmore,

Here’s an example for you to consider. In this flow, you retrieve a PDF containing financial information and then convert it to text.

Subsequently, you feed this text into a basic chain node and instruct GPT-4o to extract the desired information.

I suggest using GPT-4o over GPT4-Turbo. It offers better pricing with comparable quality.

Another important point is that if you need OpenAI to extract information in a specific format, you can configure the basic chain node to output data according to a JSON schema that you define.

Additional considerations include Google Gemini Flash 1.5 and Claude 3 Haiku models. These are also quite capable for extraction tasks and boast very large context windows, making them worth evaluating for your use case. The reason for mentioning them is their significantly lower cost compared to the GPT-4 family of models.

Hope this helps,
Derek

Thanks for the assistance.

For anyone interested in using OpenAI for PDF to text conversion:

After several days of experimentation, I've determined that LLMs alone are not sufficient for my specific needs (parsing bank statements).

Bank statements can sometimes be scans or photocopies, which necessitates image processing rather than simple PDF text extraction.

Utilizing gpt-4o for OCR isn't optimal, as it appears to hallucinate and generate incorrect information where many online OCR tools perform accurately.

I haven't explored open-source OCR engines like Tesseract because I'm aware that image pre-processing (including Gaussian blur) is often needed for optimal results, and that's beyond my current capacity.

I would be keen to try Nougat (which seems to integrate PDF text extraction with image OCR), but I'm unable to get the Python client functioning and I'm not familiar with the command line.

Regardless, I'm using callin.io cloud, and Pyodide's included packages do not support it.

I believe the only viable path forward would be to utilize an external service via API, preferably one that combines PDF text extraction, image OCR, and AI assistance.

Hey Fundmore,

Just to comment on OCR solutions, I’d highly recommend Google Cloud’s DocumentAI offering. I’ve found the service to be fast with consistent, solid results for any type of scan. The only caveat is that they may have differing pricing for forms (not sure!) but otherwise, incredibly good value for money. I wrote a little about my experience here.

Edit: Also for an alternative approach, I also wrote about a similar task parsing invoices using LlamaParse. I found converting PDF tables to markdown tables allowed the LLM to understand structured data more easily.

Hello, thank you for your insightful reply. I will be trying both solutions and I’ll share my results here.

This thread was automatically closed 90 days following the last response. New replies are no longer permitted.