Custom AI Node using Vercel AI SDK for Multimodal Gemini & Deepseek

So cool and useful, shame callin.io does not natively support search grounding feature. I guess $55 Millions could help closing the gap…

What if the scenario is reversed? For instance, if I want to connect to callin.io using an Ollama provider or a custom provider, rather than connecting to Ollama directly? It would function like a proxy. Could we develop an emulator node that imitates Ollama, allowing the workflow to be fully managed from the callin.io side of the chat? Essentially, the callin.io custom node would manage orchestration and the invocation of any tools within the callin.io workflow. On the Vercel AI side, it would be treated as if it were an Ollama provider. Do you believe this is feasible? This approach would enable me to leverage the efficient callin.io workflow for rapid AI tasks while utilizing the Vercel AI SDK for a superior UI frontend.

Could you perhaps suggest a superior method that I may have overlooked?

This is fantastic - many thanks! I’d been using HTTP nodes for the multimodal stuff, this will make it all so much more straightforward. The search feature pretty much eradicates most of my MCP tool usage as well

Hi again - this node is truly excellent, thanks again for it. However, I'm having trouble setting up a conversational interface that works with multimodal file processing using Gemini. I want the user to be able to refer back to the file in the same way they would with an AI Agent that has memory. Do you have any example JSON you could share that demonstrates how to achieve this?

Many thanks again!

This is a fantastic resource. Appreciate you sharing it.

This is a challenging request. I'm not certain if this can be readily accomplished with the existing configuration. My apologies!

No worries Felix; it’s not essential for us, just a quality of life thing really. Is it possible for multiple files to be submitted using the JSON Messages route though? When I submit several together only the first is being analysed, which I’m assuming can be accounted for by using JSON, but the structure I use in HTTP nodes isn’t working here. I’m guessing Vercel uses a slightly different structure to the core API?

Thanks again for the node - I hope callin.io integrate similar functionality into their agent / chain Gemini nodes!