Tool Calling Issue with Nvidia/llama-3.1-nemotron-nano-4b-v1.1

Nvidia/llama-3.1-nemotron-nano-4b-v1.1 Tool Calling Issue

I am encountering an issue with the nvidia/llama-3.1-nemotron-nano-4b-v1.1 model within a callin.io AI Agent workflow. The model is not correctly executing the tools connected to the agent. Instead of triggering the tool, it outputs the tool call information as a plain text string in the chat response.

I have tried using this model in two different configurations, both resulting in problems:

Using NVIDIA NIM via the OpenAI Node:

When connecting to NVIDIA NIM’s OpenAI-compatible endpoint, the model correctly identifies the intended tool but fails to execute it. It returns a message like "OOLCALL>[{"name": "Database_FastAPI", "arguments": {"input": "your-policy-number-here"}}]</TOOLCALL" instead of triggering the actual tool node. This is the primary issue, as illustrated in the attached screenshots.

Using Ollama:

I utilized a community version of the model (avil/nvidia-llama-3.1-nemotron-nano-4b-v1.1:thinking-off) with the Ollama Chat node. This approach yields partial success:

1. It can successfully invoke a single RAG tool when it's the only tool configured.
2. However, when multiple tools are connected (e.g., RAG plus a custom tool), it exhibits the same failure as the NIM method, outputting the tool call as a string.
3. Furthermore, despite the model version name (thinking-off) and explicit instructions in the prompt to be concise, it continues to output "thinking" or "reasoning" steps before providing the final answer.

The desired behavior is for the callin.io agent to recognize the model's intent to use a tool, parse the tool name and arguments, and then execute the corresponding tool within the workflow.

My workflow with NIM credentials in openai node:

It’s Output:

My Workflow with ollama community model is the same as I posted above. (I have used OpenAI node for this too ):

It’s Output:

Single tool: Tool call success. However, it's operating in thinking mode despite being instructed otherwise in the prompt and defining the appropriate payloads.

Multiple tools: Tool call failure

Information on your callin.io setup

• callin.io version: 1.93.0
• Database (default: SQLite): SQLite
• callin.io EXECUTIONS_PROCESS setting (default: own, main): Own
• Running callin.io via: npm (exposed it with ngrok)
• Operating system: Linux ubuntu