Troubleshooting: Locally hosted LLM unable to call tools

Hello,

One potential reason for this issue might be that many local LLMs aren't sophisticated enough to recognize when they need to invoke a tool and, consequently, do not support tool usage.

Have you verified if the model you're currently using supports tool use?

Please see the link below for Llama models that support tool calls.

https://www.perplexity.ai/search/which-llama-models-support-too-AdeJi20YR1GXDzwxIMOwwg

Llama Models That Support Tool Calls

Several Llama models support tool calling (also known as function calling), enabling them to interact with external APIs, functions, or services. Here's a breakdown of which Llama models offer this capability and the types of tool calling they support:

Official Meta Llama Models

• Llama 3.1

  • Supports JSON-based tool calling natively.
  • Widely implemented in platforms like Ollama and Groq, facilitating agentic automation and integration with external tools or APIs1234.
  • Available in various parameter sizes (e.g., 8B, 70B, 405B).

• Llama 3.2

  • Builds upon 3.1 with continued support for JSON-based tool calling.
  • Introduces “pythonic” tool calling, a more flexible and Python-friendly format1.

• Llama 4

  • Supports both JSON-based and the new pythonic tool calling format.
  • It's recommended to use the pythonic tool parser for optimal results.
  • Supports parallel tool calls, a feature not present in Llama 3.x1.

Community and Fine-Tuned Models

• Fine-tuned Llama 3 Models

  • Community projects have fine-tuned Llama 3 (e.g., Llama3-8b-instruct) for enhanced function/tool calling, including LoRA adapters and quantized versions for efficient local deployment56.
  • These fine-tuned models are trained on datasets specifically designed for function calling tasks and are available in different formats (16-bit, 4-bit, GGUF for llama.cpp, etc.).

• TinyLlama

  • A smaller, fine-tuned variant with tool/function calling support, suitable for resource-constrained environments6.

Comparison Table

*Community fine-tuned models, not official Meta releases.

Key Points

• Llama 3.1, 3.2, and 4 all support tool calling, with enhanced capabilities and flexibility in newer versions123.
• JSON-based tool calling is the standard across all models, while pythonic tool calling is introduced in 3.2 and recommended for Llama 41.
• Parallel tool calls are exclusively supported in Llama 41.
• Fine-tuned models such as those from the “unclecode” repository extend tool calling capabilities to smaller or more specialized Llama variants56.

In summary, if you require tool calling support, opt for Llama 3.1 or a later version. For advanced features like pythonic tool calling and parallel execution, Llama 4 is the recommended choice. Fine-tuned community models are also available for specific use cases or lightweight deployments.

1. Tool Calling - vLLM
2. Tool support · Ollama Blog
3. Tool Calling in Llama 3: A Step-by-step Guide To Build Agents - Composio
4. https://www.reddit.com/r/LocalLLaMA/comments/1eaztwv/quick reviewofllama31toolcalling/
5. unclecode/llama3-function-call-lora-adapter-240424 · Hugging Face
6. unclecode/tinyllama-function-call-Q4KM_GGFU-250424 · Hugging Face
7. unclecode/llama3-function-call-Q4KM_GGFU-240424 · Hugging Face
8. https://llama.developer.meta.com/docs/guides/tool-guide/
9. Tool calls in LLaMa 3.1 - Docs - Braintrust
10. Llama 4 | Model Cards and Prompt formats
11. Llama 3.3 | Model Cards and Prompt formats
12. Tools - LlamaIndex
13. okamototk/llama-swallow
14. Function Calling — NVIDIA NIM for Large Language Models (LLMs)

Hi,

Thank you for your prompt response.

If you review the model card from NVD, it explicitly states that the model is trained for tool calling.

I also verified its capability by switching the Agent Type to “OpenAI Functions Agent”. With this agent type, I'm encountering a 400 error (no body) from callin.io.

This is the console log when using the Model in the “OpenAI Functions Agent”:

2025-06-27T11:08:45.467Z | error | 400 status code (no body) {"file":"error-reporter.js","function":"defaultReport"}
2025-06-27T11:08:45.467Z | debug | Running node "AI Agent" finished with error {"node":"AI Agent","workflowId":"jOqu92akylxZQm06","file":"logger-proxy.js","function":"exports.debug"}
2025-06-27T11:08:45.467Z | debug | Executing hook on node "AI Agent" (hookFunctionsPush) {"executionId":"6809","pushRef":"wvad9rmsml","workflowId":"jOqu92akylxZQm06","file":"execution-lifecycle-hooks.js"}
2025-06-27T11:08:45.468Z | debug | Pushed to frontend: nodeExecuteAfter {"dataType":"nodeExecuteAfter","pushRefs":"wvad9rmsml","file":"abstract.push.js","function":"sendTo"}
2025-06-27T11:08:45.468Z | debug | Workflow execution finished with error {"error":{"level":"warning","tags":{},"context":{},"functionality":"configuration-node","name":"NodeApiError","timestamp":1751022525464,"node":{"parameters":{"notice":"","model":{"__rl":true,"value":"nvidia/llama-3.3-nemotron-super-49b-v1","mode":"list","cachedResultName":"nvidia/llama-3.3-nemotron-super-49b-v1"},"options":{}},"type":"@n8n/n8n-nodes-langchain.lmChatOpenAi","typeVersion":1.2,"position":[-840,-320],"id":"cec34fcd-ddfd-4bcb-b4bd-b97031e8ee17","name":"Local","notesInFlow":true,"credentials":{"openAiApi":{"id":"dX1EaCNOPnvtPwDG","name":"Local Reasoning Model"}}},"messages":["400 status code (no body)"],"httpCode":"400","description":"400 status code (no body)","message":"Bad request - please check your parameters","stack":"NodeApiError: Bad request - please check your parametersn    at Object.onFailedAttempt (/usr/local/lib/node_modules/n8n/node_modules/.pnpm/@n8n+n8n-nodes-langchain@file+packages+@n8n+nodes-langchain_9ca6f82764a6c40719e9f8a538948cbd/node_modules/@n8n/n8n-nodes-langchain/nodes/llms/n8nLlmFailedAttemptHandler.ts:26:21)n    at RetryOperation._fn (/usr/local/lib/node_modules/n8n/node_modules/.pnpm/p-retry@4.6.2/node_modules/p-retry/index.js:67:20)n    at processTicksAndRejections (node:internal/process/task_queues:105:5)"}},"workflowId":"jOqu92akylxZQm06","file":"logger-proxy.js","function":"exports.debug"}

!!! IMPORTANT
The part that is particularly interesting is that when I use the Plan & Execute Agent with the model and include a Wikipedia tool, it successfully utilizes the tool and returns the result to the user.

The tests above suggest that the model is theoretically capable of calling tools, but there might be an issue elsewhere that I'm unable to identify.

Questions that have arisen from this:

• Is there a difference in how tools are invoked between the Plan & Execute Agent, OpenAI Functions Agent, and Tools Agent?
• Which Agent would be the most appropriate for this scenario (given that the llama3.3 model uses the OpenAI API standard)?

If I can provide any further information or debugging logs, please let me know, and I'll be happy to assist.

Thank you for providing information. From the model card, it appears viable for tool calling. Additionally, testing with the Plan and Execute Agent type resulted in successful usage of the Wikipedia tool. Issues arise with the Tools Agent or OpenAI Functions Agent.