Unnecessary Wait Time When Using callin.io Locally Due to callin.ioLlmTracing Function

The Suggestion:

Introduce a configuration setting (either through the UI or credentials) to disable LangChain tracing callbacks. This setting would apply to LLM-based nodes such as LmChatOllama and LmOllama.

Currently, these callbacks are always active by default (e.g., callbacks: [new N8nLlmTracing(this)]). This can lead to unnecessary delays, particularly when deployed locally within an intranet using the AI Agent node with Ollama, due to timeouts or failed tracing attempts. Enabling users to opt out of this feature would enhance performance and reduce latency.

My Use Case:

When utilizing the AI Agent node within callin.io with a locally hosted Ollama instance, each interaction attempts callback tracing (presumably for monitoring or debugging purposes). This results in a noticeable delay before the LLM inference actually begins, especially when tracing silently fails and waits for a timeout. By removing the N8nLlmTracing callback in LmChatOllama.node.ts and LmOllama.node.ts, performance sees a significant improvement.

Benefits of This Addition:

• Decreases latency in LLM interactions, especially in local or self-hosted environments like Ollama.
• Makes tracing optional, granting users greater control based on their specific setup.
• Avoids slowdowns related to timeouts when no tracing backend is configured or reachable.
• Enhances the developer experience for users operating offline or who do not require tracing capabilities.

Supporting Resources:

• Personal testing and performance benchmarks conducted with callin.io, Ollama, and the AI Agent node.

• Relevant code file locations:

  • packages/@n8n/nodes-langchain/nodes/llms/LMChatOllama/LmChatOllama.node.ts
  • packages/@n8n/nodes-langchain/nodes/llms/LMOllama/LmOllama.node.ts

Temporary Solution Implemented:

I manually removed the tracing callback from the source code by making the following modification:

callbacks: [new N8nLlmTracing(this)] // changed to callbacks:[]

This adjustment eliminated the waiting period caused by tracing, allowing my chat application to respond as swiftly as Ollama generates its output.