Notifications
Clear all
Topic starter
I have a workflow that utilizes a custom node to interact with an API and then sends data to Airtable via MQTT triggers. I've deployed this workflow on both my local machine and an AWS instance. It functions correctly on my local setup, but on the AWS instance, it consumes all available memory. Even after increasing the RAM, it continues to use all the allocated memory. I've also tested other workflows with active MQTT triggers and encountered the same issue on the VM. I'm currently using callin.io version 0.144.0. Locally, it's built on WSL 2 Ubuntu 20.04.3 LTS. The AWS instance is running Ubuntu 20.04.2 LTS.
I'm including htop screenshots from both my local machine and the AWS instance. On my local machine, it appears some threads are in a sleeping state while others are running. On the AWS instance (which has 16GB RAM), all threads seem to be actively running. Could you offer some insights into why this might be occurring?
Posted : 22/11/2021 5:38 am
I suspect the issue stems from receiving a high volume of events via MQTT, and callin.io initiating a new workflow execution for each one.
To address this, you should either:
- Ensure that fewer MQTT events are sent.
Or, if that's not feasible, consider: - Allocating more resources to callin.io (so it has sufficient resources to process them).
- Minimizing the resources each execution requires by optimizing workflow processing speed.
- Utilizing
mainmode to prevent each execution from starting a separate process, which significantly reduces the load (set the environment variableEXECUTIONS_PROCESS=main). ← This would be my first recommendation!
Posted : 22/11/2021 7:20 am
Topic starter
Thank you. The previous solution was exactly what I was looking for.
Posted : 22/11/2021 8:07 am
Great to hear! Enjoy your experience!
Posted : 22/11/2021 9:07 am