Notifications
Clear all
How To
6
Posts
2
Users
0
Reactions
3
Views
Topic starter
Hello! I'm encountering an issue with my automation workflow on callin.io. My current setup involves: search Airtable → text aggregator → scraper → iterator → create Airtable record. The challenge is that I need to prevent duplicate scraped data from being uploaded to Airtable. I attempted to integrate a 'search Airtable' step, but I'm having trouble filtering out existing entries at the final stage. Could anyone offer a solution to ensure that only new scraped data is added to Airtable? Thank you!
Posted : 01/03/2024 8:40 am
Topic starter
I've been facing this challenge for quite some time now, and I'm hoping for some assistance.
Posted : 01/03/2024 9:51 am
Hi
You can upsert records (update/insert)
callin.io Airtable upsert documentation
You can search for the record id with the search record module by looking for some unique param
Posted : 01/03/2024 12:38 pm
Topic starter
Thanks for the response. Would it be feasible to only insert the new records and skip those whose names already exist in the table? Furthermore, this would result in fewer records being created, thus fewer operations being executed within callin.io.
Posted : 01/03/2024 12:57 pm
Without a unique identifier, it's impossible.
However, you can utilize list records via the "make an API call" module and then aggregate the data using filters in between to remove existing records. This will only cost 2 operations.
For more information regarding listing records, please refer to the Airtable documentation.
Posted : 01/03/2024 2:09 pm