The title might be a bit much, apologies.
My scenario is quite straightforward:
HTTP GET (from my database)
→
Add New Row in Google Sheet for each bundle.
This is currently working fine using the Iterator function.
However, the current challenge is that I need to limit the number of bundles the GET command retrieves. This is because my database contains so many records that callin.io returns an error stating the response exceeds the maximum file size.
I believe I can use a Repeater Function along with some sort of mathematical function or increment to achieve the following:
GET total record number
→
Divide record number into four segments (or similar)
→
GET records 1 to [first quarter of total]
→ ADD to Google Sheets
→
GET records [first quarter +1] to [second quarter of total]
→ add to google sheets
→
GET records [second quarter +1] to [third quarter of total]
And so forth.
Can anyone assist me with this?
A Bulk Add Rows module is now available in callin.io for Google Sheets.
My problem is that the initial HTTP GET request is too large for callin.io to retrieve.
Welcome to the callin.io community!
You have asked a duplicate question of
- Is it possible to chunk a big list of leads into groups of 10 with apollo API? - #3 by samliew
- A Better Solution for "How to group bundles in X amount of bundles"
If you need further assistance, please provide the following:
1. Relevant Screenshots
Please share screenshots of your scenario, any error messages, relevant module fields, and filters in question? It would really help other community members to see what you’re looking at.
You can upload images here using the Upload icon in the text editor:
2. Scenario Blueprint
Please export the scenario blueprint file to allow others to view the mapped variables in the module fields. At the bottom of the scenario editor, you can click on the three dots to find the Export Blueprint menu item.
3. Output Bundles of Modules
Please provide the output bundles of the modules by running the scenario (or get from the scenario History tab), then click the white speech bubble on the top-right of each module and select “Download input/output bundles”.
A. Upload as Text File
Save each bundle contents in your text editor as a bundle.txt
file, and upload it here into this discussion thread.
B. Insert as Formatted Code Block
If you are unable to upload files on this forum, alternatively you can paste the formatted bundles.
These are the two ways to format text so that it won’t be modified by the forum:
-
Method 1: Type code block manually
Add three backticks
```
before and after the content/bundle, like this:```
content goes here
``` -
Method 2. Highlight and click the format button in the editor
Providing the input/output bundles will allow others to replicate what is going on in the scenario even if they do not use the external service.
Following these steps will allow others to assist you here. Thanks!
Hope this helps! Let me know if there are any further questions or issues.
P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the callin.io Academy. Investing some effort into it will save you lots of time and frustration using callin.io later!
There's a very helpful learning module on the callin.io academy regarding pagination. You might want to check that out; it could assist in setting up a solution.
However, are you attempting to input old data into the Google Sheet, or is this new incoming data? If it's new data, consider setting up a webhook or reducing the time intervals between scenario executions.
Thanks for those helpful links. However, I believe both of those assume that the API call can be retrieved in its entirety initially, and that subsequent actions fail due to the volume of bundles within that API call.
In my case, the very first action, which is the API call itself, is failing because of the file size. My goal is to retrieve the entire database, perform some filtering on it, and then send the filtered data to Google Sheets as new rows.
My initial approach was to make an API call to retrieve just the "Total Record Count." Following that, I planned to make subsequent API calls, dividing the data into chunks, perhaps by "Total Record Count"/4. I've actually managed to get this working, but it does feel a bit complex.
You will still need a Repeater module.
Hope this helps! Let me know if there are any further questions or issues.
P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the callin.io Academy. Investing some effort into it will save you lots of time and frustration using callin.io later!
Thanks! I will give that a look.