Process large datasets in controlled batches with conditional logic
Workflow Description
Workflow that receives bulk data via manual trigger and HTTP request, divides it into sized batches, applies conditional logic and custom code to each batch, implements wait intervals to prevent system overload, and exports processed results as a downloadable file.
How it works
- 1.Receive data through manual trigger and HTTP request endpoint
- 2.Divide incoming data into predefined batch sizes
- 3.Execute conditional checks and custom code processing per batch
- 4.Insert wait intervals between batch cycles to manage load
- 5.Convert and export final results to file format
Use cases
- Process customer lists or product catalogs in large volumes without service disruption
- Export refined datasets from databases as structured file batches
Requirements
- Active n8n account with API access permissions
- Basic understanding of JSON data format and file output structures
Service Value
Ready-made workflow template for automation delivery and service execution.
Apps Used
Details
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (21)
Set Fields
Set
When clicking ‘Start’
Manual Trigger
Check Task Status
HTTP Request
Create Task
HTTP Request
If
If
Wait
Wait
Get RAW Audit Data
HTTP Request
Extract URLs
Code
Loop Over Items
Split In Batches
Query GSC API
HTTP Request
Wait1
Wait
Map GSC Data to URL
Set
Merge GSC Data with RAW Data
Code
Build Report Structure
Code
Generate HTML Report
Code
Extract 404 & 301
Code
Loop Over Items1
Split In Batches
Map URLs Data
Code
Get Source URLs Data
HTTP Request
Sticky Note
Sticky Note
Download Report
Convert To File