Developer & DevOps
Basic multi-page website scraper with .AI
16 nodes
229
145
Automatic trigger
Workflow Description
An automation template that extracts data from multi-page websites using HTTP requests and batch processing, storing results in Google Drive. Includes XML parsing, data filtering, and rate limiting between requests to prevent blocking.
How it works
- 1.Trigger the workflow manually to begin scraping
- 2.Split the URL list into batches using Split In Batches node
- 3.Send HTTP requests for each page with delays between calls
- 4.Parse HTML/XML responses and extract relevant data fields
- 5.Apply filters to retain only matching or valid rows
- 6.Save final results to a file in Google Drive
Use cases
- Extract product prices from multi-page e-commerce sites
- Gather real estate and classified listings from directory websites
- Monitor content updates and news from multiple web sources
Requirements
- Active Google Drive account for saving results
- List of target webpage URLs to scrape
- Basic knowledge of HTML structure and filtering options
Service Value
Ready-made workflow template for automation delivery and service execution.
Apps Used
Split In Batches
Google Drive
Details
Trigger
Automatic trigger
Nodes
16
Apps
2
Views
229
Downloads
145
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (16)
/
Sticky Note
Sticky Note
When clicking ‘Test workflow’
Manual Trigger
Loop Over Items
Split In Batches
Wait
Wait
Limit
Limit
Get List of Website URLs
HTTP Request
Convert to JSON
Xml
Create List of Website URLs
Split Out
Filter By Topics or Pages
Filter
Set Website URL
Set
Jina.ai Web Scraper
HTTP Request
Save Webpage Contents to Google Drive
Google Drive
Extract Title & Markdown Content
Code
Sticky Note1
Sticky Note
Sticky Note2
Sticky Note
Sticky Note3
Sticky Note