Spicrawl

v0.1.3

Last updated Oct 27, 2026

Spicrawl n8n node for web scraping: scrape websites to Markdown, HTML or JSON, extract data with CSS selectors, batch-scrape URLs and use it as an AI Agent tool

42 Weekly Downloads

1,005 Monthly Downloads

View on NPM GitHub Repository

Included Nodes

Spicrawl

Description

n8n-nodes-spicrawl: web scraping node for n8n

![smithery badge](https://smithery.ai/servers/spicrawl/web-scraper)

How do I scrape a website in n8n? Install the community node n8n-nodes-spicrawl, add the Spicrawl node to a workflow, paste a Spicrawl API key into its credential, enter the page URL and run the node. It returns the page as Markdown, HTML, text, a PDF or JSON. Set Rendering to JavaScript or Auto for pages built by JavaScript, add Extract Fields (JSON) to pull named fields out with CSS selectors, or use Batch Scrape to scrape a list of URLs as one job. An AI Agent node can call it as a tool. The full guide is at docs.spicrawl.com/agents/n8n.

This is an n8n community node. It lets you use Spicrawl in your n8n workflows.

Spicrawl is a web scraping API. It fetches any URL and returns it as clean markdown, HTML, text, a PDF or structured JSON. It renders JavaScript when a page needs it, and runs large URL lists as batch jobs.

n8n is a fair-code licensed workflow automation platform.

Also available as an MCP server: Smithery · MCP Registry

Quick start
Use cases
Installation
Operations
Credentials
Usage
Examples
FAQ
Troubleshooting
Compatibility
Resources
Version history

Quick start

1. Install the node. The node is verified by n8n. In the node panel, search Spicrawl and select Install node, on n8n Cloud or self-hosted. On self-hosted n8n you can also open Settings > Community Nodes > Install, enter n8n-nodes-spicrawl and accept the prompt. More in Installation.
2. Add your API key. Create a key at app.spicrawl.com with the scrape scope, then add a Spicrawl API credential in n8n and paste the key into API Key.
3. Scrape a page. Add a Spicrawl node, set URL to the page, leave Response Format on Markdown and select Execute step. The page is in the content field of the output item.

For a ready-made workflow, import an example from Examples.

Use cases

Feed web pages to an AI model or a RAG pipeline. Markdown output is much smaller than HTML, so it costs fewer tokens to send to a model.
Give an AI Agent a web page reader. The node can be used as an AI Agent tool, so the agent supplies the URL and reads the page itself.
Extract fields from product, listing or article pages. Map field names to CSS selectors and get structured JSON back under data. Run it on a schedule to track changes over time.
Scrape a list of URLs from a spreadsheet, feed or database. Batch Scrape takes up to 10,000 URLs from a previous node and returns one item per URL.
Archive a page. Save a page as HTML, a PDF or a screenshot and send it to Google Drive, email or a file.

Installation

The node is verified by n8n, so it installs from inside n8n on both n8n Cloud and self-hosted n8n:

1. In a workflow, open the node panel and search for Spicrawl.
2. Select the result with the Verified badge (actions Scrape page and Batch scrape pages), then select Install node.

On self-hosted n8n you can instead install the package by name. As the instance owner, open Settings > Community Nodes > Install, enter n8n-nodes-spicrawl and accept the prompt. See the installation guide in the n8n community nodes documentation.

Operations

The Spicrawl node has two operations.

| Operation | What it does |
|—|—|
| Scrape URL | Fetches one page and returns it as markdown, HTML, text, a PDF (binary) or a JSON envelope. Can also extract fields with CSS selectors, return page metadata and links, and capture a screenshot (binary). |
| Batch Scrape | Submits a list of URLs as one job, waits for it to finish, then returns one output item per URL. Takes the URLs from a text list or from the input items. |

The node is marked as usable as a tool, so an AI Agent node can call both operations.

Rendering tiers

Every page is loaded by the cheapest tier that can do what you asked for.

Rendering: Auto (Scrape URL only) starts with fetch and escalates to js only if the page needs it. You pay for the tier that worked. See Credits for current prices. A request that fails costs nothing.

Scrape URL parameters

Options:

Batch Scrape parameters

| Parameter | API field | Notes |
|—|—|—|
| URL Source | | Text List (one URL per line) or Input Items (one URL from each input item, default expression {{ $json.url }}). Up to 10,000 URLs. |
| Response Format | response_format | Markdown (default), HTML or Text. |
| Rendering | js_render | Fetch or JavaScript. |
| Simplify | | On by default. |

Options: Block Resources, Concurrency, Credit Budget, Custom Headers, Failure Threshold, Job Name, Main Content Only, Max Attempts, Max Cost Per Page (Credits), Max Wait (Minutes), Poll Interval (Seconds), Proxy URL, Wait After Load (Ms), Wait For Selector.

Batch jobs run each URL as one fetch or one render. Extraction, screenshots, PDFs, actions, Auto rendering and the result cache are not available in a batch; the API refuses them with a 400 that names each offending field, and nothing is charged. Use Scrape URL for those.

Batch Scrape runs once for all input items, not once per item. The job is polled until it finishes (or Max Wait is reached), then every finished page is returned.

Credentials

You need a Spicrawl account and an API key.

1. Sign in at app.spicrawl.com and open API Keys.
2. Create a key. Live keys start with spicrawllive and spend credits. Test keys start with spicrawltest.
3. Give the key the scrape scope for Scrape URL and the batch scope for Batch Scrape.
4. In n8n, create a Spicrawl API credential and paste the key into API Key.

n8n tests the credential by listing your recent requests, which works with any valid key. See Authentication.

Usage

Output of Scrape URL

With Simplify on you get these fields, when they apply:

Turn Simplify off to also get engine (the tier that served the page), creditsremaining, cachestate, proxy_source and the rest of the JSON envelope. PDFs and screenshots always arrive as binary data, whatever Simplify is set to.

Output of Batch Scrape

One item per finished URL, in submission order:

Failed URLs are returned too, so filter on status with an IF or Filter node. If nothing finished, the node returns the job itself so you can see why.

Batch results stay readable for 72 hours. If the job does not finish within Max Wait, the node stops with an error that names the job ID; the job keeps running and finished pages are still charged.

Notes

Scraping is not idempotent: every call is charged, including cache hits. Use n8n’s Retry On Fail for errors marked retryable, not for successful calls.
Errors from the API are shown with their stable code, for example ERR::AUTH::INSUFFICIENT_SCOPE, plus the API’s hint and the request ID.
Use the markdown format when the output goes to an AI model: it is much smaller than HTML.

Examples

Import an example with Workflow menu > Import from URL or clipboard. Select your credential on the Spicrawl node afterwards.

Scrape a page to markdown

{
  "nodes": [
    {
      "name": "When clicking Test workflow",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [0, 0],
      "parameters": {}
    },
    {
      "name": "Spicrawl",
      "type": "n8n-nodes-spicrawl.spicrawl",
      "typeVersion": 1,
      "position": [220, 0],
      "parameters": {
        "operation": "scrape",
        "url": "https://example.com",
        "responseFormat": "markdown",
        "rendering": "fetch",
        "simplify": true,
        "options": {}
      }
    }
  ],
  "connections": {
    "When clicking Test workflow": {
      "main": [[{ "node": "Spicrawl", "type": "main", "index": 0 }]]
    }
  }
}

Extract product fields from a JavaScript page

Set Rendering to JavaScript, then add the options Wait For Selector (.price) and Extract Fields (JSON):

{
  "title": "h1",
  "price": ".price",
  "images": { "selector": ".gallery img", "kind": "list", "output": "attr", "attribute": "src" },
  "next_page": "a.next@href"
}

The fields come back under data:

{
  "url": "https://example.com/products/42",
  "status": 200,
  "credits": 3,
  "data": { "title": "Walnut Desk", "price": "$349.00", "images": ["https://example.com/img/desk-1.jpg"] },
  "emptyfields": ["nextpage"]
}

Take a screenshot

Add the options Screenshot and Screenshot Full Page. The image arrives in the binary property screenshot, ready for a Write Binary File, Google Drive or Send Email node. The call uses the browser tier.

Batch scrape a list of URLs

{
  "nodes": [
    {
      "name": "When clicking Test workflow",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [0, 0],
      "parameters": {}
    },
    {
      "name": "Spicrawl",
      "type": "n8n-nodes-spicrawl.spicrawl",
      "typeVersion": 1,
      "position": [220, 0],
      "parameters": {
        "operation": "batch",
        "urlSource": "list",
        "urls": "https://example.com/products/1nhttps://example.com/products/2",
        "responseFormat": "markdown",
        "rendering": "js",
        "simplify": true,
        "batchOptions": { "jobName": "nightly-catalog", "concurrency": 20 }
      }
    }
  ],
  "connections": {
    "When clicking Test workflow": {
      "main": [[{ "node": "Spicrawl", "type": "main", "index": 0 }]]
    }
  }
}

To scrape URLs that come from a previous node (a Google Sheet, an RSS feed, a database), set URL Source to Input Items and point URL at the field, for example {{ $json.link }}.

Give an AI agent a web page reader

Add a Spicrawl node to an AI Agent node’s tools. The agent fills in the URL and picks the options it needs. Keep Response Format on Markdown, and set Max Cost (Credits) so one tool call can never spend more than you intend.

FAQ

Can it scrape JavaScript-rendered websites?

Yes. Set Rendering to JavaScript to render the page before it is returned, and add Wait For Selector or Wait After Load (Ms) when content appears late. Auto starts with a plain fetch and moves to JavaScript rendering only if the page needs it, and you pay for the tier that worked. A plain fetch costs 1 credit and JavaScript rendering costs 3, so use Fetch for server-rendered pages. Some protected sites refuse automated requests; the Proxy URL option sends the request through your own proxy. Always check status in the output, because a blocked page is still a successful node run.

Can I use it as an AI Agent tool?

Yes. The node is marked as usable as a tool, so an AI Agent node can call both Scrape URL and Batch Scrape. On self-hosted n8n, set the environment variable N8NCOMMUNITYPACKAGESALLOWTOOL_USAGE=true and restart n8n, otherwise n8n does not offer community nodes as agent tools. Keep Response Format on Markdown and set Max Cost (Credits) so one tool call cannot spend more than you intend. See Use Spicrawl as an AI Agent tool.

Is there a free tier?

Yes. Every Spicrawl organization gets 1,000 free credits per month. A plain fetch costs 1 credit and JavaScript rendering costs 3, so a month covers up to 1,000 plain pages or 333 JavaScript-rendered pages. A request that fails costs nothing. See Credits for the current price list.

How do I extract specific fields from a page?

Add the option Extract Fields (JSON) to Scrape URL and give it a map of field name to CSS selector, for example {"title": "h1", "price": ".price"}. The values come back under data, and empty_fields lists every selector that matched nothing, which is the first sign a site changed its markup. Extraction costs no extra credits. For pages built by JavaScript, set Rendering to JavaScript and add Wait For Selector. See Extract product fields from a JavaScript page.

How is this different from the HTTP Request node?

The HTTP Request node returns the raw response and does not run JavaScript. The Spicrawl node converts the page to Markdown, text or extracted fields, can render JavaScript, and runs a list of URLs as one batch job.

Troubleshooting

The full list is in the error catalogue.

Compatibility

Developed against n8n 2.x with n8n-workflow 2.42 (n8nNodesApiVersion 1). The package has no runtime dependencies.

Resources

Spicrawl in n8n: the full guide
Source code and issues
n8n community nodes documentation
Spicrawl documentation
Scrape API reference
Batch jobs guide
Structured data extraction guide

Version history

See CHANGELOG.md.

0.1.2: Documentation and package metadata for discoverability. No code change.
0.1.1: Source moved to its own repository, with the package at the repository root. No code change.
0.1.0: Scrape URL and Batch Scrape.

License

MIT