What does the Magpie IQ Scraping API scrape?

The Scraping API runs scrapes on demand: you submit URLs or product IDs and download the raw results. It covers Shopee variant-sold data (up to 50,000 items per batch), Shopee product pages (single and bulk), Shopee search, category, and merchant listing pages, plus Tokopedia and Blibli product detail pages.

Which countries does the Magpie IQ Scraping API cover?

The Scraping API supports seven markets: Indonesia, Singapore, Thailand, Philippines, Malaysia, Vietnam, and Taiwan.

Scraping API · On-demand collection

Fresh marketplace
data, scraped
on demand.

Submit URLs or product IDs, we fetch them now, and you download the raw results. Shopee across seven markets, plus Tokopedia and Blibli product pages — on the same infrastructure that has powered the Magpie pipeline since 2020.

Get started API documentation →

cURL

              $ curl -X POST \
  "https://api.magpieiq.com/scraping/v1/shopee/variant_sold_v1/submit" \
  -H "Authorization: Bearer mgk_live_••••••••••" \
  -d '{
    "country": "id",
    "items": [
      { "shop_id": 178926468, "item_id": 21630167606 },
      { "shop_id": 90721822,  "item_id": 8420715441 }
    ]
  }'
            

              Retrieve · 200 OK
              ✓ job complete
            

              {
  "job_id":   "vs_9f31c2ae",
  "status":   "success",
  "progress": {
    "submitted": 2,
    "scraped":   2,
    "failed":    0
  },
  "download": {
    "url":        "https://storage.googleapis.com/…",
    "format":     "json",
    "expires_at": "2026-07-11T09:00:00Z"
  }
}
            

How it works

Submit. Approve. Poll. Download.

Every job follows the same lifecycle. Single product-page requests run immediately with no approval; large batch jobs go through team approval so we can schedule capacity — typically within working hours (8am–6pm GMT+8).

Submit

Send URLs or product IDs in one HTTP call — up to 50,000 items for a variant-sold batch, or a single product page for instant fetch.

Approval — batch jobs only

Batch jobs are reviewed and approved by our team, typically within working hours (8am–6pm GMT+8). Single-page endpoints skip this step entirely.

Poll

Track job status and per-item progress via the retrieve endpoint — submitted, scraped, failed counts in real time.

Download

Merged results delivered as raw JSON via a signed URL — pull the whole job or fetch individual items.

Technical

Quick reference

Base URL api.magpieiq.com/scraping/v1

Job lifecycle submit → poll → download

Authentication Bearer token

Batch size Up to 50,000 items

Countries ID · SG · TH · PH · MY · VN · TW

Result format Raw JSON · signed URL

Batch processing window 8am–6pm GMT+8

Documentation data.magpieiq.com/docs ↗

Endpoints

What you can scrape

Endpoint	What it does	Limits	Approval
Variant Sold	Batch variant-sold scrape — units sold per variant, per item	≤ 50,000 items	Approval required
Shopee Product Page	Single product-page fetch — full PDP payload, runs immediately	1 / request	No approval
Shopee Product Page (bulk)	Bulk product pages by CSV of product IDs	CSV batch	No approval
Shopee Listing product	Search, category, or merchant listing pages by URL	≤ 50 URLs / job	Approval required
Tokopedia PDP	Single Tokopedia product-detail scrape	1 / request	No approval
Blibli PDP	Single Blibli product-detail scrape	1 / request	No approval

Pricing is credit-based per endpoint. Contact us for rates, volume pricing, or custom integrations.

Use cases

When monthly data isn't fast enough.

The Data API serves the maintained universe. The Scraping API is for everything else — pages we don't track, and numbers you need this hour, on the same infrastructure that cracks Shopee's anti-bot defenses.

Live price & stock monitoring

Track competitor prices, flash-sale discounts, and stock levels as they change — not as they looked last month. Re-scrape the same SKU list as often as you need.

Price tracking Stock alerts Flash sales

Pages the universe doesn't track

Long-tail listings, niche categories, new merchants, specific search-result pages — if it has a URL, submit it and get the raw payload back.

Long-tail SKUs Search results Merchant pages

Launch & campaign checks

Product launched this morning? Mega-campaign running today? Pull the numbers now — sold counts, pricing, and reviews the moment they matter.

D-day tracking 9.9 / 11.11 / 12.12 Launch audits

Fresh marketplace
data, scraped
on demand.

Submit. Approve. Poll. Download.

Quick reference

What you can scrape

When monthly data isn't fast enough.

Want cleaned, history-deep market data without running a scrape?

Any page. Any hour.
Raw results in one call.

Fresh marketplacedata, scrapedon demand.

Submit. Approve. Poll. Download.

Quick reference

What you can scrape

When monthly data isn't fast enough.

Want cleaned, history-deep market data without running a scrape?

Any page. Any hour.Raw results in one call.

Fresh marketplace
data, scraped
on demand.

Any page. Any hour.
Raw results in one call.