What should I do when a Taiwan stock data API rate-limits or blocks me?

Not every failure deserves a retry

• 429: throttled. Retry, but wait — and wait longer each time.
• 5xx: a transient server-side problem. Retry; it usually clears within seconds.
• Any other 4xx: your request is wrong — a misspelled parameter, for instance. A hundred retries produce a hundred identical failures. Read the error message first.
• Handling all three with one retry loop is the classic amplifier: a typo turns into sustained hammering.

Exponential backoff, and honour Retry-After

Double the wait on each attempt and add jitter so parallel workers do not wake together. If the response carries Retry-After, use it. Note that responses here do not currently carry rate-limit headers, so your backoff must not depend on one being present — use it if it appears, fall back to your own schedule if it does not.

import random, time, requests

RETRYABLE = {429, 500, 502, 503, 504}

def get_with_backoff(url, *, headers, params, attempts=5):
    for attempt in range(attempts):
        r = requests.get(url, headers=headers, params=params, timeout=30)
        if r.status_code == 200:
            return r.json()
        if r.status_code not in RETRYABLE:
            # A wrong parameter will not fix itself — surface it instead of retrying
            raise RuntimeError(f"{r.status_code}: {r.text[:200]}")
        wait = r.headers.get("Retry-After")          # use it when present
        delay = float(wait) if wait else (2 ** attempt) + random.random()
        time.sleep(delay)
    raise RuntimeError("out of retries")

Caching: history does not change, so fetch it once

Yesterday's prices are settled. Having pulled a symbol's 2024 daily bars once, tomorrow's run of the same analysis does not need them again. Key the response by symbol and range on disk and serve hits locally — this usually cuts request volume further than any retry strategy.

import json, pathlib

CACHE = pathlib.Path("cache")
CACHE.mkdir(exist_ok=True)

def cached_prices(symbol, start, end, *, headers):
    key = CACHE / f"{symbol}_{start}_{end}.json"
    if key.exists():
        return json.loads(key.read_text())          # a hit sends no request at all
    data = get_with_backoff(
        "https://api.twmarketdata.com/v2/datasets/twse-daily-price",
        headers=headers,
        params={"symbol": symbol, "start_date": start, "end_date": end},
    )
    key.write_text(json.dumps(data, ensure_ascii=False))
    return data

Batching: one range query instead of a loop over days

The most common waste is one request per trading day across a year — roughly 250 requests for something a single call returns. The date range is a query parameter; pull the whole span at once and request volume drops by two orders of magnitude.

# Avoid: one request per day# for day in every_trading_day(2024):#     fetch(symbol="2330", start_date=day, end_date=day) # Prefer: one request for the whole spandata = cached_prices("2330", "2024-01-01", "2024-12-31", headers=HEADERS)print("one request returned", data["count"], "rows")

The order that actually helps

• Cache first. Repeat requests are the easiest volume to remove and it changes none of your results.
• Batch second. Per-day to per-range usually resolves the problem on its own.
• Tune backoff last. Backoff is a protection mechanism, not a throughput one — it makes you slow down gracefully, it does not get you more data.

Verify before you register

Five large-cap symbols — 2330, 2317, 2454, 0050 and 2603 — need no key on free-tier datasets, which is enough to confirm your request shape, the response structure and your parsing code. Both conditions must hold: a free-tier dataset AND one of those five symbols. Any other symbol returns 401 even on the same endpoint.

# No key requiredimport requestsr = requests.get(    "https://api.twmarketdata.com/v2/datasets/twse-daily-price",    params={"symbol": "2330", "limit": 1},    timeout=20,)print(r.status_code, r.json()["rows"][0]["date"])

What we do not have

• Responses do not currently carry rate-limit headers, so you cannot read remaining quota off a response — write your backoff so it does not need one.
• No real-time quotes, so high-frequency polling buys nothing here; fetch history once and cache it.
• Per-plan quotas live on the pricing page. This page does not restate those numbers, so it cannot drift out of step with the page that owns them.

Common questions