How do I clean Taiwan open data — ROC dates and string numbers?

ROC dates come in two lengths — do not share one converter

Date fields are ROC-calendar strings. A full date looks like 1150811 (seven digits, year month day); a year-month looks like 11507 (five digits). Both add 1911 to the year, but they slice differently, and folding them into one function is an easy way to produce the wrong month.

def roc_date(value: str) -> str:
    """ROC date 1150811 -> 2026-08-11."""
    return f"{int(value[:-4]) + 1911}-{value[-4:-2]}-{value[-2:]}"

def roc_month(value: str) -> str:
    """ROC year-month 11507 -> 2026-07."""
    return f"{int(value[:-2]) + 1911}-{value[-2:]}"

print(roc_date("1150811"))   # 2026-08-11
print(roc_month("11507"))    # 2026-07

Everything is a string, and casting needs care

Open, high, low, close, volume and turnover all arrive as strings. A bare float() works most of the time and raises on an empty value or a placeholder. Give the cast a fallback, and make that fallback None rather than zero — zero is a legitimate price and a legitimate volume, so using it as the failure marker destroys the distinction later.

def to_number(value):
    """Return None when it will not parse — never 0, which is a real value."""
    if value is None:
        return None
    text = str(value).strip()
    if text in ("", "-", "--"):
        return None
    try:
        return float(text)
    except ValueError:
        return None

print(to_number("13.88"), to_number("20635383"), to_number("--"), to_number(""))

Stripping commas everywhere is the wrong reflex

Most cleaning walkthroughs open by removing commas from every field. Measured across these endpoints, the numeric fields carry no thousands separators at all; the values that do contain a comma are free-text notes, such as a remark describing a cumulative revenue change. A blanket strip does not make any number cleaner — it edits the prose. Scope comma removal to fields you have declared numeric.

# List the exact keys you know are numeric. Anything not listed is left untouched.NUMERIC_FIELDS = {"營業收入-當月營收"} def clean_row(row: dict) -> dict:    out = {}    for key, value in row.items():        if key in NUMERIC_FIELDS:                 # only fields you know are numeric            out[key] = to_number(str(value).replace(",", ""))        else:            out[key] = value                      # free text kept verbatim    return out

Two field vocabularies, so keep a mapping

Whole-market daily quotes use English field names such as Date, Code and ClosingPrice. Listed-company monthly revenue uses Chinese field names. Combining endpoints into one table therefore needs your own mapping, maintained as you add endpoints.

FIELD_MAP = {    # daily quotes (English names)    "Code": "symbol", "Date": "date", "ClosingPrice": "close", "TradeVolume": "volume",    # monthly revenue (Chinese names)    "公司代號": "symbol", "資料年月": "revenue_month", "營業收入-當月營收": "revenue",} def normalise(row: dict) -> dict:    return {FIELD_MAP.get(k, k): v for k, v in row.items()}

Or skip this layer

If none of this is where you want to spend your time, responses here already arrive converted: ISO date strings, numeric types, and the listed and OTC markets under one set of field names. The source is still official — the difference is who does the cleaning.

import requests

r = requests.get(
    "https://api.twmarketdata.com/v2/datasets/twse-daily-price",
    headers={"X-API-Key": "sk_live_your_key"},
    params={"symbol": "2330", "limit": 3},
    timeout=20,
)
for row in r.json()["rows"]:
    # date is already ISO, close is already numeric — no conversion needed
    print(row["date"], row["close"], row["volume_shares"], type(row["close"]).__name__)

What we do not have

• Parsing it yourself is entirely workable and free — the first four sections exist to make that path smooth.
• No real-time quotes; the official open endpoints are likewise post-close snapshots.
• We do not preserve the original ROC strings and string types. If you want the raw shape, call the official endpoint directly.

Common questions