ShanHaiGateway: a Go gateway that pools multiple LLM upstreams behind one OpenAI-compatible endpoint

What it is

ShanHaiGateway is a self-hosted gateway written in Go that consolidates several upstream model providers behind a single OpenAI-compatible endpoint. The project lives at github.com/ffseika0304/shanhai-gateway with its site at seika.ltd, is released under the MIT license, and its health endpoint reports version 0.5.1. Its headline mechanisms are smooth weighted round robin (SWRR), exponential cooldown for quota-class errors, task-level session affinity, shadow retention observability and a Docker deployment the README describes as three commands to production.

Why the author built it

The author describes the motivation plainly: when overseas providers open free model tiers, some users register batches of accounts, exhaust the quota and even resell the access until the provider withdraws the offer. The author says they follow a similar harvesting pattern, with two differences: they teach the method openly, and they do not freeload entirely, spending a little over ¥100 per month to keep the setup running. Commercial channels, vouchers and free pools are mixed together, what is owed is paid, and free capacity absorbs the rest. In a production test in October 2026, the author reports pressing a monthly volume of 20 billion tokens down to a cash bill of about ¥170.

The README frames every operational rule as the residue of a real incident, one pit exchanged for one rule, all collected in an operations-semantics table rather than drawn from theory.

Problems it targets

The gateway addresses failure modes the author encountered while round-robining three upstream providers:

• One upstream having a bad day takes the whole product down with 503s. Retry alone is too blunt; failures need routing by error class, with quota errors backing off exponentially, saturated backends queuing, and network errors observed without penalty.
• Prefix-cache hit rates stay low. Plain rotation scatters the second turn of a conversation to another provider, but the prefix cache lives provider-side, so switching providers voids it. The author measured a 51 to 56 percent hit rate for same-prefix clusters under loose rotation, exactly the pool sampling probability, which the README calls the ceiling of round robin.
• An upstream silently delisted a model and a fallback backend idled at zero percent success for four and a half days before anyone noticed. A 4xx is not necessarily the client's fault; someone has to claim that class of failure on the backend's behalf.
• A disconnect storm produced 1720 times 503: client disconnects were logged as backend failures, healthy backends entered cooldown one after another, and eventually no backend was left.
• Keys end up scattered everywhere and the repository can never be shown in public.

Feature set

• SWRR weighted round robin plus fallback-chain isolation: weight, concurrency and timeout are tunable per backend, and fallback backends are isolated from the main pool so the two do not contaminate each other.
• Error-taxonomy-driven cooldown: 403 and 429 responses back off exponentially, 5 minutes then 10 minutes, doubling up to a 4-hour cap; a saturated backend queues requests up to 60 seconds instead of throwing a 503; client disconnects incur no cooldown.
• Task-level session affinity: the gateway prefers the X-Job-Id header and falls back to X-Session-Key, pinning every turn of a task to the same backend so the provider-side prefix cache becomes the reuse unit. The author reports the measured hit rate rising from 27% to 51%, peaking at 69%.
• A quirk layer for vendor-specific behaviour: provider-specific error bodies, usage keys and egress customisation live in a dedicated package (core/quirk); the kernel contains zero branches on provider names, and upstreams without quirks need no configuration.
• Multi-user downstream key billing: sk-user-* quota issuance, pluggable login built on a registry pattern, and a bundled user center.
• Shadow retention observability: a single SQLite database records attempts and captures, automatically scrubbed before being written, with a fine-grained dashboard at /api/stats.
• Configuration hot reload: backends.json is reloaded on mtime changes, so weights and backends can change without a restart.
• A sidecar for slow-cycle self-healing: an independent Python process with zero third-party dependencies makes throttling and degradation decisions from windowed incremental failure rates, sharing work with the gateway's fast path.
• Quota-window probing with automatic removal: scripts/quota_probe.py queries each provider's 5h/7d/30d quota windows grouped by backend name and writes a snapshot to quota.json every 180 seconds; the sidecar evaluates every 60 seconds, removes a backend whose window is exhausted and re-enables it once it falls back below the threshold. Each subscription is metered independently, so adding a subscription touches no mapping table.
• Docker deployment: multi-stage build, pure-Go SQLite with zero CGO, non-root execution, a healthcheck and zero secrets inside the image.

The cost claim

According to the README, someone is running the gateway in production at 20 billion tokens per month for a cash cost of roughly ¥170 per month; the same volume billed at each provider's list price would come to about $3,800 per month, roughly ¥26,600.

Free pools carried about 63% of the traffic. The author describes blending highly unstable free and gifted quotas into a highly available main pool as one of the gateway's core capabilities: it does not distinguish main from helper upstreams and dispatches fully at random. The reported result is that pure commercial payment channels (Volcano Pro/Plan and similar) carried only about 37% of real requests, while free model pools bundled with low-cost subscriptions, proxy pools and cache reads absorbed about 63% of throughput.

The free pools' 429s and stalls are isolated by circuit breaking, a 600-second bidirectional timeout and failure classification; despite frequent free-pool errors, the main business still saw zero upstream 429s. Resilience comes from both ends timing out at 600 seconds: whichever side cuts first, the request is re-sent on a different lane. If the upstream cuts, that lane enters cooldown and the caller's retry lands elsewhere; if the gateway times out first, it switches lanes internally and the caller barely notices.

Each added subscription brings another free pool, since every Go subscription ships with its own free model pool, so the paid share keeps shrinking (the author measured it falling from 43.6% to about 37% after attaching one more subscription) while total throughput grows. The bill depends on which channel is cheap rather than how many tokens are consumed; providers are interchangeable routing entries in configuration, not architectural assumptions.

The author's bill recap, at a production load of 1 billion tokens per day, combines route spreading, cache affinity (70% hit rate) and rolling rotation of multi-account vouchers plus free gift pools for a monthly cash spend of about ¥170:

That works out to about ¥0.85 per 100 million tokens, with only 23% of the volume billed as fully priced new tokens; the same volume at list price would cost about $3,800 per month. The author says code alone (cache affinity plus free-tier mixing) together with request-size routing at the pool-configuration layer compresses the list-price share to between a quarter and a third, and the channel strategy on top brings the total cost reduction to 99%.

How it achieves this

• Internal session affinity: every turn of a task (X-Job-Id, falling back to X-Session-Key) is pinned back to the same lane. Under loose rotation the measured same-prefix-cluster hit rate was 51 to 56 percent, equal to the pool sampling probability; after pinning, the author measured a rise from 27% to 51% with a peak of 69%.
• External session-header passthrough: a per-backend session_header option (for example x-opencode-session) writes the session value into the vendor's own header. Some vendors' caches only honour their own header; the README says enabling the field is a one-line change in attempt.go with no code modification.
• Quirk layer: vendor-specific error bodies, usage keys and egress customisation go into internal/core/quirk while the kernel keeps zero name-based branches. A quirk-free upstream needs zero configuration, and behaviour without the quirk layer is bit-identical to a generic upstream.
• Per-subscription metering with automatic quota-window removal: the probe queries 5h/7d/30d quota windows grouped by backend name and writes a quota.json snapshot every 180 seconds; the sidecar judges every 60 seconds. Rotating an expiring subscription is a matter of disabling one and enabling another, staggered so each is fully burned without collapsing concurrency.
• Request-size routing, done at the pool-configuration layer rather than in the kernel: small requests go to per-token-billed pools and large ones to per-request-billed pools, with a measured break-even point of 22,222 tokens; the author reports 35% savings.
• Mixed free and low-tier channels: free gift pools rotate in the same pool as commercial channels; instability is acceptable precisely because a free-pool collapse does not touch the main business.

Adding a subscription takes two steps and the rest is automatic. First, add an entry to accounts[] with a unique name, api_keys[] for calling, and quota-query credentials (AK/SK delivered with the deployment configuration; if they are missing the dashboard reports missing credentials and takes no action, never borrowing another account's credentials). Second, add a backend in backends[] whose base_url points at that provider's plan endpoint and whose account references the first entry. The change hot-loads within about 30 seconds, the probe starts querying its quota windows by backend name, and the sidecar begins managing removal and restoration.

The author's own caveats

The README insists its numbers be read together with five qualifications:

1. Promotional prices have a shelf life: vouchers and half-price deals can lapse at any time; reverting to normal pricing would cost about ¥340 per month, still a reduction of more than 98%.
2. The "free and low-tier carries 63%" figure fluctuates with upstream generosity and earn rates; the measured range was 37% to 57%, higher when cache reads are counted, and no specific share is promised.
3. Free and gifted quotas are granted by upstream providers, not by this project. What the project permanently open-sources is cache affinity, reroute-on-failure retry, and per-subscription independent metering with automatic removal.
4. The figures are single-machine production measurements from 2026-10 (daily consumption of 441.3 million tokens), on the same accounting basis as the bill table; the measurement method is documented in calibration records released with the version.
5. No vendor is locked in: the main pool's share swung between 11% and 43% within three days, changing only accounts and weights in configuration, while routing, cache affinity and fallback logic went untouched.

Quick start

Docker is the recommended path. The container listens on host port :18016 by default, overridable with SHG_PORT:

git clone https://github.com/ffseika0304/shanhai-gateway.git && cd shanhai-gateway
mkdir -p runtime/config runtime/data
cp config/backends.example.json runtime/config/backends.json   # edit to your backends
cp config/keys.example.json    runtime/config/keys.json        # edit to your downstream keys
docker compose -f deploy/docker-compose.yml up -d --build
curl -s http://127.0.0.1:18016/health          # {"status":"ok","version":"0.5.1"}
curl -s http://127.0.0.1:18016/api/stats | head -c 400

Running the bare binary requires Go 1.27 or newer:

make build                          # equivalent to go build -o shg ./cmd/shg
cp config/backends.example.json config/backends.json   # fake credentials work for a first run
export LISTEN=":18006"              # billing-free LAN port
export ROUTER_VALIDATE=0            # skip the startup self-check (remove with real keys)
./shg &
curl -s http://127.0.0.1:18006/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"demo-model","messages":[{"role":"user","content":"hi"}]}'

A 200 response means the path is open; GET /health is the healthcheck and GET /api/stats the traffic dashboard. The LAN port does not require a key for POST /v1/chat/completions; downstream-key billing is only enabled on the public port and the user center. Both deployment shapes share the same path semantics: a bare run reads real (gitignored) files from /config/ and stores data under ~/.local/share/shg/, while Docker mounts ./runtime/config/ at /etc/shg and ./runtime/data/ at /var/lib/shg. Both identify as shg/0.5.1 in the user agent, /healthz and log prefixes, and both hot-reload configuration on mtime.

Architecture

Dependencies point one way, api to service to core, while config and runtime act as horizontal feed layers that depend neither on each other nor on service. cmd/shg only assembles: environment, paths, directories, SQLite and two listeners.

• internal/api — the control-plane HTTP gates: the billing-free LAN gate and the public gate (dashboard gate plus key), plus user center, admin, dashboard and public gates. Who may enter is unrelated to how a request is routed; changing entry points or frontends touches only this package.
• internal/service — data-plane and control-plane business: routing (weighted rotation, candidate filtering, fallback isolation, atomic swap), proxy (request rewriting, streaming, queue timeout, disconnect detection), auth (pluggable login providers) and apikey (downstream key billing). A routing algorithm can be replaced by P2C or least-connections without core or the config format following; a new login method is just a new provider file.
• internal/core — errclass is the single source of truth for upstream error classification plus the global wordlist, and one definition must serve cooldown, circuit breaking, statistics and fallback at once; quirk; upstream holds the backend capacity and health state machine (concurrency slots, breaking, exponential cooldown, per-class failure counters) whose magic numbers were each tuned by a production incident; store is the single SQLite database with a single write entry, so schema evolution and business logic do not drag each other.
• internal/config — what counts as a legal backend, the load/validate/mtime-hot-reload pipeline and the self-check blacklist.
• internal/runtime — the program's identity and on-disk locations: paths, env, log rotation, sweeping and SQLite maintenance.

cmd/shg is meant to stay thin forever: business sinks downward, and only startup and shutdown ordering lives there.

A minimal backends.json looks like this (all credentials fake; the full field set is in config/backends.example.json):

{
  "accounts": [
    { "name": "vendor-a", "base_url": "https://api.example-llm.com/v1",
      "api_keys": ["sk-xxxxx"] }
  ],
  "backends": [
    {
      "name": "vendor-a-main",
      "account": "vendor-a",
      "model": "demo-model",
      "base_url": "https://api.example-llm.com/v1",
      "weight": 1,
      "concurrency": 8,
      "enabled": true,
      "is_primary": true
    }
  ]
}

Environment variables: GATEWAY_DATA_DIR sets the data root (default ~/.local/share/shg), GATEWAY_CONFIG_DIR the config directory (default /config), GATEWAY_DB_FILE the main SQLite file (/shg.db), GATEWAY_BACKENDS_FILE the backend baseline (/backends.json), GATEWAY_KEYS_FILE the downstream key ledger (/keys.json) and GATEWAY_QUOTA_FILE the quota snapshot (/quota.json). ROUTER_BACKENDS injects JSON configuration directly with priority over files; ROUTER_VALIDATE defaults to 1 and skips the startup self-check at 0; LISTEN and LISTEN_PUBLIC default to :18006 and not listening; SESSION_AFFINITY defaults to on, and switching it off makes behaviour bit-identical to an unaffined gateway — AFFINITY_TTL, AFFINITY_MAX and AFFINITY_CONTENTION_LIMIT control entry lifetime, LRU capacity and the saturation-migration threshold (6h, 8192 and 3). REASONING_EFFORT defaults to low: when the client did not explicitly set reasoning_effort the gateway injects a default to cap reasoning budgets for thinking models, respects an explicit client value, and an empty string disables injection; a guard detects an already-injected value from a preceding gateway so chained deployments never double-inject.

The prefix is GATEWAY_* rather than SHG_* because the sidecar contract (py/sidecar.py reads GATEWAY_BACKENDS_FILE and GATEWAY_QUOTA_FILE) is anchored to those names; the application identity lives in internal/runtime/identity.go as AppName "shg". backends.blocked.json has no variable of its own — it shares backends.json's directory and must move with it.

Secrets and data hygiene

No secrets ship in the repository or the image: real backends.json, keys.json, backends.blocked.json and quota.env files are all gitignored and exist only at runtime locations, while the repo carries only .example.json templates. Data (.db, logs) is never committed, since captures contain raw request text and count as in-production data rather than configuration; disaster recovery therefore means code plus the runtime configuration directory. Any real key appearing in the repository is treated as a bug and redacted on sight.

Operations semantics: rules distilled from incidents

The README keeps a rule table where every entry cites the incident that produced it; SLR refers to the project's earlier self-built front router, seika_llm_router, since replaced wholesale by this gateway, with the relevant entries kept as semantic history.

1. Client disconnects never trigger cooldown. A disconnect storm on 03.10.2026 produced 1720 times 503: disconnects were logged as backend failures, healthy backends cooled down one by one and the whole pool ran dry. Disconnects are classified as network events for observation only, exempt from cooldown and circuit breaking.
2. A missing model is the backend's fault. When an upstream silently delisted a model the gateway showed 4xx and black-hole behaviour; from 06.10.2026 a fallback backend idled at zero percent success for four and a half days before discovery. Such 4xx cannot be charged to the client; it lands on the responsible backend for cooldown or fallback removal, with identity patched through model_expire and model_fallback.
3. Quota-class errors back off exponentially; saturation queues. 403 and 429 responses cool down 5 minutes, then 10, doubling to a 4-hour cap rather than ramming repeatedly; a full backend queues up to 60 seconds (QUEUE_WAIT) instead of answering 503.
4. The blacklist is write-only and cleaned manually. backends.blocked.json is the startup self-check's list of permanently dead upstreams; there is no automatic unblocking path, because auto-clearing would use failure rate as evidence while the blacklist rests on probe facts, so entries require human review and removal from the file.
5. Strikes reset only on success. The 01.10.2026 403 self-lock storm: clearing the strike before quota had recovered let the backend return to the pool, hit another 403 immediately and re-enter cooldown in a loop. The whole exponential-backoff chain depends on consecutive strikes being zeroed solely by success, so the admin hot-edit endpoint deliberately offers no strike field.
6. Session affinity is a preference layer above rotation. On 10.10.2026 production measured a 0% prefix-cache hit rate because SWRR scattered conversation turns across providers whose caches live provider-side. Established sessions pin to the same backend; new sessions still land by SWRR weight. The affinity table is written only after a final successful delivery so failed sessions are not pinned to bad backends, and eviction and retention share their source with cooldown and circuit logic (errclass.ReleaseAffinity) rather than a second health layer. Affinity keys prefer X-Job-Id, task-level, with all turns of a task sharing one id — exactly the prefix-cache reuse unit — and fall back to X-Session-Key.
7. Vendor quirks go to the quirk layer; the kernel never branches on names. Earlier pits were generalised into universal rules — unconditional Content-Length removal, modifying only the model key, the session_header field and a generic wordlist — but a non-generalisable pit must neither be folded into shared rules where it would hurt other vendors nor handled as name checks scattered through the kernel. The third path is internal/core/quirk/xxx.go consulted at three fixed seams.
8. The observability contract mirrors SLR. The attempts table carries a job_id column, migrated idempotently via ALTER on existing databases, so per-job attribution and cache-analysis scripts keep working unchanged; /api/stats exposes an SLR-shaped affinity{hits,misses,puts,pinned_jobs} object where pinned_jobs takes the raw table length (Len must precede CountPerBackend, which incidentally clears TTL entries), and the three sticky_* keys are retained as historical names the dashboard still reads. puts counts every real Put including same-backend refreshes, bit-identical to SLR.
9. Instances are started only by the service manager, never by hand. On 21.10.2026 another agent launched ./slr manually and detached (PPID=1), holding ports 18006 and 18012; the systemd instance failed to bind and crash-looped, leaving the dashboard flaky all day. Three accompanying rules: wait for the port before starting and report the occupying PID; graceful shutdown may hold the port up to 600 seconds (the single-backend timeout), so the unit needs TimeoutStopSec=45 plus Restart=always; frontend changes are go:embed, so the binary must be rebuilt and the service stopped first, since a running executable cannot be overwritten. After deployment verify the steady state — ActiveState, NRestarts not growing, the port's owning PID belonging to the service — because curling immediately after restart can be answered by the stale instance.

The quirk layer

The kernel calls it at exactly three fixed seams of the request lifecycle:

1. Decorate(req, backend) runs on egress, after generic header cleanup (removing Content-Length, client credentials and Accept-Encoding, fixing Content-Type and the user agent) and before sending upstream; it may add vendor headers or query parameters and conditionally trim the body. Default behaviour is a no-op.
2. Classify(status, body) (Class, bool) runs during classification; returning (Class, false) means the case is not claimed and the caller falls back to generic judgement, letting a vendor own only its own phrases without hijacking the shared wordlist. The default claims 400 and 404 plus the generic model-delisted wordlist, which still lives in errclass.
3. ParseUsage(body) Usage runs during usage parsing; OK=false falls back to the OpenAI default. The default parses OpenAI usage plus prompt_tokens_details.cached_tokens.

A missing or unregistered driver falls back to the package-level OpenAI singleton, bit-identical to not having the layer at all; the registry is a name lookup guarded by an RWMutex, keeping the kernel free of name branches — quirks are data, a config field, not code paths.

Adding a quirk without touching the kernel: create internal/core/quirk/xxx.go with type Xxx struct{ OpenAI } embedding the base implementation and overriding only what differs, register it in func init() { Register(Xxx{}) } — duplicate names panic — then set "driver": "xxx" on the backend in backends.json. That is a config field, not an environment variable; a wrong name merely degrades to generic behaviour and cannot stop startup. The README gives three real shapes: a provider that returns arrears messages only in Chinese (seam two claims those phrases and blacklists its 403s); a provider whose model-delisted error body uses a self-coined term (seam two claims it into fail_streak so consecutive failures trip the breaker); and a provider whose cache-hit key is prompt_cache_hit_tokens instead of cached_tokens (seam three parses it so cached-token billing does not drop to zero). The default distribution ships no concrete quirks; the interface is reserved in advance.

The captures consumption contract

captures are retained snapshots, a side path linked to attempts by rid. The README says consumers should judge a capture by exactly four criteria, a contract forced out of real traffic data:

1. usable=false is the positive signal for incomplete delivery: delivered in full is usable (1), while mid-stream disconnects, cut-off or undelivered responses are usable (0). partial_ratio, truncated and status are not positive signals for incompleteness.
2. partial_ratio only answers whether any content exists: 0 means none, above 0 means some. partial_ratio=1.0 is the normal shape of a complete delivery, not a fully partial one, and grading severity by partial_ratio is wrong because it has no P50 baseline and never will.
3. truncated=true marks truncation, not failure: truncation can occur inside a complete delivery when the body exceeds the tee limit, and unusability still follows the usable flag of rule 1.
4. Multiple rows sharing a rid are ordered by ts, and the last row with usable=true is the final delivery: retries and partials legitimately produce several rows per rid, which is not duplicated data.

The mnemonic: availability from usable, content presence from partial_ratio, truncation from truncated, deduplication from rule 4's ordering. status is only a reference when a real HTTP code exists — a partial cut mid-stream still shows 200 — so status must not be used as a completion criterion.

Development instances and neighbouring services

The production unit slr-router.service owns ports 18006 (LAN) and 18012 (public) plus the shadow-data/ directory; any hand-run ./slr steals them and crash-loops the systemd instance. Local trials should use separate ports and data directories:

LISTEN=:28006 LISTEN_PUBLIC=:28012 \
GATEWAY_DATA_DIR=/tmp/slr-dev/data \
GATEWAY_CONFIG_DIR=/tmp/slr-dev/config \
GATEWAY_DB_FILE=/tmp/slr-dev/data/slr-dev.db \
ROUTER_VALIDATE=0 \
./slr

Fake credentials such as sk-xxxxx suffice; real ones can be copied from shadow-config/backends.json, and a dev instance must never point at production directories.

The dashboard's WorkBuddy Panel link is a separate project outside this repository: WorkBuddy2API Panel wraps Tencent CodeBuddy accounts into an OpenAI-compatible multi-account gateway with a web operations panel covering account pools, circuit breaking, session stickiness and keep-alive. Its repository is github.com/linguo2625469/workbuddy2api-panel, a fork of the enhanced branch of Sliverkiss/workbuddy2api. UI and API share one port, with the wb2api process bound to *:7863: the panel sits at http://:7863/panel/ and the API feed at http://127.0.0.1:7863/v1, which is where every account=workbuddy backend points. The dashboard navigation defaults to the same host's :7863/panel/; if the panel lives elsewhere, run localStorage.setItem("wbPanelUrl","http://:7863/panel/") once in the browser console, since the real address is never stored. An older panel UI listened on :18011 and has since merged into 7863; any document still citing 18011 is outdated. wb-panel and wb2api are not restarted alongside this gateway, and upgrades on either side do not implicate the other.

Troubleshooting

• The gateway fails to start reporting no backend configuration: the environment points at the wrong directory — GATEWAY_CONFIG_DIR must contain backends.json — and under Docker the runtime/config/ mount may be empty (docker logs shg shows how many backends the startup section parsed).
• A backend never gets picked: inspect enabled (disabled in config), cooldown_remain_s (quota cooldown, exponential up to four hours) and circuit_open in /api/stats; circuit_open:false does not mean there is no cooldown, since the two counters are separate.
• Every backend is cooling and all requests fail — the exact shape rule 1 exists to prevent: check fail_breakdown; a mass of network-class failures suggests an old version misclassifying disconnect storms, while a concentrated burst of 401 means dead keys that cooldown will not fix until replaced.
• For raw requests, query the SQLite captures table after reading the consumption contract; entries are scrubbed before storage, with secrets, intranet paths, phone numbers and Bearer headers replaced by [mask], which the README calls a feature rather than a defect.
• A container restarting in a loop: docker logs shg reveals the exit cause, usually missing files in runtime/config/ or invalid JSON; repeated healthcheck failures mark the container unhealthy, but restart: unless-stopped only reacts to process exit.

Development and layout

The repository layout puts cmd/shg/ as the sole entry point (assembly plus graceful shutdown); internal/ splits into api/, service/ (routing, proxy, affinity, auth with providers/, apikey), core/ (errclass, quirk, upstream, store), config/ and runtime/; web/dashboard and web/console (the operations dashboard and user center) are embedded via go:embed; py/sidecar.py is the quota daemon in pure standard library; scripts/quota_probe.py collects quota windows; config/ holds only *.example.json templates; deploy/ contains the Dockerfile and docker-compose.yml; API_CONTRACT.md documents the api.seika.ltd contract; and the Makefile exposes build, test, fmt, vet and docker targets.

The local development baseline requires gofmt -l internal/ cmd/ web/ to return empty, go vet ./... clean, and go test ./... -race -count=1 green across a baseline of 12 packages before make build. Development without hitting real upstreams uses fake credentials like sk-xxxxx: requests fail at network errors or 401, which conveniently exercises error classification, exponential cooldown and fallback removal in order, while unit tests pin routing, breaking and statistics table-driven. An end-to-end run without outbound traffic injects a loopback upstream configuration via ROUTER_BACKENDS together with ROUTER_VALIDATE=0.

Security, responsibility and contributing

ShanHaiGateway is a self-hosted tool, and how it is deployed and credentialed is entirely the operator's decision. The repository and image ship zero secrets, so real credentials exist only in the operator's own mounted volumes or local directories and never in git history. captures are scrubbed by an ordered rule table before persistence, yet retention is inherently sensitive and custody of the database file rests with the operator. On the public port the README advises enabling the login gate and downstream-key billing; the billing-free LAN port belongs on trusted network segments only, and whether to add HTTPS or IP limits depends on the deployment surface. The project is provided as-is, with the author disclaiming liability for direct or indirect consequences of use, misconfiguration, missing hardening, upstream credential leaks or data loss.

Issues and pull requests are welcome, though the README asks contributors to read the operations-semantics table before touching routing semantics or cooldown parameters, since every rule was paid for by a real incident. The project is MIT licensed. Acknowledgements go to modernc.org/sqlite for the pure-Go SQLite that keeps the image CGO-free, rs/cors for the CORS middleware, and the workbuddy2api ecosystem described above.