SLM Hive — Text-to-SQL

Challenge

Single-model Text-to-SQL fails in a predictable way: the same model that generates SQL is the one that detects errors in it — creating a structural blind spot. A zero-shot baseline with qwen2.5-coder:7b on the Spider dataset achieved 53.4% accuracy on 500 multi-domain questions.

The engineering question: can a local swarm of specialized agents close the gap to a level that justifies the architecture's added latency — while remaining entirely private and API-free?

Architecture

The hive runs on LangGraph with four specialized nodes, each responsible for a single cognitive task:

1. Analyst — runs a lightweight LIMIT 5 peek query against the target database to sample actual row values. This gives the downstream code generator concrete categorical values and date formats instead of relying solely on schema definitions.
2. Architect — qwen2.5-coder:7b, a code-specialized model, receives the natural language question, schema, and Analyst sample data. Generates raw SQLite.
3. Sandbox — local SQLite execution environment. Captures results or structured errors. Never touches external services.
4. Auditor — llama3.1:8b-instruct reviews the Sandbox output and implements a classified feedback loop: failures are categorized as either Syntax Error (malformed query) or Empty Logic Error (query executes but returns no rows). Each category gets targeted rewrite instructions back to the Architect.

Routing: Sandbox success → return result. Sandbox failure → Auditor → Architect retry (max 2 retries).

Friction and Deep Diagnostics

The Analyst's peek query was the non-obvious architectural decision that drove the most accuracy improvement on categorical fields. Without it, the Architect would hallucinate enum values — generating WHERE status = 'active' when the actual column value is 'Active' or 'ACTIVE'. Sampling 5 rows before generating SQL surfaces the real format without requiring any schema change.

The classified feedback loop required careful prompt engineering. An Auditor that returns generic "the query failed" feedback causes the Architect to make random edits. Forcing the Auditor to classify into exactly two categories (syntax vs. logic) made rewrites targeted and measurably improved recovery rate. The 0.4 average retry count means the Auditor is recovering roughly one in four queries that the Architect initially fails.

Thermal management in the benchmarking script was a practical necessity: running 500 consecutive local inference calls on Apple Silicon without pauses causes sustained thermal throttling that distorts latency measurements. The benchmark uses a cooldown interval every 50 queries — an unglamorous but critical detail for reproducible results.

Metrics and Validation

Benchmark: 500 randomized Spider dataset samples. All inference local on Apple Silicon M5 (24GB Unified Memory).

+24 percentage points absolute improvement. The swarm recovered nearly one in four queries that the zero-shot baseline failed. The 2× latency cost (5.5s → 11.2s) is the price of the Analyst peek + Auditor classification pass — a deliberate trade-off favoring correctness over raw speed for offline batch workloads.

Estimated API cost saved vs. equivalent frontier model pricing for the test batch: $2.70 — trivial in isolation, but the architecture scales to thousands of queries per day without a billing ceiling.