Text-to-SQL demos are easy. Production is not. This report shows how to move from a plausible query to a trusted business answer across complex schemas, private definitions, access rules, and cost limits.
Why it persists: Large schemas overload context and increase irrelevant joins. Production checks should include read-only enforcement, allowed objects, cost estimation, row limits, sensitive fields, join paths, and result sanity.
What recovers it: Start with one business area where metrics and joins are governed. Select relevant tables, columns, definitions, policy, and verified queries.
Column names rarely encode metric definitions, exclusions, fiscal calendars, or approved joins. Production systems need semantic context and verified examples.
A syntactically valid query can be logically wrong. Evaluate result equivalence, policy compliance, cost, latency, and explanation quality.
The user should see only rows, columns, and metrics they are entitled to access. The model cannot be trusted to enforce policy through instruction alone.
Start with one business area where metrics and joins are governed. Create a semantic layer, glossary, examples, and approved query patterns.
Select relevant tables, columns, definitions, policy, and verified queries. Keep the model focused on the smallest sufficient context.
Use identity-aware access, query parsing, allowlists, cost limits, row limits, dry runs, sandboxing, and post-execution checks. Guardrails should exist outside the model.
Capture questions, clarifications, generated SQL, execution, corrections, and accepted answers. Use failures to expand the evaluation suite and semantic context.
Text-to-SQL is not a translation feature. It is a governed analytical product with a semantic layer, security model, evaluation harness, and user experience for uncertainty. Start narrow, make every answer inspectable, and expand only when correctness holds on real enterprise questions.
Not safely for broad production use. Add retrieval, semantic context, identity-aware access, validation, limits, and evaluation.
Use real questions and compare result sets, policy compliance, latency, cost, explanations, and clarification behavior.
A bounded area with high question volume, trusted metrics, read-only use, and engaged subject-matter owners.
For reliable business answers, yes. The model needs governed definitions and joins that schemas alone do not provide.
Yes for transparency, along with the interpreted question, sources, filters, and caveats.
