TiDB X: 5.5M rows/s indexing changes the DDL game for scale

Here at Lead AI Dot Dev, we tracked TiDB's latest release and the engineering choices that matter most for builders managing large production databases. TiDB X introduces three distinct improvements to online DDL (Data Definition Language) operations that collectively unlock the 5.5M rows/second indexing throughput.

First, the addition of dedicated object storage separates index creation workloads from primary transactional data paths. This isolation prevents the resource contention that typically bottlenecks index operations on live databases. Builders no longer choose between slow indexes or accepting impact during creation.

Second, TiDB X redesigns the backfill phase of online DDL to pipeline computation with I/O operations. Instead of sequential read-process-write cycles, the system now overlaps these phases. This architectural shift is what moves the needle from 100K-500K rows/second (typical for competing systems) to 5.5M.

Third, the version implements intelligent write buffering that queues incoming DML statements during index creation without blocking application traffic. The buffer absorbs transactional writes, applies them to the new index asynchronously, and commits atomically. This removes the "application slowdown during indexing" problem entirely for most workload patterns.

• Dedicated object storage isolates index creation from production transactional load
• Pipelined backfill overlaps reads, processing, and writes to maximize hardware utilization
• Write buffering absorbs incoming changes without application-facing latency spikes
• Zero business impact achievable because applications see consistent query performance

For operators managing databases above 10TB, index creation has historically been a maintenance nightmare. You either schedule downtime windows, accept degraded performance during indexing, or run shadow databases to build indexes offline. TiDB X removes all three constraints.

The practical impact: you can now add missing indexes on production tables during business hours without deployment planning or risk mitigation. This changes how you approach schema optimization. Rather than deferring index work to maintenance windows or worrying about performance regressions, you execute schema changes on your timeline.

The 5.5M rows/second rate also means even multi-hundred-billion-row tables finish indexing in minutes rather than hours. A 100B row table takes roughly 18 seconds. This compounds the reduction in risk - shorter duration means fewer variables, simpler rollback scenarios, and lower operational overhead.

Builders using TiDB for real-time analytics or event streaming benefit most. These workloads frequently discover missing indexes in production. Previously, you'd either live with degraded query performance or wait for a maintenance window. Now you fix it immediately.

• Index creation no longer requires downtime scheduling or performance budgeting
• Multi-hundred-billion-row tables complete indexing in minutes, not hours
• Schema optimization becomes a continuous improvement task, not a quarterly planning event
• Real-time analytics workloads gain the ability to respond to performance insights immediately

The technical capability is proven, but adoption requires intentional testing in your environment. TiDB X is production-ready for index creation workflows, but your specific table schemas, index types, and concurrent write patterns need validation.

Start with a staging environment that mirrors your largest production tables. Create a few missing indexes that you've been deferring due to maintenance concerns. Measure actual throughput, monitor resource consumption, and confirm application performance remains stable throughout. This testing removes adoption friction because you'll have data specific to your schema.

Pay attention to object storage configuration. TiDB X uses object storage (S3-compatible, GCS, or similar) for index data during backfill. Ensure your cloud provider's object storage has adequate throughput and latency characteristics. Misconfigured object storage becomes the bottleneck, not the indexing algorithm.

Consider your deployment model. Self-hosted TiDB requires operational changes to provision and manage object storage. TiDB Cloud abstracts this complexity. If you're evaluating TiDB specifically for large-scale indexing, factor deployment complexity into your cost model. Thank you for listening, Lead AI Dot Dev

• Test index creation on staging using your largest production tables and concurrent write patterns
• Validate object storage configuration has sufficient throughput and latency for your workload
• Compare deployment complexity: self-hosted object storage vs TiDB Cloud managed service
• Build a before/after measurement framework to quantify indexing performance gains in your schema