Skip to content

bench: add connection and query benchmarks for SQLite, Postgres, MySQL - #4368

Open
Map130 wants to merge 6 commits into
transact-rs:mainfrom
Map130:bench/connection
Open

bench: add connection and query benchmarks for SQLite, Postgres, MySQL#4368
Map130 wants to merge 6 commits into
transact-rs:mainfrom
Map130:bench/connection

Conversation

@Map130

@Map130 Map130 commented Aug 9, 2026

Copy link
Copy Markdown

Closes #158.

Adds criterion benchmarks covering all items from the issue checklist, for all three supported databases.

What's included

Each database gets a benches/<db>/connection.rs with five benchmarks:

Benchmark Description
new_connection Establish a fresh connection from scratch (connect only; graceful close is done outside the timed section)
pool_checkout Borrow + return a connection from a pool (min=max=1, test_before_acquire(false))
ping Ping an existing open connection
query_small_result Fetch 1 row by PK from a populated table (TechEmpower single-query style)
query_large_result Fetch 10 000 rows with TEXT and BLOB/BYTEA columns

How to run

SQLite (no external DB needed):

cargo bench --bench sqlite-connection --features sqlite,runtime-tokio

Postgres:

DATABASE_URL=postgres://... cargo bench --bench postgres-connection --features postgres,runtime-tokio

MySQL (mysql-rsa enables caching_sha2_password auth without TLS for MySQL 8+):

DATABASE_URL=mysql://... cargo bench --bench mysql-connection --features mysql-rsa,runtime-tokio

The existing tests/docker-compose.yml can be used to spin up Postgres and MySQL.

Example results

Local run (Docker postgres:17 / mysql:8.0 over loopback, SQLite :memory:; median of criterion's estimate). Absolute numbers are hardware/loopback-specific — they're here to show the benchmarks work and that the metrics are distinct, not as a cross-machine reference.

Benchmark SQLite Postgres MySQL
new_connection 289 µs 8.41 ms 2.15 ms
pool_checkout 39.3 µs 184 µs 207 µs
ping 17.8 µs 143 µs 172 µs
query_small_result 17.9 µs 185 µs 432 µs
query_large_result 39.3 ms 8.42 ms 10.3 ms

Notes on methodology

  • pool_checkout measures a full borrow + return cycle, with the pool's on-acquire ping disabled via test_before_acquire(false). With the default (true), each iteration would pay two network round-trips: a ping on acquire() and a ping on release. Note the pool always pings on release (return_to_pool) to flush time-sensitive state, and that isn't configurable — so for Postgres/MySQL this benchmark inherently includes one RTT and lands near ping. The delta over ping (e.g. Postgres 184 µs vs 143 µs) is the pool's own machinery.
  • new_connection times connect() only. Connections are closed gracefully (Terminate / COM_QUIT) outside the measured section via iter_custom, so teardown isn't included in the timing and we don't abandon thousands of sockets on the server over a run.
  • Fetched results are passed through std::hint::black_box so the optimizer can't elide the decode/allocation work being measured.
  • SQLite numbers aren't directly comparable to Postgres/MySQL. sqlite::memory: has no network round-trip, so its new_connection/ping/pool_checkout measure in-process setup rather than a real connection/RTT. (It's also why SQLite's query_large_result looks slow relative to the networked DBs: it's doing the full 10 000-row materialization with no pipelining to overlap, and the tuple decode dominates.) The three DBs share the same schema (16-byte blob, 10 000 rows) but should be read per-database.

Map130 and others added 6 commits August 9, 2026 19:17
Adds criterion benchmarks for SQLite covering all items from issue transact-rs#158:
new connection, pool checkout, ping, small-result query (1 row, TechEmpower-style),
and large-result query (10K rows with TEXT and BLOB columns).
Same benchmark suite as for SQLite: new connection, pool checkout, ping,
small-result query (1 row), and large-result query (10K rows with TEXT and BYTEA columns).
Requires DATABASE_URL pointing to a running Postgres instance.
Same benchmark suite as for SQLite and Postgres: new connection, pool checkout, ping,
small-result query (1 row), and large-result query (10K rows with TEXT and BLOB columns).
Requires DATABASE_URL pointing to a running MySQL instance.
Uses mysql-rsa feature for caching_sha2_password auth without TLS (MySQL 8+).
- pool_checkout: set test_before_acquire(false) so it measures the pool
  checkout fast path instead of an implicit ping on every acquire.
- new_connection: time connect() only via iter_custom and close the
  connection gracefully outside the measured section, so teardown isn't
  timed and sockets aren't abandoned across iterations.
- query benchmarks: pass fetched rows through std::hint::black_box to
  prevent the optimizer from eliding the decode/allocation work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_before_acquire(false) only disables the on-acquire ping; the pool
always pings on return_to_pool to flush time-sensitive state, so
pool_checkout measures a full borrow+return cycle (~1 RTT), not a pure
in-process checkout. Correct the comment accordingly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add benchmarks to inform performance-oriented refactoring

1 participant