bench: add connection and query benchmarks for SQLite, Postgres, MySQL - #4368
Open
Map130 wants to merge 6 commits into
Open
bench: add connection and query benchmarks for SQLite, Postgres, MySQL#4368Map130 wants to merge 6 commits into
Map130 wants to merge 6 commits into
Conversation
Adds criterion benchmarks for SQLite covering all items from issue transact-rs#158: new connection, pool checkout, ping, small-result query (1 row, TechEmpower-style), and large-result query (10K rows with TEXT and BLOB columns).
Same benchmark suite as for SQLite: new connection, pool checkout, ping, small-result query (1 row), and large-result query (10K rows with TEXT and BYTEA columns). Requires DATABASE_URL pointing to a running Postgres instance.
Same benchmark suite as for SQLite and Postgres: new connection, pool checkout, ping, small-result query (1 row), and large-result query (10K rows with TEXT and BLOB columns). Requires DATABASE_URL pointing to a running MySQL instance. Uses mysql-rsa feature for caching_sha2_password auth without TLS (MySQL 8+).
- pool_checkout: set test_before_acquire(false) so it measures the pool checkout fast path instead of an implicit ping on every acquire. - new_connection: time connect() only via iter_custom and close the connection gracefully outside the measured section, so teardown isn't timed and sockets aren't abandoned across iterations. - query benchmarks: pass fetched rows through std::hint::black_box to prevent the optimizer from eliding the decode/allocation work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_before_acquire(false) only disables the on-acquire ping; the pool always pings on return_to_pool to flush time-sensitive state, so pool_checkout measures a full borrow+return cycle (~1 RTT), not a pure in-process checkout. Correct the comment accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #158.
Adds criterion benchmarks covering all items from the issue checklist, for all three supported databases.
What's included
Each database gets a
benches/<db>/connection.rswith five benchmarks:new_connectionpool_checkouttest_before_acquire(false))pingquery_small_resultquery_large_resultHow to run
SQLite (no external DB needed):
Postgres:
MySQL (
mysql-rsaenablescaching_sha2_passwordauth without TLS for MySQL 8+):The existing
tests/docker-compose.ymlcan be used to spin up Postgres and MySQL.Example results
Local run (Docker
postgres:17/mysql:8.0over loopback, SQLite:memory:; median of criterion's estimate). Absolute numbers are hardware/loopback-specific — they're here to show the benchmarks work and that the metrics are distinct, not as a cross-machine reference.new_connectionpool_checkoutpingquery_small_resultquery_large_resultNotes on methodology
pool_checkoutmeasures a full borrow + return cycle, with the pool's on-acquire ping disabled viatest_before_acquire(false). With the default (true), each iteration would pay two network round-trips: a ping onacquire()and a ping on release. Note the pool always pings on release (return_to_pool) to flush time-sensitive state, and that isn't configurable — so for Postgres/MySQL this benchmark inherently includes one RTT and lands nearping. The delta overping(e.g. Postgres 184 µs vs 143 µs) is the pool's own machinery.new_connectiontimesconnect()only. Connections are closed gracefully (Terminate/COM_QUIT) outside the measured section viaiter_custom, so teardown isn't included in the timing and we don't abandon thousands of sockets on the server over a run.std::hint::black_boxso the optimizer can't elide the decode/allocation work being measured.sqlite::memory:has no network round-trip, so itsnew_connection/ping/pool_checkoutmeasure in-process setup rather than a real connection/RTT. (It's also why SQLite'squery_large_resultlooks slow relative to the networked DBs: it's doing the full 10 000-row materialization with no pipelining to overlap, and the tuple decode dominates.) The three DBs share the same schema (16-byte blob, 10 000 rows) but should be read per-database.