Increase integer column types to bigint, fix mysql index errors#138
Increase integer column types to bigint, fix mysql index errors#138ffried wants to merge 16 commits intotiagozip:mainfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR addresses database compatibility issues by increasing integer column types to bigint (to properly store Date.now() values) and changing indexed text columns to varchar(4096) for MySQL compatibility. It also adds migration logic for existing databases and refactors the periodic cleanup into a separate function.
Key Changes:
- Changed integer columns storing timestamps to bigint across all tables
- Introduced MySQL-compatible varchar(4096) for indexed text columns
- Added migration logic with
changeIntToBigInt()function to update existing databases
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
65791dc to
1d32ecd
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
thanks, ill start reviewing and testing later. just to be sure, can you confirm that varchar(4096) does indeed work on both SQLite, postgres and MySQL? |
|
It's only used for MySQL. Postgres an sqlite continue to use text. |
|
I have tested this PR and can confirm it fixes this bug |
|
After more trial and error, it seems there are still issues with Postgres compatibility |
This might totally be the case, since I've only fixed the most obvious once (namely the ones storing timestamps). |
|
One issue I encountered was on the I also noticed that the UI graph for a site I triggered a few solve requests was not displaying anything, although the count was counting them. Lastly, I think this SQL query is not working because I got |
Can you retest with the changes and let me know if they work for you? |
|
Sure, I'll give it a shot. |
|
@ancashoria Any update? |
66cb2ad to
df86deb
Compare
|
I have switched to SQLite so I won't be testing this any time soon. Sorry |
|
I've tested this now with PostgreSQL and it's working just fine. We're running our instance now using the current HEAD of this PR. |
|
thanks! could you confirm if you have tested this on mysql as well? |
|
@tiagozip I don't have a mysql setup right now. I've tested the migrations on mysql (which is where the need for the fix for the index on text columns was discovered). Other than that, I don't expect mysql to have any issues with the changes I've made here. I can't guarantee that I'll find the time soon to test MySQL. I was able to test Postgres using our (now) production setup, that I could easily use for testing since there was nothing in it, but the environment was there. |
|
@tiagozip Alright, I've spent a couple more hours on this and tested it with mysql. My dislike for MySQL grew even more. The changes needed for Postgres are simply due to it being properly typed (compared to SQLite). That's fine and expected. MySQL, however, now needs different column types (due to index limitations) and escaping all table names (which I think should be handled by Bun...?). Also note that Bun seems to have difficulties with longer MySQL passwords: oven-sh/bun#26195 Besides that, I've found a couple of other issues. Though that's not related to this PR and I'll open separate issues for that. |
|
Forgot to test the redeem. That works now as well... |
fd134f9 to
215b5b0
Compare
|
@tiagozip We've run this with Postgres for a while now w/o any apparent issues. Any chance on getting this merged? |
|
that's great, did you also test mysql? |
|
@tiagozip Yes, had a test run locally where I clicked through everything. And did a quick HTML widget test. |
|
this should be in prod soon after additional testing! |
|
Could this work with the new instrumentation challenges? |
|
There's a ton of conflicts now. Also, the current |
|
By the way, I'm planning on switching the entirety of Cap Standalone to Valkey/Redis for speed and to avoid issues with volumes.
|
|
@tiagozip you still need a database, though. ValKey / Redis isn't a good solution for long-term storage. There is hardly any clever backup mechanism, nor migrations guarantees on new major releases. Their primary use case is caching / temporary storage. |
|
Cap Standalone fortunately doesn't need any very long-term storage other than site keys, everything else is relatively temporary, from redeem tokens to JWT blocklist and sessions
|
|
Hey, we are in the process of implementing Cap for our applications. If there is no work being done on this, we could try to get the current branch working. Or if there is anything we can do to get this merged, we would be happy to help. |
|
I'd recommend keeping the default Bun SQLite db url until v3 releases. If
you really need Postgres feel free to use the fork though.
…On Tue, Mar 10, 2026, 10:03 Lenora Bilişim ***@***.***> wrote:
*lenoragpt* left a comment (tiagozip/cap#138)
<#138 (comment)>
Hey, we are in the process of implementing Cap for our applications. If
there is no work being done on this, we could try to get the current branch
working. Or if there is anything we can do to get this merged, we would be
happy to help.
—
Reply to this email directly, view it on GitHub
<#138 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQ3M5XXPDUNVJWXRKUPPA3T4P7R7HAVCNFSM6AAAAACMEHADXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DAMZQGE3DAOBXGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
|
The
integertype in non-SQLite databases is too small to holdDate.now()values.This increases the type to bigint. To deal with existing databases,
alter tablecommands were added. They should simply succeed in case the type already matches (which is the case for newly created databases).This fixes #136.
Also, in MySQL you can't use
textwith indices (unless specifying a prefix length). This is why I've changed the data type tovarchar(4096)for these columns. Note that noalter tableis needed for these, because the creation of the table already fails. The size (4096) is currently arbitrarily chosen and can be set to anything large enough (up to 65,535).