Skip to content

Bump PG minors for Citus 12.1 CI (14.24 / 15.19 / 16.15) - #243

Merged
ibrahim halatci (ihalatci) merged 1 commit into
bump-pg16-16.13-citus-12.1from
bump-pg-minors-citus-12.1-20260815
Aug 19, 2026
Merged

Bump PG minors for Citus 12.1 CI (14.24 / 15.19 / 16.15)#243
ibrahim halatci (ihalatci) merged 1 commit into
bump-pg16-16.13-citus-12.1from
bump-pg-minors-citus-12.1-20260815

Conversation

@ihalatci

@ihalatci ibrahim halatci (ihalatci) commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Bumps the PostgreSQL minors used by the Citus 12.1 CI image line to the
latest 2026-08-14 releases:

Major Was Now
14 14.19 14.24
15 15.14 15.19
16 16.13 16.15

All three are available on apt.postgresql.org for bookworm
(14.24-1.pgdg12+2, 15.19-1.pgdg12+2, 16.15-1.pgdg12+2).

Why this needs a CI look

This deliberately un-pins PG14 and PG15. Commit 98d1a12 held them back at
14.19 / 15.14 because the May-2026 coordinated minor set
(14.23 / 15.18 / 16.14) crashed PostgreSQL's own vanilla select_parallel
test on all three majors, and only PG16 was moved forward (to 16.13).

The point of this PR is to re-test that against the August set. If
select_parallel still crashes on PG14/PG15, the fallback is to keep this
line on the PG16-only bump (16.13 -> 16.15) and leave PG14/PG15 pinned.

The build-test-images run for this branch is green and published the dev
image set as image_suffix: -dev-b63fad9
(e.g. ghcr.io/citusdata/exttester:16.15-dev-b63fad9). Wire that suffix into
the Citus 12.1 test run to get the vanilla select_parallel signal — a green
image build here only proves the minors install, not that the tests pass.

Base branch

The 12.1 line has no long-lived base branch in this repo, so this targets
bump-pg16-16.13-citus-12.1, which is the current tip of that line.

Only circleci/images/PG_VERSIONS is touched.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

… / 16.15)

PG14 14.19->14.24, PG15 15.14->15.19, PG16 16.13->16.15.

This intentionally un-pins PG14/PG15, which 98d1a12 held back at
14.19/15.14 because the May-2026 coordinated set (14.23/15.18/16.14)
crashed PostgreSQL's own vanilla select_parallel test. Re-testing
against the August set to see whether that regression is gone.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ihalatci
ibrahim halatci (ihalatci) merged commit 4af5e7f into bump-pg16-16.13-citus-12.1 Aug 19, 2026
15 checks passed
ibrahim halatci (ihalatci) added a commit to citusdata/citus that referenced this pull request Aug 19, 2026
Validation PR for the-process
[#243](citusdata/the-process#243). Repoints CI
at the freshly built dev images carrying the new PostgreSQL minors
published 2026-08-14/15.

| | before | after |
|---|---|---|
| PG14 | 14.19 | 14.24 |
| PG15 | 15.14 | 15.19 |
| PG16 | 16.13 | 16.15 |

`image_suffix`: `-dev-b63fad9`

**The PG14/PG15 un-pin experiment succeeded.** PG14/PG15 were previously
pinned at 14.19/15.14 because the May-2026 minor set crashed
PostgreSQL's vanilla `select_parallel` regression test. `check-vanilla`
is green on PG14, PG15 and PG16 with the August set, so the crash is
fixed upstream and the pin can be dropped.

---

## Fallout from the new minors

Validation surfaced two independent PostgreSQL security changes in this
minor set that break Citus.

### 1. psql `COPY FROM STDIN` (fixed here)

When a `COPY` fails early, the new psql silently swallows every line up
to and including the next `\.`. Tests that relied on the old behaviour
were adjusted: a terminating `\.` was added where one was missing, stray
statements after a swallowed block were removed, and expected output was
trimmed for the lines that no longer execute.

### 2. `output_plugin_libraries` (fixed here)

PostgreSQL 14.24, 15.19, 16.15, 17.11 and 18.6 add an
`output_plugin_libraries` GUC. Only the libraries it lists may be used
as logical decoding output plugins. It defaults to `pgoutput,
test_decoding` and is `superuser`-settable, so it takes a reload rather
than a restart.

**Upgrade note for operators.** Citus uses an output plugin named
`citus` for logical replication during non-blocking shard splits. On
these minors the following fail until the plugin is allowed:

* `citus_split_shard_by_split_points(..., 'force_logical')` and `'auto'`
* `create_distributed_table_concurrently()`
* `citus_isolate_tenant_to_new_shard(..., 'force_logical'` / `'auto')`

Shard **moves** and the rebalancer are unaffected -- they use
`pgoutput`. CDC through the `pgoutput` shim is unaffected; CDC through
`wal2json` needs the same treatment, exactly as it does on vanilla
PostgreSQL.

Remediation, on **every** node:

```sql
ALTER SYSTEM SET output_plugin_libraries = pgoutput, test_decoding, citus;
SELECT pg_reload_conf();
```

The GUC is `GUC_LIST_QUOTE`, so quoting the whole list (`'pgoutput,
test_decoding, citus'`) stores it as a single name and does **not**
work. Use the bare list above, or quote each element individually.

This PR does not work around the restriction -- allowing a decoder is
deliberately an operator action. Instead Citus now **fails fast and
clearly**: a preflight check runs against the source node before any
shards, publications or replication slots are created, and raises an
error naming the plugin, the node, the current allowlist value, and a
ready-to-paste `ALTER SYSTEM` hint.

### Testing

The regression and CDC harnesses start their own clusters, so in CI *we*
are the operator: `pg_regress_multi.pl` and `cdctestlib.pm` now write
the allowlist themselves, guarded by a probe of `postgresql.conf.sample`
so they stay compatible with older minors where the GUC does not exist.

To keep the un-remediated path covered, a new
`check-split-output-plugin-denied` job deliberately skips that override
(via `CITUS_TEST_SKIP_OUTPUT_PLUGIN_ALLOWLIST=1`) and asserts that a
non-blocking split fails with the new error and leaks no shards,
publications or replication slots.

---

**Draft:** these are `-dev-` images. Do not merge until the-process
[#243](citusdata/the-process#243) is merged and
this PR is repointed at the new dev suffix that merge produces. (The
12.1 line in the-process is not a `release-*` branch, so it emits
`-dev-` images rather than `-v` ones.)

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7c6370b2-06fd-4491-bf92-ecb811d34518
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant