Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs-data/property-overrides.json
Original file line number Diff line number Diff line change
Expand Up @@ -778,7 +778,7 @@
"config_scope": "cluster"
},
"default_leaders_preference": {
"description": "Default settings for preferred location of topic partition leaders. It can be either \"none\" (no preference), or \"racks:<rack1>,<rack2>,...\" (prefer brokers with rack ID from the list).\n\nThe list can contain one or more rack IDs. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.\n\nIf config_ref:enable_rack_awareness,true,properties/cluster-properties[] is set to `false`, leader pinning is disabled across the cluster.",
"description": "Default settings for preferred location of topic partition leaders. It can be either \"none\" (no preference), or \"racks:<rack1>,<rack2>,...\" (prefer brokers with rack ID from the list).\n\nThe list can contain one or more rack IDs. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.\n\nIf prop:enable_rack_awareness[link=true] is set to `false`, leader pinning is disabled across the cluster.",
"related_topics": [
"xref:develop:produce-data/leader-pinning.adoc[Leader pinning]"
],
Expand All @@ -795,7 +795,7 @@
"version": "v26.1.1"
},
"delete.retention.ms": {
"description": "The retention time for tombstone records in a compacted topic. Redpanda removes tombstone records after the retention limit is exceeded.\n\nIf you have enabled Tiered Storage and set <<redpandaremoteread,`redpanda.remote.read`>> or <<redpandaremotewrite,`redpanda.remote.write`>> for the topic, you cannot enable tombstone removal.\n\nIf both `delete.retention.ms` and the cluster property config_ref:tombstone_retention_ms,true,properties/cluster-properties[] are set, `delete.retention.ms` overrides the cluster level tombstone retention for an individual topic.\n\nThis property supports three states:\n\n* Positive value: Sets the milliseconds to retain tombstone records before removal.\n* 0: Tombstone records are immediately eligible for removal.\n* Negative value: Disables tombstone removal entirely for this topic.",
"description": "The retention time for tombstone records in a compacted topic. Redpanda removes tombstone records after the retention limit is exceeded.\n\nIf you have enabled Tiered Storage and set <<redpandaremoteread,`redpanda.remote.read`>> or <<redpandaremotewrite,`redpanda.remote.write`>> for the topic, you cannot enable tombstone removal.\n\nIf both `delete.retention.ms` and the cluster property prop:tombstone_retention_ms[link=true] are set, `delete.retention.ms` overrides the cluster level tombstone retention for an individual topic.\n\nThis property supports three states:\n\n* Positive value: Sets the milliseconds to retain tombstone records before removal.\n* 0: Tombstone records are immediately eligible for removal.\n* Negative value: Disables tombstone removal entirely for this topic.",
"related_topics": [
"xref:reference:properties/cluster-properties.adoc#tombstone_retention_ms[`tombstone_retention_ms`]",
"xref:manage:cluster-maintenance/compaction-settings.adoc#tombstone-record-removal[Tombstone record removal]"
Expand Down Expand Up @@ -1435,7 +1435,7 @@
"config_scope": "topic"
},
"max.message.bytes": {
"description": "The maximum size of a message or batch of a topic. If a compression type is enabled, `max.message.bytes` sets the maximum size of the compressed message or batch.\n\nIf `max.message.bytes` is set to a positive value, it overrides the cluster property xref:reference:properties/cluster-properties.adoc#kafka_batch_max_bytes[`kafka_batch_max_bytes`] for the topic.\n\nSet an upper limit for `max.message.bytes` using the cluster property config_ref:kafka_max_message_size_upper_limit_bytes,true,properties/cluster-properties[`kafka_max_message_size_upper_limit_bytes`].",
"description": "The maximum size of a message or batch of a topic. If a compression type is enabled, `max.message.bytes` sets the maximum size of the compressed message or batch.\n\nIf `max.message.bytes` is set to a positive value, it overrides the cluster property xref:reference:properties/cluster-properties.adoc#kafka_batch_max_bytes[`kafka_batch_max_bytes`] for the topic.\n\nSet an upper limit for `max.message.bytes` using the cluster property prop:kafka_max_message_size_upper_limit_bytes[link=true].",
"related_topics": [
"xref:reference:properties/cluster-properties.adoc#kafka_batch_max_bytes[`kafka_batch_max_bytes`]",
"xref:develop:produce-data/configure-producers.adoc#message-batching[Message batching]",
Expand Down Expand Up @@ -1811,7 +1811,7 @@
]
},
"redpanda.leaders.preference": {
"description": "The preferred location (rack) for partition leaders of a topic.\n\nThis property inherits the value from the config_ref:default_leaders_preference,true,properties/cluster-properties[] cluster configuration property. You may override the cluster-wide setting by specifying the value for individual topics.\n\nIf the cluster configuration property config_ref:enable_rack_awareness,true,properties/cluster-properties[] is set to `false`, Leader Pinning is disabled across the cluster.",
"description": "The preferred location (rack) for partition leaders of a topic.\n\nThis property inherits the value from the prop:default_leaders_preference[link=true] cluster configuration property. You may override the cluster-wide setting by specifying the value for individual topics.\n\nIf the cluster configuration property prop:enable_rack_awareness[link=true] is set to `false`, Leader Pinning is disabled across the cluster.",
"related_topics": [
"xref:develop:produce-data/leader-pinning.adoc[Leader pinning]"
],
Expand Down Expand Up @@ -1858,7 +1858,7 @@
"config_scope": "topic"
},
"redpanda.storage.mode": {
"description": "The storage mode for a topic. Determines how topic data is stored and whether it is eligible for upload to object storage.\n\nAccepted values:\n\n* `local`: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings.\nifndef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables xref:manage:tiered-storage.adoc[Tiered Storage] for the topic.\n* `cloud`: Topic data is stored in object storage using the xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] architecture. Local storage is used only as a write buffer.\nendif::[]\nifdef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables Tiered Storage for the topic.\n* `cloud`: Topic data is stored in object storage using the Cloud Topics architecture. Local storage is used only as a write buffer.\nendif::[]\n* `unset`: Specifies that the topic's storage mode is unset, regardless of the cluster default. The topic may still have Tiered Storage enabled through the legacy properties `redpanda.remote.read` and `redpanda.remote.write`.\n\nThis property overrides the cluster-wide config_ref:default_redpanda_storage_mode,true,properties/cluster-properties[] setting for individual topics.",
"description": "The storage mode for a topic. Determines how topic data is stored and whether it is eligible for upload to object storage.\n\nAccepted values:\n\n* `local`: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings.\nifndef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables xref:manage:tiered-storage.adoc[Tiered Storage] for the topic.\n* `cloud`: Topic data is stored in object storage using the xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] architecture. Local storage is used only as a write buffer.\nendif::[]\nifdef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables Tiered Storage for the topic.\n* `cloud`: Topic data is stored in object storage using the Cloud Topics architecture. Local storage is used only as a write buffer.\nendif::[]\n* `unset`: Specifies that the topic's storage mode is unset, regardless of the cluster default. The topic may still have Tiered Storage enabled through the legacy properties `redpanda.remote.read` and `redpanda.remote.write`.\n\nThis property overrides the cluster-wide prop:default_redpanda_storage_mode[link=true] setting for individual topics.",
"related_topics": [
"self-managed-only: xref:manage:tiered-storage.adoc[Tiered Storage]",
"self-managed-only: xref:develop:manage-topics/cloud-topics.adoc[Manage Cloud Topics]"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
:page-categories: Deployment
:page-aliases: deploy:deployment-option/self-hosted/manual/production/dev-deployment.adoc

You can deploy Redpanda using well-known configuration properties optimized for a development or test environment. This configuration uses less system resources and simplifies setup, but it's not suitable for a production deployment. For example, in xref:reference:rpk/rpk-redpanda/rpk-redpanda-mode.adoc#development-mode[development mode], the default `group_topic_partitions` is 3, and the default `storage_min_free_bytes` is 1 GiB. In production mode, the default `group_topic_partitions` is 16, and the default `storage_min_free_bytes` is 5 GiB.
You can deploy Redpanda using well-known configuration properties optimized for a development or test environment. This configuration uses less system resources and simplifies setup, but it's not suitable for a production deployment. For example, in xref:reference:rpk/rpk-redpanda/rpk-redpanda-mode.adoc#development-mode[development mode], the default prop:group_topic_partitions[] is 3, and the default prop:storage_min_free_bytes[] is 1 GiB. In production mode, the default prop:group_topic_partitions[] is 16, and the default prop:storage_min_free_bytes[] is 5 GiB.

[NOTE]
====
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -429,7 +429,7 @@ This is useful when using attached disks in instances with ephemeral operating s
|
| Set bucket name to enable Tiered Storage.

| `schema_registry_replication_factor`
| prop:schema_registry_replication_factor[]
| 1
| The replication factor of Schema Registry's internal storage topic.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ administrators can access the host.
* mTLS authentication: You can generate client and server x509 certificates
before starting Redpanda for the first time, refer to them in
`redpanda.yaml`, and use the client certificate when accessing the Admin API.
* Username/password authentication: Use the combination of `admin_api_require_auth`,
* Username/password authentication: Use the combination of prop:admin_api_require_auth[],
`superusers`, and `RP_BOOTSTRAP_USER` to access the Admin API username/password
authentication. You probably still want to enable TLS on the Admin API
endpoint to protect credentials in flight.
Expand All @@ -199,23 +199,23 @@ endpoint to protect credentials in flight.

Seed servers help new brokers join a cluster by directing requests from newly-started brokers to an existing cluster. The xref:reference:properties/broker-properties.adoc#seed_servers[`seed_servers`] broker property controls how Redpanda finds its peers when initially forming a cluster. It is dependent on the xref:reference:properties/broker-properties.adoc#empty_seed_starts_cluster[`empty_seed_starts_cluster`] broker property.

Starting with Redpanda version 22.3, you should explicitly set `empty_seed_starts_cluster` to `false` on every broker, and every broker in the cluster should have the same value set for `seed_servers`. With this set of configurations, Redpanda clusters form with these guidelines:
Starting with Redpanda version 22.3, you should explicitly set prop:empty_seed_starts_cluster[] to `false` on every broker, and every broker in the cluster should have the same value set for prop:seed_servers[]. With this set of configurations, Redpanda clusters form with these guidelines:

* When a broker starts and it is a seed server (its address is in the `seed_servers` list), it waits for all other seed servers to start up, and it forms a cluster with all seed servers as members.
* When a broker starts and it is a seed server (its address is in the prop:seed_servers[] list), it waits for all other seed servers to start up, and it forms a cluster with all seed servers as members.
* When a broker starts and it is not a seed server, it sends requests to the seed servers to join the cluster.

It is essential that all seed servers have identical values for the `seed_servers` list. Redpanda strongly recommends at least three seed servers when forming a cluster. Each seed server decreases the likelihood of unintentionally forming a split brain cluster. To ensure brokers can always discover the cluster, at least one seed server should be available at all times.
It is essential that all seed servers have identical values for the prop:seed_servers[] list. Redpanda strongly recommends at least three seed servers when forming a cluster. Each seed server decreases the likelihood of unintentionally forming a split brain cluster. To ensure brokers can always discover the cluster, at least one seed server should be available at all times.

By default, for backward compatibility, `empty_seed_starts_cluster` is set to `true`, and Redpanda clusters form with the guidelines used prior to version 22.3:
By default, for backward compatibility, prop:empty_seed_starts_cluster[] is set to `true`, and Redpanda clusters form with the guidelines used prior to version 22.3:

* When a broker starts with an empty `seed_servers` list, it creates a single broker cluster with itself as the only member.
* When a broker starts with a non-empty `seed_servers` list, it sends requests to the brokers in that list to join the cluster.
* When a broker starts with an empty prop:seed_servers[] list, it creates a single broker cluster with itself as the only member.
* When a broker starts with a non-empty prop:seed_servers[] list, it sends requests to the brokers in that list to join the cluster.

You should never have more than one broker with an empty `seed_servers` list, which would result in the creation of multiple clusters.
You should never have more than one broker with an empty prop:seed_servers[] list, which would result in the creation of multiple clusters.

IMPORTANT: Redpanda expects its storage to be persistent, and it's an error
to erase a broker's drive and restart it. However, in some environments (like when migrating to a different Node pool on Kubernetes), truly persistent storage is unavailable,
and brokers may find their data volumes erased. For such environments, Redpanda recommends setting `empty_seed_starts_cluster` to false and designating a set of seed servers such that they couldn't lose their storage simultaneously.
and brokers may find their data volumes erased. For such environments, Redpanda recommends setting prop:empty_seed_starts_cluster[] to false and designating a set of seed servers such that they couldn't lose their storage simultaneously.

=== Do not configure broker IDs

Expand All @@ -229,7 +229,7 @@ Redpanda assigns unique IDs automatically to prevent issues such as:
- Brokers with empty disks rejoining the cluster.
- Conflicts during recovery or scaling.

Manually setting or reusing `node_id` values, even for decommissioned brokers, can cause cluster inconsistencies and operational failures.
Manually setting or reusing prop:node_id[] values, even for decommissioned brokers, can cause cluster inconsistencies and operational failures.
====

include::deploy:partial$self-test.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ redpanda-external NodePort 10.96.137.220 <none> 9644:31644/TCP,909
The headless Service associated with a StatefulSet gives the Pods their network identity in the form of a fully qualified domain name (FQDN). Both Redpanda brokers in the same Redpanda cluster and clients within the same Kubernetes cluster use this FQDN to communicate with each other.

An important requirement of distributed applications such as Redpanda is peer discovery: The ability for each broker to find other brokers in the same cluster.
When each Pod is rolled out, its `seed_servers` field is updated with the FQDN of each Pod in the cluster so that they can discover each other.
When each Pod is rolled out, its prop:seed_servers[] field is updated with the FQDN of each Pod in the cluster so that they can discover each other.

[,bash]
----
Expand Down
2 changes: 1 addition & 1 deletion modules/deploy/partials/requirements.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ endif::[]

- Each Redpanda broker must have at least 2 MB of memory for each topic partition replica.
+
The total memory available for partition replicas is determined as a percentage of the cluster's total memory, which is controlled by the xref:reference:tunable-properties.adoc#topic_partitions_memory_allocation_percent[`topic_partitions_memory_allocation_percent`] setting. Each partition replica consumes xref:reference:tunable-properties.adoc#topic_memory_per_partition[`topic_memory_per_partition`] bytes from this pool. If insufficient memory is available, topic operations will fail. You can adjust the allocation ratio using `topic_partitions_memory_allocation_percent`, but doing so is not recommended, as lowering it may lead to instability or degraded performance.
The total memory available for partition replicas is determined as a percentage of the cluster's total memory, which is controlled by the xref:reference:tunable-properties.adoc#topic_partitions_memory_allocation_percent[`topic_partitions_memory_allocation_percent`] setting. Each partition replica consumes xref:reference:tunable-properties.adoc#topic_memory_per_partition[`topic_memory_per_partition`] bytes from this pool. If insufficient memory is available, topic operations will fail. You can adjust the allocation ratio using prop:topic_partitions_memory_allocation_percent[], but doing so is not recommended, as lowering it may lead to instability or degraded performance.

*Recommendations*:

Expand Down
2 changes: 1 addition & 1 deletion modules/develop/pages/data-transforms/configure.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Set the following based on the number of functions you have and the amount of me
- xref:reference:properties/cluster-properties.adoc#data_transforms_per_core_memory_reservation[`data_transforms_per_core_memory_reservation`]: Increase this setting if you plan to deploy a large number of data transforms or if your transforms are memory-intensive. Reducing it may limit the number of concurrent transforms.
- xref:reference:properties/cluster-properties.adoc#data_transforms_per_function_memory_limit[`data_transforms_per_function_memory_limit`]: Adjust this setting if individual transform functions require more memory to process records efficiently. Reducing it may cause memory errors in complex transforms.

The maximum number of functions that can be deployed to a cluster is equal to `data_transforms_per_core_memory_reservation` / `data_transforms_per_function_memory_limit`. When that limit is hit, Redpanda cannot allocate memory for the VM and the transforms stay in `errored` states.
The maximum number of functions that can be deployed to a cluster is equal to prop:data_transforms_per_core_memory_reservation[] / prop:data_transforms_per_function_memory_limit[]. When that limit is hit, Redpanda cannot allocate memory for the VM and the transforms stay in `errored` states.
endif::[]

ifndef::env-cloud[]
Expand Down
Loading
Loading