From e84ab29b58833e9eaff1f08913b9b586ba52e886 Mon Sep 17 00:00:00 2001 From: zhangzujian Date: Mon, 3 Aug 2026 16:47:46 +0800 Subject: [PATCH 1/4] docs: explain Kube-OVN pinger probe targets on ACP 4.4 --- ..._Pinger_Public_Probe_Targets_on_ACP_4_4.md | 113 ++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md diff --git a/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md b/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md new file mode 100644 index 000000000..26b8f64eb --- /dev/null +++ b/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md @@ -0,0 +1,113 @@ +--- +kind: + - Troubleshooting +products: + - Alauda Container Platform +ProductsVersion: + - 4.4.x +--- + +# Replace Kube-OVN pinger public probe targets on ACP 4.4 + +## Issue + +In an Alauda Container Platform cluster that uses Kube-OVN, CoreDNS can repeatedly report failed queries for a public domain even when applications do not query that domain. For an IPv4 cluster, the log commonly mentions `kube-ovn.io.`; IPv6 and dual-stack clusters can instead show `google.com.`. A representative CoreDNS error has the following form: + +```text +[ERROR] plugin/errors: 2 kube-ovn.io. A: read udp ...: i/o timeout +``` + +The errors are especially common in air-gapped environments or private networks that cannot use an upstream public DNS server. They are probe failures and do not by themselves indicate a Pod-networking or application-DNS outage. + +## Environment + +- Alauda Container Platform 4.4.x +- **Alauda Container Platform Networking for Kube-OVN** as the cluster CNI plugin +- CoreDNS logs contain repeated failures for the Kube-OVN pinger's configured public domain + +## Resolution + +Replace the public probe targets with targets that are reachable from the cluster. + +Before updating the plugin, choose the following values: + +| Plugin field | Recommended value | +| --- | --- | +| **Pinger External Probe DNS** | A fully qualified domain name that CoreDNS can resolve without using a public upstream resolver. Include the trailing dot to prevent search-domain expansion. For example, use `kubernetes.default.svc.cluster.local.` when the cluster domain is `cluster.local`, or use an organization-owned internal FQDN. | +| **Pinger External Probe IP** | An IP address that accepts ICMP echo requests from the cluster nodes. For a dual-stack cluster, provide a comma-separated IPv4 and IPv6 pair if both address families must be monitored. | + +:::note +Leaving either field empty in the ACP 4.4 plugin form preserves the Kube-OVN default for the active network stack. An empty **Pinger External Probe DNS** field therefore does not remove the default public-domain query. Enter an explicit reachable target. +::: + +1. In the web console, navigate to **Administrator** > **Marketplace** > **Cluster Plugins** and select the affected cluster. + +2. Search for `ovn` and locate **Alauda Container Platform Networking for Kube-OVN**. + +3. In the plugin row, open the action menu (vertical ⋮) and select **Update**. + +4. Set **Pinger External Probe DNS** to the internal FQDN selected above. If the default public probe IP is also unreachable, set **Pinger External Probe IP** to the selected internal IP address. + +5. Submit the update and wait for the `kube-ovn-pinger` DaemonSet rollout to complete: + + ```bash + kubectl -n kube-system rollout status daemonset/kube-ovn-pinger + ``` + +6. Confirm that the DaemonSet contains the new targets: + + ```bash + kubectl -n kube-system get daemonset kube-ovn-pinger \ + -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' + echo + ``` + + Verify that the output includes the expected `--external-dns=` and, if changed, `--external-address=` arguments. + +7. After at least one probe interval, verify that CoreDNS no longer reports failures for the former public target: + + ```bash + kubectl -n kube-system logs -l k8s-app=kube-dns \ + --since=10m --prefix \ + | grep -E 'kube-ovn\.io\.|google\.com\.' + ``` + + No new matching error should appear after the pinger rollout. If the command also returns older entries from before the rollout, repeat it with a shorter `--since` window. + +The pinger continues to publish external-connectivity health metrics, but those metrics now describe connectivity to the configured internal targets rather than public Internet connectivity. + +## Root Cause + +Kube-OVN runs one `kube-ovn-pinger` Pod on each eligible node. The pinger periodically checks an external IP address and resolves an external DNS name so it can publish external-connectivity metrics. + +The default target depends on the cluster network stack. IPv4 uses `kube-ovn.io.` for the DNS probe, while IPv6 and dual-stack use `google.com.`. When a private cluster cannot reach a public resolver, every pinger Pod continues to issue the configured query and CoreDNS records the upstream timeout or failure. The number of log entries therefore grows with both the node count and the probe frequency. + +ACP 4.4 exposes the pinger IP and DNS targets in the Kube-OVN cluster-plugin configuration. Replacing the defaults through that configuration is persistent across plugin reconciliation and cluster-plugin updates. + +## Diagnostic Steps + +Inspect the currently rendered pinger arguments: + +```bash +kubectl -n kube-system get daemonset kube-ovn-pinger \ + -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' +echo +``` + +If the output contains `--external-dns=kube-ovn.io.` or `--external-dns=google.com.`, the Kube-OVN pinger is generating the corresponding DNS query. + +List the pinger Pods and their node placement to estimate how many independent probes are running: + +```bash +kubectl -n kube-system get pods -l app=kube-ovn-pinger -o wide +``` + +Check recent CoreDNS errors for the default public targets: + +```bash +kubectl -n kube-system logs -l k8s-app=kube-dns \ + --since=10m --prefix \ + | grep -E '\[ERROR\].*(kube-ovn\.io\.|google\.com\.)' +``` + +Before attributing a broader DNS incident to this probe, also verify that application service names resolve normally. A failure limited to the pinger's public target affects its external-DNS health metric and produces log noise, but it does not by itself show that cluster-local DNS or Pod networking is unavailable. From 7fd6a2295b30d1e30b9d8fd087cf58747d874a5d Mon Sep 17 00:00:00 2001 From: zhangzujian Date: Mon, 3 Aug 2026 17:03:49 +0800 Subject: [PATCH 2/4] docs: cover disabling Kube-OVN external probes --- ...e_OVN_Pinger_External_Probes_on_ACP_4_4.md | 131 ++++++++++++++++++ ..._Pinger_Public_Probe_Targets_on_ACP_4_4.md | 113 --------------- 2 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md delete mode 100644 docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md diff --git a/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md new file mode 100644 index 000000000..97b1b9c1a --- /dev/null +++ b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md @@ -0,0 +1,131 @@ +--- +kind: + - Troubleshooting +products: + - Alauda Container Platform +ProductsVersion: + - 4.4.x +--- + +# Configure or disable Kube-OVN pinger external probes on ACP 4.4 + +## Issue + +In an Alauda Container Platform cluster that uses Kube-OVN, the following unexpected public traffic can appear even when applications do not generate it: + +- CoreDNS repeatedly queries `kube-ovn.io.` in an IPv4 cluster or `google.com.` in an IPv6 or dual-stack cluster. A representative CoreDNS error has the following form: + + ```text + [ERROR] plugin/errors: 2 kube-ovn.io. A: read udp ...: i/o timeout + ``` + +- A firewall, flow log, or traffic-audit system records ICMP traffic to `1.1.1.1`, `2606:4700:4700::1111`, or both. The traffic recurs from nodes that run `kube-ovn-pinger`. + +These probes are especially visible in air-gapped environments and private networks that prohibit unapproved public egress. Probe failures or audit records do not by themselves indicate a Pod-networking or application-DNS outage, but an operator might want to redirect the probes to approved internal targets or disable them completely. + +## Environment + +- Alauda Container Platform 4.4.x +- **Alauda Container Platform Networking for Kube-OVN** as the cluster CNI plugin +- CoreDNS logs or network audits contain repeated traffic to the Kube-OVN pinger's default public targets + +## Resolution + +Choose one of the following policies: + +| Policy | **Pinger External Probe DNS** | **Pinger External Probe IP** | Result | +| --- | --- | --- | --- | +| Disable public probes | Leave empty | Leave empty | The pinger skips both the DNS lookup and the ICMP probe. | +| Keep probes on approved internal targets | Enter an internal FQDN, such as `kubernetes.default.svc.cluster.local.` when the cluster domain is `cluster.local` | Enter an internal address that accepts ICMP echo requests | The pinger retains external-probe metrics without sending traffic to the default public targets. | + +You can also disable only one probe by leaving its field empty. When configuring a DNS target, use a fully qualified domain name with a trailing dot to prevent search-domain expansion. For a dual-stack IP probe, provide a comma-separated IPv4 and IPv6 pair if both address families must be monitored. + +1. In the web console, navigate to **Administrator** > **Marketplace** > **Cluster Plugins** and select the affected cluster. + +2. Search for `ovn` and locate **Alauda Container Platform Networking for Kube-OVN**. + +3. In the plugin row, open the action menu (vertical ⋮) and select **Update**. + +4. Apply the selected policy: + + - To stop all public pinger traffic, clear both **Pinger External Probe DNS** and **Pinger External Probe IP**. + - To retain the probes, replace the defaults with the approved internal FQDN and IP address. + +5. Submit the update and wait for the `kube-ovn-pinger` DaemonSet rollout to complete: + + ```bash + kubectl -n kube-system rollout status daemonset/kube-ovn-pinger + ``` + +6. Confirm that the DaemonSet contains the expected arguments: + + ```bash + kubectl -n kube-system get daemonset kube-ovn-pinger \ + -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' + echo + ``` + + If the probes are disabled, the output must include `--external-dns=` and `--external-address=` with empty values. If the probes are redirected, verify the expected `--external-dns=` and `--external-address=` values. + +7. After at least one probe interval, verify that CoreDNS no longer reports queries for the former public target: + + ```bash + kubectl -n kube-system logs -l k8s-app=kube-dns \ + --since=10m --prefix \ + | grep -E 'kube-ovn\.io\.|google\.com\.' + ``` + + No new matching entry should appear after the pinger rollout. If the command also returns older entries from before the rollout, repeat it with a shorter `--since` window. + +8. Check the firewall, flow log, or traffic-audit system and confirm that no new pinger traffic reaches `1.1.1.1` or `2606:4700:4700::1111` after the rollout. + +When the fields are empty, the pinger stops updating the corresponding external-address and external-DNS health metrics but continues its other Kube-OVN health checks. When internal targets are configured, the metrics describe connectivity to those targets rather than public Internet connectivity. + +## Root Cause + +Kube-OVN runs one `kube-ovn-pinger` Pod on each eligible node. The pinger periodically sends ICMP echo requests to an external IP address and resolves an external DNS name so it can publish external-connectivity metrics. + +The default targets depend on the cluster network stack: + +| Network stack | External address | External DNS name | +| --- | --- | --- | +| IPv4 | `1.1.1.1` | `kube-ovn.io.` | +| IPv6 | `2606:4700:4700::1111` | `google.com.` | +| Dual stack | `1.1.1.1,2606:4700:4700::1111` | `google.com.` | + +Every pinger Pod produces an independent stream. Node count and probe frequency therefore multiply both the DNS queries and ICMP traffic. Public DNS failures surface in CoreDNS logs, while successful or failed ICMP probes can still be visible to egress firewalls and traffic-audit systems. + +ACP 4.4 exposes both targets in the Kube-OVN cluster-plugin configuration. A non-empty field replaces the corresponding target. An empty field renders an empty pinger argument, and the pinger skips that probe. The configuration is persistent across plugin reconciliation and cluster-plugin updates. + +## Diagnostic Steps + +Inspect the currently rendered pinger arguments: + +```bash +kubectl -n kube-system get daemonset kube-ovn-pinger \ + -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' +echo +``` + +If the output contains one of the following default arguments, the Kube-OVN pinger is generating the corresponding traffic: + +- `--external-dns=kube-ovn.io.` or `--external-dns=google.com.` generates DNS lookups. +- `--external-address=1.1.1.1`, `--external-address=2606:4700:4700::1111`, or a comma-separated pair generates ICMP probes. + +List the pinger Pods and their node placement to estimate how many independent probes are running: + +```bash +kubectl -n kube-system get pods -l app=kube-ovn-pinger -o wide +``` + +Check recent CoreDNS errors for the default public targets: + +```bash +kubectl -n kube-system logs -l k8s-app=kube-dns \ + --since=10m --prefix \ + | grep -E '\[ERROR\].*(kube-ovn\.io\.|google\.com\.)' +``` + +In the traffic-audit system, filter for the default external addresses and correlate the timestamps with the pinger Pods listed above. Depending on the egress path, the observed source can be a Pod address or a source-NAT address on the node. + +Before attributing a broader network incident to these probes, verify that application service names resolve normally and that application traffic is healthy. A finding limited to the pinger's public targets affects its external-probe metrics or creates audit noise, but it does not by itself show that cluster-local DNS or Pod networking is unavailable. diff --git a/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md b/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md deleted file mode 100644 index 26b8f64eb..000000000 --- a/docs/en/solutions/Replacing_Kube_OVN_Pinger_Public_Probe_Targets_on_ACP_4_4.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -kind: - - Troubleshooting -products: - - Alauda Container Platform -ProductsVersion: - - 4.4.x ---- - -# Replace Kube-OVN pinger public probe targets on ACP 4.4 - -## Issue - -In an Alauda Container Platform cluster that uses Kube-OVN, CoreDNS can repeatedly report failed queries for a public domain even when applications do not query that domain. For an IPv4 cluster, the log commonly mentions `kube-ovn.io.`; IPv6 and dual-stack clusters can instead show `google.com.`. A representative CoreDNS error has the following form: - -```text -[ERROR] plugin/errors: 2 kube-ovn.io. A: read udp ...: i/o timeout -``` - -The errors are especially common in air-gapped environments or private networks that cannot use an upstream public DNS server. They are probe failures and do not by themselves indicate a Pod-networking or application-DNS outage. - -## Environment - -- Alauda Container Platform 4.4.x -- **Alauda Container Platform Networking for Kube-OVN** as the cluster CNI plugin -- CoreDNS logs contain repeated failures for the Kube-OVN pinger's configured public domain - -## Resolution - -Replace the public probe targets with targets that are reachable from the cluster. - -Before updating the plugin, choose the following values: - -| Plugin field | Recommended value | -| --- | --- | -| **Pinger External Probe DNS** | A fully qualified domain name that CoreDNS can resolve without using a public upstream resolver. Include the trailing dot to prevent search-domain expansion. For example, use `kubernetes.default.svc.cluster.local.` when the cluster domain is `cluster.local`, or use an organization-owned internal FQDN. | -| **Pinger External Probe IP** | An IP address that accepts ICMP echo requests from the cluster nodes. For a dual-stack cluster, provide a comma-separated IPv4 and IPv6 pair if both address families must be monitored. | - -:::note -Leaving either field empty in the ACP 4.4 plugin form preserves the Kube-OVN default for the active network stack. An empty **Pinger External Probe DNS** field therefore does not remove the default public-domain query. Enter an explicit reachable target. -::: - -1. In the web console, navigate to **Administrator** > **Marketplace** > **Cluster Plugins** and select the affected cluster. - -2. Search for `ovn` and locate **Alauda Container Platform Networking for Kube-OVN**. - -3. In the plugin row, open the action menu (vertical ⋮) and select **Update**. - -4. Set **Pinger External Probe DNS** to the internal FQDN selected above. If the default public probe IP is also unreachable, set **Pinger External Probe IP** to the selected internal IP address. - -5. Submit the update and wait for the `kube-ovn-pinger` DaemonSet rollout to complete: - - ```bash - kubectl -n kube-system rollout status daemonset/kube-ovn-pinger - ``` - -6. Confirm that the DaemonSet contains the new targets: - - ```bash - kubectl -n kube-system get daemonset kube-ovn-pinger \ - -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' - echo - ``` - - Verify that the output includes the expected `--external-dns=` and, if changed, `--external-address=` arguments. - -7. After at least one probe interval, verify that CoreDNS no longer reports failures for the former public target: - - ```bash - kubectl -n kube-system logs -l k8s-app=kube-dns \ - --since=10m --prefix \ - | grep -E 'kube-ovn\.io\.|google\.com\.' - ``` - - No new matching error should appear after the pinger rollout. If the command also returns older entries from before the rollout, repeat it with a shorter `--since` window. - -The pinger continues to publish external-connectivity health metrics, but those metrics now describe connectivity to the configured internal targets rather than public Internet connectivity. - -## Root Cause - -Kube-OVN runs one `kube-ovn-pinger` Pod on each eligible node. The pinger periodically checks an external IP address and resolves an external DNS name so it can publish external-connectivity metrics. - -The default target depends on the cluster network stack. IPv4 uses `kube-ovn.io.` for the DNS probe, while IPv6 and dual-stack use `google.com.`. When a private cluster cannot reach a public resolver, every pinger Pod continues to issue the configured query and CoreDNS records the upstream timeout or failure. The number of log entries therefore grows with both the node count and the probe frequency. - -ACP 4.4 exposes the pinger IP and DNS targets in the Kube-OVN cluster-plugin configuration. Replacing the defaults through that configuration is persistent across plugin reconciliation and cluster-plugin updates. - -## Diagnostic Steps - -Inspect the currently rendered pinger arguments: - -```bash -kubectl -n kube-system get daemonset kube-ovn-pinger \ - -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' -echo -``` - -If the output contains `--external-dns=kube-ovn.io.` or `--external-dns=google.com.`, the Kube-OVN pinger is generating the corresponding DNS query. - -List the pinger Pods and their node placement to estimate how many independent probes are running: - -```bash -kubectl -n kube-system get pods -l app=kube-ovn-pinger -o wide -``` - -Check recent CoreDNS errors for the default public targets: - -```bash -kubectl -n kube-system logs -l k8s-app=kube-dns \ - --since=10m --prefix \ - | grep -E '\[ERROR\].*(kube-ovn\.io\.|google\.com\.)' -``` - -Before attributing a broader DNS incident to this probe, also verify that application service names resolve normally. A failure limited to the pinger's public target affects its external-DNS health metric and produces log noise, but it does not by itself show that cluster-local DNS or Pod networking is unavailable. From 2729c7ad637480663ee57710f6d64a96cc3d0f12 Mon Sep 17 00:00:00 2001 From: zhangzujian Date: Mon, 3 Aug 2026 17:19:59 +0800 Subject: [PATCH 3/4] docs: clarify external probes are disabled by default --- ...e_OVN_Pinger_External_Probes_on_ACP_4_4.md | 53 +++++++++++-------- 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md index 97b1b9c1a..831e9cc9a 100644 --- a/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md +++ b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md @@ -11,32 +11,34 @@ ProductsVersion: ## Issue -In an Alauda Container Platform cluster that uses Kube-OVN, the following unexpected public traffic can appear even when applications do not generate it: +Starting with Alauda Container Platform 4.4, the Kube-OVN pinger's external address and external DNS probes are disabled by default. Both plugin fields are empty unless an administrator explicitly configures probe targets. -- CoreDNS repeatedly queries `kube-ovn.io.` in an IPv4 cluster or `google.com.` in an IPv6 or dual-stack cluster. A representative CoreDNS error has the following form: +An operator might nevertheless find one of the following during an upgrade investigation or traffic audit: + +- CoreDNS queries for `kube-ovn.io.` or `google.com.`, sometimes accompanied by an error such as: ```text [ERROR] plugin/errors: 2 kube-ovn.io. A: read udp ...: i/o timeout ``` -- A firewall, flow log, or traffic-audit system records ICMP traffic to `1.1.1.1`, `2606:4700:4700::1111`, or both. The traffic recurs from nodes that run `kube-ovn-pinger`. +- ICMP traffic to `1.1.1.1`, `2606:4700:4700::1111`, or both. -These probes are especially visible in air-gapped environments and private networks that prohibit unapproved public egress. Probe failures or audit records do not by themselves indicate a Pod-networking or application-DNS outage, but an operator might want to redirect the probes to approved internal targets or disable them completely. +These findings do not match the ACP 4.4 default. Check the rendered pinger arguments and the current plugin configuration to determine whether probes were explicitly enabled, an update has not finished rolling out, the audit event predates the update, or another workload generated the traffic. ## Environment - Alauda Container Platform 4.4.x - **Alauda Container Platform Networking for Kube-OVN** as the cluster CNI plugin -- CoreDNS logs or network audits contain repeated traffic to the Kube-OVN pinger's default public targets +- The operator needs to verify, disable, or configure the pinger's optional external probes ## Resolution -Choose one of the following policies: +Choose one of the following policies. The first row is the ACP 4.4 default. | Policy | **Pinger External Probe DNS** | **Pinger External Probe IP** | Result | | --- | --- | --- | --- | -| Disable public probes | Leave empty | Leave empty | The pinger skips both the DNS lookup and the ICMP probe. | -| Keep probes on approved internal targets | Enter an internal FQDN, such as `kubernetes.default.svc.cluster.local.` when the cluster domain is `cluster.local` | Enter an internal address that accepts ICMP echo requests | The pinger retains external-probe metrics without sending traffic to the default public targets. | +| Keep external probes disabled (default) | Leave empty | Leave empty | The pinger skips both the DNS lookup and the ICMP probe. | +| Keep probes on approved internal targets | Enter an internal FQDN, such as `kubernetes.default.svc.cluster.local.` when the cluster domain is `cluster.local` | Enter an internal address that accepts ICMP echo requests | The pinger retains external-probe metrics without sending traffic to public targets. | You can also disable only one probe by leaving its field empty. When configuring a DNS target, use a fully qualified domain name with a trailing dot to prevent search-domain expansion. For a dual-stack IP probe, provide a comma-separated IPv4 and IPv6 pair if both address families must be monitored. @@ -48,8 +50,8 @@ You can also disable only one probe by leaving its field empty. When configuring 4. Apply the selected policy: - - To stop all public pinger traffic, clear both **Pinger External Probe DNS** and **Pinger External Probe IP**. - - To retain the probes, replace the defaults with the approved internal FQDN and IP address. + - To keep the ACP 4.4 default and stop external pinger traffic, clear both **Pinger External Probe DNS** and **Pinger External Probe IP**. + - To enable the probes, enter an approved internal FQDN and IP address. 5. Submit the update and wait for the `kube-ovn-pinger` DaemonSet rollout to complete: @@ -67,7 +69,7 @@ You can also disable only one probe by leaving its field empty. When configuring If the probes are disabled, the output must include `--external-dns=` and `--external-address=` with empty values. If the probes are redirected, verify the expected `--external-dns=` and `--external-address=` values. -7. After at least one probe interval, verify that CoreDNS no longer reports queries for the former public target: +7. If public-domain queries were previously observed, verify that CoreDNS no longer reports them after the rollout: ```bash kubectl -n kube-system logs -l k8s-app=kube-dns \ @@ -77,25 +79,32 @@ You can also disable only one probe by leaving its field empty. When configuring No new matching entry should appear after the pinger rollout. If the command also returns older entries from before the rollout, repeat it with a shorter `--since` window. -8. Check the firewall, flow log, or traffic-audit system and confirm that no new pinger traffic reaches `1.1.1.1` or `2606:4700:4700::1111` after the rollout. +8. Check the firewall, flow log, or traffic-audit system and confirm that no new traffic attributed to the pinger reaches `1.1.1.1` or `2606:4700:4700::1111` after the rollout. When the fields are empty, the pinger stops updating the corresponding external-address and external-DNS health metrics but continues its other Kube-OVN health checks. When internal targets are configured, the metrics describe connectivity to those targets rather than public Internet connectivity. ## Root Cause -Kube-OVN runs one `kube-ovn-pinger` Pod on each eligible node. The pinger periodically sends ICMP echo requests to an external IP address and resolves an external DNS name so it can publish external-connectivity metrics. +Kube-OVN runs one `kube-ovn-pinger` Pod on each eligible node. When configured, the pinger periodically sends ICMP echo requests to an external IP address and resolves an external DNS name so it can publish external-connectivity metrics. The pinger executes each probe only when the corresponding argument is non-empty. + +ACP 4.4 renders both arguments with empty values by default: + +```text +--external-address= +--external-dns= +``` -The default targets depend on the cluster network stack: +Consequently, a default ACP 4.4 installation does not generate external address or external DNS probe traffic. When probes are enabled, one pinger Pod on each eligible node produces an independent stream, so node count and probe frequency multiply the observed traffic. -| Network stack | External address | External DNS name | +The following public targets can identify a historical or explicitly configured pinger probe: + +| Network stack | Recognizable external address | Recognizable external DNS name | | --- | --- | --- | | IPv4 | `1.1.1.1` | `kube-ovn.io.` | | IPv6 | `2606:4700:4700::1111` | `google.com.` | | Dual stack | `1.1.1.1,2606:4700:4700::1111` | `google.com.` | -Every pinger Pod produces an independent stream. Node count and probe frequency therefore multiply both the DNS queries and ICMP traffic. Public DNS failures surface in CoreDNS logs, while successful or failed ICMP probes can still be visible to egress firewalls and traffic-audit systems. - -ACP 4.4 exposes both targets in the Kube-OVN cluster-plugin configuration. A non-empty field replaces the corresponding target. An empty field renders an empty pinger argument, and the pinger skips that probe. The configuration is persistent across plugin reconciliation and cluster-plugin updates. +ACP 4.4 exposes both targets in the Kube-OVN cluster-plugin configuration. A non-empty field enables the corresponding probe. An empty field disables it. The configuration is persistent across plugin reconciliation and cluster-plugin updates. ## Diagnostic Steps @@ -107,7 +116,9 @@ kubectl -n kube-system get daemonset kube-ovn-pinger \ echo ``` -If the output contains one of the following default arguments, the Kube-OVN pinger is generating the corresponding traffic: +The ACP 4.4 default is `--external-dns=` and `--external-address=`. With those empty values, the current pinger does not generate either kind of external probe. + +If the output instead contains one of the following non-empty arguments, the Kube-OVN pinger is configured to generate the corresponding traffic: - `--external-dns=kube-ovn.io.` or `--external-dns=google.com.` generates DNS lookups. - `--external-address=1.1.1.1`, `--external-address=2606:4700:4700::1111`, or a comma-separated pair generates ICMP probes. @@ -118,7 +129,7 @@ List the pinger Pods and their node placement to estimate how many independent p kubectl -n kube-system get pods -l app=kube-ovn-pinger -o wide ``` -Check recent CoreDNS errors for the default public targets: +Check recent CoreDNS errors for recognizable public probe targets: ```bash kubectl -n kube-system logs -l k8s-app=kube-dns \ @@ -126,6 +137,6 @@ kubectl -n kube-system logs -l k8s-app=kube-dns \ | grep -E '\[ERROR\].*(kube-ovn\.io\.|google\.com\.)' ``` -In the traffic-audit system, filter for the default external addresses and correlate the timestamps with the pinger Pods listed above. Depending on the egress path, the observed source can be a Pod address or a source-NAT address on the node. +In the traffic-audit system, filter for the recognizable external addresses and correlate the timestamps with the pinger Pods listed above. Depending on the egress path, the observed source can be a Pod address or a source-NAT address on the node. If the DaemonSet arguments are empty, check whether the event predates the latest pinger rollout and whether another process owns the traffic before attributing it to Kube-OVN. Before attributing a broader network incident to these probes, verify that application service names resolve normally and that application traffic is healthy. A finding limited to the pinger's public targets affects its external-probe metrics or creates audit noise, but it does not by itself show that cluster-local DNS or Pod networking is unavailable. From 9f9649357aa3cce3080df66b368da2f2fcf8601d Mon Sep 17 00:00:00 2001 From: zhangzujian Date: Mon, 3 Aug 2026 18:04:55 +0800 Subject: [PATCH 4/4] docs: deduplicate pinger diagnostic steps --- ...bling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md index 831e9cc9a..e5c61de53 100644 --- a/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md +++ b/docs/en/solutions/Configuring_or_Disabling_Kube_OVN_Pinger_External_Probes_on_ACP_4_4.md @@ -59,15 +59,7 @@ You can also disable only one probe by leaving its field empty. When configuring kubectl -n kube-system rollout status daemonset/kube-ovn-pinger ``` -6. Confirm that the DaemonSet contains the expected arguments: - - ```bash - kubectl -n kube-system get daemonset kube-ovn-pinger \ - -o jsonpath='{.spec.template.spec.containers[?(@.name=="pinger")].args}' - echo - ``` - - If the probes are disabled, the output must include `--external-dns=` and `--external-address=` with empty values. If the probes are redirected, verify the expected `--external-dns=` and `--external-address=` values. +6. Use the argument inspection under **Diagnostic Steps** to confirm that the DaemonSet matches the selected probe policy before checking logs or traffic-audit records. 7. If public-domain queries were previously observed, verify that CoreDNS no longer reports them after the rollout: