Tenant or Projects dropdown shows empty

Problem

It was observed that the tenants/projects list shows up empty when the user logs in to a Region with a large number of tenants. This was consistent for SSO and local users. The UI inspect was showing a lot of calls failing with "net error":"ERR HTTP2 PROTOCOL ERROR"

Environment

• Private Cloud Director Virtualization - till v2025.8

• Private Cloud Director Kubernetes – till v2025.8

• Self-Hosted Private Cloud Director Virtualization -till v2025.8

• Self-Hosted Private Cloud Director Kubernetes - till v2025.8

• Component: UI

Cause

The cause of the problem was the nginx response headers being too large and being dropped or truncated on response during the login process. This is now tracked on PCD-3830, and the fix is available from v2025.10-115 or above versions.

Diagnostics

1. It was noticed that the error was seen only for a user with access to a large number of tenants, which could impact the response header or response body size

2. The issue was not reproduced for users with access to a lesser number of tenants.

3. On browser, run an inspect on the page and check for "net error":"ERR HTTP2 PROTOCOL ERROR"

4. Debug logs were enabled on the ingress-nginx-controller pod on the control plane side by adding --v=5 to the startup params in the deployment. The nginx debug logs are shown below, which confirmed the issue

[info] 71#71: *484 client sent too large header field while processing HTTP_2 connection, client: [IP_ADDRESS], server: 0.0.0.0:443

[info] 71#71: *568 client sent too large header field while processing HTTP_2 connection, client: [IP_ADDRESS], server: 0.0.0.0:443

Step 4 can be performed on Self-Hosted PCD

Workaround

The following changes were made to the ingress-nginx-controller deployment on the control plane, which fixed the problem.

a) The config map was edited to add buffer params client-header-buffer-size and large-client-header-buffers as below:

$ kubectl edit cm ingress-nginx-controller -n <WORKLOAD_NAMESPACE>
apiVersion: v1
data:
 allow-snippet-annotations: "true"
 annotations-risk-level: Critical
 client-header-buffer-size: 64k
 large-client-header-buffers: 4 32k
kind: ConfigMap

b) After making the changes the ingress-nginx-controller deployment was restarted

$ kubectl rollout restart deploy_ingress-nginx-controller  -n <WORKLOAD_NAMESPACE>

Validation

After applying the changes, the tenant dropdown list was verified as loading ok for various browsers and users.