profile
viewpoint

Ask questionsKubernetes incorrectly warns with "invalid ingress configuration" when using the "allow-http: false" annotation

On GKE, creating an Ingress with a managed certificate and disabling http as described in Disabling HTTP via the annotation: kubernetes.io/ingress.allow-http: "false" generates a warning:

GKE v1.17.8-gke.17:

kubectl version |grep Server
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.8-gke.17", GitCommit:"cd7ca396c79d2e8f3fdb06c6865549770091d431", GitTreeState:"clean", BuildDate:"2020-07-20T22:12:03Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}

Steps to reproduce:

$ kubectl get -n frontend ingress 
No resources found in frontend namespace.
$ grep allow app/store/cluster/in-scope/namespaces/frontend/ingress.yaml 
    kubernetes.io/ingress.allow-http: "false"
$ kubectl -n frontend apply -f .../ingress.yaml 
ingress.networking.k8s.io/frontend created
$ kubectl -n frontend describe ingress frontend
Name:             frontend
Namespace:        frontend
Address:          34.120.228.7
Default backend:  default-http-backend:80 (10.4.1.11:8080)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /*   frontend:80 (10.4.1.4:8080)
Annotations:  ingress.gcp.kubernetes.io/pre-shared-cert: mcrt-REDACTED
              ingress.kubernetes.io/backends: {"k8s-be-31403--REDACTED":"HEALTHY","k8s-be-31818--REDACTED":"HEALTHY"}
              ingress.kubernetes.io/https-forwarding-rule: k8s2-fs-888jw4sk-frontend-frontend-REDACTED
              ingress.kubernetes.io/https-target-proxy: k8s2-ts-888jw4sk-frontend-frontend-REDACTED
              ingress.kubernetes.io/ssl-cert: mcrt-REDACTED
              ingress.kubernetes.io/url-map: k8s2-um-888jw4sk-frontend-frontend-REDACTED
              kubernetes.io/ingress.allow-http: false
              kubernetes.io/ingress.global-static-ip-name: frontend-ext-ip
              networking.gke.io/managed-certificates: frontend
Events:
  Type     Reason  Age   From                     Message
  ----     ------  ----  ----                     -------
  Normal   ADD     97s   loadbalancer-controller  frontend/frontend
  Warning  Sync    54s   loadbalancer-controller  Error during sync: error running load balancer syncing routine: loadbalancer 888jw4sk-frontend-frontend-REDACTED does not exist: invalid ingress frontend configuration, please check your usage of the 'kubernetes.io/ingress.allow-http' annotation.
  Normal   CREATE  37s   loadbalancer-controller  ip: (REDACTED)

The warning message appears even when using the annotation as documented: " Warning Sync 54s loadbalancer-controller Error during sync: error running load balancer syncing routine: loadbalancer 888jw4sk-frontend-frontend-REDACTED does not exist: invalid ingress frontend configuration, please check your usage of the 'kubernetes.io/ingress.allow-http' annotation."

Contents of ingress.yaml:

$ cat app/store/cluster/in-scope/namespaces/frontend/ingress.yaml 
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: frontend
  annotations:
    kubernetes.io/ingress.global-static-ip-name: frontend-ext-ip
    kubernetes.io/ingress.allow-http: "false"
    networking.gke.io/managed-certificates: frontend
spec:
  rules:
  - http:
      paths:
        - path: "/*"
          backend:
            serviceName: frontend
            servicePort: 80

It does not appear to be related to the status of the ManagedCertificate:

$ kubectl -n frontend describe managedcertificates.networking.gke. frontend
Name:         frontend
Namespace:    frontend
Labels:       <none>
Annotations:  API Version:  networking.gke.io/v1beta2
Kind:         ManagedCertificate
Metadata:
  Creation Timestamp:  2020-08-12T18:35:56Z
  Generation:          4
  Resource Version:    443128
  Self Link:           /apis/networking.gke.io/v1beta2/namespaces/frontend/managedcertificates/frontend
  UID:                 REDACTED
Spec:
  Domains:
    store.REDACTED.com
Status:
  Certificate Name:    mcrt-REDACTED
  Certificate Status:  Active
  Domain Status:
    Domain:     store.REDACTED.com
    Status:     Active
  Expire Time:  2020-11-10T10:47:37.000-08:00
Events:         <none>

This is possibly related to #1001, however the WillNotConfigureFrontend event is not seen.

kubernetes/ingress-gce

Answer questions jmound

Unless I'm mistaken, it's not an invalid configuration. If that's the case, then that line should be removed. From a previous comment:

...its just that the provisioning of the LB will fail for some time until the ManagedCertificate controller applies the certificate to the Ingress.

How about something along the lines of: LoadBalancer provisioning in progress, waiting for ManagedCertificate status to update ? I'm not familiar with the exact workflow of the managed certificate controller, so that probably needs to be modified to be more accurate. But the main point is that if the ingress config with the managed cert annotation is a correct, valid configuration, we shouldn't be messaging otherwise.

useful!
source:https://uonfu.com/
Github User Rank List