Ask questionsMake Route53 dns01 work with IAM roles for service accounts
If you set everything up as described in the linked AWS page and put the assumed role in, just put the web identity role into v1.10.1, you get a lot of errors:
error instantiating route53 challenge solver: unable to assume role: AccessDenied: Access denied\n\tstatus code: 403
Answer questions hendrikhalkow
Hello there, I can confirm that opening port 443 as described in https://github.com/jetstack/cert-manager/issues/2109 made creating the cluster issuer work.
I also added the
eks.amazonaws.com/role-arn annotation that made the admission controller mutate the cert manager pod. Here is an excerpt from
kubectl describe pod:
Environment: POD_NAMESPACE: cert-manager (v1:metadata.namespace) AWS_ROLE_ARN: arn:aws:iam::xxxxxxxxxxxx:role/CertManager-demo AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token Mounts: /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro) /var/run/secrets/kubernetes.io/serviceaccount from cert-manager-token-c595c (ro)
However, the challenge fails because the injected token cannot be read:
$ kubectl describe challenge ... ... Error presenting challenge: Failed to change Route 53 record set: WebIdentityErr: unable to read file at /var/run/secrets/eks.amazonaws.com/serviceaccount/token ...
$ kubectl logs cert-manager-... ... E1010 12:04:47.578424 1 controller.go:131] cert-manager/controller/challenges "msg"="re-queuing item due to error processing" "error"="Failed to change Route 53 record set: WebIdentityErr: unable to read file at /var/run/secrets/eks.amazonaws.com/serviceaccount/token\ncaused by: open /var/run/secrets/eks.amazonaws.com/serviceaccount/token: permission denied" "key"="cert-manager/demo-example-com-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx" ...
As the container has no shell I am unable to check the current permissions, but I think it's just a small thing.
Edit: enabling the securityContext solved this. Everything is working as expected as soon as the required changes are made. The only thing left is adjusting the documentation and the Kubernetes manifests accordingly.
Changes to be made:
eks.amazonaws.com/role-arnannotation to service account
securityContext: fsGroup: 1001