<p>I'm fairly new to K8s and learning through AWS EKS.  I setup a cluster with:</p>
<pre><code>cd &lt;some dir&gt;
CLUSTER_NAME=k8s-play
eksctl anywhere generate clusterconfig $CLUSTER_NAME --provider docker &gt; $CLUSTER_NAME.yaml
eksctl anywhere create cluster -f $CLUSTER_NAME.yaml
</code></pre>
<p>This takes maybe an hour and doesn't finish (no errors in terminal).  The AWS EKS learning doc says:</p>
<blockquote>
<p>After a few minutes the “cluster created” message will be displayed, which signifies that your Amazon EKS Anywhere cluster was successfully deployed on your local machine.</p>
</blockquote>
<p>The output is:</p>
<pre><code> $ eksctl anywhere create cluster -f k8s-play.yaml                                                                                                                                                                      [7:50:49]
Performing setup and validations
Warning: The docker infrastructure provider is meant for local development and testing only
✅ Docker Provider setup is valid
✅ Validate certificate for registry mirror
✅ Create preflight validations pass
Creating new bootstrap cluster
Provider specific pre-capi-install-setup on bootstrap cluster
Installing cluster-api providers on bootstrap cluster
Provider specific post-setup
Creating new workload cluster
Installing networking on workload cluster
Installing cluster-api providers on workload cluster
collecting cluster diagnostics
collecting management cluster diagnostics
⏳ Collecting support bundle from cluster, this can take a while {&quot;cluster&quot;: &quot;bootstrap-cluster&quot;, &quot;bundle&quot;: &quot;k8s-play/generated/bootstrap-cluster-2024-10-26T08:23:55+11:00-bundle.yaml&quot;, &quot;since&quot;: 1729887835238682000, &quot;kubeconfig&quot;: &quot;k8s-play/generated/k8s-play.kind.kubeconfig&quot;}
Support bundle archive created  {&quot;path&quot;: &quot;support-bundle-2024-10-25T21_23_56.tar.gz&quot;}
Analyzing support bundle    {&quot;bundle&quot;: &quot;k8s-play/generated/bootstrap-cluster-2024-10-26T08:23:55+11:00-bundle.yaml&quot;, &quot;archive&quot;: &quot;support-bundle-2024-10-25T21_23_56.tar.gz&quot;}
Analysis output generated   {&quot;path&quot;: &quot;k8s-play/generated/bootstrap-cluster-2024-10-26T08:25:20+11:00-analysis.yaml&quot;}
collecting workload cluster diagnostics
⏳ Collecting support bundle from cluster, this can take a while {&quot;cluster&quot;: &quot;k8s-play&quot;, &quot;bundle&quot;: &quot;k8s-play/generated/k8s-play-2024-10-26T08:25:25+11:00-bundle.yaml&quot;, &quot;since&quot;: 1729887925599414000, &quot;kubeconfig&quot;: &quot;k8s-play/k8s-play-eks-a-cluster.kubeconfig&quot;}
</code></pre>
<p>I had a look for errors in <code>support-bundle.../logs</code> and there are a lot but I do not know which if any point to the cause, or they are all a reason.  The errors are:</p>
<pre><code>$ ag error                                                                                                                                                                                                            [16:31:10]
cert-manager/cert-manager-8674857d7b-zgv2b.log
38:I1025 20:52:02.768190       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;selfsigned-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
40:E1025 20:52:02.772058       1 controller.go:133] cert-manager/controller/issuers &quot;msg&quot;=&quot;issuer in work queue no longer exists&quot; &quot;error&quot;=&quot;issuer.cert-manager.io \&quot;test-selfsigned\&quot; not found&quot;
41:E1025 20:52:02.789005       1 requestmanager_controller.go:136] cert-manager/controller/certificates-request-manager &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
42:E1025 20:52:02.789102       1 trigger_controller.go:137] cert-manager/controller/certificates-trigger &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
43:E1025 20:52:02.789120       1 revisionmanager_controller.go:106] cert-manager/controller/certificates-revision-manager &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
44:E1025 20:52:02.789139       1 readiness_controller.go:142] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
45:E1025 20:52:02.789199       1 issuing_controller.go:152] cert-manager/controller/certificates-issuing &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
46:E1025 20:52:02.956024       1 controller.go:163] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;re-queuing item due to error processing&quot; &quot;error&quot;=&quot;secrets \&quot;selfsigned-cert-\&quot; is forbidden: unable to create new content in namespace cert-manager-test because it is being terminated&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
47:E1025 20:52:02.956070       1 keymanager_controller.go:139] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
51:I1025 20:52:03.037561       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;capi-system/capi-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
54:I1025 20:52:03.086437       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;re...&quot; &quot;key&quot;=&quot;capi-system/capi-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
58:I1025 20:52:03.126360       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-system/capi-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
60:I1025 20:52:03.136875       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-system/capi-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
65:I1025 20:52:03.266488       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-bootstrap-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
68:I1025 20:52:03.369530       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-bootstrap-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
73:I1025 20:52:03.395063       1 controller.go:161] cert-manager/controller/certificates-trigger &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-bootstrap-provider-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
79:I1025 20:52:03.419626       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-bootstrap-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
81:I1025 20:52:03.428145       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-bootstrap-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
85:I1025 20:52:03.534188       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-bootstrap-provider-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
87:I1025 20:52:03.537428       1 controller.go:161] cert-manager/controller/certificates-issuing &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-bootstrap-provider-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
88:I1025 20:52:03.541702       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-bootstrap-provider-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
92:I1025 20:52:03.568072       1 controller.go:161] cert-manager/controller/certificates-trigger &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-controller-system/etcdadm-controller-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-controller-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
99:I1025 20:52:03.718792       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-controller-system/etcdadm-controller-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-controller-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
101:I1025 20:52:03.723278       1 controller.go:161] cert-manager/controller/certificates-issuing &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-controller-system/etcdadm-controller-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-controller-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
102:I1025 20:52:03.727309       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;etcdadm-controller-system/etcdadm-controller-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;etcdadm-controller-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
103:E1025 20:52:03.769911       1 readiness_controller.go:142] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
104:E1025 20:52:03.956955       1 keymanager_controller.go:139] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;certificate not found for key&quot; &quot;error&quot;=&quot;certificate.cert-manager.io \&quot;selfsigned-cert\&quot; not found&quot; &quot;key&quot;=&quot;cert-manager-test/selfsigned-cert&quot;
108:I1025 20:52:04.043249       1 controller.go:161] cert-manager/controller/certificates-trigger &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-control-plane-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
115:I1025 20:52:04.163286       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;r...&quot; &quot;key&quot;=&quot;capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-control-plane-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
117:I1025 20:52:04.170751       1 controller.go:161] cert-manager/controller/certificates-issuing &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-control-plane-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
118:I1025 20:52:04.172335       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capi-kubeadm-control-plane-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
123:I1025 20:52:04.836824       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capd-system/capd-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capd-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
126:I1025 20:52:04.951381       1 controller.go:161] cert-manager/controller/certificates-key-manager &quot;msg&quot;=&quot;re-queuing item due to optimistic locking on resource&quot; &quot;key&quot;=&quot;capd-system/capd-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capd-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
130:I1025 20:52:04.978760       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capd-system/capd-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capd-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;
132:I1025 20:52:04.985399       1 controller.go:161] cert-manager/controller/certificates-readiness &quot;msg&quot;=&quot;...&quot; &quot;key&quot;=&quot;capd-system/capd-serving-cert&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on certificates.cert-manager.io \&quot;capd-serving-cert\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot;

kube-system/kube-controller-manager-k8s-play-eks-a-cluster-control-plane.log
9:E1025 20:51:19.247194       1 leaderelection.go:330] error retrieving resource lock kube-system/kube-controller-manager: leases.coordination.k8s.io &quot;kube-controller-manager&quot; is forbidden: User &quot;system:kube-controller-manager&quot; cannot get resource &quot;leases&quot; in API group &quot;coordination.k8s.io&quot; in the namespace &quot;kube-system&quot;
271:E1025 20:52:07.786800       1 tokens_controller.go:262] error synchronizing serviceaccount cert-manager-test/default: secrets &quot;default-token-kcbjp&quot; is forbidden: unable to create new content in namespace cert-manager-test because it is being terminated
273:W1025 20:52:28.350307       1 endpointslice_controller.go:306] Error syncing endpoint slices for service &quot;capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-webhook-service&quot;, retrying. Error: EndpointSlice informer cache is out of date

kube-system/kube-apiserver-k8s-play-eks-a-cluster-control-plane.log
39:E1025 20:51:19.238656       1 controller.go:152] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/***HIDDEN***, ResourceVersion: 0, AdditionalErrorMsg:
130:W1025 21:08:30.063246       1 watcher.go:229] watch chan error: etcdserver: mvcc: required revision has been compacted
137:W1025 21:18:03.037553       1 watcher.go:229] watch chan error: etcdserver: mvcc: required revision has been compacted

kube-system/kube-scheduler-k8s-play-eks-a-cluster-control-plane.log
3:W1025 20:51:19.249418       1 authentication.go:345] Error looking up in-cluster authentication configuration: configmaps &quot;extension-apiserver-authentication&quot; is forbidden: User &quot;system:kube-scheduler&quot; cannot get resource &quot;configmaps&quot; in API group &quot;&quot; in the namespace &quot;kube-system&quot;

cert-manager/cert-manager-cainjector-f5b94ccdf-7k9sv.log
44:E1025 20:51:58.672741       1 controller.go:175] cert-manager/secret/mutatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;unable to update target object with new CA data&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on mutatingwebhookconfigurations.admissionregistration.k8s.io \&quot;cert-manager-webhook\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot; &quot;resource_kind&quot;=&quot;MutatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;cert-manager-webhook&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot;
45:E1025 20:51:58.672798       1 controller.go:304] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration &quot;msg&quot;=&quot;Reconciler error&quot; &quot;error&quot;=&quot;Operation cannot be fulfilled on mutatingwebhookconfigurations.admissionregistration.k8s.io \&quot;cert-manager-webhook\&quot;: the object has been modified; please apply your changes to the latest version and try again&quot; &quot;name&quot;=&quot;cert-manager-webhook&quot; &quot;namespace&quot;=&quot;&quot;
61:E1025 20:52:02.812634       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated certificate&quot; &quot;error&quot;=&quot;Certificate.cert-manager.io \&quot;capi-serving-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;clusterclasses.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot;
...
115:E1025 20:52:03.031904       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated secret&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;clusterclasses.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
...
169:E1025 20:52:03.050701       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;MutatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capi-mutating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
171:E1025 20:52:03.058308       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;ValidatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capi-validating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
173:E1025 20:52:03.090625       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;ValidatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capi-validating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
175:E1025 20:52:03.090627       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;MutatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capi-mutating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
177:E1025 20:52:03.091687       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capi-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;clusterclasses.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-system&quot;,&quot;Name&quot;:&quot;capi-webhook-service-cert&quot;}
...
195:E1025 20:52:03.104518       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated certificate&quot; &quot;error&quot;=&quot;Certificate.cert-manager.io \&quot;capi-kubeadm-bootstrap-serving-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-bootstrap-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-bootstrap-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;kubeadmconfigs.bootstrap.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot;
215:E1025 20:52:03.266486       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated secret&quot; &quot;error&quot;=&quot;Secret \&quot;capi-kubeadm-bootstrap-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-bootstrap-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-bootstrap-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;kubeadmconfigs.bootstrap.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-bootstrap-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-bootstrap-webhook-service-cert&quot;}
218:E1025 20:52:03.272898       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated secret&quot; &quot;error&quot;=&quot;...&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-bootstrap-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-bootstrap-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;kubeadmconfigtemplates.bootstrap.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-bootstrap-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-bootstrap-webhook-service-cert&quot;}
...
331:E1025 20:52:04.100442       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated certificate&quot; &quot;error&quot;=&quot;Certificate.cert-manager.io \&quot;capd-serving-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;dockerclusters.infrastructure.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot;
...
343:E1025 20:52:04.120411       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated secret&quot; &quot;error&quot;=&quot;Secret \&quot;capi-kubeadm-control-plane-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-control-plane-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-control-plane-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;kubeadmcontrolplanetemplates.controlplane.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capi-kubeadm-control-plane-system&quot;,&quot;Name&quot;:&quot;capi-kubeadm-control-plane-webhook-service-cert&quot;}
359:E1025 20:52:04.711835       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated certificate&quot; &quot;error&quot;=&quot;Certificate.cert-manager.io \&quot;capd-serving-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;dockerclustertemplates.infrastructure.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot;
...
383:E1025 20:52:04.829939       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler &quot;msg&quot;=&quot;unable to fetch associated secret&quot; &quot;error&quot;=&quot;Secret \&quot;capd-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-serving-cert&quot;} &quot;resource_kind&quot;=&quot;CustomResourceDefinition&quot; &quot;resource_name&quot;=&quot;dockerclusters.infrastructure.cluster.x-k8s.io&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-webhook-service-cert&quot;}
...
413:E1025 20:52:04.848016       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capd-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-serving-cert&quot;} &quot;resource_kind&quot;=&quot;MutatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capd-mutating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-webhook-service-cert&quot;}
415:E1025 20:52:04.855050       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler &quot;msg&quot;=&quot;...&quot; &quot;error&quot;=&quot;Secret \&quot;capd-webhook-service-cert\&quot; not found&quot; &quot;certificate&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-serving-cert&quot;} &quot;resource_kind&quot;=&quot;ValidatingWebhookConfiguration&quot; &quot;resource_name&quot;=&quot;capd-validating-webhook-configuration&quot; &quot;resource_namespace&quot;=&quot;&quot; &quot;resource_version&quot;=&quot;v1&quot; &quot;secret&quot;={&quot;Namespace&quot;:&quot;capd-system&quot;,&quot;Name&quot;:&quot;capd-webhook-service-cert&quot;}
...

capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-controller-manager-68b4c848dc-f7tfn.log
37:E1025 20:52:58.401912       1 controller.go:242] controller/kubeadmcontrolplane &quot;msg&quot;=&quot;Failed to update KubeadmControlPlane Status&quot; &quot;error&quot;=&quot;failed to create remote cluster client: error creating client and cache for remote cluster: error creating dynamic rest mapper for remote cluster \&quot;eksa-system/k8s-play\&quot;: context deadline exceeded&quot; &quot;cluster&quot;=&quot;k8s-play&quot; &quot;name&quot;=&quot;k8s-play&quot; &quot;namespace&quot;=&quot;eksa-system&quot; &quot;reconciler group&quot;=&quot;controlplane.cluster.x-k8s.io&quot; &quot;reconciler kind&quot;=&quot;KubeadmControlPlane&quot;
38:E1025 20:52:58.428989       1 controller.go:317] controller/kubeadmcontrolplane &quot;msg&quot;=&quot;Reconciler error&quot; &quot;error&quot;=&quot;...&quot; &quot;name&quot;=&quot;k8s-play&quot; &quot;namespace&quot;=&quot;eksa-system&quot; &quot;reconciler group&quot;=&quot;controlplane.cluster.x-k8s.io&quot; &quot;reconciler kind&quot;=&quot;KubeadmControlPlane&quot;
40:E1025 20:53:18.603785       1 controller.go:242] controller/kubeadmcontrolplane &quot;msg&quot;=&quot;Failed to update KubeadmControlPlane Status&quot; &quot;error&quot;=&quot;...&quot; &quot;cluster&quot;=&quot;k8s-play&quot; &quot;name&quot;=&quot;k8s-play&quot; &quot;namespace&quot;=&quot;eksa-system&quot; &quot;reconciler group&quot;=&quot;controlplane.cluster.x-k8s.io&quot; &quot;reconciler kind&quot;=&quot;KubeadmControlPlane&quot;
41:E1025 20:53:18.632466       1 controller.go:317] controller/kubeadmcontrolplane &quot;msg&quot;=&quot;Reconciler error&quot; &quot;error&quot;=&quot;...&quot; &quot;name&quot;=&quot;k8s-play&quot; &quot;namespace&quot;=&quot;eksa-system&quot; &quot;reconciler group&quot;=&quot;controlplane.cluster.x-k8s.io&quot; &quot;reconciler kind&quot;=&quot;KubeadmControlPlane&quot;

etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-controller-manager-664f699b7c-9gth9.log
43:1.7298895614229357e+09   INFO    controllers.EtcdadmConfig   bootstrap data secret for EtcdadmConfig already exists, updating    {&quot;secretError&quot;: &quot;json: unsupported type: func(string, secret.Purpose) string&quot;, &quot;EtcdadmConfig&quot;: &quot;k8s-play-etcd-vvb65&quot;}

</code></pre>
<p>Help appreciated!</p>
<hr />
<p>Terminal command completed with error - like Peter in commented suggested, it is cert manager.</p>
<pre><code>Error: initializing capi resources in cluster: executing init: Fetching providers
Using Override=&quot;core-components.yaml&quot; Provider=&quot;cluster-api&quot; Version=&quot;v1.1.3+9999d58&quot;
Using Override=&quot;bootstrap-components.yaml&quot; Provider=&quot;bootstrap-kubeadm&quot; Version=&quot;v1.1.3+e448518&quot;
Using Override=&quot;bootstrap-components.yaml&quot; Provider=&quot;bootstrap-etcdadm-bootstrap&quot; Version=&quot;v1.0.0-rc6+eece35c&quot;
Using Override=&quot;bootstrap-components.yaml&quot; Provider=&quot;bootstrap-etcdadm-controller&quot; Version=&quot;v1.0.0-rc9+45b432d&quot;
Using Override=&quot;control-plane-components.yaml&quot; Provider=&quot;control-plane-kubeadm&quot; Version=&quot;v1.1.3+9e13ae0&quot;
Using Override=&quot;infrastructure-components-development.yaml&quot; Provider=&quot;infrastructure-docker&quot; Version=&quot;v1.1.3+736b241&quot;
Installing cert-manager Version=&quot;v1.5.3+c34401d&quot;
Using Override=&quot;cert-manager.yaml&quot; Provider=&quot;cert-manager&quot; Version=&quot;v1.5.3+c34401d&quot;
Waiting for cert-manager to be available...
Error: timed out waiting for the condition
</code></pre>
<p>I am going to try with <code>minikube</code> to eliminate AWS EKS issues.</p>
<p><code>minikube</code> worked just fine (<a href="https://kubernetes.io/docs/tutorials/hello-minikube/" rel="nofollow noreferrer">https://kubernetes.io/docs/tutorials/hello-minikube/</a>) so the issue is with <code>AWS EKS</code>.</p>

CC BY-SA 4.0
2024-10-28T12:01:05.313Z
I'm not expert enough to answer definitively, but I think the second line of your error message is the real error: `40:E1025 20:52:02.772058       1 controller.go:133] cert-manager/controller/issuers "msg"="issuer in work queue no longer exists" "error"="issuer.cert-manager.io \"test-selfsigned\" not found`
Peter Turner
CC BY-SA 4.0
2024-10-28T22:33:18.743Z
Ok thanks Peter, I will see what I can work out with that.
HankCa
CC BY-SA 4.0
2024-10-29T02:12:07.500Z
Nothing fruitful turned up.

Terminal finally completed with error.  I will add this to question.
HankCa
<kubernetes><eks>
Beginner at Kubernetes - cluster creation not finising - unable to work out why
HankCa
# Beginner at Kubernetes - cluster creation not finising - unable to work out why

(Asked by: username_0 on 2024-10-28)

I'm fairly new to K8s and learning through AWS EKS.  I setup a cluster with:
cd 
CLUSTER_NAME=k8s-play
eksctl anywhere generate clusterconfig $CLUSTER_NAME --provider docker > $CLUSTER_NAME.yaml
eksctl anywhere create cluster -f $CLUSTER_NAME.yaml

This takes maybe an hour and doesn't finish (no errors in terminal).  The AWS EKS learning doc says:

After a few minutes the “cluster created” message will be displayed, which signifies that your Amazon EKS Anywhere cluster was successfully deployed on your local machine.

The output is:
 $ eksctl anywhere create cluster -f k8s-play.yaml                                                                                                                                                                      [7:50:49]
Performing setup and validations
Warning: The docker infrastructure provider is meant for local development and testing only
✅ Docker Provider setup is valid
✅ Validate certificate for registry mirror
✅ Create preflight validations pass
Creating new bootstrap cluster
Provider specific pre-capi-install-setup on bootstrap cluster
Installing cluster-api providers on bootstrap cluster
Provider specific post-setup
Creating new workload cluster
Installing networking on workload cluster
Installing cluster-api providers on workload cluster
collecting cluster diagnostics
collecting management cluster diagnostics
⏳ Collecting support bundle from cluster, this can take a while {"cluster": "bootstrap-cluster", "bundle": "k8s-play/generated/bootstrap-cluster-2024-10-26T08:23:55+11:00-bundle.yaml", "since": 1729887835238682000, "kubeconfig": "k8s-play/generated/k8s-play.kind.kubeconfig"}
Support bundle archive created  {"path": "support-bundle-2024-10-25T21_23_56.tar.gz"}
Analyzing support bundle    {"bundle": "k8s-play/generated/bootstrap-cluster-2024-10-26T08:23:55+11:00-bundle.yaml", "archive": "support-bundle-2024-10-25T21_23_56.tar.gz"}
Analysis output generated   {"path": "k8s-play/generated/bootstrap-cluster-2024-10-26T08:25:20+11:00-analysis.yaml"}
collecting workload cluster diagnostics
⏳ Collecting support bundle from cluster, this can take a while {"cluster": "k8s-play", "bundle": "k8s-play/generated/k8s-play-2024-10-26T08:25:25+11:00-bundle.yaml", "since": 1729887925599414000, "kubeconfig": "k8s-play/k8s-play-eks-a-cluster.kubeconfig"}

I had a look for errors in support-bundle.../logs and there are a lot but I do not know which if any point to the cause, or they are all a reason.  The errors are:
$ ag error                                                                                                                                                                                                            [16:31:10]
cert-manager/cert-manager-8674857d7b-zgv2b.log
38:I1025 20:52:02.768190       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="re-queuing item due to optimistic locking on resource" "key"="cert-manager-test/selfsigned-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"selfsigned-cert\": the object has been modified; please apply your changes to the latest version and try again"
40:E1025 20:52:02.772058       1 controller.go:133] cert-manager/controller/issuers "msg"="issuer in work queue no longer exists" "error"="issuer.cert-manager.io \"test-selfsigned\" not found"
41:E1025 20:52:02.789005       1 requestmanager_controller.go:136] cert-manager/controller/certificates-request-manager "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
42:E1025 20:52:02.789102       1 trigger_controller.go:137] cert-manager/controller/certificates-trigger "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
43:E1025 20:52:02.789120       1 revisionmanager_controller.go:106] cert-manager/controller/certificates-revision-manager "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
44:E1025 20:52:02.789139       1 readiness_controller.go:142] cert-manager/controller/certificates-readiness "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
45:E1025 20:52:02.789199       1 issuing_controller.go:152] cert-manager/controller/certificates-issuing "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
46:E1025 20:52:02.956024       1 controller.go:163] cert-manager/controller/certificates-key-manager "msg"="re-queuing item due to error processing" "error"="secrets \"selfsigned-cert-\" is forbidden: unable to create new content in namespace cert-manager-test because it is being terminated" "key"="cert-manager-test/selfsigned-cert"
47:E1025 20:52:02.956070       1 keymanager_controller.go:139] cert-manager/controller/certificates-key-manager "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
51:I1025 20:52:03.037561       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="re-queuing item due to optimistic locking on resource" "key"="capi-system/capi-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
54:I1025 20:52:03.086437       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="re..." "key"="capi-system/capi-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
58:I1025 20:52:03.126360       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capi-system/capi-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
60:I1025 20:52:03.136875       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capi-system/capi-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
65:I1025 20:52:03.266488       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-bootstrap-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
68:I1025 20:52:03.369530       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="..." "key"="capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-bootstrap-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
73:I1025 20:52:03.395063       1 controller.go:161] cert-manager/controller/certificates-trigger "msg"="re-queuing item due to optimistic locking on resource" "key"="etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-bootstrap-provider-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
79:I1025 20:52:03.419626       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-bootstrap-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
81:I1025 20:52:03.428145       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="..." "key"="capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-bootstrap-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
85:I1025 20:52:03.534188       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-bootstrap-provider-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
87:I1025 20:52:03.537428       1 controller.go:161] cert-manager/controller/certificates-issuing "msg"="..." "key"="etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-bootstrap-provider-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
88:I1025 20:52:03.541702       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="re-queuing item due to optimistic locking on resource" "key"="etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-bootstrap-provider-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
92:I1025 20:52:03.568072       1 controller.go:161] cert-manager/controller/certificates-trigger "msg"="..." "key"="etcdadm-controller-system/etcdadm-controller-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-controller-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
99:I1025 20:52:03.718792       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="etcdadm-controller-system/etcdadm-controller-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-controller-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
101:I1025 20:52:03.723278       1 controller.go:161] cert-manager/controller/certificates-issuing "msg"="..." "key"="etcdadm-controller-system/etcdadm-controller-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-controller-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
102:I1025 20:52:03.727309       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="..." "key"="etcdadm-controller-system/etcdadm-controller-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"etcdadm-controller-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
103:E1025 20:52:03.769911       1 readiness_controller.go:142] cert-manager/controller/certificates-readiness "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
104:E1025 20:52:03.956955       1 keymanager_controller.go:139] cert-manager/controller/certificates-key-manager "msg"="certificate not found for key" "error"="certificate.cert-manager.io \"selfsigned-cert\" not found" "key"="cert-manager-test/selfsigned-cert"
108:I1025 20:52:04.043249       1 controller.go:161] cert-manager/controller/certificates-trigger "msg"="re-queuing item due to optimistic locking on resource" "key"="capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-control-plane-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
115:I1025 20:52:04.163286       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="r..." "key"="capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-control-plane-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
117:I1025 20:52:04.170751       1 controller.go:161] cert-manager/controller/certificates-issuing "msg"="..." "key"="capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-control-plane-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
118:I1025 20:52:04.172335       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capi-kubeadm-control-plane-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
123:I1025 20:52:04.836824       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capd-system/capd-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capd-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
126:I1025 20:52:04.951381       1 controller.go:161] cert-manager/controller/certificates-key-manager "msg"="re-queuing item due to optimistic locking on resource" "key"="capd-system/capd-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capd-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
130:I1025 20:52:04.978760       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capd-system/capd-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capd-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"
132:I1025 20:52:04.985399       1 controller.go:161] cert-manager/controller/certificates-readiness "msg"="..." "key"="capd-system/capd-serving-cert" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"capd-serving-cert\": the object has been modified; please apply your changes to the latest version and try again"

kube-system/kube-controller-manager-k8s-play-eks-a-cluster-control-plane.log
9:E1025 20:51:19.247194       1 leaderelection.go:330] error retrieving resource lock kube-system/kube-controller-manager: leases.coordination.k8s.io "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-system"
271:E1025 20:52:07.786800       1 tokens_controller.go:262] error synchronizing serviceaccount cert-manager-test/default: secrets "default-token-kcbjp" is forbidden: unable to create new content in namespace cert-manager-test because it is being terminated
273:W1025 20:52:28.350307       1 endpointslice_controller.go:306] Error syncing endpoint slices for service "capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-webhook-service", retrying. Error: EndpointSlice informer cache is out of date

kube-system/kube-apiserver-k8s-play-eks-a-cluster-control-plane.log
39:E1025 20:51:19.238656       1 controller.go:152] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/***HIDDEN***, ResourceVersion: 0, AdditionalErrorMsg:
130:W1025 21:08:30.063246       1 watcher.go:229] watch chan error: etcdserver: mvcc: required revision has been compacted
137:W1025 21:18:03.037553       1 watcher.go:229] watch chan error: etcdserver: mvcc: required revision has been compacted

kube-system/kube-scheduler-k8s-play-eks-a-cluster-control-plane.log
3:W1025 20:51:19.249418       1 authentication.go:345] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

cert-manager/cert-manager-cainjector-f5b94ccdf-7k9sv.log
44:E1025 20:51:58.672741       1 controller.go:175] cert-manager/secret/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="unable to update target object with new CA data" "error"="Operation cannot be fulfilled on mutatingwebhookconfigurations.admissionregistration.k8s.io \"cert-manager-webhook\": the object has been modified; please apply your changes to the latest version and try again" "resource_kind"="MutatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1"
45:E1025 20:51:58.672798       1 controller.go:304] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration "msg"="Reconciler error" "error"="Operation cannot be fulfilled on mutatingwebhookconfigurations.admissionregistration.k8s.io \"cert-manager-webhook\": the object has been modified; please apply your changes to the latest version and try again" "name"="cert-manager-webhook" "namespace"=""
61:E1025 20:52:02.812634       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated certificate" "error"="Certificate.cert-manager.io \"capi-serving-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="clusterclasses.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1"
...
115:E1025 20:52:03.031904       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated secret" "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="clusterclasses.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
...
169:E1025 20:52:03.050701       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="MutatingWebhookConfiguration" "resource_name"="capi-mutating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
171:E1025 20:52:03.058308       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="capi-validating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
173:E1025 20:52:03.090625       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="capi-validating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
175:E1025 20:52:03.090627       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="MutatingWebhookConfiguration" "resource_name"="capi-mutating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
177:E1025 20:52:03.091687       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="..." "error"="Secret \"capi-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-system","Name":"capi-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="clusterclasses.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-system","Name":"capi-webhook-service-cert"}
...
195:E1025 20:52:03.104518       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated certificate" "error"="Certificate.cert-manager.io \"capi-kubeadm-bootstrap-serving-cert\" not found" "certificate"={"Namespace":"capi-kubeadm-bootstrap-system","Name":"capi-kubeadm-bootstrap-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="kubeadmconfigs.bootstrap.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1"
215:E1025 20:52:03.266486       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated secret" "error"="Secret \"capi-kubeadm-bootstrap-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-kubeadm-bootstrap-system","Name":"capi-kubeadm-bootstrap-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="kubeadmconfigs.bootstrap.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-kubeadm-bootstrap-system","Name":"capi-kubeadm-bootstrap-webhook-service-cert"}
218:E1025 20:52:03.272898       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated secret" "error"="..." "certificate"={"Namespace":"capi-kubeadm-bootstrap-system","Name":"capi-kubeadm-bootstrap-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="kubeadmconfigtemplates.bootstrap.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-kubeadm-bootstrap-system","Name":"capi-kubeadm-bootstrap-webhook-service-cert"}
...
331:E1025 20:52:04.100442       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated certificate" "error"="Certificate.cert-manager.io \"capd-serving-cert\" not found" "certificate"={"Namespace":"capd-system","Name":"capd-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="dockerclusters.infrastructure.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1"
...
343:E1025 20:52:04.120411       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated secret" "error"="Secret \"capi-kubeadm-control-plane-webhook-service-cert\" not found" "certificate"={"Namespace":"capi-kubeadm-control-plane-system","Name":"capi-kubeadm-control-plane-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="kubeadmcontrolplanetemplates.controlplane.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capi-kubeadm-control-plane-system","Name":"capi-kubeadm-control-plane-webhook-service-cert"}
359:E1025 20:52:04.711835       1 sources.go:114] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated certificate" "error"="Certificate.cert-manager.io \"capd-serving-cert\" not found" "certificate"={"Namespace":"capd-system","Name":"capd-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="dockerclustertemplates.infrastructure.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1"
...
383:E1025 20:52:04.829939       1 sources.go:124] cert-manager/certificate/customresourcedefinition/generic-inject-reconciler "msg"="unable to fetch associated secret" "error"="Secret \"capd-webhook-service-cert\" not found" "certificate"={"Namespace":"capd-system","Name":"capd-serving-cert"} "resource_kind"="CustomResourceDefinition" "resource_name"="dockerclusters.infrastructure.cluster.x-k8s.io" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capd-system","Name":"capd-webhook-service-cert"}
...
413:E1025 20:52:04.848016       1 sources.go:124] cert-manager/certificate/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capd-webhook-service-cert\" not found" "certificate"={"Namespace":"capd-system","Name":"capd-serving-cert"} "resource_kind"="MutatingWebhookConfiguration" "resource_name"="capd-mutating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capd-system","Name":"capd-webhook-service-cert"}
415:E1025 20:52:04.855050       1 sources.go:124] cert-manager/certificate/validatingwebhookconfiguration/generic-inject-reconciler "msg"="..." "error"="Secret \"capd-webhook-service-cert\" not found" "certificate"={"Namespace":"capd-system","Name":"capd-serving-cert"} "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="capd-validating-webhook-configuration" "resource_namespace"="" "resource_version"="v1" "secret"={"Namespace":"capd-system","Name":"capd-webhook-service-cert"}
...

capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-controller-manager-68b4c848dc-f7tfn.log
37:E1025 20:52:58.401912       1 controller.go:242] controller/kubeadmcontrolplane "msg"="Failed to update KubeadmControlPlane Status" "error"="failed to create remote cluster client: error creating client and cache for remote cluster: error creating dynamic rest mapper for remote cluster \"eksa-system/k8s-play\": context deadline exceeded" "cluster"="k8s-play" "name"="k8s-play" "namespace"="eksa-system" "reconciler group"="controlplane.cluster.x-k8s.io" "reconciler kind"="KubeadmControlPlane"
38:E1025 20:52:58.428989       1 controller.go:317] controller/kubeadmcontrolplane "msg"="Reconciler error" "error"="..." "name"="k8s-play" "namespace"="eksa-system" "reconciler group"="controlplane.cluster.x-k8s.io" "reconciler kind"="KubeadmControlPlane"
40:E1025 20:53:18.603785       1 controller.go:242] controller/kubeadmcontrolplane "msg"="Failed to update KubeadmControlPlane Status" "error"="..." "cluster"="k8s-play" "name"="k8s-play" "namespace"="eksa-system" "reconciler group"="controlplane.cluster.x-k8s.io" "reconciler kind"="KubeadmControlPlane"
41:E1025 20:53:18.632466       1 controller.go:317] controller/kubeadmcontrolplane "msg"="Reconciler error" "error"="..." "name"="k8s-play" "namespace"="eksa-system" "reconciler group"="controlplane.cluster.x-k8s.io" "reconciler kind"="KubeadmControlPlane"

etcdadm-bootstrap-provider-system/etcdadm-bootstrap-provider-controller-manager-664f699b7c-9gth9.log
43:1.7298895614229357e+09   INFO    controllers.EtcdadmConfig   bootstrap data secret for EtcdadmConfig already exists, updating    {"secretError": "json: unsupported type: func(string, secret.Purpose) string", "EtcdadmConfig": "k8s-play-etcd-vvb65"}


Help appreciated!

Terminal command completed with error - like Peter in commented suggested, it is cert manager.
Error: initializing capi resources in cluster: executing init: Fetching providers
Using Override="core-components.yaml" Provider="cluster-api" Version="v1.1.3+9999d58"
Using Override="bootstrap-components.yaml" Provider="bootstrap-kubeadm" Version="v1.1.3+e448518"
Using Override="bootstrap-components.yaml" Provider="bootstrap-etcdadm-bootstrap" Version="v1.0.0-rc6+eece35c"
Using Override="bootstrap-components.yaml" Provider="bootstrap-etcdadm-controller" Version="v1.0.0-rc9+45b432d"
Using Override="control-plane-components.yaml" Provider="control-plane-kubeadm" Version="v1.1.3+9e13ae0"
Using Override="infrastructure-components-development.yaml" Provider="infrastructure-docker" Version="v1.1.3+736b241"
Installing cert-manager Version="v1.5.3+c34401d"
Using Override="cert-manager.yaml" Provider="cert-manager" Version="v1.5.3+c34401d"
Waiting for cert-manager to be available...
Error: timed out waiting for the condition

I am going to try with minikube to eliminate AWS EKS issues.
minikube worked just fine (https://kubernetes.io/docs/tutorials/hello-minikube/) so the issue is with AWS EKS.

**Tags:** kubernetes, eks

> **Comments:**
> **username_1**: I'm not expert enough to answer definitively, but I think the second line of your error message is the real error: `40:E1025 20:52:02.772058       1 controller.go:133] cert-manager/controller/issuers "msg"="issuer in work queue no longer exists" "error"="issuer.cert-manager.io \"test-selfsigned\" not found`  
> **username_0**: Ok thanks Peter, I will see what I can work out with that.  
> **username_0**: Nothing fruitful turned up.  Terminal finally completed with error.  I will add this to question.  

---

## 0 answers

