Skip to content

Release notes for kOps 1.37 series

kOps 1.37 adds support for Kubernetes 1.37 and for private asset repositories in cloud object storage, expands support for Karpenter and Akamai (Linode), and updates the default containerd, runc, and etcd versions. It removes support for Kubernetes 1.31, gossip DNS, AWS Classic Load Balancers for the API, and Kopeio networking. Read the breaking changes before you upgrade.

Field names in these notes are from the v1alpha3 API. Where the v1alpha2 name is different, it follows in parentheses.

Significant changes

Kubernetes

  • kOps 1.37 supports Kubernetes 1.37. (#18772)

  • The new spec.kubelet.nodeStatusReportFrequency field sets how often the kubelet posts its full node status when the status does not change. When nodeStatusUpdateFrequency is set and this field is not, the kubelet uses the update frequency as the report frequency instead of its 5-minute default. (#18941)

  • The new spec.kubeAPIServer.strictTransportSecurityDirectives field sets the HTTP Strict Transport Security (HSTS) directives that kube-apiserver adds to its responses. (#18577)

Container Runtime

  • containerd 2.3.6 and runc 1.5.1 are now the defaults. If you pin spec.containerd.version to 2.3, use version 2.3.4 or newer. Versions 2.3.0 through 2.3.3 cannot start pods when kOps pins the sandbox image by digest. (#18683, #18808, containerd#13529)

  • For containerd registry mirrors whose URL has a path, such as an ECR pull-through cache at https://<account>.dkr.ecr.<region>.amazonaws.com/v2/<prefix>, kOps now sets override_path = true in hosts.toml, so that containerd does not append a second /v2 to the URL. Mirror URLs without a path are unchanged. (#18819)

  • A value in spec.containerd.nri.pluginRequestTimeout or spec.containerd.nri.pluginRegistrationTimeout no longer causes nodeup to write an empty containerd configuration file. When spec.containerd.nri.enabled is true, kOps now rejects a timeout that is zero or negative. (#18721)

Node bootstrap

  • Nodes now download an xz-compressed nodeup binary, which is approximately one quarter of the size of the uncompressed binary. The bootstrap script verifies the SHA-256 checksum of the compressed file before it decompresses it. (#18737, #18765)

  • Nodes in IPv6-only clusters that use --dns=none now bootstrap correctly. (#18684)

  • Node bootstrap now retries failed tasks and waits at most 30 seconds between kops-controller requests. Nodes recover faster when the control plane starts slowly or many nodes join at the same time. (#18675, #18718, #18742)

  • nodeup now writes /etc/sysconfig/kops-configuration with mode 0600. This file can contain state store and OpenStack credentials, and it was readable by all users. (#18654)

Private asset repositories

KOPS_BASE_URL and spec.assets.fileRepository now support private asset repositories in cloud object storage. Nodes authenticate to the storage with the identity of their instance. The asset repository documentation gives more information.

  • AWS s3:// URLs: grant s3:GetObject on the bucket to the instance profiles of the instance groups. The node image must provide curl 8.0 or newer, because the bootstrap script downloads nodeup from the s3:// location. kOps supports s3:// repositories in the commercial AWS and AWS GovCloud partitions. (#18661)

  • Google Cloud Storage gs:// URLs: grant roles/storage.objectViewer on the bucket to the instance service account. (#18466, #18623, #18713)

  • Azure azureblob://<account>/<container>/<prefix> URLs: grant the Storage Blob Data Reader role on the asset container to the system-assigned managed identity of the nodes. kOps supports azureblob:// repositories only in the Azure public cloud. (#18666)

Networking

  • kOps now uses Calico v3.32.3. When spec.networking.calico.nftablesMode is not set, Calico now selects the nftables or iptables backend from the kube-proxy configuration. (#18701, #18969)

  • kOps now uses Cilium v1.18.16, AWS VPC CNI v1.23.2, and Flannel v0.28.9. The default CNI plugins version is v1.7.1 for Kubernetes 1.34 and v1.8.0 for Kubernetes 1.35. Kubernetes 1.36 or newer uses v1.9.1. Kubernetes 1.32 and 1.33 continue to use v1.6.2. (#18692, #18704, #18705, #18874, #18970)

  • With spec.networking.cilium.chainingMode: portmap, nodes now install the standard CNI plugins bundle, which supplies the portmap plugin. (#18589)

  • The new spec.networking.amazonVPC.enableNetworkPolicy (spec.networking.amazonvpc.enableNetworkPolicy) field turns on the network policy agent in the aws-node DaemonSet and sets enable-network-policy-controller in the amazon-vpc-cni ConfigMap, so that NetworkPolicy resources are enforced. kOps does not deploy the Amazon network policy controller. Deploy it and its networking.k8s.aws CRDs as a custom addon before you set the field. Until the CRDs exist, the node agent restarts and new pods fail to start. The node agent requires Linux kernel 5.10 or newer. (#19013)

  • kOps now uses kindnet v1.0.3. kindnet now gets new local pods from containerd and does not wait for the API server. DNS caching is disabled by default. Set spec.networking.kindnet.dnsCaching: true to keep the previous behavior. (#18635, #18727, #18984)

  • kOps now uses kube-router v2.12.0, with these changes:

    • The new spec.networking.kubeRouter.useNFTablesForNetpol (spec.networking.kuberouter.useNFTablesForNetpol) field uses nftables instead of iptables and ipsets for network policy enforcement. The nftables backend is experimental. This field does not change the service proxy or router controllers. (#18708)
    • Repeated sync failures no longer restart kube-router through the liveness probe. Monitor the kube_router_controller_sync_last_success and kube_router_controller_sync_failures_total metrics for these failures. (#18961)
    • On the first sync after the upgrade, kube-router recreates its network policy chains and sets with new names and removes the old ones. (#18961)
    • The new spec.networking.kubeRouter.externalIPRanges (spec.networking.kuberouter.externalIPRanges) and spec.networking.kubeRouter.loadBalancerIPRanges (spec.networking.kuberouter.loadBalancerIPRanges) fields restrict the CIDRs that kube-router accepts for Service external and load-balancer IPs. Either field enables strict external IP validation. kube-router does not program or advertise addresses outside the specified ranges. If the fields are not set, kube-router continues to accept all addresses. (#18757)
    • The DaemonSet now uses a startup probe, so a slow start no longer causes a CrashLoopBackOff. The DaemonSet also sets hostIPC: true, which kube-router requires for direct server return (DSR). (#18757)

  • kOps now uses CoreDNS v1.15.0. The CoreDNS PodDisruptionBudget now uses maxUnavailable: 33% instead of 50%. With three or more replicas, voluntary disruptions leave at least two DNS pods running. The budget also sets unhealthyPodEvictionPolicy: AlwaysAllow. This setting lets a node drain evict CoreDNS pods that are running but not ready. (#18679, #18980)

  • When spec.externalDNS.provider (spec.externalDns.provider) is external-dns, kOps now uses external-dns v0.24.0 on AWS and GCE. external-dns v0.24.0 reads the external-dns.kubernetes.io/ annotation prefix. kOps enables --enable-legacy-annotation-prefix, so external-dns.alpha.kubernetes.io/ annotations continue to work. Upstream will remove this flag in a future release, so change these annotations to the new prefix. (#18998)

AWS

  • The new spec.cloudProvider.aws.useIPBasedNodeNames (spec.cloudConfig.useIPBasedNodeNames) field uses the EC2 private DNS name as the Kubernetes node name, for example ip-10-0-0-1.eu-west-1.compute.internal, instead of the EC2 instance ID. kOps does not support this setting on IPv6-only clusters. kOps configures the subnets that it manages to assign IP-based hostnames. Shared subnets must already assign them. The setting affects only new nodes. To rename existing nodes, replace them with kops rolling-update cluster. On a running cluster, first replace the control-plane nodes or delete the kops-controller pods, because nodes that start before kops-controller has the new setting can fail to bootstrap. (#18632)

  • Karpenter support now includes these changes:

    • kOps now uses Karpenter v1.15.1. Its IAM policy now grants ec2:RebootInstances on the instances that Karpenter manages, matching the upstream reference policy. (#18909)
    • Generated EC2NodeClass objects now map the root volume settings of the instance group to blockDeviceMappings. When a root volume uses a customer managed KMS key, Karpenter's IAM policy includes the KMS permissions for encrypted volumes. (#18655)
    • spec.mixedInstancesPolicy.instanceRequirements now creates CPU, memory, and excluded-instance requirements in the NodePool. When instance requirements are set, spec.machineType and spec.mixedInstancesPolicy.instances are optional. (#18676, #18761)
    • A Karpenter instance group can now mix GPU and non-GPU instance types. With kOps-managed NVIDIA support, use a GPU-only instance group, because kOps applies the GPU labels and taints to the whole NodePool. (#18761)
    • Karpenter's iam:PassRole permission now applies only to EC2 and the worker node role. With a custom instance profile, Karpenter can pass any role to EC2. Mutating EC2 actions now require the cluster and karpenter.sh/nodepool tags. (#18582)
    • When Karpenter is enabled, kOps now configures interruption handling: an SQS queue, the EventBridge rules for spot interruptions, instance state changes, scheduled changes, and rebalance recommendations, and the IAM permissions for Karpenter. Clusters without internet egress need a VPC endpoint for SQS. A cluster that changes from Node Termination Handler to Karpenter keeps its queue, and kOps disables the Node Termination Handler lifecycle hooks of the Auto Scaling groups. Node Termination Handler and Karpenter remain mutually exclusive. (#18607)
    • kOps now rejects a change of spec.manager on an existing instance group, because there is no migration path between Karpenter and an Auto Scaling group. Create a new instance group with the other manager and delete the old one. (#18952)

  • kOps narrows the IAM permissions of the instance roles:

    • The ec2:ModifyInstanceAttribute permission for kube-router and kindnet applies only to instances that have the cluster tag. (#18654)
    • kOps grants ec2:DescribeInstanceTypes only to instance roles that require it. NVIDIA GPU support and Cilium ENI IPAM require this permission. AWS VPC CNI clusters retain it as a CNI permission. The cluster-autoscaler permissions omit it when spec.clusterAutoscaler.awsUseStaticInstanceList is enabled. (#18656)
    • kOps no longer grants ec2:DescribeRegions to etcd-manager. etcd-manager reads the region from the instance metadata service. (#18656)
    • The sqs:DeleteMessage and sqs:ReceiveMessage permissions of Node Termination Handler apply only to queues that have the cluster tag. (#18607)
    • kOps no longer creates inline IAM policies for roles that require no permissions. This includes the bastion role. It also includes the worker role when spec.iam.allowContainerRegistry is not enabled and no other feature requires node permissions. kops update cluster deletes earlier inline policies from these roles. The Terraform target removes the related aws_iam_role_policy resources during the next apply. (#18656)

  • AWS VPC CNI node roles now include ec2:DescribeSecurityGroups. The CNI requires this permission for subnet discovery. Without it, the CNI allocates addresses only from the primary ENI's subnet. (#18693)

  • On nodes that use AWS VPC CNI or Cilium ENI IPAM, the network configuration now keeps the DNS servers of the primary interface on Graviton4 instances, and leaves the secondary ENIs unmanaged on 8th-generation Intel instances. Previously, the configuration selected interfaces by name, and these instance types use different interface names. (#18642, #18725)

  • kOps now uses pod identity webhook v0.6.19. The image now comes from public.ecr.aws/eks/amazon-eks-pod-identity-webhook, because upstream no longer publishes releases to Docker Hub. The webhook now also applies spec.iam.serviceAccountExternalPermissions entries that set namespace: "*", so pods in any namespace that use a service account with that name get the IAM role. The webhook ignores pods that have the eks.amazonaws.com/skip-pod-identity-webhook label. (#18805, #18924)

  • kOps now uses AWS Load Balancer Controller v3.6.3. The new spec.cloudProvider.aws.loadBalancerController.defaultLoadBalancerScheme (spec.awsLoadBalancerController.defaultLoadBalancerScheme) field sets the scheme of the load balancers that the controller creates for Services and Ingresses without a scheme annotation. The controller defaults to internal. (#18894, #18919)

  • The AWS Load Balancer Controller's Service webhook now handles only Services of type LoadBalancer, so other Services can be created while the controller is unavailable. The webhook previously skipped every Service in kube-system; it now also sets the load balancer class on new LoadBalancer Services there.

  • kOps now uses aws-iam-authenticator v0.7.24. The addon again uses the backendMode, clusterID, and identityMappings settings of spec.authentication.aws, which kOps 1.36.0 omitted. (#18600, #18929)

  • kOps now deploys the same cloud-controller-manager version for all supported Kubernetes versions. (#18649, #18988)

  • The AWS EBS CSI driver addon no longer creates the ebs-csi-controller-volumemodifier Service, which had no endpoints. (#18775)

  • In a shared VPC (spec.networking.networkID, spec.networkID), the cluster spec can omit the zone of an AWS subnet specified by ID. kOps gets the zone from EC2. At least one subnet must specify a zone, because kOps gets the region from the zones. (#18630)

  • kOps now rejects V2 bootstrap tokens when their pre-signed STS URLs do not use HTTPS. (#18574)

  • kOps now sends MaxInstanceLifetime to an Auto Scaling group only when the value changes. Previously, every other change to the group cleared it, and the next update set it again. (#18955)

Azure

  • Azure nodes now authenticate their bootstrap requests to kops-controller with signed instance metadata. kOps verifies the signature chain, nonce, expiration, and VM ID of each request. (#18163)

  • kops rolling-update cluster now detects the Azure VMs that do not run the latest scale set model, and replaces them. (#18669)

  • On clusters with a highly available control plane, kOps now runs two replicas of the Azure cloud-controller-manager. (#18669)

  • The Azure cloud-controller-manager v1.37 validates the annotations of load balancer Services more strictly, and rejects invalid or conflicting values that earlier versions accepted. It also rejects a Service whose service.beta.kubernetes.io/azure-pls-name annotation names a private link service that another Service uses. After the upgrade, correct the annotations of Services that report SyncLoadBalancerFailed events. (#18988)

  • On kindnet clusters, a new network security group rule lets the pod CIDR connect to the Kubernetes API. (#18604)

GCP

  • kOps now registers the SSH public key under the ubuntu user for Ubuntu images. The guest agent cannot create the admin user on these images, so the key was never installed. Connect to Ubuntu nodes as ubuntu. Other images continue to use admin. (#18581)

  • The firewall rule for traffic from nodes to control-plane instances now allows the metrics ports of kube-controller-manager, kube-scheduler, kube-proxy, etcd, and node-exporter with every networking provider. Previously, only clusters with IP alias networking had access to these ports, and not to the etcd events and node-exporter ports. With IP alias or GCE routes networking, a new rule gives the pod CIDR the same access to control-plane instances as the nodes have. (#18545, #18620, #18685, #18753)

  • During GCE cluster deletion, kOps no longer selects unrelated empty backend services. It also removes routes and load-balancer firewall rules that can block network deletion. (#18460, #18760)

  • kOps no longer ships the metadata-proxy addon. Only GCE clusters with Kubernetes versions older than 1.29 used it. kops create instancegroup no longer adds the cloud.google.com/metadata-proxy-ready=true label on GCE. Existing instance groups that carry the label are unaffected. (#19001)

Hetzner

  • The cloud-controller-manager and CSI driver versions that kOps deploys support Kubernetes 1.34 and newer. Kubernetes 1.32 and 1.33 clusters run component versions that upstream does not test with these Kubernetes versions. (#18974, #18988)

  • kops rolling-update cluster no longer reports every Hetzner server as in need of an update. The Hetzner API no longer returns the datacenter of a server, and kOps now reads the server location. kops update cluster also no longer reports changes to an API load balancer that has no changes. (#18631, #18638)

DigitalOcean

  • When kOps deletes a DigitalOcean cluster, it now removes the SSH key that it uploaded for the cluster. (#18697)

  • kOps keeps the cloud-controller-manager at v0.1.45. Newer releases resolve nodes through the InstancesV2 interface, which requires a providerID that kOps does not set, and leave every node with the uninitialized taint. (#18775)

OpenStack

  • Kubernetes 1.37 clusters use openstack-cloud-controller-manager and cinder-csi-plugin v1.37.1, because the v1.37.0 release of cloud-provider-openstack is broken. Other Kubernetes versions continue to use the .0 release of their minor version. (#18868)

  • kOps now waits longer for an instance to be deleted. Nova releases the boot volume of a volume-backed server before it deletes the server record, so the deletion takes longer. (#18899)

Akamai (Linode) (experimental)

  • The provider can now manage SSH keys, networks, subnets, instances, and block-storage volumes. This support includes etcd data volumes. The provider also supplies a default network CIDR and kops-controller configuration. (#18383, #18590, #18622, #18700, #18709, #18743, #18744)

Etcd

  • Kubernetes 1.37 clusters now use etcd 3.7.3 by default. Kubernetes 1.34 through 1.36 clusters use etcd 3.6.16. Kubernetes 1.32 and 1.33 clusters use etcd 3.5.35. (#18557, #18813, #18914)

  • kOps now uses etcd-manager v3.0.20261022. (#18905)

  • etcd-manager now mounts the bundled etcd images as image volumes on all supported Kubernetes versions. For versions older than Kubernetes 1.36, kOps enables the ImageVolume kubelet feature gate on control-plane nodes, unless the kubelet configuration already sets this feature gate. (#18688)

  • A new cluster can run an etcd version that kOps does not bundle. Set both version and image on each etcd cluster when you create the cluster. etcd-manager then takes the etcd binaries from that image. The image cannot be added, changed, or removed after the etcd cluster exists, and the version cannot be changed while the image is set. kOps does not support the restore of backups for an etcd cluster that uses a custom image. The cluster spec documentation gives the requirements. (#18525)

Operating System Support

  • On RHEL 10-family operating systems, nodeup installs kernel-modules-extra. This package supplies the kernel modules that networking components require. (#18724)

  • kOps adds experimental support for Amazon Linux 2027, which is in public preview. AWS does not recommend the preview for production workloads and will deprecate the preview AMIs at general availability. (#18777)

Cloud controller managers and CSI drivers

kOps updates the default versions of the cloud controller managers and CSI drivers that it deploys. (#18775, #18974, #18988)

Provider Component Version
AWS cloud-controller-manager v1.37.1
AWS EBS CSI driver v1.67.2
GCP cloud-controller-manager v37.2.5
GCP PD CSI driver v1.28.1
Azure cloud-controller-manager and cloud-node-manager v1.37.3
Azure azuredisk CSI driver v1.36.1
DigitalOcean CSI driver v4.18.0
Hetzner cloud-controller-manager v1.39.2
Hetzner CSI driver v2.24.1
  • On AWS, Azure, DigitalOcean, GCP, and Scaleway, cloud-controller-manager pods now tolerate all taints. (#18510)

Other components

  • cluster-autoscaler defaults to v1.37.0 for Kubernetes 1.34 and newer, and to v1.34.6 for Kubernetes 1.32 and 1.33, because cluster-autoscaler v1.35 and newer does not start on these Kubernetes versions. (#18968, #19008)

  • kOps now uses cert-manager v1.21.3. The new spec.certManager.webhookFeatureGates and spec.certManager.cainjectorFeatureGates fields set feature gates on the webhook and cainjector Deployments. spec.certManager.featureGates continues to apply to the controller only. (#18787, #18997)

  • kOps now uses metrics-server v0.9.1. (#18996)

  • kOps is now built with Go 1.27.3. This Go release distrusts the compromised eMudhra intermediate certificates (CVE-2026-111943), a fix that the Go project handled as urgent, and it includes several other security fixes. Most of the addons that kOps deploys are also updated to versions that are built with a fixed Go version. (#18883)

Other changes of note

  • The new kops validate cluster --max-unready-nodes flag lets validation tolerate a specified number of non-ready worker nodes. (#18614)

  • An addon object with a False condition that does not report readiness, for example a KEDA ScaledObject with Paused: "False", no longer causes kops validate cluster to fail. The addon health check now uses only the condition types that report readiness, such as Ready and Available. (#18657)

  • kops update cluster --instance-group and --instance-group-roles, and the first step of kops reconcile cluster, now keep the resources that the selected instance groups share with the other instance groups. On Hetzner, the cluster-autoscaler node groups of the instance groups that are not being updated are still left out until an update includes them. (#18850, #18851, #18855, #18856, #18857, #18860, #18952)

  • kops create cluster now rejects an empty --project value. Omit the flag to use the default project of the gcloud configuration. (#18575)

  • kOps now preserves spec.cloudProvider.aws.binariesLocation and spec.cloudProvider.gce.binariesLocation when it stores cluster specs through the v1alpha2 API, in the new spec.cloudConfig.awsBinariesLocation and spec.cloudConfig.gceBinariesLocation fields. Previously, kOps removed these settings without a warning. (#18740)

  • Release binaries are smaller. nodeup no longer includes unused cloud and Kubernetes clients, and the linker removes unused methods. (#18552, #18554, #18571, #18572, #18687, #18695, #18710, #18721)

Breaking changes

  • kOps no longer supports Kubernetes 1.31. Upgrade clusters to Kubernetes 1.32 or newer before an upgrade to kOps 1.37. (#18598)

  • kOps no longer supports gossip DNS, which kOps deprecated in 1.29. kOps rejects a cluster whose name ends in .k8s.local, unless spec.networking.topology.dns (spec.topology.dns.type) is None. Use kOps 1.36 to migrate existing gossip clusters to None-DNS before the upgrade. The migration keeps the cluster name. The gossip documentation gives the migration steps. kOps also rejects cluster specs that set spec.gossipConfig or spec.dnsControllerGossipConfig. Remove these fields before the upgrade. (#18601)

  • kOps no longer supports an AWS Classic Load Balancer (CLB) for the API, which kOps deprecated in 1.26. A cluster fails validation if spec.api.loadBalancer.class is Classic or is not set, which previously selected Classic. kOps also removes the deprecated kops create cluster --api-loadbalancer-class flag. Existing clusters must use kOps 1.36 or earlier to migrate to a Network Load Balancer (NLB). See the CLB to NLB migration guide. kOps ignores spec.api.loadBalancer.idleTimeoutSeconds and spec.api.loadBalancer.accessLog.interval, which applied only to CLBs. The v1alpha3 API no longer has these fields. This change does not affect externally managed CLBs that use spec.externalLoadBalancers[].loadBalancerName. (#18617)

  • kOps no longer supports Kopeio networking because the project is no longer maintained. Clusters with spec.networking.kopeio now fail validation. kops create cluster no longer accepts --networking kopeio or --networking kopeio-vxlan. Before an upgrade to kOps 1.37, use kOps 1.36 or earlier to switch to another CNI and replace all nodes with kops rolling-update cluster. The switch interrupts pod networking. kOps does not remove the old CNI, so delete the kopeio-networking-agent DaemonSet in the kube-system namespace after the switch. The kindnet CNI provides similar minimal networking. (#18706)

  • Calico 3.32 enforces ClusterNetworkPolicy resources and no longer enforces AdminNetworkPolicy and BaselineAdminNetworkPolicy resources. It also removes the adminnetworkpolicy and baselineadminnetworkpolicy static tiers. Calico 3.31 in kOps 1.36 does not enforce ClusterNetworkPolicy resources, and kOps 1.36 does not install the ClusterNetworkPolicy CRD. To keep these policies in effect during the upgrade, install the CRD from the standard channel of network-policy-api v0.2.0, which is the version that kOps 1.37 installs with Calico, and create equivalent ClusterNetworkPolicy resources before the upgrade. Keep the AdminNetworkPolicy and BaselineAdminNetworkPolicy resources until all nodes run Calico 3.32, then delete them. kOps does not delete these resources or their CRDs. (#18969)

  • kOps no longer supports containerd versions older than 2.1.0. A spec.containerd.version value below 2.1.0 fails validation. Remove the pinned version to use the default version, 2.3.6, or pin version 2.3.4 or newer. Nodes install the new runtime when they are replaced. (#18683)

  • etcd-manager now requires image volumes, and an explicit ImageVolume: "false" setting stops etcd-manager. Before you upgrade a cluster that runs Kubernetes 1.32 through 1.35, remove ImageVolume: "false" from spec.kubelet.featureGates and spec.controlPlaneKubelet.featureGates (spec.masterKubelet.featureGates) in the cluster spec, and from spec.kubelet.featureGates in every control-plane instance group. (#18688)

  • kOps no longer has the SkipEtcdVersionCheck feature flag. kOps rejects an etcd version that it does not bundle unless the etcd cluster also sets image, which is possible only when the cluster is created. An existing etcd cluster with a version that kOps does not bundle must change to a bundled version before the upgrade. (#18525)

  • cert-manager v1.21 no longer includes the cert-manager-tokenrequest Role and RoleBinding, which let the cert-manager controller create tokens for its own service account. kOps deletes them during the upgrade. If an issuer sets serviceAccountRef.name: cert-manager, create a Role and RoleBinding before the upgrade that grant the cert-manager controller create on serviceaccounts/token for that service account. cert-manager recommends a dedicated service account for the issuer, which needs the same grant. The cert-manager-edit ClusterRole, which is aggregated to the edit and admin roles, no longer grants create on challenges, or create, patch, and update on orders. The metrics port of the cert-manager controller Service is now named http-metrics instead of tcp-prometheus-servicemonitor. Update the ServiceMonitor objects and scrape configurations that select the port by name. (#18997)

  • On AWS, kOps removes instance role permissions that other workloads can depend on. Before the upgrade, grant these workloads the permissions through spec.additionalPolicies or a service account IAM role:

    • kOps no longer grants Route 53 permissions to the control-plane role when spec.iam.useServiceAccountExternalPermissions is enabled. dns-controller and external-dns use their service account IAM roles. (#18654)
    • KMS policies for customer managed keys restrict kms:CreateGrant to grants that AWS services create for the role. The policies restrict the other KMS actions to calls through EC2 or S3, so a workload that calls KMS directly with an instance role loses this access. When spec.encryptionConfig is enabled, control-plane roles keep unrestricted data actions so that a KMS plugin can call KMS directly. (#18671)

  • kOps now sets containerd configuration version 4 when it installs containerd 2.3 or newer. Flatcar and ContainerOS continue to use version 3 because they use the operating system's containerd package. Check custom containerd configuration when you upgrade:

    • containerd ignores the top-level grpc, ttrpc, metrics, and debug sections in spec.containerd.configAdditions. Set these values under the plugins."io.containerd.server.v1.grpc", plugins."io.containerd.server.v1.ttrpc", plugins."io.containerd.server.v1.metrics", and plugins."io.containerd.server.v1.debug" plugins instead.
    • As with configuration version 3, containerd does not migrate entries under the legacy plugins."io.containerd.grpc.v1.cri" plugin. Use the plugins."io.containerd.cri.v1.runtime" and plugins."io.containerd.cri.v1.images" paths for these entries.
    • kOps does not change spec.containerd.configOverride. The same rules apply to an override that declares version = 4.
    • containerd versions 2.3.0 through 2.3.4 do not start if a drop-in configuration has a higher version than the root configuration. The NVIDIA container toolkit can write such a drop-in configuration. With these versions, an override must declare version = 4 if a drop-in configuration declares version 4.
    • If spec.containerd.packages overrides the package URL, spec.containerd.version must match the packaged binary, because kOps uses this field to select the configuration version. (#18728)

  • kOps releases no longer include the protokube binary, which only gossip DNS used, the kops-utils-cp image, which copied the etcd binaries, or the uncompressed nodeup binary. Releases publish nodeup.xz and its SHA-256 checksum instead. kops get assets --copy copies the current set of files. If you copy release assets to a private repository in another way, update the list of files. The bootstrap script requires xz and curl, so custom node images must provide both. All supported distributions already provide them. (#18653, #18688, #18737, #18765)

  • On DigitalOcean, Hetzner, and Scaleway, control-plane user data no longer contains cloud API credentials (DIGITALOCEAN_ACCESS_TOKEN, HCLOUD_TOKEN, and the SCW_* variables). kOps supplies credentials to applicable components through generated manifests. Scripts in additionalUserData and units in spec.hooks that use these credentials must now set them. (#18654)

  • kops update cluster --target=terraform now rejects --instance-group and --instance-group-roles, because Terraform treats omitted resources as deletions. Generate the full configuration, and limit the apply with terraform apply -target=<resource address> instead. (#18829)

  • On Azure, the Terraform target now requires version 5.0.0 or newer of the azurerm provider. After you generate the Terraform configuration with kOps 1.37, run terraform init -upgrade to install it. The first plan replaces the azurerm_storage_blob resources, because the provider now identifies their storage container by ID. (#18636)

  • kops toolbox template no longer resolves template expressions to methods on rendered values. This restriction also applies to values from template functions. Rewrite these expressions to use struct fields, map keys, or registered template functions. For example, replace {{ now.Unix }} with {{ now | unixEpoch }}. (#18695)

Known Issues

  • None at this time

Deprecations

  • Support for Kubernetes version 1.31, AWS Classic Load Balancer (CLB) for the API, gossip DNS, and Kopeio networking is removed in kOps 1.37.

  • Support for Kubernetes version 1.32 is deprecated and will be removed in kOps 1.38. Upgrade clusters to Kubernetes 1.33 or newer before an upgrade to kOps 1.38.

  • Support for containerd 2.1, which is end of life upstream, is deprecated and will be removed in kOps 1.38. Remove the pinned spec.containerd.version to use the default version, or pin version 2.3.4 or newer.

  • Support for external-dns on OpenStack is deprecated and will be removed in kOps 1.38, because of security concerns. OpenStack clusters that set spec.externalDNS.provider (spec.externalDns.provider) to external-dns use external-dns v0.16.1, the last release that has the OpenStack Designate provider. This release is built with Go 1.24.1 and does not have the fix for CVE-2026-111943. Use dns-controller instead. kOps does not remove external-dns when you change the provider, so delete the external-dns Deployment in the kube-system namespace after the change. (#18998)

  • The Spotinst integration is deprecated. kOps 1.38 will block the creation of new Spotinst clusters, and will continue to support the deletion of existing clusters. kOps 1.39 will remove the integration. If you use the integration, comment on the issue with your kOps version and the Spot features that you use. (#18717)

  • Cloud storage ACL support is deprecated. kOps will remove it in stages through 1.39. This deprecation includes KOPS_STATE_S3_ACL, GCS object ACL writes, the GoogleCloudBucketAcl feature flag, and the discovery-store public-read fallback. kOps does not change existing ACL grants, so grant the same access through bucket policies or IAM before you disable ACLs. For AWS, use S3 Object Ownership. For GCS, use uniform bucket-level access. (#18716)