Release notes for kops 1.17 series ¶
Significant changes ¶
-
To address the issue of IPv4 only clusters being susceptible to MitM attacks via IPv6 rogue router advertisements, the affected components have been upgraded as follows:
- Docker version 19.03.11 - CVE-2020-13401
- CNI plugins 0.8.6 - CVE-2020-10749
- Calico 3.13.4 - CVE-2020-13597
- Weave Net 2.6.5 - CVE-2020-11091
-
The default instance type for AWS has been changed to t3.medium. This should provide better performance and reduced costs in clusters where the average CPU usage is low.
-
Support for Ubuntu 20.04 (Focal) has been added.
Breaking changes ¶
-
Support for Docker versions 1.11, 1.12 and 1.13 has been removed because of the dockerproject.org shut down. Those affected must upgrade to a newer Docker version.
-
Terraform users on AWS may need to rename some resources in their state file in order to prepare for future Terraform 0.12 support. See Required Actions below.
-
Please see the notes in the 1.15 release about the apiGroup changing from kops to kops.k8s.io
-
Since 1.16, a controller is now used to apply labels to nodes. If you are not using AWS, GCE or OpenStack your (non-master) nodes may not have labels applied correctly.
Required Actions ¶
- Terraform users on AWS may need to rename resources in their terraform state file in order to prepare for future Terraform 0.12 support. Terraform 0.12 no longer supports resource names starting with digits. In kOps, both the default route and additional VPC CIDR associations are affected. See #7957 for more information.
- The default route was named
aws_route.0-0-0-0--0
and will now be namedaws_route.route-0-0-0-0--0
. - Additional CIDR blocks associated with a VPC were similarly named the hyphenated CIDR block with two hyphens for the
/
, for exampleaws_vpc_ipv4_cidr_block_association.10-1-0-0--16
. These will now be prefixed withcidr-
, for exampleaws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16
.
To prevent downtime, follow these steps with the new version of kOps:
kops update cluster --target terraform ...
terraform plan
# Observe any aws_route or aws_vpc_ipv4_cidr_block_association resources being destroyed and recreated
# Run these commands as necessary. The exact names may differ; use what is outputted by terraform plan
terraform state mv aws_route.0-0-0-0--0 aws_route.route-0-0-0-0--0
terraform state mv aws_vpc_ipv4_cidr_block_association.10-1-0-0--16 aws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16
terraform plan
# Ensure these resources are no longer being destroyed and recreated
terraform apply
- Kubernetes 1.9 users will need to enable the PodPriority feature gate. This is required for newer versions of kOps.
To enable the Pod priority feature, follow these steps:
kops edit cluster
# Add the following section
spec:
kubelet:
featureGates:
PodPriority: "true"
- If either a kOps 1.17 alpha release or a custom kOps build was used on a cluster,
a kops-controller Deployment may have been created that should get deleted because it has been replaced with a DaemonSet.
Run
kubectl -n kube-system delete deployment kops-controller
after upgrading to kOps 1.17.0-alpha.2 or later.
Deprecations ¶
-
Support for Kubernetes releases prior to 1.9 is deprecated and will be removed in kops 1.18.
-
The
kops/v1alpha1
API is deprecated and will be removed in kops 1.18. Users ofkops replace
will need to supply v1alpha2 resources. -
Support for Ubuntu 16.04 (Xenial) has been deprecated and will be removed in future versions of kOps.
-
Support for Debian 8 (Jessie) has been deprecated and will be removed in future versions of kOps.
-
Support for CoreOS has been deprecated and will be removed in future versions of kOps. Those affected should consider using Flatcar as a replacement.
-
Support for the "Legacy" etcd provider has been deprecated. It will not be supported for Kubernetes 1.18 or later. To migrate to the default "Manager" etcd provider see the etcd migration documentation.
-
The default StorageClass
gp2
prior to kOps 1.17.0 is no longer the default, replaced by StorageClasskops-ssd-1-17
.
Known Issues ¶
-
kOps 1.17.0-beta.1 included an update for AWS IAM Authenticator to 0.5.0. This version fails to use the volume mounted ConfigMap causing API authentication issues for clients with aws-iam-authenticator credentials. Any cluster with
spec.authentication.aws
defined according to the docs without overriding thespec.authentication.aws.image
is affected. The workaround is to specify the old 0.4.0 image withspec.authentication.aws.image=602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.4.0
. For the 1.17.0 release, this change was rolled back, and the AWS IAM authenticator defaults to version 0.4.0 -
kOps 1.17.0 includes a new StorageClass
kops-ssd-1-17
which is set as the default via the annotation"storageclass.beta.kubernetes.io/is-default-class":"true"
. If you have modified the previousgp2
StorageClass, it could conflict with the defaulting behavior. To resolve, patch thegp2
StorageClass to have the annotation"storageclass.beta.kubernetes.io/is-default-class":"false"
, which aligns with a patch to kOps 1.17.1 as well.kubectl patch storageclass.storage.k8s.io/gp2 --patch '{"metadata": {"annotations": {"storageclass.beta.kubernetes.io/is-default-class": "false"}}}'
Full change list since 1.16.0 release ¶
1.16.0-alpha.1 to 1.17.0-alpha.1 ¶
- Add release notes for 1.16.0-alpha.1 @justinsb #7896
- stable channel: promote kubernetes 1.13.12, 1.14.8 etc @justinsb #7891
- Don't update first node in instancegroup if cluster fails validation @johngmyers,@justinsb #7872
- add missing priorityClassName to flannel DaemonSet @EladDolev #7842
- fix broken links @dj80hd #7901
- Fix rendering of the Node Authorizer template @KashifSaadat #7916
- Fix fork bomb in Makefile @johngmyers #7935
- Unhide docs make logging @mikesplain #7936
- Upgrade AWS VPC CNI to 1.5.5 @rifelpet #7938
- Correct spelling mistakes @yuxiaobo96 #7922
- Fix flannel CNI version to use 0.2.0 @srikiz #7924
- Update vendoring documentation for go modules @rifelpet #7937
- Remove duplication and update release details @mikesplain #7939
- Updated documentation on how to move from single to multi master @mccare #7439
- Create PodDisruptionBudget for kube-dns in kube-system namespace @hakman #7856
- Add support for newer Docker versions @hakman #7860
- Machine types updates @mikesplain #7947
- fix 404 urls in docs @tanjunchen #7943
- Fix generation of documentation /sitemap.xml file @aledbf #7949
- kOps site link @mikesplain #7950
- Fix netlify mixed content @mikesplain #7953
- Fix goimports errors @rifelpet #7955
- Upate Lyft CNI to v0.5.1 @maruina #7402
- Add relnotes for 1.16.0-alpha.2 @justinsb #7962
- Bump version of alpha @mikesplain #7963
- Add relnotes for 1.15.0 @justinsb #7964
- Update feature flag documentation @rifelpet #7969
- Bazel upgrade @mikesplain #7933
- Upgrade AWS SDK @rifelpet #7972
- Fix panic when ssh key not exists on digitalocean @prksu #7941
- Upgrade go to 1.13 @rifelpet #7973
- Use correct values for CALICO_IPV4POOL_IPIP in Calico v3 @zacblazic #7899
- Put kubernetes 1.17.0-beta.2 into channels @justinsb #7982
- Update compatibility matrix @mikesplain #7984
- Promote peter & ryan & zetaab to approvers @justinsb #7983
- upgrade the time api @tanjunchen #7910
- sysctls.go: Fix some comments @eest #7923
- Ignore devcontainer for vscode remote-containers @granular-ryanbonham #7987
- Dont run travis with Go 1.11 @rifelpet #7988
- Change doc cross-references from absolute to relative links @johngmyers #7907
- Correct link error:404 @yuxiaobo96 #7954
- Update apiVersion in docs and tests @johngmyers #7906
- [aws-iam-authenticator] Docs - Steps to disable DaemonSet Temporarily @bhegazy #7926
- Add indent template function and use it to fix KubeDNS.ExternalCoreFile rendering @rochacon #7979
- fix golint failures @FayerZhang #7894
- [Issue-7870] kops controller support for digital ocean @srikiz #7961
- cleanup whitespace in root.go @joshbranham #7997
- Run goimports from locked version in go.mod @justinsb #7998
- Remove note about 1.15 not being released @kaspernissen #8000
- fix(openstack): fix additional security groups on instance groups @mitch000001 #8004
- DOCS: fix simple typo in readme @lpmi-13 #8005
- Spotinst: Upgrade the Spotinst SDK to version 1.36 @liranp #8003
- Release 1.17.0-alpha.1 @justinsb #7985
1.17.0-alpha.1 to 1.17.0-alpha.2 ¶
- Fix mounting Calico "flexvol-driver-host" in CoreOS @hakman #8062
- Cherry-pick #8074 to release-1.17 @johngmyers #8084
- Bump cilium version to 1.6.4 @olemarkus #8022
- Complete support for Flatcar @mazzy89 #7545
- Canal v3.10 manifest for k8s v1.15+ @KashifSaadat,@hakman #7917
- Cherry pick #8095 @zetaab #8096
- test validateCluster twice to make sure it does not flap @zetaab,@johngmyers #8088
- Add inf1 isntances @mikesplain #8128
- Add CapacityOptimized to list of supported spot allocation strategies @gjtempleton #7406
- Update Calico to v3.10.2 @hakman #8104
- Openstack: Fix cluster floating ips @mitch000001 #8115
- cilium: don't try to mount sys/fs/bpf if already mounted @justinsb #7832
- Update copyrights for 2020 @johngmyers #8241
- Fix protokube osx build @mikesplain #8263
- Set CLUSTER_NAME env var on amazon-vpc-cni pods @rifelpet #8274
- Add deprecation warning for older k8s versions @rifelpet #8176
- Remove kops-controller deployment @rifelpet #8273
- Don't output empty sections in the manifests @justinsb #8317
- Cloud controller template function @DavidSie #7992
- Configuration to specify no SSH key @austinmoore- #7096
- tests: increase timeout in rolling update tests @justinsb #8139
- Fix crossbuild-nodeup-in-docker @johngmyers #8343
- update gophercloud dependency @zetaab #8347
- Update Terraform resource names to be 0.12 compatible. @rifelpet #7957
- Allow local filesystem state stores (to aid CI pull-request workflows) @ari-becker,@rifelpet #6465
- Fix issues with older versions of k8s for basic clusters @hakman #8248
- Use IAMPrefix() for hostedzone @lazzarello #8366
- Fix scheduler policy configmap args @vvbogdanov87 #8386
- Add Cilium.EnablePolicy back into templates @olemarkus #8379
- Bump etcd-manager to 3.0.20200116 (#8310) @mmerrill3 #8399
- CoreDNS default image bump to 1.6.6 to resolve CVE @gjtempleton #8333
- Don't load nonexistent calico-client cert when CNI is Cilium @johngmyers #8338
- kOps releases - prefix git tags with v @rifelpet #8373
- EBS Root Volume Termination @tioxy #7865
- Alicloud: etcd-manager support @bittopaz #8016
1.17.0-alpha.2 to 1.17.0-alpha.3 ¶
- Add missing priorityClassName for critical pods @johngmyers #8200
- Alicloud: allow use RAM role for OSS client @bittopaz #8025
- Update coredns to 1.6.7 @maruina #8452
- Fix Github download url for nodeup @adri,@justinsb #8468
1.17.0-alpha.3 to 1.17.0-alpha.4 ¶
- Cilium - Add missing Identity Allocation Mode to Operator Template @daviddyball #8445
- Revert "Update coredns to 1.6.7" @gjtempleton #8502
- GCS: Don't try to set ACLs if bucket-policy only is set @justinsb #8493
- Make it possible to enable Prometheus metrics for Cilium @olemarkus #8433
- Update cilium to 1.6.6 @olemarkus #8484
1.17.0-alpha.4 to 1.17.0-beta.1 ¶
- Stabilize sequence of "export xx=xxx" statements @bittopaz #8247
- Add events RBAC permissions to kops-controller @rifelpet #8535
- Fix DNS loop on Ubuntu 18.04 (Bionic) @hakman #8353
- Update AWS IAM Authenticator to 0.5.0 @rifelpet #8423
- Update amazon-vpc-cni-k8s to v1.6.0 @hakman #8538
- Update IAM permissions for amazon-vpc-cni-k8s 1.6.0 @rifelpet #8548
- Switch AWS IAM Authenticator to use non-scratch image @rifelpet #8555
- add s3 region @zetaab #8592
- Update coredns to 1.6.7 @maruina #8602
- Cilium fix bpffs check @olemarkus #8599
- Fix periodic e2e test for Ubuntu 16.04 @hakman #8160
- Bump Cilium to 1.7 for k8s 1.12+ @olemarkus #8589
- Pass the cloud object to validator from caller @johngmyers #7925
- Custom sysctl Parameters @ripta #7730
- Automatically install dependencies with local packages @hakman #8020
- Add support for custom env vars in amazon-vpc-cni @rifelpet #8276
1.17.0-beta.1 to 1.17.0-beta.2 ¶
- Update Calico and Canal to v3.12.0 @hakman #8618
- Revert AWS IAM Authenticator Update in release-1.17 @rifelpet #8653
- Update default instance types for AWS @hakman #8282
- Implementing audit dynamic configuration (#7392) @mmerrill3 #7424
- amazon-vpc-routed-eni env variable config @mikesplain #8682
- Fix uploading of file assets @johngmyers #8694
- Create New Default StorageClass: kops-ssd-1-17 @joshbranham #8582
- Set kube-proxy-replacement to partial @olemarkus #8745
- Allow configuration of enable-remote-node-identity @olemarkus #8731
- Make cilium operator health check go against localhost IP @olemarkus #8789
- Tag EBS volumes when using launch templates with AWS API target @johngmyers,@hakman #8462
- Update lyft CNI to 0.6.0 @maruina #8757
- Fix Handling of LaunchTemplate Versions for MixedInstancePolicy @granular-ryanbonham,@KashifSaadat,@qqshfox #8038
- Enable stamping on bazel image builds @rifelpet #8835
- Add support for Docker 19.03.8 in kOps 1.17 @hakman #8845
- Remove support for Docker 1.11, 1.12 and 1.13 @hakman #8855
- Fix kuberouter for k8s 1.16+ @UnderMyBed,@hakman #8697
- Fix tests for obsolete Docker versions in 1.17 @hakman #8889
- Add CloudLabels tags to additional AWS resources @rifelpet #8903
- Canal v3.13.1 @KashifSaadat,@hakman #8795
- Add support for Ubuntu 20.04 (Focal) @hakman #8727
- Fix CloudFormation template tags for NatGateway @hakman #8051
- Remove irrelevant TODO comment from userdata @rifelpet #8936
- Load the correct certificate before deleting @olemarkus #8945
- Enabling JSON output for Terraform instead of writing the HCL syntax … @mccare #8145
- Use non-experimental version of encryption provider config flag in 1.13+ @zacblazic #7900
- feat(openstack): propagate cloud labels to machines @mitch000001 #8999
- cherry pick of #8967 upstream release 1.17 @michalschott #8979
- Update Calico and Canal to latest patch versions @hakman #8961
- Upgrade AWS SDK @rifelpet,@MoShitrit #8721
- Bump cilium to 1.7.2 @olemarkus #8982
- Back-port well known owner aliases and SSH users to 1.17 @hakman #9035
- Use Ubuntu 18.04 Docker packages for Ubuntu 20.04 setups @hakman #9047
- Update to etcd-manager 3.0.20200429 @justinsb #9016
- Bump cilium to 1.7.3 @olemarkus #9026
- Adding ability to configure resources for weave (#8113) @mmerrill3,@hakman #8216
- Update Weave Net to version 2.6.0 @bboreham,@hakman #7898
1.17.0-beta.2 to 1.17.0 ¶
- Allow cluster maintenance when channel is unavailable @johngmyers #9053
- Added support for configuring disable-attach-detach-reconcile-sync in… @andersosthus,@hakman #9068
- Revert "Automated cherry pick of #8999: feat(openstack): propagate cloud labels to machines" @zetaab #9088
- manual cherry-pick #8994 into 1.17. @michalschott #9090
- Disable TX checksum offload for Flannel VXLAN @hakman #9074
- Use Ubuntu 18.04 Docker 19.03.8 packages for Ubuntu 20.04 @hakman #9085
- kube-apiserver: healthcheck via sidecar container @justinsb #9069
- Reduce the number of TravisCI jobs for release branch @hakman #9080
- EnsureFileOwner: cleanup logic @justinsb #8576
- kube-apiserver-healthcheck: actually enable on 1.17 @justinsb #9095
- Fix zsh completion @olemarkus #9108
- Add EC2 Instance LifeCycle label @atmosx #9121
- Upgrade amazon vpc cni to 1.6.1 @rifelpet #9020
- Cherrypick 8402 release 1.17 @rdrgmnzs #9161
- Revert "Add EC2 instance lifecycle label to nodes" @johngmyers #9162
- [Issue-7956] - [Digital Ocean] Minor fix to have proper indexing for digital ocean regions @srikiz #8002
- [DigitalOcean] Add load balancer support for master HA @srikiz #8237
- Use systemd-timesyncd for Ubuntu 20.04 @hakman #9182
- Update etcd-manager to 3.0.20200527 @justinsb #9184
- Update DigitalOcean cloud-controller-manager to v0.1.24 @timoreimann #9179
- Use debian as default image for DO images @srikiz #9181
- Refactor: Add Region() method to fi.Cloud @justinsb,@srikiz #8180
- Remove all versions of a file form the S3 bucket @hakman #9171
- Allow listing versions for objects in the S3 bucket @hakman #9205
1.17.0 to 1.17.1 ¶
- Update etcd-manager to 3.0.20200531 @hakman #9237
- [Digital Ocean] Update RBAC for DO CCM @srikiz #9249
- Update Calico and Canal for CVE-2020-13597 @hakman #9268
- Update Weave for CVE-2020-13597 @hakman #9285
- Use CNI 0.8.6 for Kubernetes 1.15+ @hakman #9256
- Use Docker 19.03.11 for Kubernetes 1.17+ @hakman #9314
- Update Weave Net to 2.6.5 @hakman #9330
- Update default users for kubeconfig with supported distros @hakman,@KashifSaadat #8798
- Bump cilium to 1.7.5 @olemarkus #9367
- Bug: Explicitly set default StorageClass to support upgrades @joshbranham #9337
- Add support for c5a aws ec2 instance types @coolstang #9386
- Fix: dns-controller: 3999 port address already in use @vgunapati #9404