144. 如何在 K3s 上部署 AWS EBS CSI 驱动

Situation 地理位置Task 任务This knowledge base article will provide the directions for deploying and testing the AWS EBS CSI driver and storage class on K3s.本知识库文章将提供如何在 K3s 上部署和测试 AWS EBS CSI 驱动及存储类的指导。Requirements 要求K3s 1.18 (may apply to other versions)K3s 1.18可能适用于其他版本Amazon Web Services (AWS) account with privileges to launch EC2 instances and create IAM policies.拥有启动 EC2 实例和创建 IAM 策略的权限的亚马逊网络服务AWS账户。Background 背景K3s has all in-tree storage providers removed since Kubernetes is shifting to out of tree providers for Container Storage Interface (CSI) and Cloud Provider Interface (CPI). While in-tree providers are convenient, they add a lot of bloat to Kubernetes and will eventually be removed from upstream Kubernetes, possibly in 2021.K3s 移除了所有树内存储提供商因为 Kubernetes 正在转向树外的容器存储接口CSI和云服务提供商接口CPI。虽然树内提供者很方便但它们会给 Kubernetes 带来大量臃肿最终可能会在 2021 年从上游 Kubernetes 中移除。This how-to guide will instruct you on installing and configuring the AWS EBS CSI driver and storage class. This will allow you to dynamically provision and attach an EBS volume to your pod without having to manually create a persistent volume (PV) and EBS volume in advance. In the event that your node crashes and your pod is re-launched on another node, your pod will be reattached to the volume assuming that node is running in the same availability zone used by the defunct node.本指南将指导你如何安装和配置 AWS EBS CSI 驱动及存储类。这将使你能够动态配置并附加 EBS 卷到你的 Pod 上而无需提前手动创建持久卷PV和 EBS 卷。如果你的节点崩溃且你的 Pod 在另一个节点上重新启动假设该节点运行在已停用节点使用的同一可用区你的 Pod 会重新连接到该卷。Solution 解决方案Assuming you want the CSI and storage class automatically deployed by K3s, copy the following YAML to a file in your manifests folder on one or all of your K3s servers. For example,/var/lib/rancher/k3s/server/manifests/aws-ebs-csi.yaml:假设你希望 K3s 自动部署 CSI 和存储类将以下 YAML 复制到你 K3s 服务器上 manifests 文件夹中的文件。例如/var/lib/rancher/k3s/server/manifests/aws-ebs-csi.yamlspan stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-yamlapiVersion: a>First, note at the time of this writing, v0.5.0 is the latest version of the driver. If there is a newer version available, you can replace this in the chart and version tags. See the AWS EBS CSI readme for documentation on the versions currently available. Second, you can customize theenableVolumeScheduling,enableVolumeResizing,enableVolumeSnaphost, andextraVolumeTagsbased on your needs. These parameters and others are documented in the Helm chart.首先撰写本文时v0.5.0 是驱动的最新版本。如果有更新版本可以在图表和版本标签中替换该版本。请参阅 AWS EBS CSI 说明文件 了解当前可用版本的文档。其次你可以根据需求自定义enableVolumeScheduling、enableVolumeResizing、enableVolumeSnaphost和extraVolumeTags。这些参数及其他参数均有记录在 Helm 图表中。Next, you need to give the driver IAM permissions to manage EBS volumes. This can be done one of two ways. You can either feed your AWS access key and secret key as a Kubernetes secret, or use an AWS instance profile. Since the first option involves passing sensitive keys in clear text and storing them directly in Kubernetes, the second option is usually preferred. I will go over both options. For either option, make sure your access keys or instance profile has the following permissions set in IAM:接下来你需要赋予驱动 IAM 权限来管理 EBS 卷。这有两种方式。你可以将 AWS 访问密钥和秘密密钥作为 Kubernetes 秘密输入或者使用 AWS 实例配置文件。由于第一种方式是通过明文传递敏感密钥并直接存储在 Kubernetes 中第二种通常更为推荐。我会介绍这两种选项。无论哪种方式确保你的访问密钥或实例配置文件在 IAM 中设置了以下权限span stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-json{ Version: 2012-10-17, Statement: [ { Effect: Allow, Action: [ ec2:AttachVolume, ec2:CreateSnapshot, ec2:CreateTags, ec2:CreateVolume, ec2:DeleteSnapshot, ec2:DeleteTags, ec2:DeleteVolume, ec2:DescribeAvailabilityZones, ec2:DescribeInstances, ec2:DescribeSnapshots, ec2:DescribeTags, ec2:DescribeVolumes, ec2:DescribeVolumesModifications, ec2:DetachVolume, ec2:ModifyVolume ], Resource: * } ] }/code/span/spanReference: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/example-iam-policy.json参考资料https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/example-iam-policy.jsonOption 1: Kubernetes Secret选项 1Kubernetes SecretYou can place your AWS access key and secret key into a Kubernetes secret. Create a YAML file with the following contents and run a kubectl apply. You can also place this inside your/var/lib/rancher/k3s/server/manifests/aws-ebs-csi.yamlfile. Keep in mind this is not a terribly secure option and anyone with access to these files or secrets in the kube-system namespace will be able to obtain your AWS access keys.你可以把 AWS 访问密钥和秘密密钥放进 Kubernetes 密钥里。创建一个包含以下内容的 YAML 文件运行 kubectl 应用。你也可以把它放在/var/lib/rancher/k3s/server/manifests/aws-ebs-csi.yaml文件里。请记住这并不是一个很安全的选项任何在 kube 系统命名空间中访问这些文件或秘密的人都能获得你的 AWS 访问密钥。span stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-yamlapiVersion: v1 kind: Secret metadata: name: aws-secret namespace: kube-system stringData: key_id: AKI********** access_key: **********/code/span/spanOption 2: Instance Profile选项2实例配置文件This option to more secure and should not expose your keys in clear text or in a Kubernetes secret object. Youll need to make sure when your EC2 instances are launched, youve attached an instance profile that has the permissions defined above in the JSON block.这个选项更安全不应该以明文或 Kubernetes 秘密对象的形式暴露你的密钥。你需要确保在 EC2 实例启动时附加了一个实例配置文件 里面有上述 JSON 模块中定义的权限。Verifying and Testing 验证与测试You can now check your pods to see if the CSI pods are running. You should see something like this:你现在可以检查你的播客看看 CSI 播客是否在运行。你应该会看到类似这样的内容span stylecolor:#000000span stylebackground-color:#ffffffcode# kubectl get pods -n kube-system | grep ebs ebs-snapshot-controller-0 1/1 Running 0 15m ebs-csi-node-k2gh5 3/3 Running 0 15m ebs-csi-node-xdcvn 3/3 Running 0 15m ebs-csi-controller-6f799b5548-46jqr 6/6 Running 0 15m ebs-csi-controller-6f799b5548-h4nbb 6/6 Running 0 15m/code/span/spanTime to test things out. The following command can be run that should provision a 1GB EBS and attach it to your pod:是时候测试一下了。可以执行以下命令应该能配置一个 1GB 的 EBS并将其连接到你的 Pod 上span stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-yamlcat EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: myclaim spec: accessModes: - ReadWriteOnce storageClassName: ebs-storageclass resources: requests: storage: 1Gi --- apiVersion: v1 kind: Pod metadata: name: storage-test spec: containers: - name: storage-test image: ubuntu:latest command: [/bin/sleep] args: [infinity] volumeMounts: - name: myebs mountPath: /mnt/test volumes: - name: myebs persistentVolumeClaim: claimName: myclaim EOF/code/span/spanIn your AWS console, you should see a new EBS volume has been created. After about a minute, you should be able to exec into your pod and see the volume mounted in your pod:在你的 AWS 控制台中你应该会看到一个新的 EBS 卷被创建。大约一分钟后你应该能执行进入你的 Pod看到 pod 上挂载的卷span stylecolor:#000000span stylebackground-color:#ffffffcode# kubectl exec storage-test -- df -h Filesystem Size Used Avail Use% Mounted on overlay 31G 6.2G 25G 20% / tmpfs 64M 0 64M 0% /dev tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/nvme2n1 976M 2.6M 958M 1% /mnt/test /dev/root 31G 6.2G 25G 20% /etc/hosts shm 64M 0 64M 0% /dev/shm tmpfs 3.8G 12K 3.8G 1% /run/secrets/a>Cleaning Up 清理Remove the test pod by running the following:通过以下操作拆卸测试舱span stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-bashkubectl delete pod storage-test/code/span/spanRemove the PVC by running:通过以下方式去除 PVCspan stylecolor:#000000span stylebackground-color:#ffffffcode classlanguage-bashkubectl delete pvc myclaim/code/span/spanCheck the AWS console and you should see your EBS volume has been removed automatically by the AWS EBS CSI driver.查看 AWS 控制台你应该会看到你的 EBS 卷已被 AWS EBS CSI 驱动自动移除。Reference 参考文献K3s documentation K3s 文档AWS EBS CSI documentationAWS EBS CSI 文档访问Rancher-K8S解决方案博主企业合作伙伴 https://blog.csdn.net/lidw2009