Kubernetes存储与进阶

1 K8s Storage

https://kubernetes.io/docs/concepts/storage/

1.1 Volume

（1）定义一个pod，其中包含两个container

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: v1
kind: Pod
metadata:
  name: volume-pod
spec:
  containers:
  - name: nginx-container
    image: nginx
    ports:
    - containerPort: 80
    volumeMounts:
    - name: volume-pod # 使用哪个volume
      mountPath: /nginx-volume # 对应container中的路径
  - name: busybox-container
    image: busybox
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
    volumeMounts:
    - name: volume-pod # 使用哪个volume
      mountPath: /busybox-volume # 对应container中的路径
  volumes:
  - name: volume-pod # volume名称
    hostPath:
      path: /tmp/volume-pod # volume对应的宿主机路径

（2）根据文件创建Pod并查看运行情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 创建Pod
kubectl apply -f volume-pod.yaml
# 查看Pod运行情况
kubectl get pods -o wide
kubectl describe pod volume-pod
# 来到Pod运行的worker节点，比如w1
docker ps | grep volume-pod
# 查看Pod Volume对应的宿主机路径
ls /tmp/volume-pod
# 进入到两个容器中，查看容器对应的路径
docker exec -it containerid sh
ls /busybox-volume
ls /nginx-volume
# 在容器中创建文件或者在宿主机上创建文件，看看是否同步，发现是同步的，说明Pod里面的Container是共享volume的
# 查看两个container中的hosts文件是否一样，发现也是一样的，说明Pod里面的Container是共享network的
docker exec -it containerid cat /etc/hosts

1.2 PV PVC Pod介绍

1.2.1 PersistentVolume-PV

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistent-volumes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi # 存储空间大小
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce # 只允许一个Pod进行独占式读写操作
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: slow
  mountOptions:
    - hard
    - nfsvers=4.1
  nfs:
    path: /tmp # 远端服务器的目录
    server: 172.17.0.2 # 远端的服务器

1.2.2 PersistentVolumeClaim-PVC

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myclaim
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 8Gi
  storageClassName: slow
  selector:
    matchLabels:
      release: "stable"
    matchExpressions:
      - {key: environment, operator: In, values: [dev]}

1.2.3 Pod使用PVC

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#claims-as-volumes

（1）前提是PV和PVC已经绑定在一起，也就是PV能够满足PVC的要求

（2）同时PV和PVC的storageClassName要一样

（3）PersistentVolumeController会发现两者有关系，然后将PV的名字添加在PVC中，这样就建立了绑定关系

1.3 PV PVC Pod实战

需求：Nginx持久化存储，步骤：1.共享存储使用NFS，比如选择在master节点 。2.创建PV和PV。3.Nginx Pod中使用PVC

NFC简介：NFS(network file system)网络文件系统，是FreeBSD支持的文件系统中的一种，允许网络中的计算机之间通过TCP/IP网络共享资源

  1. Master节点搭建NFS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#安装nfs
yum install -y nfs-utils
# 创建nfs目录
mkdir -p /nfs/data/
mkdir -p /nfs/data/nginx
# 授予权限
chmod -R 777 /nfs/data
# 配置exports文件
: vi /etc/exports
/nfs/data *(rw,no_root_squash,sync)
# 使得配置生效
exportfs -r
# 查看生效
exportfs
# 启动rpcbind、nfs服务
systemctl restart rpcbind && systemctl enable rpcbind
systemctl restart nfs && systemctl enable nfs
# 查看rpc服务的注册情况
rpcinfo -p localhost
# showmount测试
showmount -e master-ip
  1. 定义PV，PVC和Pod文件

nginx-pv-pvc-demo.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# 定义PV
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nginx-pv
spec:
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 2Gi
  nfs:
    path: /nfs/data/nginx
    server: 192.168.0.51 # master节点的ip
---
# 定义PVC，用于绑定PV,自动根据accessModes和storage绑定，多种参数绑定形式。
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nginx-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 2Gi
---
# 定义Nginx Pod，指定需要使用的PVC
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-persistent-storage
          mountPath: /usr/share/nginx/html
      volumes:
        - name: nginx-persistent-storage
          persistentVolumeClaim:
          claimName: nginx-pvc
  1. 创建资源并测试
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 根据yaml文件创建资源：pod、pv、pvc
kubectl apply -f nginx-pv-pvc-demo.yaml
# 查看资源
kubectl get pv,pvc
kubectl get pods -o wide
kubectl describe pod nginx
kubectl describe pv nginx-pv
kubectl describe pvc nginx-pvc
# 在master对应的nfs/data/nginx中新建jack.html
echo "hello jack pv pvc pod." > jack.html
# 进入nginx容器查看/usr/share/nginx/html
kubectl get pods -o wide
docker exec -it containerid sh
ls /usr/share/nginx/html
cat /usr/share/nginx/html/jack.html
# 查看nginx pod ip并访问资源
kubectl get pods -o wide
curl nginx_pod_ip/jack.html
# 删除nginx pod并查看新nginx pod的资源
kubectl delete pod nginx-pod
curl nginx-new-pod-ip/jack.html

手动创建PV是比较麻烦的，可以交给StorageClass操作。

2 Resouce

https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#example-1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
---
apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: app
    image: images.my-company.example/app:v4
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  - name: log-aggregator
    image: images.my-company.example/log-aggregator:v6
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

3 Dashboard

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

（1）创建dashboard.yaml文件，记得把image替换成阿里云的，以及配置成NodePort访问形式.

（2）定义访问dashboard需要的account文件并运行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard

（3）创建访问dashboard需要的token

1
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

（4）访问K8s_ip:30018，输入token

4 Controller进阶

https://kubernetes.io/docs/concepts/workloads/controllers/

4.1 Job

https://kubernetes.io/docs/concepts/workloads/controllers/job/

对于RS，RC之类的控制器，能够保持Pod按照预期数目持久地运行下去，它们针对的是持久性的任 务，比如web服务。而有些操作其实不需要持久，比如压缩文件，希望任务完成之后，Pod就结束运行，不需要保持在系统中，此时就需要用到Job。所以可以这样理解，Job是对RS、RC等持久性控制器的补充。负责批量处理短暂的一次性任务，仅执行一次，并保证处理的一个或者多个Pod成功结束。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: batch/v1
kind: Job
metadata:
  name: job-demo
spec:
  template:
    metadata:
      name: job-demo
    spec:
      restartPolicy: Never
      containers:
      - name: counter
        image: busybox
        command:
        - "bin/sh"
        - "-c"
        - "for i in 9 8 7 6 5 4 3 2 1; do echo $i; done"

4.2 CronJob

https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/

相当于定时任务

4.3 StatefulSet

https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

之前接触的Pod的管理对象比如RC、Deployment、DaemonSet和Job都是面向无状态的服务，但是现实中有很多服务是有状态的，比如MySQL集群、MongoDB集群、ZK集群等。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# 定义Service
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
# 定义StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web

4.4 DaemonSet

https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/

1
2
3
4
# 查看kube-proxy类型
kubectl get pods kube-proxy-9jgb6 -n kube-system -o yaml | grep kind
# 查看calico.yaml文件
# 搜索"DaemonSet"，发现calico-node也是DaemonSet类型
云原生(容器化) > Kubernetes
#Kubernetes #k8s
Kubernetes存储与进阶
http://www.zivjie.cn/2023/04/30/云原生(容器化)/k8s/Kubernetes存储与进阶/
作者
Francis
发布于
2023年4月30日
许可协议