Kubernetes 应用编排、管理与运维

一、kubectl 运维命令

kubectl controls the Kubernetes cluster manager.

Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/

Basic Commands (Beginner):
  create        Create a resource from a file or from stdin.
  expose        使用 replication controller, service, deployment 或者 pod 并暴露它作为一个 新的 Kubernetes Service
  run           在集群中运行一个指定的镜像
  set           为 objects 设置一个指定的特征

Basic Commands (Intermediate):
  explain       查看资源的文档
  get           显示一个或更多 resources
  edit          在服务器上编辑一个资源
  delete        Delete resources by filenames, stdin, resources and names, or by resources and label selector

Deploy Commands:
  rollout       Manage the rollout of a resource
  scale         Set a new size for a Deployment, ReplicaSet or Replication Controller
  autoscale     自动调整一个 Deployment, ReplicaSet, 或者 ReplicationController 的副本数量

Cluster Management Commands:
  certificate   修改 certificate 资源.
  cluster-info  显示集群信息
  top           Display Resource (CPU/Memory/Storage) usage.
  cordon        标记 node 为 unschedulable
  uncordon      标记 node 为 schedulable
  drain         Drain node in preparation for maintenance
  taint         更新一个或者多个 node 上的 taints

Troubleshooting and Debugging Commands:
  describe      显示一个指定 resource 或者 group 的 resources 详情
  logs          输出容器在 pod 中的日志
  attach        Attach 到一个运行中的 container
  exec          在一个 container 中执行一个命令
  port-forward  Forward one or more local ports to a pod
  proxy         运行一个 proxy 到 Kubernetes API server
  cp            复制 files 和 directories 到 containers 和从容器中复制 files 和 directories.
  auth          Inspect authorization

Advanced Commands:
  diff          Diff live version against would-be applied version
  apply         通过文件名或标准输入流(stdin)对资源进行配置
  patch         使用 strategic merge patch 更新一个资源的 field(s)
  replace       通过 filename 或者 stdin替换一个资源
  wait          Experimental: Wait for a specific condition on one or many resources.
  convert       在不同的 API versions 转换配置文件
  kustomize     Build a kustomization target from a directory or a remote url.

Settings Commands:
  label         更新在这个资源上的 labels
  annotate      更新一个资源的注解
  completion    Output shell completion code for the specified shell (bash or zsh)

Other Commands:
  alpha         Commands for features in alpha
  api-resources Print the supported API resources on the server
  api-versions  Print the supported API versions on the server, in the form of "group/version"
  config        修改 kubeconfig 文件
  plugin        Provides utilities for interacting with plugins.
  version       输出 client 和 server 的版本信息

Usage:
  kubectl [flags] [options]

Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).


二、kubectl 命令详解：

#查看

#查看所有对象
kubectl get all
#查看namespace
kubectl get namespace
# 查看 node
kubectl get nodes
kubectl get node <node-name> -o yaml  ##查看node的 yaml 文件
kubectl get node --show-labels        ##查看node的标签
#查看所有namespace的pods运行情况
kubectl get pods --all-namespaces
查看Pod所有标签
kubectl get pods --show-labels
查看标签env=dev的Pods
kubectl get pods --show-labels -l env=dev
查看标签env=dev并tie=front的Pods
kubectl get pods --show-labels -l env=dev,tie=front
查看标签env=test,dev的Pods
kubectl get pods --show-labels -l 'env in (test,dev)'
根据标签查看pods
kubectl get pods -l app=nginx
#查看具体pods，记得后边跟namespace名字哦
kubectl get pods  kubernetes-dashboard-76479d66bb-nj8wr --namespace=kube-system
# 查看pods具体信息
kubectl get pods -o wide kubernetes-dashboard-76479d66bb-nj8wr --namespace=kube-system
# 查看 pod 的 yaml 信息
#kubectl get pod kubernetes-dashboard-latest-3665071062-b5k84 --namespace=kube-system  -o yaml
# 查看 pod 动态
kubectl get pod --watch  ## kubectl get pod -w
# 查看 pod 调度信息
kubectl get pod -o wide
# 查看集群健康状态
kubectl get cs
# 获取所有deployment
kubectl get deployment --all-namespaces
# 查看具体的 deployment
kubectl get deployment nginx-app
# 查看 replicaset
kubectl get replicasets   ## kubectl get rs
# 查看rc和servers
kubectl get rc,svc
kubectl get replicaset  ## kubectl get rc
kubectl get service     ## kubectl get svc
# 查看DaemonSet
Kubectl get ds   
kubectl get daemonsets
# 查看pods结构信息（重点，通过这个看日志分析错误）
# 对控制器和服务，node同样有效
kubectl describe pods xxxxpodsname --namespace=xxxnamespace
# 查看pod日志
kubectl logs $POD_NAME
# 查看pod变量
kubectl exec my-nginx-5j8ok -- printenv | grep SERVICE
# 查看deployment动态
kubectl get --watch deployments
# 查看 statefulset
kubectl get sts ## kubectl get statefulsets
# 查看endpoint
kubectl get endpoints
kubectl get endpoints --all-namespaces
# 查看job,cronjobs
kubectl get jobs
kubectl describe job tj-org-import-job-1596090600
kubectl get cronjobs
kubectl get cronjobs tj-org-import-job -o yaml
# 查看pvc,pv
kubectl get pvc
kubectl get pv
# 查看 serviceaccout
kubectl get serviceaccout
kubectl get serviceaccout default
kubectl get serviceaccout default -o yaml
kubectl get serviceaccount --all-namespace  ## kubectl get sa --all-namespace
# 查看 secret
kubectl get secret
kubectl get secret default-token-xxxx -o yaml
echo -n "xxxxx" | base64 --decode  ## base64解码
# 查看 role,rolebinding
kubectl get role
kubectl get clusterrole
kubectl get rolebinding
kubectl get clusterrolebinding
# 查看 configmap
kubectl get configmap
# 查看 component status
kubectl get cs  ## kubectl get componentstatus
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                 
scheduler            Healthy   ok                 
etcd-0               Healthy   {"health":"true"}
# 查看资源的文档
kubectl explain pods            ##查看pod提供的完整配置字段
kubectl explain pods.metadata   ##查看pod.metadata提供的完整配置字段
kubectl explain pod.spec        ##查看pod.sepc提供的完整配置字段
kubectl explain pod.spec.containers
kubectl explain pod.spec.containers.livenessProbe


# 集群
# 查看集群健康状态
kubectl get cs
# 集群核心组件运行情况        
kubectl cluster-info   
# 表空间名
kubectl get namespaces 
# Server，Client版本 
kubectl version     
# API 版本    
kubectl api-versions 
# 查看事件   
kubectl get events     
//获取全部节点
kubectl get nodes     
//删除节点
kubectl delete node k8s2 
# 查看变更进度
kubectl rollout status deploy nginx-test

# 创建
# 创建资源
kubectl create -f pod-demo.yaml   ## -f file类型
kubectl create -f ./nginx.yaml    
# 创建当前目录下的所有yaml资源     
kubectl create -f .          
# 使用多个文件创建资源                
kubectl create -f ./nginx1.yaml -f ./mysql2.yaml    
# 使用目录下的所有清单文件来创建资源
kubectl create -f ./dir      
# 使用 url 来创建资源               
kubectl create -f https://git.io/vPieo        
#创建带有终端的pod
kubectl run -i --tty busybox --image=busybox   
# 启动一个 nginx 实例
kubectl run nginx --image=nginx          
#启动多个pod    
kubectl run mybusybox --image=busybox --replicas=5   
# 获取 pod 和 svc 的文档
kubectl explain pods
kubectl explain svc                     

# 更新
# 滚动更新 pod
kubectl rolling-update python-v1 -f python-v2.json           frontend-v1
# 更新资源名称并更新镜像
kubectl rolling-update python-v1 python-v2 --image=image:v2 
# 更新 pod 中的镜像
kubectl rolling-update python --image=image:v2                
# 退出已存在的进行中的滚动更新
kubectl rolling-update python-v1 python-v2 --rollback       
#为 nginx RC 创建服务，启用本地 80 端口连接到容器上的 8000 端口
kubectl expose rc nginx --port=80 --target-port=8000
#更新单容器 pod 的镜像版本（tag）到 v4
kubectl get pod nginx-pod -o yaml | sed 's/\(image: myimage\):.*$/\1:v4/' | kubectl replace -f -
# 添加标签
kubectl label pods nginx-pod new-label=awesome 
# 添加注解                   
kubectl annotate pods nginx-pod icon-url=http://goo.gl/XXBTWq   
升级Deployment
#kubectl set image deployment nginx-deployment nginx=nginx:1.9.1
查询升级状态
#kubectl rollout status deployment nginx-deployment
弹性扩/缩容
#kubectl scale deployment nginx-deployment --replicas=10
# 自动扩展 deployment              
kubectl autoscale deployment nginx-deployment --min=10 --max=15 --cpu-percent=80  #回滚上一个的版本
kubectl rollout undo deployment nginx-deployment 
#回滚到以前制定的版本 
kubectl rollout undo deployment nginx-deployment --to-revision=3 
#查看deployment历史修订版本
kubectl rollout history deployment nginx-deployment  
#查看制定的版本deployment历史修订版本
kubectl rollout history deployment nginx-deployment --reversion=3 
暂停/恢复
#kubectl rollout pause deployment nginx-deployment
#kubectl rollout resume deployment nginx-deployment


# 编辑资源
# 编辑名为 docker-registry 的 service
kubectl edit svc/docker-registry  
# 编辑名为 nginx-deployment的deployment                 
kubectl edit deployment/nginx-deployment            
# 动态伸缩，将foo副本集变成3个
kubectl scale --replicas=3 rs/foo 
# 缩放“foo”中指定的资源。                              
kubectl scale --replicas=3 -f foo.yaml    
# 将deployment/mysql从2个变成3个                      
kubectl scale --current-replicas=2 --replicas=3 deployment/mysql 
# 变更多个控制器的数量
kubectl scale --replicas=5 rc/foo rc/bar rc/baz                  
# 查看变更进度
kubectl rollout status deploy deployment/mysql

kubectl edit configmap [configmapname]
kubectl edit deployment [deploymentname]                        

# 删除
# 删除 pod.json 文件中定义的类型和名称的 pod
kubectl delete -f ./pod.json                                             
# 删除名为“baz”的 pod 和名为“foo”的 service
kubectl delete pod,service baz foo                                       
# 删除具有 name=myLabel 标签的 pod 和 serivce
kubectl delete pods,services -l name=myLabel                             
# 删除 my-ns namespace下的所有 pod 和 serivce，包括尚未初始化的  
kubectl -n my-ns delete po,svc --all
# 强制删除pod
kubectl delete pods prometheus-7fcfcb9f89-qkkf7 --grace-period=0 --force
强制删除rc
#kubectl delete rc nginx-controller --force --cascade=false ##删除RC，不删除Pod，默认是级联删除
强制删除deployment
#kubectl delete deployment kubernetes-dashboard-latest  --namespace=kube-system --force=true --cascade=false
# 删除yaml 文件对应的pod
kubectl delete -f pod-demo.yaml  ## -f file类型

kubectl delete rc --cascade=false  ##删除RC，不删除Pod，默认是级联删除  kubectl delete replicationcontrollers

# 交互
# dump 输出 pod 的日志（stdout）
kubectl logs nginx-pod    
# dump 输出 pod 中容器的日志（stdout，pod 中有多个容器的情况下使用） 
kubectl logs -f <pod-name> -c <container-name>                      
kubectl logs nginx-pod -c my-container    
docker logs -f <docker-name>  
# 流式输出 pod 的日志（stdout）          
kubectl logs -f nginx-pod  
# 流式输出 pod 中容器的日志（stdout，pod 中有多个容器的情况下使用）                          
kubectl logs -f nginx-pod -c my-container    
# 交互式 shell 的方式运行 pod        
kubectl run -i --tty busybox --image=busybox -- sh 
# 连接到运行中的容器
kubectl attach nginx-pod -i  
# 转发 pod 中的 6000 端口到本地的 5000 端口                        
kubectl port-forward nginx-pod 5000:6000  
# 在已存在的容器中执行命令（只有一个容器的情况下）           
kubectl exec nginx-pod -- ls /       
# 在已存在的容器中执行命令（pod 中有多个容器的情况下）                
kubectl exec nginx-pod -c my-container -- ls /    
# 显示指定 pod和容器的指标度量   
kubectl top pod POD_NAME --containers    
进入一个正在运行的Pod
#kubectl exec -it pod-name /bin/bash
进入一个正在运行的包含多个容器的Pod
kubectl exec -it <pod-name> -c <container-name> /bin/sh   ##进入容器
kubectl exec -it pod-name -c container-name /bin/bash   
docker exec -it <container-name> /bin/sh   ##进入容器

[root@k8s-master ~]# kubectl exec -it nginx-deployment-5658f5b9cb-z8dgp -c nginx /bin/bash
[root@k8s-master ~]# kubectl exec -it nginx-deployment-5658f5b9cb-pntsk -c nginx /bin/bash
root@nginx-deployment-5658f5b9cb-z8dgp:/usr/sbin#./nginx -s reload 
root@nginx-deployment-5658f5b9cb-pntsk:/usr/sbin#./nginx -s reload     

# 调度配置
# 标记 my-node 不可调度
$ kubectl cordon k8s-node  
# 清空 my-node 以待维护                                           
$ kubectl drain k8s-node  
# 标记 my-node 可调度                                             
$ kubectl uncordon k8s-node      
# 显示 my-node 的指标度量                                      
$ kubectl top node k8s-node 
# 将当前集群状态输出到 stdout                                           
$ kubectl cluster-info dump     
# 将当前集群状态输出到 /path/to/cluster-state                                      
$ kubectl cluster-info dump --output-directory=/path/to/cluster-state  
#如果该键和影响的污点（taint）已存在，则使用指定的值替换
$ kubectl taint nodes foo dedicated=special-user:NoSchedule

标签
# 通过标签过滤
kubectl get pods -l app  ##有app key的标签
kubectl get pods -l app,release ## 有app,release key的标签2
kubectl get pods -l app=ngnix  ##标签app=nginx
kubectl get pods -l app=nginx,release=stable  ##标签app=nginx且release=stable
kubectl get pods -l app=nginx,release!=stable  ##标签app=nginx且release!=stable
kubectl get pods -l "release in(alpha,beta)"
kubectl get pods -l "release noin(alpha,beta)"
# 添加标签
kubectl label pods pod-demo release=alpha
# 修改标签
kubectl label pods pod-demo release=stable --overwrite
# 显示标签
kubectl get pods pod-demo --show-labels
# 删除标签
kubectl label pod pod-demo release-

提示：其他资源对象同理

kubectl get 资源如下:
 
  * all 
  * certificatesigningrequests (aka 'csr') 
  * clusterrolebindings 
  * clusterroles 
  * componentstatuses (aka 'cs') 
  * configmaps (aka 'cm') 
  * controllerrevisions 
  * cronjobs 
  * customresourcedefinition (aka 'crd') 
  * daemonsets (aka 'ds') 
  * deployments (aka 'deploy') 
  * endpoints (aka 'ep') 
  * events (aka 'ev') 
  * horizontalpodautoscalers (aka 'hpa') 
  * ingresses (aka 'ing') 
  * jobs 
  * limitranges (aka 'limits') 
  * namespaces (aka 'ns') 
  * networkpolicies (aka 'netpol') 
  * nodes (aka 'no') 
  * persistentvolumeclaims (aka 'pvc') 
  * persistentvolumes (aka 'pv') 
  * poddisruptionbudgets (aka 'pdb') 
  * podpreset 
  * pods (aka 'po') 
  * podsecuritypolicies (aka 'psp') 
  * podtemplates 
  * replicasets (aka 'rs') 
  * replicationcontrollers (aka 'rc') 
  * resourcequotas (aka 'quota') 
  * rolebindings 
  * roles 
  * secrets 
  * serviceaccounts (aka 'sa') 
  * services (aka 'svc') 
  * statefulsets (aka 'sts') 
  * storageclasses (aka 'sc')


三、应用编排与管理

配置清单，四个一级清单：

apiVersion: group/version

kind:

metadata:

spec:

创建 Nginx Deployment 和 Service(nginx.yaml)

### define deployment info ###
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
  labels:
    app: nginx
    env: dev
    tie: front
### define pod info ###
spec:
# define the pod count
  replicas: 2
  revisionHistoryLimit: 5  #保存5个历史版本
# define update strategy 
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable; 25%
    type: RollingUpdate   #滚动升级
# select the pod which is controlled
  selector:
    matchLabels:
      app: nginx
      env: dev
      tie: front
# define the pod
  template:
    metadata:
# define the pod label,which must be same with selector's matchLabels
      labels:
        app: nginx
        env: dev
        tie: front
### define container info ###
    spec:
      containers:
# define the container name
      - name: nginx
        image: nginx:1.17.5
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - name: conf
          mountPath: /etc/nginx
        - name: opt
          mountPath: /opt
      ##定义pod反亲和,硬亲和
      affinity:
        podAntiAffinity:  ##podAffinity:  亲和
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpression:
              - key: app
                operator: In
                values:
                - keyin-demo
            topologyKey: kubernetes.io/hostname
      ##定义pod反亲和,软亲和
      affinity:
        podAntiAffinity:    ##podAffinity:  亲和
          preferredDuringSchedulingIgnoredDuringExecution:
  - weight: 100
    podAffinityTerm:
              labelSelector:
                matchExpression:
                - key: app
                  operator: In
                  values:
                  - keyin-demo
              topologyKey: rack     ## rack 是 node 的一个label
      #node label selector
      nodeSelector:
        type: nginx
      volumes:
      - name: conf
        hostPath:
          path: /etc/nginx
          type: Directory
      - name: opt
        hostPath:
          path: /opt
          type: Directory

---

apiVersion: v1
# delcare it's service
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx
    env: dev
    tie: front
spec:
  externalIPs:
  - 192.101.10.80
  - 192.101.10.81
  - 192.101.10.82
  ports:
# define the service's port
  - port: 80
    name: nginx-service-80 
    protocol: TCP
# define the container's port
    targetPort: 80
#    nodePort: 30080
# define the pod label which the selector match with
  selector:
    app: nginx
    env: dev
    tie: front
# three type : ClusterIP,NodePort,LoadBalancer
  type: ClusterIP


创建：
kubectl create -f nginx.yaml