maosheng
- 浏览: 573435 次
- 性别:
- 来自: 北京
-
文章分类
- 全部博客 (267)
- 随笔 (4)
- Spring (13)
- Java (61)
- HTTP (3)
- Windows (1)
- CI(Continuous Integration) (3)
- Dozer (1)
- Apache (11)
- DB (7)
- Architecture (41)
- Design Patterns (11)
- Test (5)
- Agile (1)
- ORM (3)
- PMP (2)
- ESB (2)
- Maven (5)
- IDE (1)
- Camel (1)
- Webservice (3)
- MySQL (6)
- CentOS (14)
- Linux (19)
- BI (3)
- RPC (2)
- Cluster (9)
- NoSQL (7)
- Oracle (25)
- Loadbalance (7)
- Web (5)
- tomcat (1)
- freemarker (1)
- 制造 (0)
最新评论
-
panamera：
如果设置了连接需要密码，Dynamic Broker-Clus ...
ActiveMQ 集群配置 -
panamera：
请问你的最后一种模式Broker-C节点是不是应该也要修改持久 ...
ActiveMQ 集群配置 -
maosheng：
longshao_feng 写道楼主使用 文件共享 模式的ma ...
ActiveMQ 集群配置 -
longshao_feng：
楼主使用 文件共享 模式的master-slave，produ ...
ActiveMQ 集群配置 -
tanglanwen：
感触很深，必定谨记！
少走弯路的十条忠告
一、概述
kubeadm 是 kubernetes 提供的一个初始化集群的工具，使用起来非常方便，但是它创建的 apiserver、controller-manager 等证书默认只有一年的有效期，同时 kubelet 证书也只有一年有效期，一年之后 kubernetes 将停止服务
Kubernetes 集群根证书：
/etc/kubernetes/pki/ca.crt
/etc/kubernetes/pki/ca.key
......
kubeadm 默认生成的ca证书有效期是10年，其他证书（如etcd证书，apiserver证书）有效期均为1年。
二、准备工作
[root@k8s-master etcd]# tree /etc/kubernetes/
/etc/kubernetes/
├── admin.conf
├── controller-manager.conf
├── kubelet.conf
├── manifests
│ ├── etcd.yaml
│ ├── kube-apiserver.yaml
│ ├── kube-controller-manager.yaml
│ └── kube-scheduler.yaml
├── pki
│ ├── apiserver.crt
│ ├── apiserver-etcd-client.crt
│ ├── apiserver-etcd-client.key
│ ├── apiserver.key
│ ├── apiserver-kubelet-client.crt
│ ├── apiserver-kubelet-client.key
│ ├── ca.crt
│ ├── ca.key
│ ├── etcd
│ │ ├── ca.crt
│ │ ├── ca.key
│ │ ├── healthcheck-client.crt
│ │ ├── healthcheck-client.key
│ │ ├── peer.crt
│ │ ├── peer.key
│ │ ├── server.crt
│ │ └── server.key
│ ├── front-proxy-ca.crt
│ ├── front-proxy-ca.key
│ ├── front-proxy-client.crt
│ ├── front-proxy-client.key
│ ├── sa.key
│ └── sa.pub
└── scheduler.conf
3 directories, 30 files
查看证书:
[root@k8s-master]# cd /etc/kubernetes/pki
[root@k8s-master pki]# openssl x509 -in front-proxy-client.crt -noout -text |grep Not
Not Before: Jul 29 12:07:53 2020 GMT
Not After : Jul 29 12:07:54 2021 GMT
[root@k8s-master pki]# openssl x509 -in apiserver.crt -noout -text |grep Not
Not Before: Jul 29 12:07:52 2020 GMT
Not After : Jul 29 12:07:53 2021 GMT
[root@k8s-master pki]# openssl x509 -in front-proxy-client.crt -noout -text |grep Not
Not Before: Aug 1 08:36:22 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master pki]# openssl x509 -in apiserver.crt -noout -text |grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
............
[root@k8s-master kubernetes]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep Not'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep Not
Not Before: Aug 1 08:36:22 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master kubernetes]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep After'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master pki]# ls -l
总用量 56
-rw-r--r-- 1 root root 1224 7月 31 18:47 apiserver.crt
-rw-r--r-- 1 root root 1090 7月 31 18:47 apiserver-etcd-client.crt
-rw------- 1 root root 1675 7月 31 18:47 apiserver-etcd-client.key
-rw------- 1 root root 1675 7月 31 18:47 apiserver.key
-rw-r--r-- 1 root root 1099 7月 31 18:47 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 18:47 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 8月 1 2019 ca.crt
-rw------- 1 root root 1675 8月 1 2019 ca.key
drwxr-xr-x 2 root root 162 7月 31 18:47 etcd
-rw-r--r-- 1 root root 1038 8月 1 2019 front-proxy-ca.crt
-rw------- 1 root root 1679 8月 1 2019 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 7月 31 18:47 front-proxy-client.crt
-rw------- 1 root root 1675 7月 31 18:47 front-proxy-client.key
-rw------- 1 root root 1679 8月 1 2019 sa.key
-rw------- 1 root root 451 8月 1 2019 sa.pub
[root@k8s-master pki]# openssl x509 -in ca.crt -noout -text |grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 29 08:36:24 2029 GMT
[root@k8s-master pki]# cd etcd/
[root@k8s-master etcd]# ls -l
总用量 32
-rw-r--r-- 1 root root 1017 8月 1 2019 ca.crt
-rw------- 1 root root 1679 8月 1 2019 ca.key
-rw-r--r-- 1 root root 1094 7月 31 18:47 healthcheck-client.crt
-rw------- 1 root root 1675 7月 31 18:47 healthcheck-client.key
-rw-r--r-- 1 root root 1135 7月 31 18:47 peer.crt
-rw------- 1 root root 1679 7月 31 18:47 peer.key
-rw-r--r-- 1 root root 1127 7月 31 18:47 server.crt
-rw------- 1 root root 1675 7月 31 18:47 server.key
[root@k8s-master etcd]# openssl x509 -in ca.crt -noout -text |grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 29 08:36:23 2029 GMT
备份工作（非常重要）:
[root@k8s-master ]# cd /etc/kubernetes
[root@k8s-master kubernetes]# mkdir ./pki_bak
[root@k8s-master kubernetes]# mkdir ./pki_bak/etcd
[root@k8s-master kubernetes]# mkdir ./conf_bak
[root@k8s-master kubernetes]# cp pki/apiserver* ./pki_bak/
[root@k8s-master kubernetes]# cp pki/front-proxy-client.* ./pki_bak/
[root@k8s-master kubernetes]# cp pki/etcd/healthcheck-client.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp pki/etcd/peer.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp pki/etcd/server.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp ./admin.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./kubelet.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./controller-manager.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./scheduler.conf ./conf_bak/
三、证书类别
1、集群根证书：
[root@k8s-master images]# ll /etc/kubernetes/pki/ca*
-rw-r--r-- 1 root root 1025 8月 1 2019 /etc/kubernetes/pki/ca.crt
-rw------- 1 root root 1675 8月 1 2019 /etc/kubernetes/pki/ca.key
2、由此集群根证书签发的证书有：
1）kube-apiserver 组件持有的服务端证书
[root@k8s-master pki]# ll /etc/kubernetes/pki/apiserver.*
-rw-r--r-- 1 root root 1224 7月 31 18:47 /etc/kubernetes/pki/apiserver.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver.key
2）kubelet 组件持有的客户端证书
[root@k8s-master pki]# ll /etc/kubernetes/pki/apiserver-kubelet-client.*
-rw-r--r-- 1 root root 1099 7月 31 18:47 /etc/kubernetes/pki/apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver-kubelet-client.key
提示：kubelet的/var/lib/kubelet/config.yaml配置文件中一般不会明确指定服务端证书，而是只指定 ca 根证书, 让 kubelet 根据本地主机信息自动生成服务端证书并保存到配置的 cert-dir文件夹中。
[root@k8s-master kubelet]# pwd
/var/lib/kubelet
[root@k8s-master kubelet]# cat config.yaml
address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 2m0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 5m0s
cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
imagefs.available: 15%
memory.available: 100Mi
nodefs.available: 10%
nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kind: KubeletConfiguration
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s
3）汇聚层证书
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/front-proxy-ca.*
-rw-r--r-- 1 root root 1038 8月 1 2019 /etc/kubernetes/pki/front-proxy-ca.crt
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/front-proxy-ca.key
由此汇聚层根证书签发的证书有：
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/front-proxy-client.*
-rw-r--r-- 1 root root 1058 7月 31 18:47 /etc/kubernetes/pki/front-proxy-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/front-proxy-client.key
3、etcd集群根证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/ca.*
-rw-r--r-- 1 root root 1017 8月 1 2019 /etc/kubernetes/pki/etcd/ca.crt
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/etcd/ca.key
由此etcd根证书签发的证书有：
etcd server服务端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/server.*
-rw-r--r-- 1 root root 1127 7月 31 18:47 /etc/kubernetes/pki/etcd/server.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/etcd/server.key
etcd 集群中peer节点互相通信使用的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/peer.*
-rw-r--r-- 1 root root 1135 7月 31 18:47 /etc/kubernetes/pki/etcd/peer.crt
-rw------- 1 root root 1679 7月 31 18:47 /etc/kubernetes/pki/etcd/peer.key
pod 中定义 Liveness 探针使用的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/healthcheck-client.*
-rw-r--r-- 1 root root 1094 7月 31 18:47 /etc/kubernetes/pki/etcd/healthcheck-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/etcd/healthcheck-client.key
配置在 kube-apiserver 中用来与 etcd server 做双向认证的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/apiserver-etcd-client.*
-rw-r--r-- 1 root root 1090 7月 31 18:47 /etc/kubernetes/pki/apiserver-etcd-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver-etcd-client.key
4、Serveice Account密钥:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/sa.*
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/sa.key
-rw------- 1 root root 451 8月 1 2019 /etc/kubernetes/pki/sa.pub
Serveice Account密钥对仅提供给 kube-controller-manager 使用. kube-controller-manager 通过 sa.key 对 token 进行签名, master 节点通过公钥 sa.pub 进行签名的验证。
API Server身份验证过程：
API Server的authenticating环节支持多种身份校验方式：client cert、bearer token、static password auth等，这些方式中只要有一种方式通过authenticating（Kubernetes API Server会逐个方式尝试），那么身份校验就会通过。
一旦API Server发现client发起的request使用的是service account token的方式，API Server就会自动采用signed bearer token方式进行身份校验。而request则使用携带的service account token参与验证。该token是API Server在创建service account时用API server启动参数：–service-account-key-file的值签署(sign)生成的。如果–service-account-key-file未传入任何值，那么将默认使用–tls-private-key-file的值，即API Server的私钥（server.key）。
通过authenticating后，API Server将根据Pod username所在的group：system:serviceaccounts和system:serviceaccounts:(NAMESPACE)的权限对其进行authority 和admission control两个环节的处理。在这两个环节中，cluster管理员可以对service account的权限进行细化设置。
kubeadm 创建的集群，kube-proxy、flannel、coreDNS是以 pod 形式运行的，在 pod 中，直接使用 service account 与 kube-apiserver 进行认证，此时就不需要再单独为 kube-proxy 创建证书。
---------------------------------------------kubernetes 1.15 版本 以下方案---------------------------------------------
提示：1.12.1 使用可用
1、查看kubeadm-config配置
提示：不同的master节点使用的kubeadm配置有细微的差异，执行更新证书是，每个master在--config后面使用原来集群创建时，当前master对应的kubeadm配置文件。
[root@k8s-master kubernetes]# kubectl -n kube-system get cm kubeadm-config -oyaml
apiVersion: v1
data:
ClusterConfiguration: |
apiServerExtraArgs:
authorization-mode: Node,RBAC
apiVersion: kubeadm.k8s.io/v1alpha3
auditPolicy:
logDir: /var/log/kubernetes/audit
logMaxAge: 2
path: ""
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: ""
etcd:
local:
dataDir: /var/lib/etcd
image: ""
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.12.1
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
unifiedControlPlaneImage: ""
ClusterStatus: |
apiEndpoints:
k8s-master:
advertiseAddress: 192.101.10.80
bindPort: 6443
apiVersion: kubeadm.k8s.io/v1alpha3
kind: ClusterStatus
kind: ConfigMap
metadata:
creationTimestamp: 2019-08-01T08:36:48Z
name: kubeadm-config
namespace: kube-system
resourceVersion: "174"
selfLink: /api/v1/namespaces/kube-system/configmaps/kubeadm-config
uid: 80959d9d-b437-11e9-8e73-6c92bfa51bf6
2、创建kubeadm-cluster.yaml
# touch kubeadm-cluster.yaml
# vi kubeadm-cluster.yaml
apiServer:
apiServerExtraArgs:
authorization-mode: Node,RBAC
apiVersion: kubeadm.k8s.io/v1alpha3
auditPolicy:
logDir: /var/log/kubernetes/audit
logMaxAge: 2
path: ""
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: ""
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.12.1
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
unifiedControlPlaneImage: ""
3、查看帮助
提示：不同版本的kubeadm对于证书renew的命令有细微的差异，具体情况需要依据已经安装的kubeadm来判断。通过命令行kubeadm alpha --help输出类似如下信息：
[root@k8s-master yaml]# kubeadm alpha --help
Experimental sub-commands not yet fully functional.
Usage:
kubeadm alpha [command]
Available Commands:
phase Invoke subsets of kubeadm functions separately for a manual install.
Flags:
-h, --help help for alpha
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha [command] --help" for more information about a command.
[root@k8s-master yaml]# kubeadm alpha phase certs --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase certs [command]
Aliases:
certs, certificates
Available Commands:
all Generates all PKI assets necessary to establish the control plane
apiserver Generates the certificate for serving the kubernetes API
apiserver-etcd-client Generates the client apiserver uses to access etcd
apiserver-kubelet-client Generates the Client certificate for the API server to connect to kubelet
ca Generates the self-signed kubernetes CA to provision identities for other kuberenets components
etcd-ca Generates the self-signed CA to provision identities for etcd
etcd-healthcheck-client Generates the client certificate for liveness probes to healtcheck etcd
etcd-peer Generates the credentials for etcd nodes to communicate with each other
etcd-server Generates the certificate for serving etcd
front-proxy-ca Generates the self-signed CA to provision identities for front proxy
front-proxy-client Generates the client for the front proxy
renew Renews certificates for a Kubernetes cluster
sa Generates a private key for signing service account tokens along with its public key
Flags:
-h, --help help for certs
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase certs [command] --help" for more information about a command.
[root@k8s-master yaml]# kubeadm alpha phase certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase certs renew [flags]
kubeadm alpha phase certs renew [command]
Available Commands:
all renew all available certificates
apiserver Generates the certificate for serving the kubernetes API
apiserver-etcd-client Generates the client apiserver uses to access etcd
apiserver-kubelet-client Generates the Client certificate for the API server to connect to kubelet
etcd-healthcheck-client Generates the client certificate for liveness probes to healtcheck etcd
etcd-peer Generates the credentials for etcd nodes to communicate with each other
etcd-server Generates the certificate for serving etcd
front-proxy-client Generates the client for the front proxy
Flags:
-h, --help help for renew
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase certs renew [command] --help" for more information about a command.
4、重新生成master各个证书
kubeadm alpha phase certs renew etcd-healthcheck-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew etcd-peer --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew etcd-server --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew front-proxy-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver-etcd-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver-kubelet-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver --config kubeadm-cluster.yaml
5、验证证书有效期更新
[root@k8s-master images]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep After'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep After
Not After : Aug 1 10:20:09 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep After
Not After : Aug 1 10:19:54 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep After
Not After : Aug 1 10:20:02 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep After
Not After : Aug 1 10:20:25 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep After
Not After : Aug 1 10:20:32 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep After
Not After : Aug 1 10:20:39 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep After
Not After : Aug 1 10:20:18 2021 GMT
6、更新各个配置文件
1）查看帮助：
[root@k8s-master manifests]# kubeadm alpha phase kubeconfig --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase kubeconfig [command]
Available Commands:
admin Generates a kubeconfig file for the admin to use and for kubeadm itself
all Generates all kubeconfig files necessary to establish the control plane and the admin kubeconfig file
controller-manager Generates a kubeconfig file for the controller manager to use
kubelet Generates a kubeconfig file for the kubelet to use. Please note that this should be used *only* for bootstrapping purposes
scheduler Generates a kubeconfig file for the scheduler to use
user Outputs a kubeconfig file for an additional user
Flags:
-h, --help help for kubeconfig
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase kubeconfig [command] --help" for more information about a command.
2）备份集群配置文件并重新生成：
[root@k8s-master yaml]# find /etc/kubernetes/ -name '*.conf'|xargs -i mv {}{,bak}
[root@k8s-master yaml]# kubeadm alpha phase kubeconfig all --config kubeadm-cluster.yaml
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
或者分步执行：
kubeadm alpha phase kubeconfig kubelet --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig admin --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig scheduler --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig controller-manager --config kubeadm-cluster.yaml
3）重新配置kubectl权限信息：
mv $HOME/.kube/config $HOME/.kube/config.old
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
7、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
---------------------------------------------kubernetes 1.15 版本 以上方案---------------------------------------------
提示：kubernetes 1.16.3 ；1.18.2 使用可用，1.12.1 使用不可用
第一种方案：
1、查看具体过期时间
kubeadm alpha certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Jul 29, 2021 12:07 UTC 362d no
apiserver Jul 29, 2021 12:07 UTC 362d no
apiserver-etcd-client Jul 29, 2021 12:07 UTC 362d no
apiserver-kubelet-client Jul 29, 2021 12:07 UTC 362d no
controller-manager.conf Jul 29, 2021 12:07 UTC 362d no
etcd-healthcheck-client Jul 29, 2021 12:07 UTC 362d no
etcd-peer Jul 29, 2021 12:07 UTC 362d no
etcd-server Jul 29, 2021 12:07 UTC 362d no
front-proxy-client Jul 29, 2021 12:07 UTC 362d no
scheduler.conf Jul 29, 2021 12:07 UTC 362d no
2、查看帮助
[root@hadoop009 images]# kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha certs renew [flags]
kubeadm alpha certs renew [command]
Available Commands:
admin.conf Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
all Renew all available certificates
apiserver Renew the certificate for serving the Kubernetes API
apiserver-etcd-client Renew the certificate the apiserver uses to access etcd
apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
controller-manager.conf Renew the certificate embedded in the kubeconfig file for the controller manager to use
etcd-healthcheck-client Renew the certificate for liveness probes to healthcheck etcd
etcd-peer Renew the certificate for etcd nodes to communicate with each other
etcd-server Renew the certificate for serving etcd
front-proxy-client Renew the certificate for the front proxy client
scheduler.conf Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:
-h, --help help for renew
Global Flags:
--add-dir-header If true, adds the file directory to the header
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm alpha certs renew [command] --help" for more information about a command.
提示：由help可知，证书更新可针对单个证书更新
3、更新证书
更新所有证书，对证书进行续期,续期一年：
$ kubeadm alpha certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
提示：更新操作需要在所有master节点执行
4、验证续期
# ls -l
总用量 56
-rw-r--r-- 1 root root 1220 7月 31 22:48 apiserver.crt
-rw-r--r-- 1 root root 1090 7月 31 22:48 apiserver-etcd-client.crt
-rw------- 1 root root 1679 7月 31 22:48 apiserver-etcd-client.key
-rw------- 1 root root 1675 7月 31 22:48 apiserver.key
-rw-r--r-- 1 root root 1099 7月 31 22:48 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 22:48 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 7月 29 20:07 ca.crt
-rw------- 1 root root 1679 7月 29 20:07 ca.key
drwxr-xr-x 2 root root 162 7月 29 20:07 etcd
-rw-r--r-- 1 root root 1038 7月 29 20:07 front-proxy-ca.crt
-rw------- 1 root root 1675 7月 29 20:07 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 7月 31 22:48 front-proxy-client.crt
-rw------- 1 root root 1679 7月 31 22:48 front-proxy-client.key
-rw------- 1 root root 1675 7月 29 20:07 sa.key
-rw------- 1 root root 451 7月 29 20:07 sa.pub
# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Aug 01, 2021 07:38 UTC 364d no
apiserver Aug 01, 2021 07:38 UTC 364d ca no
apiserver-etcd-client Aug 01, 2021 07:38 UTC 364d etcd-ca no
apiserver-kubelet-client Aug 01, 2021 07:38 UTC 364d ca no
controller-manager.conf Aug 01, 2021 07:38 UTC 364d no
etcd-healthcheck-client Aug 01, 2021 07:38 UTC 364d etcd-ca no
etcd-peer Aug 01, 2021 07:38 UTC 364d etcd-ca no
etcd-server Aug 01, 2021 07:38 UTC 364d etcd-ca no
front-proxy-client Aug 01, 2021 07:38 UTC 364d front-proxy-ca no
scheduler.conf Aug 01, 2021 07:38 UTC 364d no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Jul 27, 2030 13:01 UTC 9y no
etcd-ca Jul 27, 2030 13:01 UTC 9y no
front-proxy-ca Jul 27, 2030 13:01 UTC 9y no
[root@hadoop010 etcd]# kubectl -n kube-system get cm kubeadm-config -oyaml
apiVersion: v1
data:
ClusterConfiguration: |
apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 192.101.11.162:6443
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.18.2
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
ClusterStatus: |
apiEndpoints:
hadoop010:
advertiseAddress: 192.101.11.162
bindPort: 6443
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterStatus
kind: ConfigMap
metadata:
creationTimestamp: "2020-07-29T13:02:16Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:ClusterConfiguration: {}
f:ClusterStatus: {}
manager: kubeadm
operation: Update
time: "2020-07-29T13:02:16Z"
name: kubeadm-config
namespace: kube-system
resourceVersion: "157"
selfLink: /api/v1/namespaces/kube-system/configmaps/kubeadm-config
uid: 2e049082-fa64-4e2e-ad73-af9fc94a051e
5、启用证书
在每台Master上执行重启kube-apiserver、kube-controller、kube-scheduler、etcd这4个容器，以便使证书生效。
# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
或者
# docker ps | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash
提示：启用操作需要在所有master节点执行。
6、更新.kube下的配置文件
$ mv $HOME/.kube/config $HOME/.kube/config.old
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
7、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期。
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
第二种方案：
1、备份导出kubeadm集群配置
# kubeadm config view > kubeadm-cluster.yaml
apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.16.3
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
2、查看具体过期时间
kubeadm alpha certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Jul 29, 2021 12:07 UTC 362d no
apiserver Jul 29, 2021 12:07 UTC 362d no
apiserver-etcd-client Jul 29, 2021 12:07 UTC 362d no
apiserver-kubelet-client Jul 29, 2021 12:07 UTC 362d no
controller-manager.conf Jul 29, 2021 12:07 UTC 362d no
etcd-healthcheck-client Jul 29, 2021 12:07 UTC 362d no
etcd-peer Jul 29, 2021 12:07 UTC 362d no
etcd-server Jul 29, 2021 12:07 UTC 362d no
front-proxy-client Jul 29, 2021 12:07 UTC 362d no
scheduler.conf Jul 29, 2021 12:07 UTC 362d no
3、查看帮助
[root@hadoop009 images]# kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha certs renew [flags]
kubeadm alpha certs renew [command]
Available Commands:
admin.conf Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
all Renew all available certificates
apiserver Renew the certificate for serving the Kubernetes API
apiserver-etcd-client Renew the certificate the apiserver uses to access etcd
apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
controller-manager.conf Renew the certificate embedded in the kubeconfig file for the controller manager to use
etcd-healthcheck-client Renew the certificate for liveness probes to healthcheck etcd
etcd-peer Renew the certificate for etcd nodes to communicate with each other
etcd-server Renew the certificate for serving etcd
front-proxy-client Renew the certificate for the front proxy client
scheduler.conf Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:
-h, --help help for renew
Global Flags:
--add-dir-header If true, adds the file directory to the header
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm alpha certs renew [command] --help" for more information about a command.
提示：由help可知，证书更新可针对单个证书更新
4、更新证书
# kubeadm alpha certs renew all --config=kubeadm-cluster.yaml #更新所有证书
提示：在保存kubeadm-cluster.yaml文件的目录下执行
提示：更新操作需要在所有master节点执行
5、确认验证
# kubeadm alpha certs check-expiration
6、启用证书
在每台Master上执行重启kube-apiserver、kube-controller、kube-scheduler、etcd这4个容器，以便使证书生效。
# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
或者
# docker ps | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash
提示：启用操作需要在所有master节点执行。
7、更新.kube下的配置文件
$ mv $HOME/.kube/config $HOME/.kube/config.old
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
8、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期。
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
kubeadm 是 kubernetes 提供的一个初始化集群的工具，使用起来非常方便，但是它创建的 apiserver、controller-manager 等证书默认只有一年的有效期，同时 kubelet 证书也只有一年有效期，一年之后 kubernetes 将停止服务
Kubernetes 集群根证书：
/etc/kubernetes/pki/ca.crt
/etc/kubernetes/pki/ca.key
......
kubeadm 默认生成的ca证书有效期是10年，其他证书（如etcd证书，apiserver证书）有效期均为1年。
二、准备工作
[root@k8s-master etcd]# tree /etc/kubernetes/
/etc/kubernetes/
├── admin.conf
├── controller-manager.conf
├── kubelet.conf
├── manifests
│ ├── etcd.yaml
│ ├── kube-apiserver.yaml
│ ├── kube-controller-manager.yaml
│ └── kube-scheduler.yaml
├── pki
│ ├── apiserver.crt
│ ├── apiserver-etcd-client.crt
│ ├── apiserver-etcd-client.key
│ ├── apiserver.key
│ ├── apiserver-kubelet-client.crt
│ ├── apiserver-kubelet-client.key
│ ├── ca.crt
│ ├── ca.key
│ ├── etcd
│ │ ├── ca.crt
│ │ ├── ca.key
│ │ ├── healthcheck-client.crt
│ │ ├── healthcheck-client.key
│ │ ├── peer.crt
│ │ ├── peer.key
│ │ ├── server.crt
│ │ └── server.key
│ ├── front-proxy-ca.crt
│ ├── front-proxy-ca.key
│ ├── front-proxy-client.crt
│ ├── front-proxy-client.key
│ ├── sa.key
│ └── sa.pub
└── scheduler.conf
3 directories, 30 files
查看证书:
[root@k8s-master]# cd /etc/kubernetes/pki
[root@k8s-master pki]# openssl x509 -in front-proxy-client.crt -noout -text |grep Not
Not Before: Jul 29 12:07:53 2020 GMT
Not After : Jul 29 12:07:54 2021 GMT
[root@k8s-master pki]# openssl x509 -in apiserver.crt -noout -text |grep Not
Not Before: Jul 29 12:07:52 2020 GMT
Not After : Jul 29 12:07:53 2021 GMT
[root@k8s-master pki]# openssl x509 -in front-proxy-client.crt -noout -text |grep Not
Not Before: Aug 1 08:36:22 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master pki]# openssl x509 -in apiserver.crt -noout -text |grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
............
[root@k8s-master kubernetes]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep Not'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep Not
Not Before: Aug 1 08:36:22 2019 GMT
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master kubernetes]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep After'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep After
Not After : Jul 31 10:47:03 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep After
Not After : Jul 31 10:47:04 2021 GMT
[root@k8s-master pki]# ls -l
总用量 56
-rw-r--r-- 1 root root 1224 7月 31 18:47 apiserver.crt
-rw-r--r-- 1 root root 1090 7月 31 18:47 apiserver-etcd-client.crt
-rw------- 1 root root 1675 7月 31 18:47 apiserver-etcd-client.key
-rw------- 1 root root 1675 7月 31 18:47 apiserver.key
-rw-r--r-- 1 root root 1099 7月 31 18:47 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 18:47 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 8月 1 2019 ca.crt
-rw------- 1 root root 1675 8月 1 2019 ca.key
drwxr-xr-x 2 root root 162 7月 31 18:47 etcd
-rw-r--r-- 1 root root 1038 8月 1 2019 front-proxy-ca.crt
-rw------- 1 root root 1679 8月 1 2019 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 7月 31 18:47 front-proxy-client.crt
-rw------- 1 root root 1675 7月 31 18:47 front-proxy-client.key
-rw------- 1 root root 1679 8月 1 2019 sa.key
-rw------- 1 root root 451 8月 1 2019 sa.pub
[root@k8s-master pki]# openssl x509 -in ca.crt -noout -text |grep Not
Not Before: Aug 1 08:36:24 2019 GMT
Not After : Jul 29 08:36:24 2029 GMT
[root@k8s-master pki]# cd etcd/
[root@k8s-master etcd]# ls -l
总用量 32
-rw-r--r-- 1 root root 1017 8月 1 2019 ca.crt
-rw------- 1 root root 1679 8月 1 2019 ca.key
-rw-r--r-- 1 root root 1094 7月 31 18:47 healthcheck-client.crt
-rw------- 1 root root 1675 7月 31 18:47 healthcheck-client.key
-rw-r--r-- 1 root root 1135 7月 31 18:47 peer.crt
-rw------- 1 root root 1679 7月 31 18:47 peer.key
-rw-r--r-- 1 root root 1127 7月 31 18:47 server.crt
-rw------- 1 root root 1675 7月 31 18:47 server.key
[root@k8s-master etcd]# openssl x509 -in ca.crt -noout -text |grep Not
Not Before: Aug 1 08:36:23 2019 GMT
Not After : Jul 29 08:36:23 2029 GMT
备份工作（非常重要）:
[root@k8s-master ]# cd /etc/kubernetes
[root@k8s-master kubernetes]# mkdir ./pki_bak
[root@k8s-master kubernetes]# mkdir ./pki_bak/etcd
[root@k8s-master kubernetes]# mkdir ./conf_bak
[root@k8s-master kubernetes]# cp pki/apiserver* ./pki_bak/
[root@k8s-master kubernetes]# cp pki/front-proxy-client.* ./pki_bak/
[root@k8s-master kubernetes]# cp pki/etcd/healthcheck-client.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp pki/etcd/peer.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp pki/etcd/server.* ./pki_bak/etcd/
[root@k8s-master kubernetes]# cp ./admin.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./kubelet.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./controller-manager.conf ./conf_bak/
[root@k8s-master kubernetes]# cp ./scheduler.conf ./conf_bak/
三、证书类别
1、集群根证书：
[root@k8s-master images]# ll /etc/kubernetes/pki/ca*
-rw-r--r-- 1 root root 1025 8月 1 2019 /etc/kubernetes/pki/ca.crt
-rw------- 1 root root 1675 8月 1 2019 /etc/kubernetes/pki/ca.key
2、由此集群根证书签发的证书有：
1）kube-apiserver 组件持有的服务端证书
[root@k8s-master pki]# ll /etc/kubernetes/pki/apiserver.*
-rw-r--r-- 1 root root 1224 7月 31 18:47 /etc/kubernetes/pki/apiserver.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver.key
2）kubelet 组件持有的客户端证书
[root@k8s-master pki]# ll /etc/kubernetes/pki/apiserver-kubelet-client.*
-rw-r--r-- 1 root root 1099 7月 31 18:47 /etc/kubernetes/pki/apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver-kubelet-client.key
提示：kubelet的/var/lib/kubelet/config.yaml配置文件中一般不会明确指定服务端证书，而是只指定 ca 根证书, 让 kubelet 根据本地主机信息自动生成服务端证书并保存到配置的 cert-dir文件夹中。
[root@k8s-master kubelet]# pwd
/var/lib/kubelet
[root@k8s-master kubelet]# cat config.yaml
address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 2m0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 5m0s
cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
imagefs.available: 15%
memory.available: 100Mi
nodefs.available: 10%
nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kind: KubeletConfiguration
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s
3）汇聚层证书
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/front-proxy-ca.*
-rw-r--r-- 1 root root 1038 8月 1 2019 /etc/kubernetes/pki/front-proxy-ca.crt
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/front-proxy-ca.key
由此汇聚层根证书签发的证书有：
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/front-proxy-client.*
-rw-r--r-- 1 root root 1058 7月 31 18:47 /etc/kubernetes/pki/front-proxy-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/front-proxy-client.key
3、etcd集群根证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/ca.*
-rw-r--r-- 1 root root 1017 8月 1 2019 /etc/kubernetes/pki/etcd/ca.crt
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/etcd/ca.key
由此etcd根证书签发的证书有：
etcd server服务端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/server.*
-rw-r--r-- 1 root root 1127 7月 31 18:47 /etc/kubernetes/pki/etcd/server.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/etcd/server.key
etcd 集群中peer节点互相通信使用的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/peer.*
-rw-r--r-- 1 root root 1135 7月 31 18:47 /etc/kubernetes/pki/etcd/peer.crt
-rw------- 1 root root 1679 7月 31 18:47 /etc/kubernetes/pki/etcd/peer.key
pod 中定义 Liveness 探针使用的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/etcd/healthcheck-client.*
-rw-r--r-- 1 root root 1094 7月 31 18:47 /etc/kubernetes/pki/etcd/healthcheck-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/etcd/healthcheck-client.key
配置在 kube-apiserver 中用来与 etcd server 做双向认证的客户端证书:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/apiserver-etcd-client.*
-rw-r--r-- 1 root root 1090 7月 31 18:47 /etc/kubernetes/pki/apiserver-etcd-client.crt
-rw------- 1 root root 1675 7月 31 18:47 /etc/kubernetes/pki/apiserver-etcd-client.key
4、Serveice Account密钥:
[root@k8s-master kubelet]# ll /etc/kubernetes/pki/sa.*
-rw------- 1 root root 1679 8月 1 2019 /etc/kubernetes/pki/sa.key
-rw------- 1 root root 451 8月 1 2019 /etc/kubernetes/pki/sa.pub
Serveice Account密钥对仅提供给 kube-controller-manager 使用. kube-controller-manager 通过 sa.key 对 token 进行签名, master 节点通过公钥 sa.pub 进行签名的验证。
API Server身份验证过程：
API Server的authenticating环节支持多种身份校验方式：client cert、bearer token、static password auth等，这些方式中只要有一种方式通过authenticating（Kubernetes API Server会逐个方式尝试），那么身份校验就会通过。
一旦API Server发现client发起的request使用的是service account token的方式，API Server就会自动采用signed bearer token方式进行身份校验。而request则使用携带的service account token参与验证。该token是API Server在创建service account时用API server启动参数：–service-account-key-file的值签署(sign)生成的。如果–service-account-key-file未传入任何值，那么将默认使用–tls-private-key-file的值，即API Server的私钥（server.key）。
通过authenticating后，API Server将根据Pod username所在的group：system:serviceaccounts和system:serviceaccounts:(NAMESPACE)的权限对其进行authority 和admission control两个环节的处理。在这两个环节中，cluster管理员可以对service account的权限进行细化设置。
kubeadm 创建的集群，kube-proxy、flannel、coreDNS是以 pod 形式运行的，在 pod 中，直接使用 service account 与 kube-apiserver 进行认证，此时就不需要再单独为 kube-proxy 创建证书。
---------------------------------------------kubernetes 1.15 版本 以下方案---------------------------------------------
提示：1.12.1 使用可用
1、查看kubeadm-config配置
提示：不同的master节点使用的kubeadm配置有细微的差异，执行更新证书是，每个master在--config后面使用原来集群创建时，当前master对应的kubeadm配置文件。
[root@k8s-master kubernetes]# kubectl -n kube-system get cm kubeadm-config -oyaml
apiVersion: v1
data:
ClusterConfiguration: |
apiServerExtraArgs:
authorization-mode: Node,RBAC
apiVersion: kubeadm.k8s.io/v1alpha3
auditPolicy:
logDir: /var/log/kubernetes/audit
logMaxAge: 2
path: ""
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: ""
etcd:
local:
dataDir: /var/lib/etcd
image: ""
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.12.1
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
unifiedControlPlaneImage: ""
ClusterStatus: |
apiEndpoints:
k8s-master:
advertiseAddress: 192.101.10.80
bindPort: 6443
apiVersion: kubeadm.k8s.io/v1alpha3
kind: ClusterStatus
kind: ConfigMap
metadata:
creationTimestamp: 2019-08-01T08:36:48Z
name: kubeadm-config
namespace: kube-system
resourceVersion: "174"
selfLink: /api/v1/namespaces/kube-system/configmaps/kubeadm-config
uid: 80959d9d-b437-11e9-8e73-6c92bfa51bf6
2、创建kubeadm-cluster.yaml
# touch kubeadm-cluster.yaml
# vi kubeadm-cluster.yaml
apiServer:
apiServerExtraArgs:
authorization-mode: Node,RBAC
apiVersion: kubeadm.k8s.io/v1alpha3
auditPolicy:
logDir: /var/log/kubernetes/audit
logMaxAge: 2
path: ""
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: ""
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.12.1
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
unifiedControlPlaneImage: ""
3、查看帮助
提示：不同版本的kubeadm对于证书renew的命令有细微的差异，具体情况需要依据已经安装的kubeadm来判断。通过命令行kubeadm alpha --help输出类似如下信息：
[root@k8s-master yaml]# kubeadm alpha --help
Experimental sub-commands not yet fully functional.
Usage:
kubeadm alpha [command]
Available Commands:
phase Invoke subsets of kubeadm functions separately for a manual install.
Flags:
-h, --help help for alpha
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha [command] --help" for more information about a command.
[root@k8s-master yaml]# kubeadm alpha phase certs --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase certs [command]
Aliases:
certs, certificates
Available Commands:
all Generates all PKI assets necessary to establish the control plane
apiserver Generates the certificate for serving the kubernetes API
apiserver-etcd-client Generates the client apiserver uses to access etcd
apiserver-kubelet-client Generates the Client certificate for the API server to connect to kubelet
ca Generates the self-signed kubernetes CA to provision identities for other kuberenets components
etcd-ca Generates the self-signed CA to provision identities for etcd
etcd-healthcheck-client Generates the client certificate for liveness probes to healtcheck etcd
etcd-peer Generates the credentials for etcd nodes to communicate with each other
etcd-server Generates the certificate for serving etcd
front-proxy-ca Generates the self-signed CA to provision identities for front proxy
front-proxy-client Generates the client for the front proxy
renew Renews certificates for a Kubernetes cluster
sa Generates a private key for signing service account tokens along with its public key
Flags:
-h, --help help for certs
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase certs [command] --help" for more information about a command.
[root@k8s-master yaml]# kubeadm alpha phase certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase certs renew [flags]
kubeadm alpha phase certs renew [command]
Available Commands:
all renew all available certificates
apiserver Generates the certificate for serving the kubernetes API
apiserver-etcd-client Generates the client apiserver uses to access etcd
apiserver-kubelet-client Generates the Client certificate for the API server to connect to kubelet
etcd-healthcheck-client Generates the client certificate for liveness probes to healtcheck etcd
etcd-peer Generates the credentials for etcd nodes to communicate with each other
etcd-server Generates the certificate for serving etcd
front-proxy-client Generates the client for the front proxy
Flags:
-h, --help help for renew
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase certs renew [command] --help" for more information about a command.
4、重新生成master各个证书
kubeadm alpha phase certs renew etcd-healthcheck-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew etcd-peer --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew etcd-server --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew front-proxy-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver-etcd-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver-kubelet-client --config kubeadm-cluster.yaml
kubeadm alpha phase certs renew apiserver --config kubeadm-cluster.yaml
5、验证证书有效期更新
[root@k8s-master images]# find /etc/kubernetes/pki/ -type f -name "*.crt" -print|egrep -v 'ca.crt$'|xargs -L 1 -t -i bash -c 'openssl x509 -noout -text -in {}|grep After'
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/server.crt|grep After
Not After : Aug 1 10:20:09 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/healthcheck-client.crt|grep After
Not After : Aug 1 10:19:54 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/etcd/peer.crt|grep After
Not After : Aug 1 10:20:02 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-etcd-client.crt|grep After
Not After : Aug 1 10:20:25 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver-kubelet-client.crt|grep After
Not After : Aug 1 10:20:32 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt|grep After
Not After : Aug 1 10:20:39 2021 GMT
bash -c openssl x509 -noout -text -in /etc/kubernetes/pki/front-proxy-client.crt|grep After
Not After : Aug 1 10:20:18 2021 GMT
6、更新各个配置文件
1）查看帮助：
[root@k8s-master manifests]# kubeadm alpha phase kubeconfig --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha phase kubeconfig [command]
Available Commands:
admin Generates a kubeconfig file for the admin to use and for kubeadm itself
all Generates all kubeconfig files necessary to establish the control plane and the admin kubeconfig file
controller-manager Generates a kubeconfig file for the controller manager to use
kubelet Generates a kubeconfig file for the kubelet to use. Please note that this should be used *only* for bootstrapping purposes
scheduler Generates a kubeconfig file for the scheduler to use
user Outputs a kubeconfig file for an additional user
Flags:
-h, --help help for kubeconfig
Global Flags:
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
-v, --v Level log level for V logs
Use "kubeadm alpha phase kubeconfig [command] --help" for more information about a command.
2）备份集群配置文件并重新生成：
[root@k8s-master yaml]# find /etc/kubernetes/ -name '*.conf'|xargs -i mv {}{,bak}
[root@k8s-master yaml]# kubeadm alpha phase kubeconfig all --config kubeadm-cluster.yaml
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
或者分步执行：
kubeadm alpha phase kubeconfig kubelet --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig admin --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig scheduler --config kubeadm-cluster.yaml
kubeadm alpha phase kubeconfig controller-manager --config kubeadm-cluster.yaml
3）重新配置kubectl权限信息：
mv $HOME/.kube/config $HOME/.kube/config.old
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
7、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
---------------------------------------------kubernetes 1.15 版本 以上方案---------------------------------------------
提示：kubernetes 1.16.3 ；1.18.2 使用可用，1.12.1 使用不可用
第一种方案：
1、查看具体过期时间
kubeadm alpha certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Jul 29, 2021 12:07 UTC 362d no
apiserver Jul 29, 2021 12:07 UTC 362d no
apiserver-etcd-client Jul 29, 2021 12:07 UTC 362d no
apiserver-kubelet-client Jul 29, 2021 12:07 UTC 362d no
controller-manager.conf Jul 29, 2021 12:07 UTC 362d no
etcd-healthcheck-client Jul 29, 2021 12:07 UTC 362d no
etcd-peer Jul 29, 2021 12:07 UTC 362d no
etcd-server Jul 29, 2021 12:07 UTC 362d no
front-proxy-client Jul 29, 2021 12:07 UTC 362d no
scheduler.conf Jul 29, 2021 12:07 UTC 362d no
2、查看帮助
[root@hadoop009 images]# kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha certs renew [flags]
kubeadm alpha certs renew [command]
Available Commands:
admin.conf Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
all Renew all available certificates
apiserver Renew the certificate for serving the Kubernetes API
apiserver-etcd-client Renew the certificate the apiserver uses to access etcd
apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
controller-manager.conf Renew the certificate embedded in the kubeconfig file for the controller manager to use
etcd-healthcheck-client Renew the certificate for liveness probes to healthcheck etcd
etcd-peer Renew the certificate for etcd nodes to communicate with each other
etcd-server Renew the certificate for serving etcd
front-proxy-client Renew the certificate for the front proxy client
scheduler.conf Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:
-h, --help help for renew
Global Flags:
--add-dir-header If true, adds the file directory to the header
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm alpha certs renew [command] --help" for more information about a command.
提示：由help可知，证书更新可针对单个证书更新
3、更新证书
更新所有证书，对证书进行续期,续期一年：
$ kubeadm alpha certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
提示：更新操作需要在所有master节点执行
4、验证续期
# ls -l
总用量 56
-rw-r--r-- 1 root root 1220 7月 31 22:48 apiserver.crt
-rw-r--r-- 1 root root 1090 7月 31 22:48 apiserver-etcd-client.crt
-rw------- 1 root root 1679 7月 31 22:48 apiserver-etcd-client.key
-rw------- 1 root root 1675 7月 31 22:48 apiserver.key
-rw-r--r-- 1 root root 1099 7月 31 22:48 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 7月 31 22:48 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 7月 29 20:07 ca.crt
-rw------- 1 root root 1679 7月 29 20:07 ca.key
drwxr-xr-x 2 root root 162 7月 29 20:07 etcd
-rw-r--r-- 1 root root 1038 7月 29 20:07 front-proxy-ca.crt
-rw------- 1 root root 1675 7月 29 20:07 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 7月 31 22:48 front-proxy-client.crt
-rw------- 1 root root 1679 7月 31 22:48 front-proxy-client.key
-rw------- 1 root root 1675 7月 29 20:07 sa.key
-rw------- 1 root root 451 7月 29 20:07 sa.pub
# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Aug 01, 2021 07:38 UTC 364d no
apiserver Aug 01, 2021 07:38 UTC 364d ca no
apiserver-etcd-client Aug 01, 2021 07:38 UTC 364d etcd-ca no
apiserver-kubelet-client Aug 01, 2021 07:38 UTC 364d ca no
controller-manager.conf Aug 01, 2021 07:38 UTC 364d no
etcd-healthcheck-client Aug 01, 2021 07:38 UTC 364d etcd-ca no
etcd-peer Aug 01, 2021 07:38 UTC 364d etcd-ca no
etcd-server Aug 01, 2021 07:38 UTC 364d etcd-ca no
front-proxy-client Aug 01, 2021 07:38 UTC 364d front-proxy-ca no
scheduler.conf Aug 01, 2021 07:38 UTC 364d no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Jul 27, 2030 13:01 UTC 9y no
etcd-ca Jul 27, 2030 13:01 UTC 9y no
front-proxy-ca Jul 27, 2030 13:01 UTC 9y no
[root@hadoop010 etcd]# kubectl -n kube-system get cm kubeadm-config -oyaml
apiVersion: v1
data:
ClusterConfiguration: |
apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 192.101.11.162:6443
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.18.2
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
ClusterStatus: |
apiEndpoints:
hadoop010:
advertiseAddress: 192.101.11.162
bindPort: 6443
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterStatus
kind: ConfigMap
metadata:
creationTimestamp: "2020-07-29T13:02:16Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:ClusterConfiguration: {}
f:ClusterStatus: {}
manager: kubeadm
operation: Update
time: "2020-07-29T13:02:16Z"
name: kubeadm-config
namespace: kube-system
resourceVersion: "157"
selfLink: /api/v1/namespaces/kube-system/configmaps/kubeadm-config
uid: 2e049082-fa64-4e2e-ad73-af9fc94a051e
5、启用证书
在每台Master上执行重启kube-apiserver、kube-controller、kube-scheduler、etcd这4个容器，以便使证书生效。
# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
或者
# docker ps | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash
提示：启用操作需要在所有master节点执行。
6、更新.kube下的配置文件
$ mv $HOME/.kube/config $HOME/.kube/config.old
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
7、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期。
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
第二种方案：
1、备份导出kubeadm集群配置
# kubeadm config view > kubeadm-cluster.yaml
apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.16.3
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
2、查看具体过期时间
kubeadm alpha certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Jul 29, 2021 12:07 UTC 362d no
apiserver Jul 29, 2021 12:07 UTC 362d no
apiserver-etcd-client Jul 29, 2021 12:07 UTC 362d no
apiserver-kubelet-client Jul 29, 2021 12:07 UTC 362d no
controller-manager.conf Jul 29, 2021 12:07 UTC 362d no
etcd-healthcheck-client Jul 29, 2021 12:07 UTC 362d no
etcd-peer Jul 29, 2021 12:07 UTC 362d no
etcd-server Jul 29, 2021 12:07 UTC 362d no
front-proxy-client Jul 29, 2021 12:07 UTC 362d no
scheduler.conf Jul 29, 2021 12:07 UTC 362d no
3、查看帮助
[root@hadoop009 images]# kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
kubeadm alpha certs renew [flags]
kubeadm alpha certs renew [command]
Available Commands:
admin.conf Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
all Renew all available certificates
apiserver Renew the certificate for serving the Kubernetes API
apiserver-etcd-client Renew the certificate the apiserver uses to access etcd
apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
controller-manager.conf Renew the certificate embedded in the kubeconfig file for the controller manager to use
etcd-healthcheck-client Renew the certificate for liveness probes to healthcheck etcd
etcd-peer Renew the certificate for etcd nodes to communicate with each other
etcd-server Renew the certificate for serving etcd
front-proxy-client Renew the certificate for the front proxy client
scheduler.conf Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:
-h, --help help for renew
Global Flags:
--add-dir-header If true, adds the file directory to the header
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm alpha certs renew [command] --help" for more information about a command.
提示：由help可知，证书更新可针对单个证书更新
4、更新证书
# kubeadm alpha certs renew all --config=kubeadm-cluster.yaml #更新所有证书
提示：在保存kubeadm-cluster.yaml文件的目录下执行
提示：更新操作需要在所有master节点执行
5、确认验证
# kubeadm alpha certs check-expiration
6、启用证书
在每台Master上执行重启kube-apiserver、kube-controller、kube-scheduler、etcd这4个容器，以便使证书生效。
# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
或者
# docker ps | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash
提示：启用操作需要在所有master节点执行。
7、更新.kube下的配置文件
$ mv $HOME/.kube/config $HOME/.kube/config.old
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
8、更新集群验证
验证kubernetes 集群：运行 kubectl cluster-info 和 kubectl get nodes 符合预期。
[root@k8s-master images]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 1y v1.12.1
k8s-node1 Ready <none> 1y v1.12.1
k8s-node2 Ready <none> 1y v1.12.1
确性kubernetes 系统相关的服务运行正常(核心是kube-apiserver,kube-controller-manager,kube-proxy, kube-flannel)：kubectl get pods -n kube-system
[root@k8s-master images]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-576cbf47c7-fk7j9 1/1 Running 0 1y
coredns-576cbf47c7-p4f4q 1/1 Running 0 1y
etcd-k8s-master 1/1 Running 2 1y
kube-apiserver-k8s-master 1/1 Running 0 2h
kube-controller-manager-k8s-master 1/1 Running 1 2h
kube-flannel-ds-amd64-f2csl 1/1 Running 0 1y
kube-flannel-ds-amd64-wm2b6 1/1 Running 0 1y
kube-flannel-ds-amd64-wrnnk 1/1 Running 1 1y
kube-proxy-cz5xg 1/1 Running 0 1y
kube-proxy-fnr96 1/1 Running 0 1y
kube-proxy-xbrcb 1/1 Running 0 1y
kube-scheduler-k8s-master 1/1 Running 1 1y
kubernetes-dashboard-77fd78f978-jl98q 1/1 Running 0 218d
检查pod的运行状态：kubectl get pods --all-namespaces
[root@k8s-master images]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bbcx-service-7c4bb5456-bksqb 1/1 Running 0 7h
default bbcx-vue-5569488679-zd75m 1/1 Running 1 7h
default cip-data-service-66ffd668dd-2l4wv 1/1 Running 0 17d
default cip-job-5fb59f9d84-4lrb4 1/1 Running 0 17d
default consul-0 1/1 Running 0 193d
...................
分享到：
发表评论
-
HTTPS的加密原理解读
2021-12-31 11:25 318一、为什么需要加密？ 因为http的内容是明文传输的，明文数据 ... -
容器技术的基石： cgroup、namespace和联合文件系统
2021-12-09 10:47 770Docker 是基于 Linux Kernel 的 Names ... -
链路追踪skywalking安装部署
2021-10-21 12:06 842APM 安装部署： 一、下载 版本目录地址：http://a ... -
自动化运维 Ansible 安装部署
2021-08-20 19:06 860一、概述 Ansible 实现了批量系统配置、批量程序部署、 ... -
Linux 下 Kafka Cluster 搭建
2021-07-08 11:23 1006概述 http://kafka.apachecn.org/q ... -
ELK RPM 安装配置
2021-06-22 18:59 645相关组件： 1)filebeat。用于收集日志组件，经测试其 ... -
在Kubernetes上部署 Redis 三主三从 集群
2021-03-10 16:25 719NFS搭建见： Linux NFS搭建与配置(https:// ... -
docker-compose 部署ELK(logstash->elasticsearch->kibana)
2020-11-11 18:02 1659概述： ELK是三个开源软件的缩写，分别表示：elastic ... -
Kubernetes1.16.3下部署node-exporter+alertmanager+prometheus+grafana 监控系统
2020-10-28 10:48 1115准备工作 建议将所有的yaml文件存在如下目录： # mkd ... -
Linux NFS 搭建与配置
2020-10-21 17:58 445一、NFS 介绍 NFS 是 Network FileSys ... -
K8S 备份及升级
2020-10-20 15:48 911一、准备工作 查看集群版本： # kubectl get no ... -
API 网关 kong 的 konga 配置使用
2020-09-23 10:46 4325一、Kong 概述： kong的 ... -
云原生技术 Docker、K8S
2020-09-02 16:53 579容器的三大好处 1.资源 ... -
Kubernetes 应用编排、管理与运维
2020-08-24 16:40 607一、kubectl 运维命令 kubectl control ... -
API 网关 kong/konga 安装部署
2020-08-25 17:34 644一、概述 Kong是Mashape开 ... -
Linux 下 Redis Cluster 搭建
2020-08-13 09:14 799Redis集群演变过程： 单 ... -
Kubernetes离线安装的本地yum源构建
2020-08-08 22:41 586一、需求场景 在K8S的使用过程中有时候会遇到在一些无法上网 ... -
kubeadm方式部署安装kubernetes
2020-07-29 08:01 2454一、前提准备： 0、升级更新系统（切记升级一下，曾被坑过） ... -
Kubernetes 部署 Nginx 集群
2020-07-20 09:32 920一.设置标签 为了保证nginx之能分配到nginx服务器需要 ... -
Prometheus 外部监控 Kubernetes 集群
2020-07-10 15:59 2130大多情况都是将 Prometheus 通过 yaml 安装在 ...
相关推荐
但这需要对Kubernetes证书管理有深入理解，操作过程较复杂。 2. **使用kubeadm**：kubeadm提供了一个方便的命令`kubeadm alpha phase kubeconfig all --certificate-renewal=true`来尝试自动更新控制平面的证书。...
【kubernetes】环境准备及K8S安装【最新完整版】 1.证书延期10年 2../update-kubeadm-cert.sh all
计算机二级公共基础知识模 拟试题及答案详解.pdf
内容概要：本文档详细介绍了语音发射机的设计与实现，涵盖了从硬件电路到具体元件的选择和连接方式。文档提供了详细的电路图，包括电源管理、信号处理、音频输入输出接口以及射频模块等关键部分。此外，还展示了各个引脚的功能定义及其与其他组件的连接关系，确保了系统的稳定性和高效性能。通过这份文档，读者可以全面了解语音发射机的工作原理和技术细节。 适合人群：对电子工程感兴趣的初学者、从事嵌入式系统开发的技术人员以及需要深入了解语音发射机制的专业人士。 使用场景及目标：适用于希望构建自己的语音发射设备的研究人员或爱好者，帮助他们掌握相关技术和实际操作技能。同时，也为教学机构提供了一个很好的案例研究材料。 其他说明：文档不仅限于理论讲解，还包括具体的实施步骤，使读者能够动手实践并验证所学知识。
内容概要：本文详细介绍了用易语言编写的单线程全功能注册机源码，涵盖了接码平台对接、滑块验证处理、IP代理管理以及料子导入等多个核心功能。文章首先展示了主框架的初始化配置和事件驱动逻辑，随后深入探讨了接码平台（如打码兔）的API调用及其返回数据的处理方法。对于滑块验证部分，作者分享了如何利用易语言的绘图功能模拟真实用户的操作轨迹，并提高了验证通过率。IP代理模块则实现了智能切换策略，确保代理的有效性和稳定性。此外，料子导入功能支持多种格式的数据解析和去重校验，防止脏数据污染。最后，文章提到了状态机设计用于控制注册流程的状态持久化。 适合人群：有一定编程基础，尤其是熟悉易语言的开发者和技术爱好者。 使用场景及目标：适用于希望深入了解易语言注册机开发的技术细节，掌握接码、滑块验证、IP代理等关键技术的应用场景。目标是帮助读者理解并优化现有注册机的功能，提高其稳定性和效率。 其他说明：文中提到的部分技术和实现方式可能存在一定的风险，请谨慎使用。同时，建议读者在合法合规的前提下进行相关开发和测试。
计算机绘图实用教程 第三章.pdf
计算机辅助设计—AutoCAD 2018中文版基础教程 各章CAD图纸及相关说明汇总.pdf
C++相关书籍，计算机相关书籍，linux相关及http等计算机学习、面试书籍。
计算机二级mysql数据库程序设计练习题(一).pdf
计算机发展史.pdf
计算机二级课件.pdf
计算机概论第三讲：计算机组成.pdf
内容概要：本文档由中国移动通信集团终端有限公司、北京邮电大学、中国信息通信研究院和中国通信学会共同发布，旨在探讨端侧算力网络（TCAN）的概念、架构、关键技术及其应用场景。文中详细分析了终端的发展现状、基本特征和发展趋势，阐述了端侧算力网络的定义、体系架构、功能架构及其主要特征。端侧算力网络通过整合海量泛在异构终端的算力资源，实现分布式多级端侧算力资源的高效利用，提升网络整体资源利用率和服务质量。关键技术涵盖层次化端算力感知图模型、资源虚拟化、数据压缩、多粒度多层次算力调度、现场级AI推理和算力定价机制。此外，还探讨了端侧算力网络在智能家居、智能医疗、车联网、智慧教育和智慧农业等领域的潜在应用场景。 适合人群：从事通信网络、物联网、边缘计算等领域研究和开发的专业人士，以及对6G网络和端侧算力网络感兴趣的学者和从业者。 使用场景及目标：适用于希望深入了解端侧算力网络技术原理、架构设计和应用场景的读者。目标是帮助读者掌握端侧算力网络的核心技术，理解其在不同行业的应用潜力，推动端侧算力网络技术的商业化和产业化。 其他说明：本文档不仅提供了端侧算力网络的技术细节，还对其隐私与安全进行了深入探讨
学习java的心得体会.docx
计算机二级考试(南开100题齐全).pdf
内容概要：本文详细介绍了计算机二级C语言考试的内容和备考方法。首先概述了计算机二级考试的意义及其在计算机技能认证中的重要性，重点讲解了C语言的基础语法，包括程序结构、数据类型、运算符和表达式等。接着深入探讨了进阶知识，如函数、数组、指针、结构体和共用体的应用。最后分享了针对选择题、填空题和编程题的具体解题技巧，强调了复习方法和实战演练的重要性。 适合人群：准备参加计算机二级C语言考试的学生和技术爱好者。 使用场景及目标：①帮助考生系统地掌握C语言的核心知识点；②提供有效的解题策略，提高应试能力；③指导考生制定合理的复习计划，增强实战经验。 其他说明：本文不仅涵盖了理论知识，还提供了大量实例代码和详细的解释，有助于读者更好地理解和应用所学内容。此外，文中提到的解题技巧和复习建议对实际编程也有很大帮助。
论文格式及要求.doc
内容概要：本文详细介绍了如何使用三菱FX3U PLC及其485BD通信板与四台台达VFD-M系列变频器进行通信的设置与应用。主要内容涵盖硬件连接注意事项、通信参数配置、RS指令的应用、CRC校验算法的实现以及频率给定和状态读取的具体方法。文中提供了多个实用的编程示例，展示了如何通过梯形图和结构化文本编写通信程序，并讨论了常见的调试技巧和优化建议。此外，还提到了系统的扩展性和稳定性措施，如增加温度传感器通信功能和应对电磁干扰的方法。 适合人群：从事工业自动化领域的工程师和技术人员，尤其是那些熟悉三菱PLC和台达变频器的使用者。 使用场景及目标：适用于需要实现多台变频器联动控制的工业应用场景，旨在提高生产效率和系统可靠性。通过学习本文，读者可以掌握如何构建稳定的RS485通信网络，确保变频器之间的高效协同工作。 其他说明：本文不仅提供了详细的理论指导，还包括了许多来自实际项目的经验教训，帮助读者避免常见错误并提升编程技能。
计算机服务规范.pdf
Discuz_X3.2_TC_UTF8.zip LNMP搭建安装包