K8S(1)Set Up a Customer Cluster

K8S(1)Set Up a Customer Cluster

Following this document
https://jimmysong.io/kubernetes-handbook/practice/install-kubernetes-on-ubuntu-server-16.04-with-kubeadm.html
I have 2 Ubuntu machines
192.168.56.101 ubuntu-master
192.168.56.103 ubuntu-dev4
Make Sure and Install Docker
> apt-get install docker.io
Check version
> docker --version
Docker version 18.06.1-ce, build e68fc7a
Install the tools, but fail
> sudo apt-get install kubelet kubeadm kubectl
No apt package "kubeadm", but there is a snap with that name.
Try "snap install kubeadm"
Solution:
https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl
> sudo apt-get update && sudo apt-get install -y apt-transport-https
> curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
> echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
> sudo apt-get update
> sudo apt-get install kubelet kubeadm kubectl
Use Kubeadm to install K8S cluster
Set Up Master First
> sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.56.101
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR Swap]: running with swap on is not supported. Please disable swap
Disable the swap
> sudo swapoff -a
> sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.56.101
[init] Using Kubernetes version: v1.13.3
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
  kubeadm join 192.168.56.101:6443 --token hwg09t.gmd6t2f7s3f18z2t --discovery-token-ca-cert-hash sha256:9eab712056ad59e628e852e40ab5889071865f6f925c8ef54c1a257eaf02f3d9
Follow the docs
> mkdir -p $HOME/.kube
> sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
> sudo chown $(id -u):$(id -g) /home/carl/.kube/config
Set Up Slaves
> sudo swapoff -a
> sudo kubeadm join 192.168.56.101:6443 --token hwg09t.gmd6t2f7s3f18z2t --discovery-token-ca-cert-hash sha256:9eab712056ad59e628e852e40ab5889071865f6f925c8ef54c1a257eaf02f3d9
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.
We can run command on master to see the nodes
> kubectl get nodes
NAME            STATUS     ROLES    AGE    VERSION
ubuntu-dev4     NotReady   <none>   75s    v1.13.3
ubuntu-master   NotReady   master   164m   v1.13.3
We can also check from master
> kubectl get pod -n kube-system -o wide
Install the network plugin canal
> kubectl apply -f  https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
clusterrole.rbac.authorization.k8s.io/calico created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/canal-flannel created
clusterrolebinding.rbac.authorization.k8s.io/canal-calico created
> kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/canal/canal.yaml
configmap/canal-config created
daemonset.extensions/canal created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
serviceaccount/canal created
After apply these two commands above, we check the status again, it is running
> kubectl get pod -n kube-system -o wide
Check node status
> kubectl get node
NAME            STATUS   ROLES    AGE    VERSION
ubuntu-dev4     Ready    <none>   20m    v1.13.3
ubuntu-master   Ready    master   3h3m   v1.13.3
Enable running pod on master as well
> kubectl taint nodes --all node-role.kubernetes.io/master-
Turn off the swap even after restart
> sudo swapoff -a
> cat /etc/fstab | grep -v '^#' | grep -v 'swap' | sudo tee /etc/fstab
Checking documents
https://juejin.im/post/5bb45d63f265da0a9e532128
Set Up the UI
> kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created
> kubectl get pods --namespace=kube-system
kubernetes-dashboard-57df4db6b-qs87d    1/1     Running   0          69s
Check all the pod running
> kubectl get pods --namespace=kube-system -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP               NODE            NOMINATED NODE   READINESS GATES
canal-cfbbs                             3/3     Running   6          125m    192.168.56.103   ubuntu-dev4     <none>           <none>
canal-vsg6n                             3/3     Running   7          125m    192.168.56.101   ubuntu-master   <none>           <none>
coredns-86c58d9df4-9gs9l                1/1     Running   1          4h53m   10.244.0.4       ubuntu-master   <none>           <none>
coredns-86c58d9df4-ch28b                1/1     Running   1          4h53m   10.244.0.5       ubuntu-master   <none>           <none>
etcd-ubuntu-master                      1/1     Running   4          4h52m   192.168.56.101   ubuntu-master   <none>           <none>
kube-apiserver-ubuntu-master            1/1     Running   4          4h52m   192.168.56.101   ubuntu-master   <none>           <none>
kube-controller-manager-ubuntu-master   1/1     Running   4          4h52m   192.168.56.101   ubuntu-master   <none>           <none>
kube-proxy-hp9n2                        1/1     Running   4          4h53m   192.168.56.101   ubuntu-master   <none>           <none>
kube-proxy-nmmvz                        1/1     Running   2          130m    192.168.56.103   ubuntu-dev4     <none>           <none>
kube-scheduler-ubuntu-master            1/1     Running   4          4h52m   192.168.56.101   ubuntu-master   <none>           <none>
kubernetes-dashboard-57df4db6b-qs87d    1/1     Running   0          2m5s    10.244.1.2       ubuntu-dev4     <none>           <none>
Start a Proxy on ubuntu-master
> kubectl proxy --address='0.0.0.0' --accept-hosts='^*'
https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
Checking network
https://www.cnblogs.com/xzkzzz/p/9952716.html
> kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
> kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/canal.yaml

Copy the file to my host machine
> scp ubuntu-master:/home/carl/.kube/config ~/.kube/config-ubuntu
> export KUBECONFIG=~/.kube/config-ubuntu
Then I can run these commands on my host machine as well
> kubectl get pods --namespace=kube-system -o wide
> kubectl proxy
Starting to serve on 127.0.0.1:8001
This command need to be on ubuntu-master
> kubectl proxy --address='0.0.0.0' --accept-hosts='^*'
These URLs do not work.
http://localhost:8001/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/#!/overview?namespace=default
http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/overview?namespace=default
I think it is related the network settings, that I can not ping 10.244.1.2 which the admin console is running
https://stackoverflow.com/questions/50401355/requests-timing-out-when-accesing-a-kubernetes-clusterip-service
Download the 2 setting files
> wget https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/canal.yaml
Change that file
  canal_iface: "enp0s8"
> kubectl apply -f ./canal.yaml
Reboot the ubuntu-master, then it works.
> ping 10.244.1.4
PING 10.244.1.4 (10.244.1.4) 56(84) bytes of data.
64 bytes from 10.244.1.4: icmp_seq=1 ttl=63 time=0.739 ms
Run the proxy on master
> kubectl proxy --address='0.0.0.0' --accept-hosts='^*'
YEAH, this UI will work
http://ubuntu-master:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login
Check TOKEN
> kubectl get secret -n kube-system | grep dashboard
kubernetes-dashboard-certs                       Opaque                                0         1h
kubernetes-dashboard-csrf                        Opaque                                1         1h
kubernetes-dashboard-key-holder                  Opaque                                2         1h
kubernetes-dashboard-token-d4p5x                 kubernetes.io/service-account-token   3         1h
Check the Dashboard I installed
> wget https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
I use this command already find out the token
> kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep kubernetes-dashboard-token|awk '{print $1}')|grep token:|awk '{print $2}'
Give rights to skip
>cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system
EOF
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
Not working, checking this
https://www.cnblogs.com/RainingNight/p/deploying-k8s-dashboard-ui.html
It seems the latest documents are here
https://github.com/kubernetes/dashboard
https://github.com/kubernetes/dashboard/wiki/Creating-sample-user

References:
https://kubernetes.io/docs/setup/scratch/
http://docs.kubernetes.org.cn/774.html
https://www.jianshu.com/p/7d1fb03b8925
https://juejin.im/post/5bb45d63f265da0a9e532128
https://my.oschina.net/u/1013857/blog/2991314
https://zhanghongtong.github.io/2018/10/09/ubuntu-%E4%BD%BF%E7%94%A8kubeadm%E5%AE%89%E8%A3%85kubernetes/
https://jimmysong.io/kubernetes-handbook/practice/install-kubernetes-on-ubuntu-server-16.04-with-kubeadm.html