K8S部署，运行第一个容器 – 春风桃李花开夜，秋雨梧桐叶落时

1.环境准备

准备三台主机 1台控制节点 2台计算节点 (环境必须是纯净的否则会有很多坑)

1.1：修改IP地址、主机和host解析

10.0.0.11 k8s-master
10.0.0.12 k8s-node-1
10.0.0.13 k8s-node-2
所有节点都需要做hosts解析

1.2：Master节点安装etcd （数据存储，注意需要Base源）

yum install etcd -y

etcd 原生支持做集群
修改etcd配置文件

[root@k8s-master ~]# vim /etc/etcd/etcd.conf

6行：ETCD_LISTEN_CLIENT_URLS="[http://0.0.0.0:2379](http://0.0.0.0:2379)"  
21行：ETCD_ADVERTISE_CLIENT_URLS="[http://10.0.0.11:2379](http://10.0.0.11:2379)"

## 启动并开机自启
[root@k8s-master ~]# systemctl start etcd.service 
[root@k8s-master ~]# systemctl enable etcd.service 

## 检查
[root@k8s-master ~]# netstat -lntup|grep 2379
tcp6       0      0 :::2379                 :::*                    LISTEN      17352/etcd  

### 测试（存数据是目录结构   和键值对相似）

[root@k8s-master ~]# etcdctl set /test/word 123456    ### 插入数据 键（目录）值（123456）
123456

[root@k8s-master ~]# etcdctl ls /
/test
[root@k8s-master ~]# etcdctl ls /test
/test/word
[root@k8s-master ~]# etcdctl get /test/word    ### 查看值
123456

### 检查集群健康状态
[root@k8s-master ~]# etcdctl -C http://10.0.0.11:2379 cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://10.0.0.11:2379
cluster is healthy   ## 因为现在只有一个节点所有只有一行

#[Member]  
ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ###数据存放目录  #
ETCD_LISTEN_PEER_URLS="[http://localhost:2380](http://localhost:2380)" ## 做集群的时候需要配置 
 ETCD_LISTEN_CLIENT_URLS="[http://localhost:2379](http://localhost:2379)" ### 监听客户端的地址 
 ETCD_NAME="default" ## 如果要配置集群的话不能是default，每个节点的名字都不一样  
#[Clustering] 
 ETCD_ADVERTISE_CLIENT_URLS="[http://localhost:2379](http://localhost:2379)" ### 再集群中显示的信息，客户端(etcdctl/curl等)跟etcd服务进行交互时请求的url

1.3 Master节点安装kubernetes

yum install kubernetes-master.x86_64 -y

修改API配置文件

[root@k8s-master ~]# vim /etc/kubernetes/apiserver 

8行: KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"  ## 服务的监听地址
11行：KUBE_API_PORT="--port=8080"     ## 服务监听的端口
14行：KUBELET_PORT="--kubelet-port=10250"   ## 通过10250端口控制kubelet
17行：KUBE_ETCD_SERVERS="--etcd-servers=[http://10.0.0.11:2379]  ## APIserver是通过那个地址和端口连接etcd数据 
23行：KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"   ## 默认的管理控制插件ServiceAccount删掉

修改其他两个服务通用的配置文件

vim /etc/kubernetes/config

22行：KUBE_MASTER="--master=[http://10.0.0.11:8080]
通过那个地址端口找到API服务

启动master所有服务

systemctl enable kube-apiserver.service
systemctl start kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl start kube-controller-manager.service
systemctl enable kube-scheduler.service
systemctl start kube-scheduler.service

测试集群是否正常

[root@k8s-master ~]# kubectl get componentstatus

image.png

1.4 Node节点安装kubernetes

yum install kubernetes-node.x86_64 -y   (自动会安装docker)

修改kube-proxy服务配置文件

vim /etc/kubernetes/config
22行：KUBE_MASTER="--master=[http://10.0.0.11:8080]

修改kubelet服务服务配置文件

vim /etc/kubernetes/kubelet

5行：KUBELET_ADDRESS="--address=0.0.0.0"  ### 监听的地址
8行：KUBELET_PORT="--port=10250"   ### kubelet端口 
11行：KUBELET_HOSTNAME="--hostname-override=10.0.0.12"   ### 给自己定义唯一的名字 不能冲突 IP地址或者主机名（各自节点改各自节点的）
14行：KUBELET_API_SERVER="--api-servers=[http://10.0.0.11:8080]  ### 连接api的地址

启动node节点所有服务

 systemctl  start  kubelet.service 
 systemctl enable kubelet.service 

 systemctl start kube-proxy.service
 systemctl enable kube-proxy.service

在master测试是否有节点加入集群

[root@k8s-master ~]# kubectl get nodes
NAME        STATUS    AGE
10.0.0.12   Ready     1m
10.0.0.13   Ready     2m

1.5 所有节点安装flannel（网络通讯）插件需要etcd

（这里有个疑问，配置文件里启用了一个网段，这边又生成一个网段，会导致多个网卡问题。）

yum install flannel -y

### 修改flannel配置文件

sed -i 's#http://127.0.0.1:2379#http://10.0.0.11:2379#g' /etc/sysconfig/flanneld

Master节点：

## 创建etcd值

etcdctl mk /atomic.io/network/config  '{ "Network": "172.16.0.0/16" }'
etcdctl get /atomic.io/network/config

## 安装docker（主要是兼职私有仓库（可以单独起一台））
yum install docker -y
## 启动flannel网络并重启所有服务

Node节点

### 启动所有服务
systemctl enable flanneld.service 
systemctl restart flanneld.service 
service docker restart
systemctl restart kubelet.service
systemctl restart kube-proxy.service

这时ifconfig会发现flannel出现172段网络

所有节点测试是否跨宿主机通信

### 这里再测试之前先修改 iptables 规则 1.13版docker和flannel之间有BUG，把默认的forward规则 ACCERT   改成了DROP导致无法上网

查看： 
iptables -L -n
iptables -P FORWARD ACCERT

### 由于busybox有ping 的功能占用空间小所以用box测试
docker load -i docker_busybox.tar.gz
docker run -it docker.io/busybox:latest 
## 进入容器之后查看一下IP地址（全局唯一）

image.png

由于修改iptables 规则是临时生效所以要修改成永久生效

vim /usr/lib/systemd/system/docker.service 
18行：ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT
### 让docker启动时先执行 以上命令
systemctl daemon-reload 
systemctl restart docker

1.6 为Master节点配置一个私有仓库

1.61 现为所有节点配置镜像加速，和私有仓库位置

vim /etc/sysconfig/docker
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false --registry-mirror=https://registry.docker-cn.com --insecure-registry=10.0.0.11:5000'

systemctl restart docker

1.62 Master启动私有仓库

### 导入registry镜像
docker load  -i registry.tar.gz 
### 后台启动容器并开机自启
docker run -d -p 5000:5000 --restart=always --name registry -v /opt/myregistry:/var/lib/registry registry

docker ps -a ## 查看一下

### 再node测试仓库是否能上传
docker tag docker.io/busybox:latest 10.0.0.11:5000/busybox:latest
docker push 10.0.0.11:5000/busybox:latest

作者：Aza_Lxxx
链接：https://www.jianshu.com/p/99d09406373e
来源：简书
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

到这后发现，报错Back-off pulling image “registry.access.redhat.com/rhel7/pod-infrastructure:latest”

报错信息：

Jan 14 12:26:57 k8snode01 kubelet: E0114 12:26:57.295735 8604 docker_manager.go:2159] Failed to create pod infra container: ErrImagePull; Skipping pod “nginx-3449338310-nmvsz_default(f67e741e-36f2-11ea-98b8-000c299594c3)”: image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request. details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)
Jan 14 12:26:57 k8snode01 kubelet: E0114 12:26:57.295770 8604 pod_workers.go:184] Error syncing pod f67e741e-36f2-11ea-98b8-000c299594c3, skipping: failed to “StartContainer” for “POD” with ErrImagePull: “image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request. details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)”

Jan 14 12:41:16 k8snode01 dockerd-current: time=”2020-01-14T12:41:16.156396599-05:00″ level=error msg=”Handler for GET /v1.26/images/registry.access.redhat.com/rhel7/pod-infrastructure:latest/json returned error: No such image: registry.access.redhat.com/rhel7/pod-infrastructure:latest”
Jan 14 12:41:16 k8snode01 kubelet: E0114 12:41:16.157271 17563 docker_manager.go:2159] Failed to create pod infra container: ImagePullBackOff; Skipping pod “nginx-3449338310-nmvsz_default(f67e741e-36f2-11ea-98b8-000c299594c3)”: Back-off pulling image “registry.access.redhat.com/rhel7/pod-infrastructure:latest”
Jan 14 12:41:16 k8snode01 kubelet: E0114 12:41:16.157298 17563 pod_workers.go:184] Error syncing pod f67e741e-36f2-11ea-98b8-000c299594c3, skipping: failed to “StartContainer” for “POD” with ImagePullBackOff: “Back-off pulling image \”registry.access.redhat.com/rhel7/pod-infrastructure:latest\””

解决方法：
1.在node节点执行 yum install *rhsm* -y
2、直接获取 rpm文件 wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
3、运行脚本 rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv –to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem 这两个命令会生成/etc/rhsm/ca/redhat-uep.pem文件.
4、重新下载文件 docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
————————————————
版权声明：本文为CSDN博主「zhaikaiyun」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/zhaikaiyun/java/article/details/103983766

开始run第一个容器：

kubectl run my-nginx --image=nginx --replicas=2 --port=80

参考：

https://www.kubernetes.org.cn/doc-44