k8s搭建

K8s官方文档地址:https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/  如果用云主机部署,一定要提前开启端口

1. 服务器规划

角色

IP

组件

k8s-master1

192.168.31.63

kube-apiserver

kube-controller-manager

kube-scheduler

etcd

k8s-master2

192.168.31.64

kube-apiserver

kube-controller-manager

kube-scheduler

 

k8s-node1

192.168.31.65

kubelet

kube-proxy

docker

etcd

k8s-node2

192.168.31.66

kubelet

kube-proxy

docker

etcd

Load Balancer(Master)

192.168.31.61

192.168.31.60 (VIP)

Nginx L4

Load Balancer(Backup)

192.168.31.62

Nginx L4

1.系统初始化

修改主机名称:

hostnamectl set-hostname k8s-master1


关闭防火墙:

# systemctl stop firewalld

# systemctl disable firewalld

 

关闭selinux:

# setenforce 0 # 临时

# sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久

 

关闭swap:

# swapoff -a  # 临时

# vim /etc/fstab  # 永久

 

同步系统时间:

# ntpdate time.windows.com

2.2 部署三个Etcd节点   

TLS、etcd地址:

链接:https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg
提取码:o101

# tar zxvf etcd.tar.gz

# cd etcd

# cp TLS/etcd/ssl/{ca,server,server-key}.pem ssl

 

分别拷贝到Etcd三个节点:

# scp –r etcd root@192.168.31.63:/opt

# scp etcd.service root@192.168.31.63:/usr/lib/systemd/system/

 

登录三个节点修改配置文件 名称和IP:

 # vi /opt/etcd/cfg/etcd.conf

#[Member]

ETCD_NAME="etcd-1"  名称一定要替换

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_PEER_URLS="https://192.168.31.63:2380"  内网ip

ETCD_LISTEN_CLIENT_URLS="https://192.168.31.63:2379"

 

#[Clustering]

ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.31.63:2380"

ETCD_ADVERTISE_CLIENT_URLS="https://192.168.31.63:2379"

ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.31.63:2380,etcd-2=https://192.168.31.64:2380,etcd-3=https://192.168.31.65:2380"   部署3个节点的内网ip

ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"

ETCD_INITIAL_CLUSTER_STATE="new"  集群状态

 

# systemctl daemon-reload

# systemctl start etcd

# ps -ef|grep etcd 查看etcd进程

# systemctl enable etcd 设置开机启动

# tail /var/log/messages -f 查看系统日志

2.3 查看集群状态

 

# /opt/etcd/bin/etcdctl \

> --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem \

> --endpoints="https://192.168.31.63:2379,https://192.168.31.64:2379,https://192.168.31.65:2379" \    部署3个节点的内网ip一定要替换

> cluster-health

如果出现下面字段,说明集群状态是健康的

member 37f20611ff3d9209 is healthy: got healthy result from https://192.168.31.63:2379

member b10f0bac3883a232 is healthy: got healthy result from https://192.168.31.64:2379

member b46624837acedac9 is healthy: got healthy result from https://192.168.31.65:2379

cluster is healthy

1.部署Master Node

1.1 生成apiserver证书

# cd TLS/k8s

 

修改请求文件中hosts字段包含所有etcd节点IP:

# vi server-csr.json

{

    "CN": "kubernetes",

    "hosts": [

      "10.0.0.1",

      "127.0.0.1",

      "kubernetes",

      "kubernetes.default",

      "kubernetes.default.svc",

      "kubernetes.default.svc.cluster",

      "kubernetes.default.svc.cluster.local",

      "192.168.31.60",    你的内网ip

      "192.168.31.61",

      "192.168.31.62",

      "192.168.31.63",

      "192.168.31.64",

      "192.168.31.65",

      "192.168.31.66"

    ],

    "key": {

        "algo": "rsa",

        "size": 2048

    },

    "names": [

        {

            "C": "CN",

            "L": "BeiJing",

            "ST": "BeiJing",

            "O": "k8s",

            "OU": "System"

        }

    ]

}

 

# ./generate_k8s_cert.sh

# ls *pem

ca-key.pem  ca.pem  kube-proxy-key.pem  kube-proxy.pem  server-key.pem  server.pem

 

3.2 部署apiserver,controller-manager和scheduler

Master节点完成以下操作。

 

二进制包下载地址:https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.16.md#v1161

 master安装包下载地址:链接:https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg 
提取码:o101 

二进制文件位置:kubernetes/serverr/bin

 

# tar zxvf k8s-master.tar.gz

# cd kubernetes

# cp TLS/k8s/ssl/*.pem ssl

# cp  –r  kubernetes /opt

# cp kube-apiserver.service kube-controller-manager.service kube-scheduler.service /usr/lib/systemd/system

 

# cat /opt/kubernetes/cfg/kube-apiserver.conf

KUBE_APISERVER_OPTS="--logtostderr=false \

--v=2 \

--log-dir=/opt/kubernetes/logs \

--etcd-servers=https://192.168.31.63:2379,https://192.168.31.64:2379,https://192.168.31.65:2379 \  替换etcd节点的内网ip

--bind-address=192.168.31.63 \ 替换master节点的ip

--secure-port=6443 \

--advertise-address=192.168.31.63 \ 替换master节点的ip

……

 

# systemctl start kube-apiserver

# systemctl start kube-controller-manager

# systemctl start kube-scheduler

# systemctl enable kube-apiserver

# systemctl enable kube-controller-manager

# systemctl enable kube-scheduler

# systemctl  start kube-apiserver

# ls /opt/kubernetes/logs  查看日志

# less /opt/kubernetes/logs/kube-apiserver.INFO

# tail -f /opt/kubernetes/logs/kube-controller-manager.INFO

# for i in $(ls /opt/kubernetes/bin);do systemctl enable $i;done开机启动

# mv /opt/kubernetes/bin/kubectl  /usr/local/bin/kubectl 移动到环境变量

# chmod a+x /usr/local/bin/kubect

# kubectl get cs查看组件状态

# 查看3个组件的进程 ps -ef|grep kube

3.3 启用TLS Bootstrapping

kubelet TLS Bootstrapping 授权

 

# cat /opt/kubernetes/cfg/token.csv

c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper"

 

格式:token,用户,uid,用户组

 

kubelet-bootstrap授权:

 

kubectl create clusterrolebinding kubelet-bootstrap \

--clusterrole=system:node-bootstrapper \

--user=kubelet-bootstrap

 

token也可自行生成替换:

 

head -c 16 /dev/urandom | od -An -t x | tr -d ' '

 

apiserver配置的token必须要与node节点bootstrap.kubeconfig配置里一致。

1.部署Worker Node

1.1 安装Docker

二进制包下载地址:https://download.docker.com/linux/static/stable/x86_64/

docker下载地址: 链接:https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg 
提取码:o101 

# tar zxvf k8s-node.tar.gz

# tar zxvf docker-18.09.6.tgz

# mv docker/* /usr/bin

# mkdir /etc/docker

# mv daemon.json /etc/docker

# mv docker.service /usr/lib/systemd/system

# systemctl start docker

# systemctl enable docker

# docker info 通过docker info查看docker是否启动成功

 

 

 执行docker info出现如下警告

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled


解决办法:

vi /etc/sysctl.conf


添加以下内容

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1


最后再执行

sysctl -p


此时docker info就看不到此报错了

执行docker info出现如下警告

 

4.2 部署kubelet和kube-proxy

拷贝证书到Node:

 

# cd TLS/k8s

# scp ca.pem kube-proxy*.pem root@192.168.31.65:/opt/kubernetes/ssl/

# cp kube-apiserver.service kube-controller-manager.service kube-

# tar zxvf k8s-node.tar.gz

# mv kubernetes /opt

# cp kubelet.service kube-proxy.service /usr/lib/systemd/system

 

查看以下三个文件中IP地址:

[root@k8s-node2 kubernetes]# grep 192 * 

 修改以下两个文件中主机名:

[root@k8s-node2 cfg]# vim bootstrap.kubeconfig

 [root@k8s-node2 cfg]# vim kubelet.conf

 

 [root@k8s-node2 cfg]# vim kubelet.kubeconfig

[root@k8s-node2 cfg]# vim kube-proxy-config.yml

 

[root@k8s-node2 cfg]# vim kube-proxy.kubeconfig

 

 

 

 

# systemctl start kubelet

# systemctl start kube-proxy

# systemctl enable kubelet

# systemctl enable kube-proxy

# tail /opt/kubernetes/logs/kubelet.INFO 查看日志

 

4.3 允许给Node颁发证书

# kubectl get csr

 

 

# kubectl certificate approve node-csr-MYUxbmf_nmPQjmH3LkbZRL2uTO-_FCzDQUoUfTy7YjI  替换你的node名称

# kubectl get node 

 

 

 

 

 

 

 

posted @ 2020-02-10 12:27  taotao12  阅读(243)  评论(0编辑  收藏  举报