K8s笔记

k8s

K8s笔记
1.启动docker容器
docker run -d -it --privileged --name=hadoop02 --hostname=hadoop02 centos:7 /usr/sbin/init
如果要是用systemctl 管理服务就要加上参数 --privileged 来增加权，并且不能使用默认的bash，换成init。

2.进入容器
docker exec -it hadoop02 /bin/bash

3.安装k8s
yum install -y etcd kubernetes
yum install -y iptables

4.修改配置
vi /etc/sysconfig/docker
OPTIONS='--selinux-enabled=false --insecure-registry gcr.io'
vi /etc/kubernetes/apiserver 
删除ServeiceAccount

5.运行程序
systemctl start etcd
systemctl start docker   --->启动失败，可能忘记安装iptables
systemctl start kube-apiserver   --->修改配置/etc/kubernetes/apiserver，KUBE_API_ADDRESS改成本机ip
                                             /etc/kubernetes/config,KUBE_MASTER改成本机ip
                                             /etc/kubernetes/kubelet,KUBELET_HOSTNAME、KUBELET_API_SERVER改成本机ip
systemctl start kube-controller-manager
systemctl start kube-scheduler
systemctl start kubelet
systemctl start kube-proxy

6.mysql-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:5
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "123456"

7.发布到k8s集群中
kubectl create -f mysql-rc.yml   
报错：The connection to the server localhost:8080 was refused - did you specify the right host or port?
解决：alias kubectl="kubectl -s http://k8s:8080"
k8s是本机名

8.查看创建的RC
kubectl get rc
kubectl get pods  --->一直ContainerCreating状态，使用kubectl describe pod mysql

报错：Back-off pulling image \"registry.access.redhat.com/rhel7/pod-infrastructure:latest\""
下载不到镜像registry.access.redhat.com/rhel7/pod-infrastructure:latest，用别的代替
解决：docker search pod-infrastructure
      docker pull docker.io/tianyebj/pod-infrastructure
      docker tag docker.io/tianyebj/pod-infrastructure registry.access.redhat.com/rhel7/pod-infrastructure:latest

9.mysql-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  ports:
    - port: 3306
  selector:
    app: mysql


安装mysql客户端
rpm -ivh https://repo.mysql.com//mysql57-community-release-el7-11.noarch.rpm
yum install mysql-community-client.x86_64


10.myweb-rc.yaml
kind: ReplicationController
metadata:
  name: myweb
spec:
  replicas: 5
  selecotr:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
        - name: myweb
          image: kubeguide/tomcat-app:v1
          ports:
          - containerPort: 8080
          env:
          - name: MYSQL_SERVICE_HOST
            value: 'mysql'
          - name: MYSQL_SERVICE_PORT
            value: '3306'

11.myweb-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: myweb
spec:
  type: NodePort
  ports:
    - port: 8080
      nodePort: 30001
  selector:
    app: myweb


报错：Error:com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
原因：镜像mysql:lastest的版本是8.0.18
      myweb容器中使用的mysql驱动是mysql-connector-java-5.1.37.jar
      MySQL8.0版本需要更换驱动为“com.mysql.cj.jdbc.Driver”，之前的“com.mysql.jdbc.Driver”已经不能在MySQL 8.0版本使用了
解决：docker pull mysql:5


Deployment
tomcat-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: frontend
    matchExpressions:
      - {key: tier, operator: In, values: [frontend]}
  template:
    metadata:
      labels:
        app: app-demo
        tier: frontend
    spec:
      containers:
      - name: tomcat-demo
        image: tomcat
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080

 

二进制文件部署
Master
1.下载etcd
https://github.com/etcd-io/etcd/releases/tag/v3.3.18
etcd-v3.3.18-linux-amd64.tar.gz
宿主机创建/data/zkdocker/k8s/cdroom
docker run -it --privileged --name=k8smaster --hostname=k8smaster -v /data/zkdocker/k8s/cdroom:/dev/shm -p 30001-30010:30001-30010 centos:7 /usr/sbin/init

docker run -it --privileged --name=k8snode --hostname=k8snode -v /data/zkdocker/k8s/cdroom:/dev/shm -p 31001-31010:31001-31010 centos:7 /usr/sbin/init
格式化查看
docker ps -a --format "table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.Names}}\t{{.Status}}"
进入容器master
docker exec -it k8smaster /bin/bash
上传etcd压缩包到容器内，解压
将etcd和etcdctl复制到/usr/bin目录下
配置/usr/lib/systemd/system/etcd.service
==================================================
[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd

[Install]
WantedBy=multi-user.target
==================================================
注意：要先创建/var/lib/etcd/，否则启动不了
将etcd加入开机自启
systemctl daemon-reload
systemctl enable etcd.service
systemctl start etcd.service
验证etcd是否启动正确
etcdctl cluster-health

2.kube-apiserver服务
cp kubernetes-1.3.0/cluster/ubuntu/master/init_scripts/kube-apiserver /usr/bin/
配置/usr/lib/systemd/system/kube-apiserver.service
====================================
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service

[Service]
Type=notify
LimitNOFILE=65536
Restart=on-failure
EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS

[Install]
WantedBy=multi-user.target
====================================
配置/etc/kubernetes/apiserver
===================
KUBE_API_ARGS="--etcd_servers=http://127.0.0.1:2379 --insecure-bind-address=0.0.0.0 --insecure-port=8080 --service-cluster-ip-range=169.169.0.0/16 --service-node-port-range=1-65535 --admission_control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
===================

3.kube-controller-manager服务
cp kubernetes-1.3.0/cluster/ubuntu/master/init_scripts/kube-controller-manager /usr/bin/
配置/usr/lib/systemd/system/kube-controller-manager.service
====================================
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
LimitNOFILE=65536
Restart=on-failure
EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS

[Install]
WantedBy=multi-user.target
====================================
配置/etc/kubernetes/controller-manager
===================
KUBE_CONTROLLER_MANAGER_ARGS="--master=http://172.17.0.4:8080 --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
===================

4.kube-scheduler服务
配置/usr/lib/systemd/system/kube-scheduler.service
====================================
[Unit]
Description=Kubernetes Scheduler Manager
Documentation=https://github.com/kubernetes/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
LimitNOFILE=65536
Restart=on-failure
EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS

[Install]
WantedBy=multi-user.target
====================================
配置/etc/kubernetes/scheduler
===================
KUBE_SCHEDULER_ARGS="--master=http://172.17.0.4:8080 --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
===================

完成后，开机自动启动设置
systemctl daemon-reload
systemctl enable kube-apiserver.service
systemctl start kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl start kube-controller-manager.service
systemctl enable kube-scheduler
systemctl start kube-scheduler

检查状态
systemctl status kube-apiserver


Node上的服务
5.kubelet服务
配置/usr/lib/systemd/system/kubelet.service
====================================
[Unit]
Description=Kubernetes Kubelete Server
Documentation=https://github.com/kubernetes/kubernetes
After=docker.service
Requires=docker.service
 
 
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
 
[Install]
WantedBy=multi-user.target
====================================
配置/etc/kubernetes/kubelet
===================
KUBELET_ARGS="--api-server=http://172.17.0.4:8080 --hostname-override=172.17.0.4 --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
===================
ip为node的地址

5.kube-proxy服务
配置/usr/lib/systemd/system/kube-proxy.service
====================================
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target
 
[Service]
EnvironmentFile=/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65535
 
[Install]
WantedBy=multi-user.target
====================================
配置/etc/kubernetes/kube-proxy
===================
KUBE_PROXY_ARGS="--master=http://172.17.0.4:8080 --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
===================

启动
systemctl start kubelet



基于CA签名的双向数字证书认证方式
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=yourcompany.com" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048
vi master_ssl.cnf
===================================
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = k8smaster
IP.1 = 169.169.0.1
IP.2 = 172.17.0.4
===================================
IP.1---》是k8s集群ip
IP.2---》是masterip

openssl req -new -key server.key -subj "/CN=k8smaster" -config master_ssl.cnf -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt
当前目录下有六个文件：ca.key,ca.srl,ca.crt,server.key,server.crt,server.csr
cp ca* /var/run/kubernetes/
cp server* /var/run/kubernetes/
配置kube-apiserver，添加以下参数
--client_ca_file=/var/run/kubernetes/ca.crt
--tls-private-key-file=/var/run/kubernetes/server.key
--tls-cert-file=/var/run/kubernetes/server.crt
--insecure-port=0
--secure-port=443

设置kube-controller-manager的客户端




--master=https://172.17.0.4:443
--service_account_private_key_file=/var/run/kubernetes/server.key
--root-ca-file=/var/run/kubernetes/ca.crt
--kubeconfig=/etc/kubernetes/kubeconfig


--master=https://172.17.0.4:443
--kubeconfig=/etc/kubernetes/kubeconfig



Node端配置（每个节点都按此操作）
1.拷贝master端生产的ca.crt,ca.key
    文件拷贝在/dev/shm目录下
2.生成证书，第二步的ip换成node地址，
    openssl genrsa -out kubelet_client.key 2048
    openssl req -new -key kubelet_client.key -subj "/CN=172.17.0.5" -out kubelet_client.csr
    openssl x509 -req -in kubelet_client.csr -CA /dev/shm/ca.crt -CAkey /dev/shm/ca.key -CAcreateserial -out kubelet_client.crt -days 5000
3.将生成的这些证书拷贝到/var/run/kubernetes/
    cp kubelet_client* /var/run/kubernetes/
    cp /dev/shm/ca* /var/run/kubernetes/
4.修改/etc/kubernetes/kubelet
    KUBELET_ADDRESS="--address=172.17.0.7"
    KUBELET_HOSTNAME="--hostname-override=k8snode2"
    KUBELET_API_SERVER="--api-servers=https://172.17.0.4:443"
    KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig"
5.修改/etc/kubernetes/config
    KUBE_MASTER="--master=https://172.17.0.4:443"
6.启动
    systemctl start kubelet

服务端查看节点
kubectl --server=https://172.17.0.4:443 --certificate-authority=/var/run/kubernetes/ca.crt --client-certificate=/var/run/kubernetes/server.crt --client-key=/var/run/kubernetes/server.key get nodes