linux的cgroup理论:https://www.cnblogs.com/menkeyi/p/10941843.html

Linux CGroup全称Linux Control Group, 是Linux内核的一个功能,用来限制,控制与分离一个进程组群的资源。

容器的资源限制就是基于linux的cgroup实现。

docker版本:1.13.1(可通过 docker info查看)

1.实践linux cgroup

[crmop@localhost ~]$ mount -t cgroup ##查看cgroup挂载,子目录(cpuset即cpu核数,cpu,memory)表示可以被cgroup限制的资源种类
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)


[root@localhost ~]# cd /sys/fs/cgroup [root@localhost cgroup]# ls ##子目录(cpuset即cpu核数,cpu,memory,blkio块设置io)表示可以被cgroup限制的资源种类 blkio cpuacct cpuset freezer memory net_cls,net_prio perf_event rdma cpu cpu,cpuacct devices hugetlb net_cls net_prio pids systemd [root@localhost cgroup]# cd cpu ##对cpu子系统来说,提供具体可以被限制的方法,如cfs_period_us与cfs_quota_us组合,
##表示在cfs_period_us时间长度内分得cfs_quota_us的cpu时间,即限制cpu的最高使用率为cfs_quota_us/cfs_period_us*100% [root@localhost cpu]#
ls cgroup.clone_children cpuacct.usage cpuacct.usage_percpu_user cpu.cfs_quota_us cpu.stat cgroup.procs cpuacct.usage_all cpuacct.usage_sys cpu.rt_period_us notify_on_release cgroup.sane_behavior cpuacct.usage_percpu cpuacct.usage_user cpu.rt_runtime_us release_agent cpuacct.stat cpuacct.usage_percpu_sys cpu.cfs_period_us cpu.shares tasks [root@localhost cpu]# ll total 0 -rw-r--r-- 1 root root 0 Aug 15 00:55 cgroup.clone_children -rw-r--r-- 1 root root 0 Jun 4 10:18 cgroup.procs -r--r--r-- 1 root root 0 Aug 15 00:55 cgroup.sane_behavior -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.stat -rw-r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_all -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu_sys -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu_user -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_sys -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_user -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.cfs_period_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.cfs_quota_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.rt_period_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.rt_runtime_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.shares -r--r--r-- 1 root root 0 Aug 15 00:55 cpu.stat -rw-r--r-- 1 root root 0 Aug 15 00:55 notify_on_release -rw-r--r-- 1 root root 0 Aug 15 00:55 release_agent -rw-r--r-- 1 root root 0 Aug 15 00:55 tasks [root@localhost cpu]# mkdir container ##这个目录称为一个“控制组”,在该目录下,自动生成该子系统对应的资源限制文件,
##删除时先yum -y install libcgroup再cgdelete cpu:container [root@localhost cpu]# ll total
0 -rw-r--r-- 1 root root 0 Aug 15 00:55 cgroup.clone_children -rw-r--r-- 1 root root 0 Jun 4 10:18 cgroup.procs -r--r--r-- 1 root root 0 Aug 15 00:55 cgroup.sane_behavior drwxr-xr-x 2 root root 0 Aug 15 00:56 container -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.stat -rw-r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_all -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu_sys -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_percpu_user -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_sys -r--r--r-- 1 root root 0 Aug 15 00:55 cpuacct.usage_user -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.cfs_period_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.cfs_quota_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.rt_period_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.rt_runtime_us -rw-r--r-- 1 root root 0 Aug 15 00:55 cpu.shares -r--r--r-- 1 root root 0 Aug 15 00:55 cpu.stat -rw-r--r-- 1 root root 0 Aug 15 00:55 notify_on_release -rw-r--r-- 1 root root 0 Aug 15 00:55 release_agent -rw-r--r-- 1 root root 0 Aug 15 00:55 tasks [root@localhost cpu]# cd container/ [root@localhost container]# ls cgroup.clone_children cpuacct.usage cpuacct.usage_percpu_sys cpuacct.usage_user cpu.rt_period_us cpu.stat cgroup.procs cpuacct.usage_all cpuacct.usage_percpu_user cpu.cfs_period_us cpu.rt_runtime_us notify_on_release cpuacct.stat cpuacct.usage_percpu cpuacct.usage_sys cpu.cfs_quota_us cpu.shares tasks [root@localhost container]# ls cgroup.clone_children cpuacct.usage cpuacct.usage_percpu_sys cpuacct.usage_user cpu.rt_period_us cpu.stat cgroup.procs cpuacct.usage_all cpuacct.usage_percpu_user cpu.cfs_period_us cpu.rt_runtime_us notify_on_release cpuacct.stat cpuacct.usage_percpu cpuacct.usage_sys cpu.cfs_quota_us cpu.shares tasks [root@localhost container]# cat cpu.cfs_period_us 100000 [root@localhost container]# cat cpu.cfs_quota_us ## -1表示不作限制 -1 [root@localhost container]# cat tasks [root@localhost container]# top top - 00:58:19 up 2 days, 7:05, 2 users, load average: 0.01, 0.03, 0.00 Tasks: 187 total, 1 running, 133 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 977804 total, 149268 free, 608916 used, 219620 buff/cache KiB Swap: 1048572 total, 1031848 free, 16724 used. 178692 avail Mem

写一个死循环且不作限制,cpu被打满

[root@localhost container]# while : ; do : ; done &
[1] 35789
[root@localhost container]# top
top - 00:59:04 up 2 days,  7:05,  2 users,  load average: 0.16, 0.06, 0.01
Tasks: 187 total,   2 running, 132 sleeping,   0 stopped,   0 zombie
%Cpu(s):100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   977804 total,   151268 free,   606924 used,   219612 buff/cache
KiB Swap:  1048572 total,  1031848 free,    16724 used.   180692 avail Mem 


PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
35789 root 20 0 116376 1328 0 R 99.3 0.1 0:13.47 bash

作资源隔离限制

[root@localhost container]# pwd
/sys/fs/cgroup/cpu/container
[root@localhost container]# cat cpu.cfs_period_us
100000
[root@localhost container]# cat cpu.cfs_quota_us
-1
[root@localhost container]# echo 20000 > cpu.cfs_quota_us ##及限制cpu最高作用率为20%
[root@localhost container]# cat cpu.cfs_quota_us
20000
[root@localhost container]# cat tasks 
[root@localhost container]# echo 35798 > tasks ## 将该进程对应的pid写入对应控制组container的tasks文件
[root@localhost container]# cat tasks 
35798
[root@localhost container]# top
top - 01:04:12 up 2 days,  7:11,  2 users,  load average: 0.80, 0.46, 0.19
Tasks: 189 total,   2 running, 132 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.9 us,  0.4 sy,  0.0 ni, 86.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   977804 total,   149876 free,   608284 used,   219644 buff/cache
KiB Swap:  1048572 total,  1031848 free,    16724 used.   179316 avail Mem 

   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                         
 35798 root      20   0  116376   1328      0 R 19.9  0.1   2:16.42 bash                                                            
  1157 gdm       20   0 2975120 186892  26480 S  1.0 19.1   3:46.17 gnome-shell  

 

2.docker实践

启动一个容器 docker启动busybox

-i表示输入有效。

-t表示提供一个伪终端,用于我们输入内容。

[root@localhost container]# docker run -it --cpu-period=100000 --cpu-quota=20000 --name my-busybox1 docker.io/busybox  /bin/sh 
/ # top
Mem: 841296K used, 136508K free, 18500K shrd, 0K buff, 151804K cached
CPU:  0.0% usr 10.0% sys  0.0% nic 90.0% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 0.00 0.00 0.02 2/365 6
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    1     0 root     S     1308  0.1   0  0.0 /bin/sh
    6     1 root     R     1304  0.1   0  0.0 top
Mem: 842244K used, 135560K free, 18500K shrd, 0K buff, 151892K cached
CPU:  1.2% usr  0.6% sys  0.0% nic 97.5% idle  0.6% io  0.0% irq  0.0% sirq
Load average: 0.00 0.00 0.02 3/365 6
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    1     0 root     S     1308  0.1   0  0.0 /bin/sh
    6     1 root     R     1304  0.1   0  0.0 top
Mem: 842276K used, 135528K free, 18500K shrd, 0K buff, 151892K cached
CPU:  0.4% usr  0.4% sys  0.0% nic 99.1% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 0.00 0.00 0.02 2/365 6
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    1     0 root     S     1308  0.1   0  0.0 /bin/sh
    6     1 root     R     1304  0.1   0  0.0 top
/ # ps -ef
PID   USER     TIME  COMMAND
    1 root      0:00 /bin/sh
    7 root      0:00 ps -ef

退出后重新启动已暂停的容器

docker ps查看正在运行的容器列表

[root@localhost container]# docker ps -a ##查看已经暂停的容器
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                       PORTS               NAMES
9b8c4b4cce0d        docker.io/busybox   "/bin/sh"           About an hour ago   Exited (130) 6 minutes ago                       my-busybox1
[root@localhost container]# docker start 9b8c4b4cce0d ## 启动
9b8c4b4cce0d
[root@localhost container]# 

进入容器,加入一个死循环,容器内查看cpu使用率

[root@localhost cpu]# docker exec -it 9b8c4b4cce0d /bin/sh
/ # while : ; do : ; done &
/ # top
Mem: 869356K used, 108448K free, 18536K shrd, 0K buff, 167576K cached
CPU: 20.0% usr  0.0% sys  0.0% nic 80.0% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 0.03 0.02 0.00 3/374 12
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
   11     6 root     R     1308  0.1   0 20.0 /bin/sh
    6     0 root     S     1312  0.1   0  0.0 /bin/sh
    1     0 root     S     1308  0.1   0  0.0 /bin/sh
   12     6 root     R     1304  0.1   0  0.0 top

容器外查看,说明容器内的top命令查看的资源是宿主机上,验证了容器只是挂载文件目录,内核还是用的宿主机操作系统内核

同时37732进程即为容器内死循环的进程,说明容器本质就是一个特殊进程

[root@localhost ~]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
9b8c4b4cce0d        docker.io/busybox   "/bin/sh"           About an hour ago   Up 8 minutes                            my-busybox1
[root@localhost ~]# top
top - 02:56:52 up 2 days,  9:03,  2 users,  load average: 0.04, 0.02, 0.00
Tasks: 197 total,   5 running, 141 sleeping,   0 stopped,   0 zombie
%Cpu(s): 18.8 us,  0.3 sy,  0.0 ni, 80.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   977804 total,    84432 free,   683024 used,   210348 buff/cache
KiB Swap:  1048572 total,   999976 free,    48596 used.   133088 avail Mem 

   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                         
 37732 root      20   0    1308     60      4 R 19.9  0.0   0:36.01 sh                                                              
  1164 mysql     20   0 1119840 156260      0 S  0.3 16.0   2:22.78 mysqld       

在容器外的cgroup并找到了相关的控制组/sys/fs/cgroup/cpu/system.slice/docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope,但在容器内找到对应的限制pid 11

[root@localhost cpu]# docker ps ##查看宿主机上docker进程
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
9b8c4b4cce0d        docker.io/busybox   "/bin/sh"           About an hour ago   Up 40 minutes                           my-busybox1
[root@localhost cpu]# docker container top 9b8c4b4cce0d ##37609为docker run,37732运行死循环程序的进程,父进程ppid为37609,其它进程为docker exec命令进入已经运行的同一容器运行的命令
##即同一运行中的容器,可以被不同的连接(进程37609,38519,它们并没有新生成容器)进入
UID PID PPID C STIME TTY TIME CMD root
37609 37593 0 02:47 pts/0 00:00:00 /bin/sh root 37732 37609 19 02:53 ? 00:06:59 /bin/sh root 38159 38142 0 03:09 pts/4 00:00:00 /bin/sh

 


[root@localhost cpu]# top
top - 03:17:02 up 2 days,  9:23,  2 users,  load average: 0.41, 0.32, 0.21
Tasks: 198 total,   2 running, 143 sleeping,   0 stopped,   0 zombie
%Cpu(s): 16.7 us,  5.6 sy,  0.0 ni, 72.2 id,  5.6 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   977804 total,    83188 free,   686168 used,   208448 buff/cache
KiB Swap:  1048572 total,   972028 free,    76544 used.   106920 avail Mem 

   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                         
 37732 root      20   0    1308     56      0 R 12.5  0.0   4:37.74 sh                                                              
 38908 root      20   0  162028   4580   3864 R  6.2  0.5   0:00.02 top                                                             
     1 root      20   0  191316   3152   1524 S  0.0  0.3   0:09.62 systemd                                                         
     2 root      20   0       0      0      0 S  0.0  0.0   0:00.05 kthreadd                                                        
     3 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 rcu_gp                                                          
     4 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 rcu_par_gp                                                      
     6 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 kworker/0:0H-kb                                                 
     8 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 mm_percpu_wq                                                    
     9 root      20   0       0      0      0 S  0.0  0.0   0:08.29 ksoftirqd/0                                                     
    10 root      20   0       0      0      0 I  0.0  0.0   0:06.56 rcu_sched                                                       
    11 root      rt   0       0      0      0 S  0.0  0.0   0:01.99 migration/0                                                     
    13 root      20   0       0      0      0 S  0.0  0.0   0:00.00 cpuhp/0                                                         
    14 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kdevtmpfs                                                       
    15 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 netns                                                           
    16 root      20   0       0      0      0 S  0.0  0.0   0:00.03 kauditd                                                         
    17 root      20   0       0      0      0 S  0.0  0.0   0:00.20 khungtaskd                                                      
    18 root      20   0       0      0      0 S  0.0  0.0   0:00.00 oom_reaper                                                      
    19 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 writeback                                                       
    20 root      20   0       0      0      0 S  0.0  0.0   0:00.59 kcompactd0                                                      
    21 root      25   5       0      0      0 S  0.0  0.0   0:00.00 ksmd                                                            
    22 root      39  19       0      0      0 S  0.0  0.0   0:04.96 khugepaged                                                      
   112 root       0 -20       0      0      0 I  0.0  0.0   0:00.00 kintegrityd   
[root@localhost cpu]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
9b8c4b4cce0d        docker.io/busybox   "/bin/sh"           About an hour ago   Up 22 minutes                           my-busybox1
[root@localhost cpu]# cd /sys/fs/cgroup/ 
[root@localhost cgroup]#
ls
blkio cpuacct cpuset freezer memory net_cls,net_prio perf_event rdma cpu cpu,cpuacct devices hugetlb net_cls
net_prio pids systemd
[root@localhost cgroup]# cd cpu
[root@localhost cpu]# find . -name tasks|xargs grep 37609 ##在容器外37609为docker run进程即容器本身进程,找到控制组
./system.slice/docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope/tasks:37609
[root@localhost cpu]# cd
system.slice/docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope
[root@localhost docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope]# pwd
/sys/fs/cgroup/cpu/system.slice/docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope
[root@localhost docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope]# ls
cgroup.clone_children  cpuacct.usage         cpuacct.usage_percpu_sys   cpuacct.usage_user  cpu.rt_period_us   cpu.stat
cgroup.procs           cpuacct.usage_all     cpuacct.usage_percpu_user  cpu.cfs_period_us   cpu.rt_runtime_us  notify_on_release
cpuacct.stat           cpuacct.usage_percpu  cpuacct.usage_sys          cpu.cfs_quota_us    cpu.shares         tasks
[root@localhost docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope]# cat tasks
37609
[root@localhost docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope]# cat cpu.cfs_quota_us
20000
[root@localhost docker-9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62.scope]# cat cpu.cfs_period_us
100000
 

进入容器内

[root@localhost ~]# docker exec -it 9b8c4b4cce0d /bin/sh ##进入容器
/ # cd /sys/fs/cgroup/
/sys/fs/cgroup # ls
blkio             cpuacct           freezer           net_cls           perf_event        systemd
cpu               cpuset            hugetlb           net_cls,net_prio  pids
cpu,cpuacct       devices           memory            net_prio          rdma
/sys/fs/cgroup # cd cpu
/sys/fs/cgroup/cpu,cpuacct # ls
cgroup.clone_children      cpu.rt_runtime_us          cpuacct.usage_all          cpuacct.usage_user
cgroup.procs               cpu.shares                 cpuacct.usage_percpu       notify_on_release
cpu.cfs_period_us          cpu.stat                   cpuacct.usage_percpu_sys   tasks
cpu.cfs_quota_us           cpuacct.stat               cpuacct.usage_percpu_user
cpu.rt_period_us           cpuacct.usage              cpuacct.usage_sys
/sys/fs/cgroup/cpu,cpuacct # cd ..
/sys/fs/cgroup # ls
blkio             cpuacct           freezer           net_cls           perf_event        systemd
cpu               cpuset            hugetlb           net_cls,net_prio  pids
cpu,cpuacct       devices           memory            net_prio          rdma
/sys/fs/cgroup # cd cpu
/sys/fs/cgroup/cpu,cpuacct # ls
cgroup.clone_children      cpu.rt_runtime_us          cpuacct.usage_all          cpuacct.usage_user
cgroup.procs               cpu.shares                 cpuacct.usage_percpu       notify_on_release
cpu.cfs_period_us          cpu.stat                   cpuacct.usage_percpu_sys   tasks
cpu.cfs_quota_us           cpuacct.stat               cpuacct.usage_percpu_user
cpu.rt_period_us           cpuacct.usage              cpuacct.usage_sys
/sys/fs/cgroup/cpu,cpuacct # ll
/bin/sh: ll: not found
/sys/fs/cgroup/cpu,cpuacct # ls 
cgroup.clone_children      cpu.rt_runtime_us          cpuacct.usage_all          cpuacct.usage_user
cgroup.procs               cpu.shares                 cpuacct.usage_percpu       notify_on_release
cpu.cfs_period_us          cpu.stat                   cpuacct.usage_percpu_sys   tasks
cpu.cfs_quota_us           cpuacct.stat               cpuacct.usage_percpu_user
cpu.rt_period_us           cpuacct.usage              cpuacct.usage_sys
/sys/fs/cgroup/cpu,cpuacct # ls -l
total 0
-rw-r--r--    1 root     root             0 Aug 15 10:10 cgroup.clone_children
-rw-r--r--    1 root     root             0 Aug 15 10:09 cgroup.procs
-rw-r--r--    1 root     root             0 Aug 15 09:47 cpu.cfs_period_us
-rw-r--r--    1 root     root             0 Aug 15 09:47 cpu.cfs_quota_us
-rw-r--r--    1 root     root             0 Aug 15 10:10 cpu.rt_period_us
-rw-r--r--    1 root     root             0 Aug 15 10:10 cpu.rt_runtime_us
-rw-r--r--    1 root     root             0 Aug 15 09:47 cpu.shares
-r--r--r--    1 root     root             0 Aug 15 10:10 cpu.stat
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.stat
-rw-r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_all
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_percpu
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_percpu_sys
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_percpu_user
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_sys
-r--r--r--    1 root     root             0 Aug 15 10:10 cpuacct.usage_user
-rw-r--r--    1 root     root             0 Aug 15 10:10 notify_on_release
-rw-r--r--    1 root     root             0 Aug 15 10:10 tasks
/sys/fs/cgroup/cpu,cpuacct # cat tasks 
1
11
13
24
/sys/fs/cgroup/cpu,cpuacct # top
Mem: 897488K used, 80316K free, 18616K shrd, 0K buff, 181188K cached
CPU: 27.2% usr  0.0% sys  0.0% nic 72.7% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 0.40 0.34 0.18 4/389 25
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
   11     1 root     R     1308  0.1   0 27.2 /bin/sh
   13     0 root     S     1316  0.1   0  0.0 /bin/sh
    1     0 root     S     1308  0.1   0  0.0 /bin/sh
   25    13 root     R     1304  0.1   0  0.0 top
/sys/fs/cgroup/cpu,cpuacct # 
/sys/fs/cgroup/cpu,cpuacct # 
/sys/fs/cgroup/cpu,cpuacct # ll
/bin/sh: ll: not found
/sys/fs/cgroup/cpu,cpuacct # ls
cgroup.clone_children      cpu.rt_runtime_us          cpuacct.usage_all          cpuacct.usage_user
cgroup.procs               cpu.shares                 cpuacct.usage_percpu       notify_on_release
cpu.cfs_period_us          cpu.stat                   cpuacct.usage_percpu_sys   tasks
cpu.cfs_quota_us           cpuacct.stat               cpuacct.usage_percpu_user
cpu.rt_period_us           cpuacct.usage              cpuacct.usage_sys
/sys/fs/cgroup/cpu,cpuacct # cat cpu.cfs_quota_us ##发现在输入的限制参数
20000
/sys/fs/cgroup/cpu,cpuacct # cat cpu.cfs_period_us
100000
/sys/fs/cgroup/cpu,cpuacct # cat tasks ##发现容器内执行死循环的进程号11,被限制了
1
11
13
29
/sys/fs/cgroup/cpu,cpuacct # 

 

比较容器内外top命令结果,发现docker内部top命令展示的资源为宿主机上的资源信息(是宿主机上/proc信息,并不是限制后的容器内部资源使用率,容器内部cpu应为100%),原因为docker隔离不彻底造成,要解决此问题可参照lxcfs https://www.jianshu.com/p/c99611bffe6f

ps:
pid=1的进程为 docker run -it --cpu-period=100000 --cpu-quota=20000 --name my-busybox1 docker.io/busybox /bin/sh 命令进程
pid=11具父进程为1的进程是我在 上一条docker run -it 进入容器后运行 while : ; do : ; done &的进程,cpu使用率为20%。
pid=13命令为docker exec -it 9b8c4b4cce0d /bin/sh,的结果进程,进入一个正在运行的进程。
pid=32为docker exec -it后运行top命令的进程。

 在宿主机上查看容器进程,docker进程=containerd+runc(namespace和cgroup)

[root@localhost ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9b8c4b4cce0d docker.io/busybox "/bin/sh" 20 hours ago Up 19 hours my-busybox1
[root@localhost ~]# docker top my-busybox1
UID PID PPID C STIME TTY TIME CMD
root 37609(对应容器进程pid=1 docker run运行容器在宿主机上的进程号,单进程模式要求只应该存在此进程,即不要再容器内运行多于一个应用) 37593 0 19:45 pts/0 00:00:00 /bin/sh 
root
37732(对应容器进程pid=11,运行死循环的那个进程,上面图片说明有误) 37609 19 19:51 ? 00:27:58 /bin/sh
root
38159(对应容器进程pid=13) 38142 0 20:07 pts/4 00:00:00 /bin/sh
root 39206(对应容器进程pid=32) 38159 0 20:38 pts/4 00:00:01 top

[root@localhost cpu]# ps -ef|grep 37609 ##进程追踪
root 37609   1657 0 22:46 pts/1 00:00:00 /bin/sh
[root@localhost cpu]# ps -ef|grep 1657
root 1657 1111 0 22:46 ? 00:00:00 /usr/bin/docker-containerd-shim-current 9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62 /var/run/docker/libcontainerd/9b8c4b4cce0d64476ec3370dc1c0a264054a86eb52440d63ccdaee1a18562b62 /usr/libexec/docker/docker-runc-current
[root@localhost cpu]# ps -ef|grep 1111
root 1111 1055 0 22:44 ? 00:00:03 /usr/bin/docker-containerd-current -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc --runtime-args --systemd-cgroup=true
[root@localhost cpu]# ps -ef|grep 1055
root 1055 1 0 22:44 ? 00:00:05 /usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/docker/docker-proxy-current --init-path=/usr/libexec/docker/docker-init-current --seccomp-profile=/etc/docker/seccomp.json --selinux-enabled --log-driver=journald --signature-verification=false --storage-driver overlay2
[root@localhost cpu]# ps -ef|grep 1
root 1 0 0 22:44 ? 00:00:02 /usr/lib/systemd/systemd --switched-root --system --deserialize 22

 

 

 3.结论:

docker的资源限制cgroup是通过容器内部的cgroup来限制的,因容器与宿主机共用内核,即限制是直接作用于宿主机上的,并没有叠加或者转化运算

 

posted on 2020-08-15 18:25  pu20065226  阅读(520)  评论(0编辑  收藏  举报