文本处理三剑客之sed

 

 

 

 


 

 

 

 

 

文本处理三剑客之sed知识概述

 

三剑客sed命令概述介绍
三剑客sed命令执行原理命令基本语法
三剑客sed命令实操练习增删改查

 

 

 

 

 

一sed命令概述

 

 

sed - stream editor for filtering and transforming text

1)可以查找过滤筛选出指定的信息
2)擅长编辑文件内容信息
3)擅长对文件行进行操作

 

 

 

命令语法结构:sed     [参数]      ' 条件   指令'     文件信息

定义条件:根据行号   根据内容    根据正则信息

定义指令:满足条件的信息处理动作(添加删除修改输出)

 

 

 

二sed命令执行原理

 

 

sed命令执行过程核心原则:找什么信息,做什么事情

找什么信息==定义查找条件

做什么事情==操作执行命令(替换删除增加输出)

 

 

 

 

 

 

三sed命令的查询操作

 

实际上是输入操作

 

(一)根据行号筛选单行信息

[root@centos71 ~]# sed  -n   '3p'   /etc/selinux/config
# SELINUX= can take one of these three values:

 

 

 

 

 

(二)根据行号筛选多行信息

连续的多行信息

[root@centos71 ~]# sed  -n   '2,4p'   /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.

 

 

 

 

 

不连续的多行信息

[root@centos71 ~]# cat  -n /etc/selinux/config
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=enforcing
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 ~]#  sed  -n   '1p;3p;5p'  /etc/selinux/config

# SELINUX= can take one of these three values:
#     permissive - SELinux prints warnings instead of enforcing.

 

 

 

 

 

 

 

(三)根据字符筛选单行信息   

 

/ /这种结构和搜索替换是类似的,搜索替换包含了搜索

搜索替换sed   's///g'

[root@centos71 test]# sed   -n  '/dis/p'   selinux.txt 
#     disabled - No SELinux policy is loaded.
SELINUX=disabled

 

 

 

 

 

 

(四)根据字符筛选多行信息

 

连续的多行信息

筛选从dis开头,target结尾的行

[root@centos71 test]# cat  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

[root@centos71 test]# sed   -n  '/dis/,/target/p'   selinux.txt 
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
[root@centos71 test]# 

 

 

 

 

 

不连续的多行信息

[root@centos71 ~]# sed  -n  '/dis/p;/en/p'  /etc/selinux/config  
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled

 

 

 

 

 

 

区间范围查找,搜索的起始字符一定要的,结束字符不一定要

有头无尾可以(壁虎),有尾无头不可以,直接死了

是贪婪匹配,每行进行过滤

 

 

 

 

 

 

 

四sed命令删除操作

 

 

(一)根据行号进行单行删除

 

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   '6d'  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

(二)根据行号进行多行删除

 

连续多行

 

注意操作是在内存的模式空间操作,对磁盘的block操作才真正操作

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   '1,11d'  selinux.txt
SELINUXTYPE=targeted

 

 

 

 

 

 

不连续多行

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   '7d;12d'  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.

 

 

 

 

 

 

 

五sed命令添加操作

 

(一)根据行号单行添加信息

 

巧记:a表示add+after

[root@centos71 test]# cat  -n  selinux.txt
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 test]# sed   '1a  hahahah'   selinux.txt 

hahahah
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

(二)根据行号多行添加信息

 

连续多行

[root@centos71 ~]# cat  -n  /etc/issue  
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed  '1,2aHAPPY'  /etc/issue
\S
HAPPY
Kernel \r on an \m
HAPPY

 

 

 

 

 

 

不连续多行

 

要加-e

[root@centos71 ~]# cat -n  /etc/issue
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed  -e  '1aHAPPY'   -e  '2aHAPPY'    -e  '3aHAPPY'  /etc/issue
\S
HAPPY
Kernel \r on an \m
HAPPY

HAPPY

 

 

 

 

 

 

 

 

在指定行后面添加多行信息

 

[root@centos71 test]# sed   '1a  hahaha\nxixixi'   selinux.txt 

hahaha
xixixi
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

在指定行前面插入信息

 

i插入,联系vim的i就是插入

[root@centos71 test]# sed   '1i  hahaha\nxixixi'   selinux.txt 
hahaha
xixixi

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

企业常见配置文件添加信息方法

 

在文件的最后一行添加内容

$表示最后一行;在最后一行后面附加新的信息

 

[root@centos71 ~]# cat -n  /etc/issue
     1    \S
     2    Kernel \r on an \m
     3    
[root@centos71 ~]# sed   '$ahahaxixi'  /etc/issue
\S
Kernel \r on an \m

hahaxixi

 

 

 

 

 

 


基础命令是最低级别的,不能识别正则符号\n\t,会认为是字符

grep只能识别基础正则

sed只能识别基础正则

awk统计分析命令,是高级命令,可以识别基础正则和扩展正则

这种情况类似于高级语言和机器语言

 

 

 

 

 

 

六sed命令修改操作

 

(一)根据单行信息搜索替换

[root@centos71 test]# sed  '7s#disabled#enforcing#g'  selinux.txt

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

(二)利用正则将信息匹配出来再做替换

 

 

[root@centos71 ~]# cat  -n  /etc/selinux/config
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15    
[root@centos71 ~]# sed '2,6s/#/abc/g'   /etc/selinux/config

abc This file controls the state of SELinux on the system.
abc SELINUX= can take one of these three values:
abc     enforcing - SELinux security policy is enforced.
abc     permissive - SELinux prints warnings instead of enforcing.
abc     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 

 

 

 

 

 

 

 

 

(三)利用后向引用前项做替换——修改网卡地址

 

 

[root@centos71 ~]# sed -r '/IPADDR/s#(.*=).*#\1192.168.111.200#g' /etc/sysconfig/network-scripts/ifcfg-eth0 
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=9d9e2656-f3ac-4f75-8722-3136d239985d
DEVICE=eth0
ONBOOT=yes
IPADDR=192.168.111.200
PREFIX=24
GATEWAY=10.0.0.254
IPV6_PRIVACY=no
DNS1=223.5.5.5

 

 

 

 

 

 

 

 

练习——sed批量创建用户信息test01 .. test10并设置随机密码(6位)

 
 

 

(一)创建用户

[root@centos71 ~]# seq  -w  10
01
02
03
04
05
06
07
08
09
10

 

 

 

 

 

[root@centos71 test]# seq -w   10  |  sed  -r   's#(.*)# useradd  test\1#g'
 useradd  test01
 useradd  test02
 useradd  test03
 useradd  test04
 useradd  test05
 useradd  test06
 useradd  test07
 useradd  test08
 useradd  test09
 useradd  test10

 

 

 

 

 

(二)设置随机密码

[root@centos71 ~]# # seq -w 10|sed -r 's#(.*)#useradd test\1;echo  password  | passwd --stdin test\1#g'

 

 

 

 

 

 

(三)保存随机密码

 注意因为设置的是随机密码,所以要使用变量

[root@centos71 ~]# seq  -w 10 |  sed -r  's#(.*)#useradd test\1;passwd=$(tr -cd 'a-zA-Z0-9' </dev/urandom|head -c 6);
echo $passwd | passwd --stdin test\1; echo test\1 $passwd >>/tmp/user_passwd#g'

 

 

 

 

 

 

 

 

练习——实现批量修改文件名称

 

 


将文件名称扩展名改为jpg

问题我的操作可以吗?

[root@centos71 test]# touch   happy{01..10}.txt
[root@centos71 test]# ls  happy*
happy01.txt  happy03.txt  happy05.txt  happy07.txt  happy09.txt  happy.txt
happy02.txt  happy04.txt  happy06.txt  happy08.txt  happy10.txt
[root@centos71 test]# ls  happy*  |  sed   -r  's#(.*\.)txt#\1jpg#g' 
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg
happy.jpg

 

 

 

 

 

[root@centos71 test]# sed   -r  's#(.*\.)txt#\1jpg#g'  happy.txt
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg

 

 

 

 

 

法二

[root@centos71 test]# ls  happy*
happy01.txt  happy03.txt  happy05.txt  happy07.txt  happy09.txt  happy.txt
happy02.txt  happy04.txt  happy06.txt  happy08.txt  happy10.txt
[root@centos71 test]# ls  happy* | sed   -r  's#(.*)txt#mv \1txt \1jpg#g'mv happy01.txt happy01.jpg
mv happy02.txt happy02.jpg
mv happy03.txt happy03.jpg
mv happy04.txt happy04.jpg
mv happy05.txt happy05.jpg
mv happy06.txt happy06.jpg
mv happy07.txt happy07.jpg
mv happy08.txt happy08.jpg
mv happy09.txt happy09.jpg
mv happy10.txt happy10.jpg
[root@centos71 test]# ls  happy* | sed   -r  's#(.*)txt#mv \1txt \1jpg#g'    | bash
[root@centos71 test]# ls  happy*
happy01.jpg  happy03.jpg  happy05.jpg  happy07.jpg  happy09.jpg  happy.txt
happy02.jpg  happy04.jpg  happy06.jpg  happy08.jpg  happy10.jpg

 

 

 

 

 

 

 

 

练习——利用sed命令取出IP地址

 

(一)取出有IP地址所在行

[root@centos71 ~]#   ip a s eth0|sed -n '3p'
    inet 10.0.0.200/24 brd 10.0.0.255 scope global noprefixroute eth0

 

 

 

 

 

(二)取出IP地址

将前面/后面减掉,简言之掐头去尾留中间

[root@centos71 ~]# ip a  s  eth0  |    sed   -n  '3p' |   sed   -r  's#.* (.*)\/.*#\1#g'
10.0.0.200

 

 

 

 

巧记:

双引号比单引号更多,作用更大,比如可以调用变量

 

 

 

 

 

 

七sed命令使用忠告

 

 

(一)sed操作时候要做备份

 

真正删除

 -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if SUFFIX supplied)

-i---将模式空间操作影响到磁盘中也就是将模式空间信息--覆盖--磁盘文件中

在编辑文件时,直接多文件进行备份sed   -i.bak

注意sed命令在使用时用到多个参数,要将-i参数写在所有参数后面

下面对文件进行搜索替换的同时也进行了备份

[root@centos71 test]# cat  -n /etc/selinux/config 
     1    
     2    # This file controls the state of SELinux on the system.
     3    # SELINUX= can take one of these three values:
     4    #     enforcing - SELinux security policy is enforced.
     5    #     permissive - SELinux prints warnings instead of enforcing.
     6    #     disabled - No SELinux policy is loaded.
     7    SELINUX=disabled
     8    # SELINUXTYPE= can take one of three values:
     9    #     targeted - Targeted processes are protected,
    10    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    11    #     mls - Multi Level Security protection.
    12    SELINUXTYPE=targeted
    13    
    14    
    15
[root@centos71 test]#  sed  -ri.bak    '7s#disabled#enforcing#'   /etc/selinux/config
[root@centos71 test]# ls  /etc/selinux/config*
/etc/selinux/config  /etc/selinux/config.bak
[root@centos71 test]# ls  /etc/selinux/config* -l
-rw-r--r-- 1 root root 543 Dec 19 19:43 /etc/selinux/config
-rw-r--r-- 1 root root 542 Dec 17 16:59 /etc/selinux/config.bak
[root@centos71 test]# diff   /etc/selinux/config*
7c7
< SELINUX=enforcing
---
> SELINUX=disabled

 

 

 

 

 

 

(二)sed在使用参数时不要一起使用参数-i、-n

 

如果做了第1步,那么第2步做了还可以恢复,但是对企业还是会有损失

所以小心驶得万年船

 

 

 

 

 

 

 

 

 

八sed特别说明

 

注意单引号、双引号、三引号的作用是一样的,查看执行过程

[root@centos71 test]# cat   sed.test.sh
#/bin/bash
echo  single quotes 
sed    -n   '/127.0.0.1/p'   /etc/hosts
echo  tow quotes 
sed    -n   ''/127.0.0.1/p''   /etc/hosts
echo  three quotes 
sed    -n   '''/127.0.0.1/p'''   /etc/hosts

 

 

 

[root@centos71 test]# bash  -x  sed.test.sh
+ echo single quotes
single quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
+ echo tow quotes
tow quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
+ echo three quotes
three quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
[root@centos71 test]# cat  -n   /etc/hosts
     1    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
     2    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

 

posted on 2019-12-19 21:04  likeblogbsgs  阅读(68)  评论(0编辑  收藏

导航