HDFS的快照snapshot的使用
一,snapshot命令
允许这个文件路径可以创建snapshots：
hdfs dfsadmin -allowSnapshot <路径>
不允许创建目录的快照。必须先删除目录的所有快照，然后再禁止快照。
hdfs dfsadmin -disallowSnapshot <路径>
创建快照
hdfs dfs -createSnapshot <路径> [<快照名称>]
[root@cm1 ~]# hdfs dfs -createSnapshot /data/mytest mytest-snap
删除快照
hdfs dfs -deleteSnapshot <路径> <快照名称>
[root@cm1 ~]# hdfs dfs -deleteSnapshot /data/mytest mytest-snap
重命名快照
hdfs dfs -renameSnapshot <路径> <旧名称> <新名称>
[root@cm1 ~]# hdfs dfs -renameSnapshot /data/mytest mytest-snap mytest-snap-new
获取Snapshottable目录列表
hdfs lsSnapshottableDir
获取快照差异报告
hdfs snapshotDiff <路径> <fromSnapshot> <toSnapshot>
二,snapshot的具体使用案例
创建data目录
hdfs dfs -mkdir /user/hadoop-cw/data
hdfs dfs -ls /user/hadoop-cw
hdfs dfs -touchz /user/hadoop-cw/data/file1.txt
hdfs dfs -touchz /user/hadoop-cw/data/file2.txt
hdfs dfs -put word.txt test.txt /user/hadoop-cw/data
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data
Found 4 items
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file1.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file2.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/test.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/word.txt
假设上面是昨天写进去的，想要把这些数据进行备份，使用快照的方式进行备份，想要对/user/hadoop-cw/data目录进行备份，需要允许对其创建快照。允许创建快照：
[root@cm1 ~]# hdfs dfsadmin -allowSnapshot /user/hadoop-cw/data
Allowing snapshot on /user/hadoop-cw/data succeeded
对/user/hadoop-cw/data创建快照，并对快照命名：
[root@cm1 ~]# hdfs dfs -createSnapshot /user/hadoop-cw/data data-20200514-snapshots
Created snapshot /user/hadoop-cw/data/.snapshot/data-20200514-snapshots
查看快照备份的文件：
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data/.snapshot/data-20200514-snapshots
Found 4 items
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:29 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/file1.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:29 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/file2.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:31 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/test.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:31 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/word.txt
通过WebUI可以查看备份情况
今天0515又往data里面写了数据：
hdfs dfs -touchz /user/hadoop-cw/data/file3.txt
hdfs dfs -touchz /user/hadoop-cw/data/file4.txt
hdfs dfs -put test2.txt /user/hadoop-cw/data
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data
Found 7 items
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file1.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file2.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:38 /user/hadoop-cw/data/file3.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:38 /user/hadoop-cw/data/file4.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/test.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:39 /user/hadoop-cw/data/test2.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/word.txt
创建今天的快照（备份）
$ hdfs dfs -createSnapshot /user/hadoop-cw/data data-20200515-snapshots
Created snapshot /user/hadoop-cw/data/.snapshot/data-20200515-snapshots
恢复14号的数据：
$ hdfs dfs -cp -ptopax /user/hadoop-cw/data/.snapshot/data-20200514-snapshots /user/hadoop-cw
查看恢复的数据：
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/
Found 2 items
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:40 /user/hadoop-cw/data
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:33 /user/hadoop-cw/data-20200514-snapshots
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data-20200514-snapshots
Found 4 items
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:29 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/file1.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:29 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/file2.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:31 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/test.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:31 /user/hadoop-cw/data/.snapshot/data-20200514-snapshots/word.txt
查看当前数据：
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data
Found 7 items
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file1.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-14 18:29 /user/hadoop-cw/data/file2.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:38 /user/hadoop-cw/data/file3.txt
-rw-r--r-- 3 hdfs hdfs 0 2020-05-15 18:38 /user/hadoop-cw/data/file4.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/test.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-15 18:39 /user/hadoop-cw/data/test2.txt
-rw-r--r-- 3 hdfs hdfs 12 2020-05-14 18:31 /user/hadoop-cw/data/word.txt
两个快照之间进行对比：
[root@cm1 ~]# hdfs snapshotDiff /user/hadoop-cw/data data-20200514-snapshots data-20200515-snapshots
Difference between snapshot data-20200514-snapshots and snapshot data-20200515-snapshots under directory /user/hadoop-cw/data:
M .
+ ./file3.txt
+ ./file4.txt
+ ./test2.txt
Results:
| + | The file/directory has been created. |
|---|---|
| - | The file/directory has been deleted. |
| M | The file/directory has been modified. |
| R | The file/directory has been renamed. |
重命名快照
hdfs dfs -renameSnapshot /user/hadoop-cw/data data-20200514-snapshots data-20200514-snapshot
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data/.snapshot
Found 2 items
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:33 /user/hadoop-cw/data/.snapshot/data-20200514-snapshot
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:40 /user/hadoop-cw/data/.snapshot/data-20200515-snapshots
删除14号的快照：
[root@cm1 ~]# hdfs dfs -deleteSnapshot /user/hadoop-cw/data data-20200514-snapshot
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data/.snapshot
Found 1 items
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:40 /user/hadoop-cw/data/.snapshot/data-20200515-snapshots
当未删除snapshot时,进行禁止快照时会报错
[root@cm1 ~]# hdfs dfsadmin -disallowSnapshot /user/hadoop-cw/data
disallowSnapshot: The directory /user/hadoop-cw/data has snapshot(s). Please redo the operation after removing all the snapshots.
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data/.snapshot
Found 1 items
drwxr-xr-x - hdfs hdfs 0 2020-05-15 18:40 /user/hadoop-cw/data/.snapshot/data-20200515-snapshots
此时需要将.snapshot下的快照删除后,才能进行禁止操作
[root@cm1 ~]# hdfs dfs -deleteSnapshot /user/hadoop-cw/data data-20200515-snapshots
[root@cm1 ~]# hdfs dfs -ls /user/hadoop-cw/data/.snapshot
[root@cm1 ~]# hdfs dfsadmin -disallowSnapshot /user/hadoop-cw/data
Disallowing snapshot on /user/hadoop-cw/data succeeded
