格灵 - 笔记 - 草稿 - 杂七杂八
arcee-service.arcee.svc.cluster.local:8502
@莫非 你们关闭 NewPersonLowQuality 开关吧。
下面是新人质量额外的条件。是为了选择一张质量较好的代表图：
1. AlignScore > 0.6（非新人AlignScore > 0.3）
2. Blur > 0.2（非新人AlignScore > 0.1）
3. AlignScore+Blur > 1.0 （（非新人无此条件）
之前深瞳云用在商业场景，摄像头安装高度、角度、配置等可控，丢弃了的新人很可能被另一个抓拍机拍到。但是安防场景不一样，可能丢了就不能再被抓拍到了
所以你们默认关闭NewPersonLowQuality吧
后面我们会优化这块逻辑，让代表图不要这么死板，可以动态调整
==========================================
「 吴春梅: @张海玉 有关安防组反馈的Position字段，麻烦今天更新下大家为什么这个字段之前加上校验吧，如果去掉校验会有什么额外影响 」
- - - - - - - - - - - - - - -
@吴春梅 我刚确认了一下, 目前是有实际的使用, 就是在聚类使用了. 主要有两个地方:
1. 可以不等超时就判断为新人条件.
// 可以不等超时就判断为新人的条件：
// alignscore > 0.99 and blur > 0.3 and width > 100
2. 判断是否值得进行聚类
// AlignScore 比最好状态提高了0.1
// 脸的宽度比之前最好状态扩大了10%
==========================================
浏览器中刷新、强制刷新和重新载入的区别是什么？
从 http 协议的角度来看
F5 和 Ctrl+F5 都是普通刷新，若页面之前访问过，就会发一个空请求到服务器，服务器返回 302 ，表示资源未更新，可以使用浏览器缓存。
强制刷新 (ctrl+F5) 则不管浏览器是否缓存，都要重新去源站服务器请求资源，成功则返回 200。
==========================================
安装curl时报错 curl : Depends: libcurl3-gnutls (= 7.47.0-1ubuntu2.12) but 7.58.0-2ubuntu3.6 is to be installed
提示安装curl依赖的libcurl版本不一致，这个时候用purge命令重新安装就好了
sudo apt-get purge libcurl3-gnutls
sudo apt-get install curl
==========================================
ssh-keygen -f "/home/ubuntu/.ssh/known_hosts" -R 192.168.2.228
==========================================
一张图片进来，如果 yaw 值很高，> 35， 那么就会 unqualified (yaw 过高），然后就结束了。如果一张图片进来， yaw < 35, 过了质量（产生 feature), 那么就会向后走，进行本地聚类 > 全局聚类。如果本地聚类和全局都没比中，但是 yaw < 25 (以及满足其他阈值), 那么就产生一个合格的新人；但是如果 35 > yaw > 25, 那么就会产生一个 NewPersonLowQuality 的图。
==========================================
ubuntu@k8s-master-227:/$ cd /opt/bigtoe/deepcloud/config/deepcloud/templates
ubuntu@k8s-master-227:/opt/bigtoe/deepcloud/config/deepcloud/templates$ ls
analyze-cluster clustersearch deepcloudmanager fcollector godeye-inner namespace.yaml oss-guard vseisd
cloud-importer cmasconsole dgc-plugin figure iot nightelf rrw yaconnector
clustermerge crw event-pusher godeye keycloak NOTES.txt syncer yaxnet
ubuntu@k8s-master-227:/opt/bigtoe/deepcloud/config/deepcloud/templates$ cd yaxnet/
ubuntu@k8s-master-227:/opt/bigtoe/deepcloud/config/deepcloud/templates/yaxnet$ ls
deployment.yaml
ubuntu@k8s-master-227:/opt/bigtoe/deepcloud/config/deepcloud/templates/yaxnet$ vim deployment.yaml
==========================================
kubectl scale deploy/crw --replicas=1 -n deepcloud
kubectl scale deploy/crw --replicas=10 -n deepcloud
kubectl -n deepcloud logs -f yaxnet-0 --tail 100
kubectl get cm -n vse
kubectl logs -f cloud-importer-6878fc76cd-b7spx -n deepcloud --tail 10
kubectl get pod -n vse -o wide | grep vse-video-vehicle-23-huawei-spqjk
kubectl edit sts yaxnet -n deepcloud
==========================================
1873 kubectl edit -n deepcloud deploy cmasconsole
1874 kubectl edit -n deepcloud deploy deepcloudmanager
1875 kubectl get pods
1876 docker pull ccr.ccs.tencentyun.com/face_cloud/yaxnet:fix_dup_msg
1877 kubectl edit sts yaxnet -n deepcloud
1878 watch kubectl get pod -n deepcloud
1879 lsblk
1880 history|grep docker
1881 docker pull ccr.ccs.tencentyun.com/face_cloud/yaxnet:test_perf
1882 ls
1883 kubectl edit sts yaxnet -n deepcloud
1884 watch kubectl get pod -n deepcloud
1885 kubectl edit sts yaxnet -n deepcloud
1886 watch kubectl get pod -n deepcloud
1887 kubectl port-forward sts/yaxnet 32112:6060
1888 kubectl port-forward sts/yaxnet 32112:6060 -n deepcloud
1889 go version
1890 apt install golang-go
1891 kubectl get svc -n deepcloud
1892 kubectl port-forward sts/yaxnet 32112:6060 -n deepcloud
1965 kubectl edit sts yaxnet -n deepcloud
1967 kubectl get po -n deepcloud -w
1968 kubectl get po -n deepcloud
1969 htop
1970 iostat -x 2
1971 htop
1972 iostat -x 2
1973 kubectl exec -it pgsql-6bb456799-mgg8p -n pgsql /bin/bash
1974 kubectl get svc -n deepcloud
1975 kubectl get svc rrw -n deepcloud
1976 kubectl get svc rrw -n deepcloud -o yaml
1977 kubectl get deploy crw -n deepcloud -o yaml
1978 kubectl get po -n deepcloud -o wide
1979 telnet 10.244.0.154 6060
1980 telnet 10.244.0.52 8900
1981 kubectl exec -it yaxnet-1 -n deepcloud sh
1982 kubectl exec -it crw-54db7956ff-qp8v4 -n deepcloud sh
1981 kubectl exec -it yaxnet-1 -n deepcloud sh
1982 kubectl exec -it crw-54db7956ff-qp8v4 -n deepcloud sh
1895 kubectl get po -n deepcloud
1896 kubectl expose po yaxnet-0 --type NodePort -n deepcloud
1897 kubectl expose po yaxnet-0 --port 6060 --type NodePort -n deepcloud
1898 kubectl get svc -n deepcloud | grep yaxnet
1899 kubectl edit deploy vseisd -n deepcloud
1900 kubectl edit deploy cloud-importer -n deepcloud
1901 kubectl get svc -n deepcloud
1902 kubectl edit deploy syncer
1903 kubectl edit deploy syncer -n deepcloud
1904 kubectl edit deploy crw -n deepcloud
1905 kubectl edit svc yaxnet-0 -n deepcloud
1906 kubectl edit svc crw -n deepcloud
1907 kubectl config
1908 kubectl config get-contexts
1909 vim ~/.kube/
1910 cd ~/.kube/
==========================================
247 cd deepcloud/
248 ls
249 vim templates/yaxnet/deployment.yaml
250 docker images | grep yaxnet
==========================================
ubuntu@k8s-master-227:~$ docker images | grep yaxnet
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.38/images/json: dial unix /var/run/docker.sock: connect: permission denied
ubuntu@k8s-master-227:~$ sudo docker images |grep yaxnet
[sudo] password for ubuntu:
ccr.ccs.tencentyun.com/face_cloud/yaxnet test_perf 38fef6a12d7e 12 hours ago 20.2MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet test_perf1 38fef6a12d7e 12 hours ago 20.2MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet <none> 9b7400a3c707 12 hours ago 20.2MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet fix_dup_msg 138abcdf8bcd 31 hours ago 19.1MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet 2.1.38.442-master 3295712ccf14 36 hours ago 19.1MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet 2.1.38.439-master c26419ed2784 3 days ago 19.1MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet 2.1.38.440-master c26419ed2784 3 days ago 19.1MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet 2.1.38.437-master 7a91914c136d 6 days ago 19.1MB
ccr.ccs.tencentyun.com/face_cloud/yaxnet 2.1.38.432-master 8c5b2c5c5ef7 11 days ago 19.1MB
==========================================
htop - interactive process viewer
==========================================
root@k8s-master-227:/home/ubuntu# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 48
On-line CPU(s) list: 0-47
Thread(s) per core: 2
Core(s) per socket: 12
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Stepping: 1
CPU MHz: 2419.044
CPU max MHz: 2900.0000
CPU min MHz: 1200.0000
BogoMIPS: 4399.89
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 30720K
NUMA node0 CPU(s): 0-11,24-35
NUMA node1 CPU(s): 12-23,36-47
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d
root@k8s-master-227:/home/ubuntu#
-----------------------------------------------------
iotop
==========================================
向 deepcloud 数据库添加记录：
./load_persons -pghost=192.168.2.227 -pgport=51453 -num=1000000 -tenant=eaacab08-007d-4b05-8879-f1fec6dc9664
==========================================
npu-smi info
==========================================
curl 10.244.0.66:7458/vse/task/status
==========================================
./kafka-console-consumer
------------------------
ln -s /fse-ssd/postgres/ /postgres
-----------------------------
vim /etc/resolv.conf
文件内容注释掉
---------------------------
apt-get update
apt install openjdk-8-jdk
----------------------
iostat -x 2
每 2 秒刷一下 iostat
-------------------------
ls -lh : 看文件大小，会友好显示 M, G 单位等
ll --block-size=M : 强制以 M 为单位
------------------------------
查看端口是干什么的？
root@k8s-master-223:/home/ubuntu# netstat -tupln | grep 60001
tcp 0 0 0.0.0.0:60001 0.0.0.0:* LISTEN 10549/rpc.mountd
----------------------------------------
vse 人车非是单独的模型
0.7.7.0: 是加密的
0.7.5.0: 不是加密的；很多客户都是 0750 的。
0770 0750 互不兼容。就像 1930 和 2730, 需要重新提特征。
-------------------------
描述一个 pod 的具体内容：
root@k8s-master-227:/home/ubuntu# kubectl describe pod vse-video-allobj-2-nvidia-dlrz6 -n vse
==============================
Nvidia pascal 卡: 1080/1070 系列
Nvidia turling 卡: 2080/2070 系列
==============================
Importer 发送各种类型图片：
http://importer.test.nemoface.com/general?img=https://dgtest1-1253924368.cos.ap-beijing.myqcloud.com/00_smoketest/1_0_1567398874_0_MO.jpg&sensor=DFC-RZ-99887766test&width=416&cached=1&height=389&captype=motor_image
-----------------------
role "0->SystemWrite:可读可写", "1->SystemRead:只读", "2->TenantWrite:租户级别只写", "3->TenantRead:租户级别只读"
configlevel "0->SystemLevel:系统级别", "1->TenantLevel:租户级别", "2->DeviceGroupLevel:设备组级别", "3->DeviceLevel:设备级别"
type "0->HotConfig:热更新配置", "1->TimerConfig:定时配置", "2->OperatorConfig:手动配置", "3->InitConfig:初始化系统配置"
--------------------
如何使用 local file 做视频
把视频上传到 /data/bigtoe/upload-videos。然后 xapi:
curl --location --request POST 'http://192.168.2.223:3154/x-api/v1/tasks' \
--header 'Content-Type: application/json' \
--data-raw '{
"name": "DFC-DC-01",
"source": {
"url": "file:///data/upload_videos/hanguang.mp4", 【红色部分千万不要变】
"type": "file",
"base_time": 1543378478000,
"start_time": 1543378478000,
"end_time": -1
},
"analysis_key_frame": false,
"rate": 1,
"detect_type": "all",
"extra": {
"sensorid": "DFC-DC-01"
}
}
--------------------
retail 中（也就是 console 中的 抓拍查询 ）一个人的人脸/人体的代表图选择的是：
先是用 trip/listbypid 中第 0 个元素的 bestImageUri/bestHumanbodyImageUri。bestImageUri ，也就是人脸的，肯定存在，否则就不会进 retail 中被比中了。
而对于 bestHumanbodyImageUri, 如果 /data/trip/listbypid 的第 0 个元素这个字段为空，则回去寻找 /person/info 中的 "HumanBodyImageUrl" 字段的值。如果也是空，那么就不显示人体了。
--------------------
当一张图片被抓拍了进入系统，也过了质量（AlignScore：默认是 0.6，Blur：默认是 0.1，Yaw：默认是 -25 至 25 ），但是没有和比对库和聚类库中的任何一条比中。那么系统会等待最多 5 秒。这期间是为了等待是否能够新的更好的和这个图片极其相似的 feature 的图片进来；也许如果有这样的图片，那么可以与比对库和聚类库中的一条对上，那么这时候就可以给第一张图片用第二张图片的 pid 给他的 pid 赋值。这样就可以避免聚散的现象。
也有一个例外，就是当第一张图片就非常好（alignscore>0.99 and blur>0.3 and width>100）， 那么就不会走等待 5 秒这个逻辑。
-----------------------
这个没有过新人质量阈值，新增Person的条件:
1. AlignScore+Blur > 1.0
2. AlignScore > 0.6
3. Blur > 0.2
------------------------
AlignScore：默认是 0.6，Blur：默认是 0.1，Yaw：默认是 -25 至 25，所以你这张图片质量不过
-------------------------
查看分开环境 ftp 端口：下面的 21002 就是
ubuntu@k8s-master-226:~$ su test1
Password:
test1@k8s-master-226:/home/ubuntu$ cd ~
test1@k8s-master-226:~$ pwd
/home/test1
test1@k8s-master-226:~$ ls
allen deepcloud ingress.yaml ladder
test1@k8s-master-226:~$ cat ladder/config/config-xiayun-k8s.yaml
maxconnections: 1000
server:
listenaddr: '0.0.0.0:21002'
idletimeout: 900
publichost: '192.168.2.226'
dataportrange:
start: 26020
end: 26029
users:
- user: 'deepglint'
pass: 'deepglint'
dir: 'seaweedfs'
istmpfs: true
dirsize: 1024 # M
filtername: 'seaweedfs_rule'
targets:
- targetname: 'seaweedfs_default_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
- targetname: 'seaweedfs_face_image_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'face_image'
- targetname: 'seaweedfs_humanbody_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'body_image'
- targetname: 'seaweedfs_face_with_body_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'face_with_bodyimageid'
- targetname: 'seaweedfs_face_with_fullimage_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'face_with_fullimageid'
- targetname: 'seaweedfs_body_with_fullimage_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'body_with_fullimageid'
- targetname: 'seaweedfs_face_with_body_with_fullimg_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'face_with_bodyimageid_with_fullimageid'
- targetname: 'seaweedfs_motor_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'motor_image'
- targetname: 'seaweedfs_motor_with_full_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'motor_with_fullimageid'
- targetname: 'seaweedfs_nonmotor_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'nonmotor_image'
- targetname: 'seaweedfs_nonmotor_with_full_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'nonmotor_with_fullimageid'
- targetname: 'seaweedfs_full_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'full'
- targetname: 'seaweedfs_face_target'
type: 'seaweedfs'
endpoint: 'http://192.168.2.226:30934/deepcloud/faces'
bucket: ''
fullbucket: ''
ak: ""
sk: ""
importer: "http://importer.test1.dginternal.com/general"
cbtype: 'face'
filters:
- filtername: 'seaweedfs_rule'
rules:
- rulename: 'dfc-hk-face'
regex: '(DFC-HK-.+)_FACE.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_image_target'
- rulename: 'dfc-hk-humanbody'
regex: '(DFC-HK-.+)_HUMAN.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_humanbody_target'
- rulename: 'dfc-hk-motor'
regex: '(DFC-HK-.+)_VEHICLE.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_motor_target'
- rulename: 'dfc-hk-nonmotor'
regex: '(DFC-HK-.+)_NONMOTOR.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_nonmotor_target'
- rulename: 'dfc-rz-face'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_0\_RL.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_image_target'
- rulename: 'dfc-rz-humanbody'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_0\_RT.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_humanbody_target'
- rulename: 'dfc-rz-face_with_body'
regex: '(DFC-RZ-.+)(([1-9][0-9]{1,4}|[1-9]{1,5})\_){2}\d{10}\_0\_(RL|RT).(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_with_body_target'
- rulename: 'dfc-rz-face_with_fullimageid'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_([1-9][0-9]{1,4}|[1-9]{1,5})\_RL.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_with_fullimage_target'
- rulename: 'dfc-rz-body_with_fullimageid'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_([1-9][0-9]{1,4}|[1-9]{1,5})\_RT.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_body_with_fullimage_target'
- rulename: 'dfc-rz-face_with_body_full'
regex: '(DFC-RZ-.+)(([1-9][0-9]{1,4}|[1-9]{1,5})\_){2}\d{10}\_([1-9][0-9]{1,4}|[1-9]{1,5})\_(RL|RT).(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_with_body_with_fullimg_target'
- rulename: 'dfc-rz-motor'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_0\_MO.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_motor_target'
- rulename: 'dfc-rz-motor-with-fullimg'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_([1-9][0-9]{1,4}|[1-9]{1,5})\_MO.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_motor_with_full_target'
- rulename: 'dfc-rz-nonmotor'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_0\_NM.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_nonmotor_target'
- rulename: 'dfc-rz-nonmotor-with-fullimg'
regex: '(DFC-RZ-.+)([1-9][0-9]{1,4}|[1-9]{1,5})\_0\_\d{10}\_([1-9][0-9]{1,4}|[1-9]{1,5})\_NM.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_nonmotor_with_full_target'
- rulename: 'dfc-rz-full'
regex: '(DFC-RZ-.+)_YT\.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_full_target'
- rulename: 'dfc-rz-face-default'
regex: '((DFC-RZ-|DFC-FI-).+)_RL\.(jpg|tmp)'
panoramaPattern: '${1}_YT'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_target'
- rulename: 'dfc-me-face-default'
regex: '(DFC-ME-.+)_1\.(jpg|tmp)'
panoramaPattern: '${1}_0'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_target'
- rulename: 'dfc-me-full'
regex: '(DFC-ME-.+)_0\.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_full_target'
- rulename: 'dfc-me-face'
regex: '(DFC-ME-.+)_2\.(jpg|tmp)'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_face_target'
- rulename: 'default'
doupd: true
aging: 'size'
keep: 60
targetname: 'seaweedfs_default_target'
test1@k8s-master-226:~$
-----------------
后聚类 cronjob
kubectl edit cronjob -n deepcloud clustermerge
-----------------------------------
root@k8s-master-227:/opt/bigtoe# ./kafka-console-consumer -h
root@k8s-master-227:/opt/bigtoe# ./kafka-console-consumer -topic capture
root@k8s-master-227:/opt/bigtoe# ./kafka-console-consumer -topic capture | grep 2e931176-4b5c-4608-a7ab-b0eddff80b99
======================================
对于删除租户 tenant/delete, 可以
方法1： 直接查到 godeye-inner 的 cluster ip 地址，然后在本机直接执行 curl 就行即可。
root@k8s-master-227:/home/ubuntu# kubectl get svc -n deepcloud
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cloud-importer NodePort 10.105.78.13 <none> 80:31080/TCP,8081:31081/TCP 2d13h
cmasconsole NodePort 10.96.247.13 <none> 80:35080/TCP 2d13h
crw LoadBalancer 10.104.245.31 <pending> 8900:14472/TCP 2d13h
deepcloudmanager NodePort 10.97.107.69 <none> 80:34080/TCP 2d13h
event-pusher NodePort 10.100.203.115 <none> 80:32080/TCP,8081:32081/TCP 2d13h
figure NodePort 10.97.134.67 <none> 80:37500/TCP 2d13h
godeye NodePort 10.111.178.183 <none> 80:39876/TCP,8081:39081/TCP 2d13h
godeye-inner ClusterIP 10.96.220.151 <none> 80/TCP,8081/TCP 2d13h
postman 中使用 curl 格式：
curl -X POST \
http://10.96.220.151/api/tenant/delete \
-H 'Content-Type: application/json' \
-H 'Postman-Token: 2065b7bb-6206-402f-8292-1cc262e791bb' \
-H 'Token: token eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1Mjk0Njg2OTIsImlkIjoic3VwZXJBZG1pbiIsIm9yaWdfaWF0IjoxNTI5NDY2ODkyfQ.5Ib-XgGIWj5DWWg0tSs5D6HRQnDBYWh3uEbRWh6fIW8' \
-H 'cache-control: no-cache' \
-d '{
"TenantID": "5353ac4e-28fe-475b-b9a5-ec8ff21f4197"
}'
如下：
root@k8s-master-227:/home/ubuntu# curl -X POST \
> http://10.96.220.151/api/tenant/delete \
> -H 'Content-Type: application/json' \
> -H 'Postman-Token: 2065b7bb-6206-402f-8292-1cc262e791bb' \
> -H 'Token: token eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1Mjk0Njg2OTIsImlkIjoic3VwZXJBZG1pbiIsIm9yaWdfaWF0IjoxNTI5NDY2ODkyfQ.5Ib-XgGIWj5DWWg0tSs5D6HRQnDBYWh3uEbRWh6fIW8' \
> -H 'cache-control: no-cache' \
> -d '{
> "TenantID": "5353ac4e-28fe-475b-b9a5-ec8ff21f4197"
> }'
{"Code":1,"Msg":"success","Data":{"TenantID":"5353ac4e-28fe-475b-b9a5-ec8ff21f4197","Cts":1576470726130533617,"Uts":1576472129998404065,"TenantName":"租户2","AccessKey":"5353ac4e-28fe-475b-b9a5-ec8ff21f4197@d2399978-58aa-456f-adbe-86e50c79ee2f","SecretKey":"86e49d55-d03f-4f11-9f3e-ad76bae6c398@2cc50998-e0a5-45ec-8c26-2679244135a2","DeviceNum":10,"RegisterRepoNum":10,"RegisterRepoCapacity":10,"ClusterRepoCapacity":2000000000,"DeviceUsedNum":0,"RegisterRepoUsedNum":0,"Comment":"租户2","Status":"deleted","MainUser":"","Ets":0}}root@k8s-master-227:/home/ubuntu#
----------
方法 2： 或者登录到 deepcloudmanager, 然后直接用 godeye-inner 的域名删除：
root@k8s-master-227:/home/ubuntu# kubectl get pod -n deepcloud
NAME READY STATUS RESTARTS AGE
cloud-importer-cf7978886-8t4x5 1/1 Running 2 2d13h
cmasconsole-58c78dfb45-9j7t2 1/1 Running 0 2d13h
crw-d99797fcb-qzl6g 1/1 Running 0 2d13h
deepcloudmanager-67d7f4f6c7-h6ph9 1/1 Running 0 2d13h
。。。。。。
root@k8s-master-227:/home/ubuntu# kubectl exec -it deepcloudmanager-67d7f4f6c7-h6ph9 -n deepcloud bash
root@deepcloudmanager-67d7f4f6c7-h6ph9:/deepCloudManager#
root@deepcloudmanager-67d7f4f6c7-h6ph9:/deepCloudManager# curl -X POST \
> http://godeye-inner/api/tenant/delete \
> -H 'Content-Type: application/json' \
> -H 'Postman-Token: 449ee4df-b2b4-45e7-bd4c-028370bb737b' \
> -H 'Token: token eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1Mjk0Njg2OTIsImlkIjoic3VwZXJBZG1pbiIsIm9yaWdfaWF0IjoxNTI5NDY2ODkyfQ.5Ib-XgGIWj5DWWg0tSs5D6HRQnDBYWh3uEbRWh6fIW8' \
> -H 'cache-control: no-cache' \
> -d '{
> "TenantID": "995ad1d0-e894-46d8-9e80-5935a85c7777"
> }'
{"Code":1,"Msg":"success","Data":{"TenantID":"995ad1d0-e894-46d8-9e80-5935a85c7777","Cts":1576472356591710747,"Uts":1576472649261971072,"TenantName":"租户3","AccessKey":"995ad1d0-e894-46d8-9e80-5935a85c7777@67fa38be-9c22-4f36-bddc-5161ec036129","SecretKey":"1fe55321-1ba2-497c-822d-0f7c531bb578@99f6d207-2117-4b9b-9950-b5960cc24a2d","DeviceNum":1,"RegisterRepoNum":1,"RegisterRepoCapacity":1,"ClusterRepoCapacity":2000000000,"DeviceUsedNum":0,"RegisterRepoUsedNum":0,"Comment":"租户3","Status":"deleted","MainUser":"","Ets":0}}root@deepcloudmanager-67d7f4f6c7-h6ph9:/deepCloudManager#
root@deepcloudmanager-67d7f4f6c7-h6ph9:/deepCloudManager#
======================================
root@k8s-master-227:/home/ubuntu/allen# docker login --username=100006586462 --password=Deepglint2018@ ccr.ccs.tencentyun.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
root@k8s-master-227:/home/ubuntu/allen#
======================================
root@k8s-master-227:/home/ubuntu/allen# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ccr.ccs.tencentyun.com/face_cloud/syncer syncer_fixOom_jch_1115_1 0f9edf92c233 6 hours ago 206MB
ccr.ccs.tencentyun.com/face_cloud/syncer syncer_fixOom_jch_1115 fbc42b93913a 8 hours ago 206MB
ccr.ccs.tencentyun.com/face_cloud/syncer syncer_fixoom_jch_1115 fbc42b93913a 8 hours ago 206MB
ccr.ccs.tencentyun.com/face_cloud/syncer syncer_oom_jch_1115 fbc42b93913a 8 hours ago 206MB
delete_expired_data latest 102808a9078b 45 hours ago 701MB
======================================
kubectl edit deployment syncer -n deepcloud
======================================
atop -dl 2
iotop
top
iostat
======================================
root@k8s-master-227:/home/ubuntu/allen/csv# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master-227 Ready master 157m v1.12.2
===========================
nvidia-smi
root@ubuntu:/opt/bigtoe# nvidia-smi
Fri Dec 13 23:33:53 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56 Driver Version: 418.56 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 108... Off | 00000000:02:00.0 Off | N/A |
| 23% 22C P8 8W / 250W | 5024MiB / 11178MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 108... Off | 00000000:03:00.0 Off | N/A |
| 23% 22C P8 9W / 250W | 4968MiB / 11178MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GeForce GTX 108... Off | 00000000:82:00.0 Off | N/A |
| 23% 23C P8 8W / 250W | 4830MiB / 11178MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 GeForce GTX 108... Off | 00000000:83:00.0 Off | N/A |
| 23% 21C P8 9W / 250W | 4830MiB / 11178MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 29670 C ./vse 5014MiB |
| 1 29333 C ./vse 4958MiB |
| 2 29269 C ./vse 4820MiB |
| 3 28801 C ./vse 4820MiB |
+-----------------------------------------------------------------------------+
======================================
iotop
top
iostat
atop -dl 2
======================================
踢出其他登录用户
root@k8s-master-227:/home/ubuntu/allen# w
20:57:26 up 2 days, 4:23, 4 users, load average: 36.28, 28.30, 24.48
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
ubuntu pts/0 192.168.104.72 17:40 3:15m 22.08s 0.03s sshd: ubuntu [priv]
ubuntu pts/1 10.0.1.167 14:57 0.00s 17:54 0.03s sshd: ubuntu [priv]
ubuntu pts/2 192.168.102.162 19:33 11:18 3.69s 0.02s sshd: ubuntu [priv]
ubuntu pts/3 192.168.102.162 17:48 9:58 2.38s 0.02s sshd: ubuntu [priv]
root@k8s-master-227:/home/ubuntu/allen# skill -9 -t pts/0
root@k8s-master-227:/home/ubuntu/allen# skill -9 -t pts/2
root@k8s-master-227:/home/ubuntu/allen# skill -9 -t pts/3
----------------------------
当我们创建视频流任务并发给深瞳云 device 的时候，发给 vseisd 的都带 feature 值，也就是说，对于 vse 是没有压力的（因为已经有 feature 了)
======================================
查看 log
kubectl -n deepcloud logs godeye-5684b5bf75-8th77 -f --since=1m
======================================
想 top 一下yaxnet 进程的 cpu 使用情况， yaxnet 有多个
先用 pidof yaxnet 查询，在用 top -p
root@k8s-master-225:/usr/share/jmeter/bin# pidof yaxnet
11352 10965 10612 9691 9215 8829 8499 7922 7409 6815
root@k8s-master-225:/usr/share/jmeter/bin# top -p 11352,10965,10612,9691,9215,8829,8499,7922,7409,681
======================================
修改 redis 内存值的大小。先修改值再安装产品
：修改/opt/bigtoe/pkg/k8s-conf/redis/10redis-configmap.yaml 这个文件的 117行，改为 maxmemory 10gb
如果已经先安装了产品，然后才修改值，那么先 replace 配置文件
kubectl replace -f /opt/bigtoe/pkg/k8s-conf/redis/10redis-configmap.yaml
再删除已有 pod 重启 (生成新的 pod）
kubectl -nredis delete pods --all
======================================
从远程拷贝文件到本地
ubuntu@VM-2-41-ubuntu:~/allen$ scp ubuntu@192.168.2.227:~/tool/Register_Add_IgnoreQuality_ForcePersonID_noGUIreport2.jmx <localPath>
从本地拷贝文件到远程
ubuntu@VM-2-41-ubuntu:~/allen$ scp <localfile> ubuntu@192.168.2.227:~
======================================
curl -X POST xxxxxx
======================================
SELECT * FROM config WHERE name like '%ClusterSearchConfidenceThreshold%'
这里 ClusterSearchConfidenceThreshold 的值影响到注册用户时， 达到什么相似度才能注册进去。
======================================
无论 人脸还是人体的大图 (FullImage) , /data/flow/capture 都不会处理，直接 drop, 但是同时把 post 请求中的信息原封不动的带到返回中
新安装的 ubuntu 1605.1 ， 想装java, 可能需要先 apt-get update 升级一下 apt-get, 再 apt install default-jre 就行了。
======================================
Katalon Recorder (Selenium IDE alternative)
======================================
安装两个 Firefox 在一个机器上：需安装一个 32bit Firefox 45, 一个 64 bit 68ESR
firefox -p 去创建新的 profile
======================================
Helm ChartMuseum Tiller
======================================
goroutine: golang 的轻量级线程。https://blog.csdn.net/u011957758/article/details/81159481
定义：在go里面，每一个并发执行的活动成为goroutine。
详解：goroutine可以认为是轻量级的线程，与创建线程相比，创建成本和开销都很小，每个goroutine的堆栈只有几kb，并且堆栈可根据程序的需要增长和缩小(线程的堆栈需指明和固定)，所以go程序从语言层面支持了高并发。
程序执行的背后：当一个程序启动的时候，只有一个goroutine来调用main函数，称它为主goroutine，新的goroutine通过go语句进行创建。
======================================
unzip source.zip -d /etc/allen/target
这里的 -d 为 directory 的意思。会直接解压到这里。所以文件多的话实现创建一下文件夹
======================================
root@k8s-master-22:/home/ubuntu/jmeter# find /etc -name '*srm*'
======================================
vim 中：
yy p
w W
21G
gg
G
======================================
vim /etc/vim/vimrc
set number
set nu
======================================
你知道怎么在offline 环境中增加一个 importer 么？
root@k8s-master-22:/# kubectl scale -n deepcloud deployment cloud-importer --replicas 2
deployment.extensions/cloud-importer scaled
======================================
tailf 等于 tail -f -n 10（tail -f或-F默认也是打印最后10行，然后追踪文件），与tail -f不同的是，如果文件不增长，它不会去访问磁盘文件，所以tailf特别适合那些便携机上跟踪日志文件，因为它减少了磁盘访问，可以省电。
======================================
import time
time.ctime()
print(time.time()) # 打印时间戳
浙公网安备 33010602011771号