spark提交jar包时候会传到所有节点work文件夹,提交的jar包占用主从服务器空间很大, 所以有两种办法: 1.将jar包传到hdfs(hdfs没玩熟,以后研究) 2.jar包传到主节点,同步到从节点 找了一下资料inotify可以监听文件,但是使用时会有重复通知,引起多次重复同步, 此脚本使 Read More
cpu信息 dmidecode -t processor # dmidecode 3.0 Getting SMBIOS data from sysfs. SMBIOS 2.8 present. Handle 0x0400, DMI type 4, 42 bytes Processor Informa Read More
./bin/spark-sql -e "select count(1),count(distinct ip),substr(url,0,44) from tongji_log where domain ='xxx.com' and ds ='20170303' group by substr(url Read More