python-实时分析log文件每秒记录数

 

文件名：gencdr.py

作用：在指定的时间里每秒向testcdr.txt文件中写N行记录，N为随机数。模拟access.log。

# -*- coding: utf-8 -*-  
""" 
 zhangbo2012
 http://www.cnblogs.com/zhangbo2012/
"""

import time
import datetime
import random

filepath = "testcdr.txt"

def time2yyyymmddhhmiss():
    return datetime.datetime.now().strftime('%Y%m%d%H%M%S')

with open(filepath,'w') as wf:
    for i in range(150):
        time.sleep(1)
        linecnt = int(random.random()*20)
        for i in range(linecnt):
            ol = "%s|%04d|%04d|%04d\n" % (time2yyyymmddhhmiss(),int(random.random()*9999),int(random.random()*9999),i)
            wf.write(ol)
            print ol,
        wf.flush()

 

运行效果

 

文件名：analyze_cdrfile.py

作用：    实时分析testcdr.txt文件中的记录，输出每秒记录数。目前配置为延迟30秒输出。

# -*- coding: utf-8 -*-  
""" 
 zhangbo2012
 http://www.cnblogs.com/zhangbo2012/
"""

import time
import datetime

filepath = "testcdr.txt"
delaysec = 30

seccnt = {}
timepos = 0

def time2yyyymmddhhmiss():
    return datetime.datetime.now().strftime('%Y%m%d%H%M%S')

def yyyymmddhhmiss2time(yyyymmddhhmiss):
    return time.mktime(time.strptime(yyyymmddhhmiss,'%Y%m%d%H%M%S'))

print "---start---"
nowrectime='9999999999'
with open(filepath,'r') as rf:
    for line in rf:
        rectime = str(line).split("|")[timepos]
        if nowrectime<rectime:
            print nowrectime,seccnt[nowrectime]

        while (time.time() - yyyymmddhhmiss2time(rectime) < delaysec):
            time.sleep(1)

        if rectime in seccnt.keys():
            seccnt[rectime] +=1
        else:
            seccnt[rectime] = 1

        nowrectime = rectime

print "---end---"

 

运行效果