s10期Tornao分享

Tornado介绍

Tornado 是 FriendFeed 使用的可扩展的非阻塞式 web 服务器及其相关工具的开源版本。这个 Web 框架看起来有些像web.py 或者 Google 的 webapp,不过为了能有效利用非阻塞式服务器环境,这个 Web 框架还包含了一些相关的有用工具 和优化。

Tornado 和现在的主流 Web 服务器框架(包括大多数 Python 的框架)有着明显的区别:它是非阻塞式服务器,而且速度相当快。得利于其 非阻塞的方式和对 epoll 的运用,Tornado 每秒可以处理数以千计的连接,这意味着对于实时 Web 服务来说,Tornado 是一个理想的 Web 框架。我们开发这个 Web 服务器的主要目的就是为了处理 FriendFeed 的实时功能 ——在 FriendFeed 的应用里每一个活动用户都会保持着一个服务器连接。(关于如何扩容 服务器,以处理数以千计的客户端的连接的问题,请参阅 C10K problem。)

Tornado快速上手

一、第一次

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

application = tornado.web.Application([
    (r"/index", MainHandler),
])


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

第一步:执行脚本,监听 8888 端口

第二步:浏览器客户端访问 /index  -->  http://127.0.0.1:8888/index

第三步:服务器接受请求,并交由对应的类处理该请求

第四步:类接受到请求之后,根据请求方式(post / get / delete ...)的不同调用并执行相应的方法

第五步:方法返回值的字符串内容发送浏览器

二、路由系统

路由系统其实就是 url 和 类 的对应关系

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

class StoryHandler(tornado.web.RequestHandler):
    def get(self, story_id):
        self.write("You requested the story " + story_id)

class BuyHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("buy.wupeiqi.com/index")

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/story/([0-9]+)", StoryHandler),
])

application.add_handlers('buy.wupeiqi.com$', [
    (r'/index',BuyHandler),
])

if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()

三、模板

Tornao中的模板语言和django中类似,模板引擎将模板文件载入内存,然后将数据嵌入其中,最终获取到一个完整的字符串,再将字符串返回给请求者。

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('home/index.html')

settings = {
    'template_path': 'template',
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)


if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()
  • escapetornado.escape.xhtml_escape 的別名
  • xhtml_escapetornado.escape.xhtml_escape 的別名
  • url_escapetornado.escape.url_escape 的別名
  • json_encodetornado.escape.json_encode 的別名
  • squeezetornado.escape.squeeze 的別名
  • linkifytornado.escape.linkify 的別名
  • datetime: Python 的 datetime 模组
  • handler: 当前的 RequestHandler 对象
  • requesthandler.request 的別名
  • current_userhandler.current_user 的別名
  • localehandler.locale 的別名
  • _handler.locale.translate 的別名
  • static_url: for handler.static_url 的別名
  • xsrf_form_htmlhandler.xsrf_form_html 的別名
  • reverse_urlApplication.reverse_url 的別名
  • Application 设置中 ui_methods 和 ui_modules 下面的所有项目

四、静态文件

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('home/index.html')

settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)


if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()
<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
</body>
</html>
    def get_content_version(cls, abspath):
        """Returns a version string for the resource at the given path.

        This class method may be overridden by subclasses.  The
        default implementation is a hash of the file's contents.

        .. versionadded:: 3.1
        """
        data = cls.get_content(abspath)
        hasher = hashlib.md5()
        if isinstance(data, bytes):
            hasher.update(data)
        else:
            for chunk in data:
                hasher.update(chunk)
        return hasher.hexdigest()
缓存实现

五、cookie

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_cookie("mycookie"):
            self.set_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")

Cookie 很容易被恶意的客户端伪造。加入你想在 cookie 中保存当前登陆用户的 id 之类的信息,你需要对 cookie 作签名以防止伪造。Tornado 通过 set_secure_cookie 和 get_secure_cookie 方法直接支持了这种功能。 要使用这些方法,你需要在创建应用时提供一个密钥,名字为 cookie_secret。 你可以把它作为一个关键词参数传入应用的设置中:

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_secure_cookie("mycookie"):
            self.set_secure_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")
            
application = tornado.web.Application([
    (r"/", MainHandler),
], cookie_secret="61oETzKXQAGaYdkL5gEmGeJJFuYh7EQnp2XdTP1o/Vo=")
def _create_signature_v1(secret, *parts):
    hash = hmac.new(utf8(secret), digestmod=hashlib.sha1)
    for part in parts:
        hash.update(utf8(part))
    return utf8(hash.hexdigest())


def _create_signature_v2(secret, s):
    hash = hmac.new(utf8(secret), digestmod=hashlib.sha256)
    hash.update(utf8(s))
    return utf8(hash.hexdigest())
内部使用算法
def decode_signed_value(secret, name, value, max_age_days=31, clock=None, min_version=None):
    if clock is None:
        clock = time.time
    if min_version is None:
        min_version = DEFAULT_SIGNED_VALUE_MIN_VERSION
    if min_version > 2:
        raise ValueError("Unsupported min_version %d" % min_version)
    if not value:
        return None

    # Figure out what version this is.  Version 1 did not include an
    # explicit version field and started with arbitrary base64 data,
    # which makes this tricky.
    value = utf8(value)
    m = _signed_value_version_re.match(value)
    if m is None:
        version = 1
    else:
        try:
            version = int(m.group(1))
            if version > 999:
                # Certain payloads from the version-less v1 format may
                # be parsed as valid integers.  Due to base64 padding
                # restrictions, this can only happen for numbers whose
                # length is a multiple of 4, so we can treat all
                # numbers up to 999 as versions, and for the rest we
                # fall back to v1 format.
                version = 1
        except ValueError:
            version = 1

    if version < min_version:
        return None
    if version == 1:
        return _decode_signed_value_v1(secret, name, value, max_age_days, clock)
    elif version == 2:
        return _decode_signed_value_v2(secret, name, value, max_age_days, clock)
    else:
        return None


def _decode_signed_value_v1(secret, name, value, max_age_days, clock):
    parts = utf8(value).split(b"|")
    if len(parts) != 3:
        return None
    signature = _create_signature_v1(secret, name, parts[0], parts[1])
    if not _time_independent_equals(parts[2], signature):
        gen_log.warning("Invalid cookie signature %r", value)
        return None
    timestamp = int(parts[1])
    if timestamp < clock() - max_age_days * 86400:
        gen_log.warning("Expired cookie %r", value)
        return None
    if timestamp > clock() + 31 * 86400:
        # _cookie_signature does not hash a delimiter between the
        # parts of the cookie, so an attacker could transfer trailing
        # digits from the payload to the timestamp without altering the
        # signature.  For backwards compatibility, sanity-check timestamp
        # here instead of modifying _cookie_signature.
        gen_log.warning("Cookie timestamp in future; possible tampering %r", value)
        return None
    if parts[1].startswith(b"0"):
        gen_log.warning("Tampered cookie %r", value)
        return None
    try:
        return base64.b64decode(parts[0])
    except Exception:
        return None


def _decode_signed_value_v2(secret, name, value, max_age_days, clock):
    def _consume_field(s):
        length, _, rest = s.partition(b':')
        n = int(length)
        field_value = rest[:n]
        # In python 3, indexing bytes returns small integers; we must
        # use a slice to get a byte string as in python 2.
        if rest[n:n + 1] != b'|':
            raise ValueError("malformed v2 signed value field")
        rest = rest[n + 1:]
        return field_value, rest
    rest = value[2:]  # remove version number
    try:
        key_version, rest = _consume_field(rest)
        timestamp, rest = _consume_field(rest)
        name_field, rest = _consume_field(rest)
        value_field, rest = _consume_field(rest)
    except ValueError:
        return None
    passed_sig = rest
    signed_string = value[:-len(passed_sig)]
    expected_sig = _create_signature_v2(secret, signed_string)
    if not _time_independent_equals(passed_sig, expected_sig):
        return None
    if name_field != utf8(name):
        return None
    timestamp = int(timestamp)
    if timestamp < clock() - max_age_days * 86400:
        # The signature has expired.
        return None
    try:
        return base64.b64decode(value_field)
    except Exception:
        return None
内部使用算法

写cookie过程:

  • 将值进行base64加密
  • 对除值意外的内容进行签名,哈希算法(无法逆向解析)
  • 拼接 签名 + 加密值

读cookie过程:

  • 读取 签名 + 加密值
  • 对签名进行验证
  • base64解密,获取值内容

注:许多API验证机制和安全cookie的实现机制相同

六、csrf

Tornado中的夸张请求伪造和Django中的相似,跨站伪造请求(Cross-site request forgery)

settings = {
    "xsrf_cookies": True,
}
application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
], **settings)
<form action="/new_message" method="post">
  {{ xsrf_form_html() }}
  <input type="text" name="message"/>
  <input type="submit" value="Post"/>
</form>
function getCookie(name) {
    var r = document.cookie.match("\\b" + name + "=([^;]*)\\b");
    return r ? r[1] : undefined;
}

jQuery.postJSON = function(url, args, callback) {
    args._xsrf = getCookie("_xsrf");
    $.ajax({url: url, data: $.param(args), dataType: "text", type: "POST",
        success: function(response) {
        callback(eval("(" + response + ")"));
    }});
};
ajax请求

注意:ajax需要使用引入jquery

七、UI方法

# uimethods.py

def tab(self):
    return 'UIMethod'
# uimodules.py

from tornado.web import UIModule

class custom(UIModule):

    def render(self, *args, **kwargs):
        return 'UIModule'
settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'ui_methods': mt,
    'ui_modules': md,
}
<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
    {% module custom(123) %}
    {{ tab() }}
</body>
</html>

八、用户验证

1、通过cookie进行用户验证

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):

    def get(self):
        login_user = self.get_secure_cookie("login_user", None)
        if login_user:
            self.write(login_user)
        else:
            self.redirect('/login')


class LoginHandler(tornado.web.RequestHandler):
    def get(self):
        self.current_user()

        self.render('login.html', **{'status': ''})

    def post(self, *args, **kwargs):

        username = self.get_argument('name')
        password = self.get_argument('pwd')
        if username == 'wupeiqi' and password == '123':
            self.set_secure_cookie('login_user', '武沛齐')
            self.redirect('/')
        else:
            self.render('login.html', **{'status': '用户名或密码错误'})

settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh'
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

2、Tornado内部提供cookie验证的机制

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class BaseHandler(tornado.web.RequestHandler):

    def get_current_user(self):
        return self.get_secure_cookie("login_user")

class MainHandler(BaseHandler):

    @tornado.web.authenticated
    def get(self):
        login_user = self.current_user
        self.write(login_user)



class LoginHandler(tornado.web.RequestHandler):
    def get(self):
        self.current_user()

        self.render('login.html', **{'status': ''})

    def post(self, *args, **kwargs):

        username = self.get_argument('name')
        password = self.get_argument('pwd')
        if username == 'wupeiqi' and password == '123':
            self.set_secure_cookie('login_user', '武沛齐')
            self.redirect('/')
        else:
            self.render('login.html', **{'status': '用户名或密码错误'})

settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
    'login_url': '/login'
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()
def authenticated(method):
    """Decorate methods with this to require that the user be logged in.

    If the user is not logged in, they will be redirected to the configured
    `login url <RequestHandler.get_login_url>`.

    If you configure a login url with a query parameter, Tornado will
    assume you know what you're doing and use it as-is.  If not, it
    will add a `next` parameter so the login page knows where to send
    you once you're logged in.
    """
    @functools.wraps(method)
    def wrapper(self, *args, **kwargs):
        if not self.current_user:
            if self.request.method in ("GET", "HEAD"):
                url = self.get_login_url()
                if "?" not in url:
                    if urlparse.urlsplit(url).scheme:
                        # if login url is absolute, make next absolute too
                        next_url = self.request.full_url()
                    else:
                        next_url = self.request.uri
                    url += "?" + urlencode(dict(next=next_url))
                self.redirect(url)
                return
            raise HTTPError(403)
        return method(self, *args, **kwargs)
    return wrapper
验证装饰器

3、自定义session验证

由于在之前已经了解到,cookie是这样搞的:

写cookie过程:

  • 将值进行base64加密
  • 对除值意外的内容进行签名,哈希算法(无法逆向解析)
  • 拼接 签名 + 加密值

读cookie过程:

  • 读取 签名 + 加密值
  • 对签名进行验证
  • base64解密,获取值内容

所以,我们会将base64加密的值返回给用户。而对于session,他只会将签名返回给用户,然后根据签名获取redis或数据库中保存的其他值。即:在session中,签名和session值的集合是一一对应的。

uapodjfaksfuka;skdj   ==> {'username': 'wupeiqi', 'age': 18}

uapodasdfasdasdfai   ==> {'username': 'alex', 'age': 16}

a).知识储备

#!/usr/bin/env python
# -*- coding:utf-8 -*-

class Foo(object):

    def __getitem__(self, key):
        print  '__getitem__',key

    def __setitem__(self, key, value):
        print '__setitem__',key,value

    def __delitem__(self, key):
        print '__delitem__',key



obj = Foo()
result = obj['k1']
#obj['k2'] = 'wupeiqi'
#del obj['k1'] 

b). session实现机制

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time

session_container = {}

create_session_id = lambda: sha1('%s%s' % (os.urandom(16), time.time())).hexdigest()


class Session(object):

    session_id = "__sessionId__"

    def __init__(self, request):
        session_value = request.get_cookie(Session.session_id)
        if not session_value:
            self._id = create_session_id()
        else:
            self._id = session_value
        request.set_cookie(Session.session_id, self._id)

    def __getitem__(self, key):
        return session_container[self._id][key]

    def __setitem__(self, key, value):
        if session_container.has_key(self._id):
            session_container[self._id][key] = value
        else:
            session_container[self._id] = {key: value}

    def __delitem__(self, key):
        del session_container[self._id][key]


class BaseHandler(tornado.web.RequestHandler):

    def initialize(self):
        # my_session['k1']访问 __getitem__ 方法
        self.my_session = Session(self)


class MainHandler(BaseHandler):

    def get(self):
        print self.my_session['c_user']
        print self.my_session['c_card']
        self.write('index')

class LoginHandler(BaseHandler):

    def get(self):
        self.render('login.html', **{'status': ''})

    def post(self, *args, **kwargs):

        username = self.get_argument('name')
        password = self.get_argument('pwd')
        if username == 'wupeiqi' and password == '123':

            self.my_session['c_user'] = 'wupeiqi'
            self.my_session['c_card'] = '12312312309823012'

            self.redirect('/index')
        else:
            self.render('login.html', **{'status': '用户名或密码错误'})

settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
    'login_url': '/login'
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

c). session框架?

#!/usr/bin/env python
#coding:utf-8

import sys
import math
from bisect import bisect


if sys.version_info >= (2, 5):
    import hashlib
    md5_constructor = hashlib.md5
else:
    import md5
    md5_constructor = md5.new


class HashRing(object):
    """一致性哈希"""
    
    def __init__(self,nodes):
        '''初始化
        nodes : 初始化的节点,其中包含节点已经节点对应的权重
                默认每一个节点有32个虚拟节点
                对于权重,通过多创建虚拟节点来实现
                如:nodes = [
                        {'host':'127.0.0.1:8000','weight':1},
                        {'host':'127.0.0.1:8001','weight':2},
                        {'host':'127.0.0.1:8002','weight':1},
                    ]
        '''
        
        self.ring = dict()
        self._sorted_keys = []

        self.total_weight = 0
        
        self.__generate_circle(nodes)
        
            
            
    def __generate_circle(self,nodes):
        for node_info in nodes:
            self.total_weight += node_info.get('weight',1)
            
        for node_info in nodes:
            weight = node_info.get('weight',1)
            node = node_info.get('host',None)
                
            virtual_node_count = math.floor((32*len(nodes)*weight) / self.total_weight)
            for i in xrange(0,int(virtual_node_count)):
                key = self.gen_key_thirty_two( '%s-%s' % (node, i) )
                if self._sorted_keys.__contains__(key):
                    raise Exception('该节点已经存在.')
                self.ring[key] = node
                self._sorted_keys.append(key)
            
    def add_node(self,node):
        ''' 新建节点
        node : 要添加的节点,格式为:{'host':'127.0.0.1:8002','weight':1},其中第一个元素表示节点,第二个元素表示该节点的权重。
        '''
        node = node.get('host',None)
        if not node:
                raise Exception('节点的地址不能为空.')
                
        weight = node.get('weight',1)
        
        self.total_weight += weight
        nodes_count = len(self._sorted_keys) + 1
        
        virtual_node_count = math.floor((32 * nodes_count * weight) / self.total_weight)
        for i in xrange(0,int(virtual_node_count)):
            key = self.gen_key_thirty_two( '%s-%s' % (node, i) )
            if self._sorted_keys.__contains__(key):
                raise Exception('该节点已经存在.')
            self.ring[key] = node
            self._sorted_keys.append(key)
        
    def remove_node(self,node):
        ''' 移除节点
        node : 要移除的节点 '127.0.0.1:8000'
        '''
        for key,value in self.ring.items():
            if value == node:
                del self.ring[key]
                self._sorted_keys.remove(key)
    
    def get_node(self,string_key):
        '''获取 string_key 所在的节点'''
        pos = self.get_node_pos(string_key)
        if pos is None:
            return None
        return self.ring[ self._sorted_keys[pos]].split(':')
    
    def get_node_pos(self,string_key):
        '''获取 string_key 所在的节点的索引'''
        if not self.ring:
            return None
            
        key = self.gen_key_thirty_two(string_key)
        nodes = self._sorted_keys
        pos = bisect(nodes, key)
        return pos
    
    def gen_key_thirty_two(self, key):
        
        m = md5_constructor()
        m.update(key)
        return long(m.hexdigest(), 16)
        
    def gen_key_sixteen(self,key):
        
        b_key = self.__hash_digest(key)
        return self.__hash_val(b_key, lambda x: x)

    def __hash_val(self, b_key, entry_fn):
        return (( b_key[entry_fn(3)] << 24)|(b_key[entry_fn(2)] << 16)|(b_key[entry_fn(1)] << 8)| b_key[entry_fn(0)] )

    def __hash_digest(self, key):
        m = md5_constructor()
        m.update(key)
        return map(ord, m.digest())


"""
nodes = [
    {'host':'127.0.0.1:8000','weight':1},
    {'host':'127.0.0.1:8001','weight':2},
    {'host':'127.0.0.1:8002','weight':1},
]

ring = HashRing(nodes)
result = ring.get_node('98708798709870987098709879087')
print result

"""
一致性哈希
from hashlib import sha1
import os, time

session_container = {}

create_session_id = lambda: sha1('%s%s' % (os.urandom(16), time.time())).hexdigest()


class Session(object):

    session_id = "__sessionId__"

    def __init__(self, request):
        session_value = request.get_cookie(Session.session_id)
        if not session_value:
            self._id = create_session_id()
        else:
            self._id = session_value
        request.set_cookie(Session.session_id, self._id)

    def __getitem__(self, key):
        # 根据 self._id 找到相对应的redis服务器,如: r = redis.StrictRedis(host='localhost', port=6379, db=0)
        # 使用python redis api 链接
        # 获取数据,即:return self._redis.hget(self._id, name)

        return session_container[self._id][key]

    def __setitem__(self, key, value):
        # 根据 self._id 找到相对应的redis服务器
        # 使用python redis api 链接
        # 设置session
        # self._redis.hset(self._id, name, value)

        if session_container.has_key(self._id):
            session_container[self._id][key] = value
        else:
            session_container[self._id] = {key: value}

    def __delitem__(self, key):
        # 根据 self._id 找到相对应的redis服务器
        # 使用python redis api 链接
        # 删除
        # return self._redis.hdel(self._id, name)
        del session_container[self._id][key]
session

九、自定义模型绑定

模型绑定有两个主要功能:

  • 自动生成html表单
  • 用户输入验证

在之前学习的Django中为程序员提供了非常便捷的模型绑定功能,但是在Tornado中,一切需要自己动手!!!

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time
import re


class MainForm(object):
    def __init__(self):
        self.host = "(.*)"
        self.ip = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$"
        self.port = '(\d+)'
        self.phone = '^1[3|4|5|8][0-9]\d{8}$'

    def check_valid(self, request):
        form_dict = self.__dict__
        for key, regular in form_dict.items():
            post_value = request.get_argument(key)
            # 让提交的数据 和 定义的正则表达式进行匹配
            ret = re.match(regular, post_value)
            print key,ret,post_value


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('index.html')
    def post(self, *args, **kwargs):
        obj = MainForm()
        result = obj.check_valid(self)
        self.write('ok')



settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
    'login_url': '/login'
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()
<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
    <form action="/index" method="post">

        <p>hostname: <input type="text" name="host" /> </p>
        <p>ip: <input type="text" name="ip" /> </p>
        <p>port: <input type="text" name="port" /> </p>
        <p>phone: <input type="text" name="phone" /> </p>
        <input type="submit" />
    </form>
</body>
</html>
html

再次进行调整和优化:

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time
import re


class IPField(object):
    regular = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$"

    def __init__(self, required=True):
        self._required = required

    def valid(self, name, value):
        if not self._required:
            return value
        else:
            ret = re.match(IPField.regular, value)
            if not ret:
                raise Exception('%s Illegal' % name)
            return ret.group()



class MainForm(object):
    def __init__(self):
        self.ip = IPField(required=False)
        self.new_ip = IPField(required=True)

    def check_valid(self, request):
        is_success = True
        form_dict = self.__dict__
        for key, regular in form_dict.items():
            try:
                post_value = request.get_argument(key)
                # 让提交的数据 和 定义的正则表达式进行匹配
                result = regular.valid(key, post_value)
            except Exception,e:
                print e
                is_success = False

        return is_success

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('index.html')
    def post(self, *args, **kwargs):
        obj = MainForm()
        result = obj.check_valid(self)
        print result
        self.write('ok')



settings = {
    'template_path': 'template',
    'static_path': 'static',
    'static_url_prefix': '/static/',
    'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
    'login_url': '/login'
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)


if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()
Web框架内部验证机制

Tornado应用

利用Tornado 的原生restfull,轻松的开发API

需求:Haproxy服务化,即:提供API可以对Ha配置文件进行获取,添加,删除

global       
        log 127.0.0.1 local2
        daemon
        maxconn 256
        log 127.0.0.1 local2 info
defaults
        log global
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms
        option  dontlognull

listen stats :8888
        stats enable
        stats uri       /admin
        stats auth      admin:1234

frontend oldboy.org
        bind 0.0.0.0:80
        option httplog
        option httpclose
        option  forwardfor
        log global
        acl www hdr_reg(host) -i www.oldboy.org
        use_backend www.oldboy.net if www

backend www.oldboy.org
        server 10.1.70.9 10.1.70.9 weight 20 maxconn 3000
配置文件
#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from lib import commons
import json

class MainHandler(tornado.web.RequestHandler):

    def get(self):
        backend_title = self.get_argument('origin', None)
        obj = commons.GetController(backend_title)
        response = obj.execute(backend_title)
        self.write(json.dumps(response.__dict__))

    def post(self, *args, **kwargs):
        data = self.get_argument('origin', None)
        obj = commons.AddController(data)
        response = obj.execute()
        self.write(json.dumps(response.__dict__))

    def delete(self, *args, **kwargs):
        data = self.get_argument('origin', None)
        obj = commons.DelController(data)
        response = obj.execute()
        self.write(json.dumps(response.__dict__))


application = tornado.web.Application([
    (r"/haproxy/", MainHandler),
    (r"/haproxy", MainHandler),
])

if __name__ == "__main__":
    application.listen(port=9999, address='0.0.0.0')
    tornado.ioloop.IOLoop.instance().start()
#!/usr/bin/env python
# -*- coding:utf-8 -*-

from __future__ import with_statement
import threading
import json
import os
import commands
from config import settings


class BaseResponse(object):
    """ 封装返回值"""

    def __init__(self):
        self.status = False
        self.message = ''
        self.data = ''


class BaseController(object):
    """ 操作基础类 """
    rLock = threading.RLock()

    def __init__(self, origin):
        self.config_path = settings['haproxy_config']
        self.config_path_bak = settings['haproxy_config_bak']
        self.origin = origin

    def get_request(self):
        try:
            post_data = self.origin
            if not post_data or not post_data.strip():
                raise Exception('parameters can not be empty')
            data = json.loads(post_data)
            return data
        except Exception, e:
            raise Exception('parameter is not valid')

    def check_post_data_format(self, post_dict):

        if not post_dict.has_key('rules') or not post_dict.has_key('backend'):
            raise Exception('parameter is not valid, not contain rules and backend')
        for item in post_dict['rules']:
            if not item.has_key('server') or not item.has_key('weight') or not item.has_key('maxconn') or not item.has_key('port'):
                raise Exception('parameter is not valid, not contain server or weight or maxconn or port')


    def check_post_data_format_part(self, post_dict):

        if not post_dict.has_key('rules') or not post_dict.has_key('backend'):
            raise Exception('parameter is not valid, not contain rules and backend')
        for item in post_dict['rules']:
            if not item.has_key('server') or not item.has_key('port'):
                raise Exception('parameter is not valid, not contain server or port')


    def file_block_list(self, frontend_title, backend_title):
        file_backend_list = []
        file_frontend_list = []
        file_backend_flag = False
        file_frontend_flag = False

        with open(self.config_path, 'r') as f:
            for line in f:
                if line.strip() == 'frontend %s' % (frontend_title,):
                    file_frontend_flag = True
                    file_frontend_list.append('frontend %s' % (frontend_title,))
                    continue
                elif line.strip() == 'backend %s' % (backend_title,):
                    file_backend_list.append('backend %s' % (backend_title,))
                    file_backend_flag = True
                    continue
                exec_line = line.expandtabs(8)
                if exec_line.startswith('%s' % (' '*8,)):
                    if file_frontend_flag:
                        file_frontend_list.append(exec_line.strip('\n'))
                        continue
                    if file_backend_flag:
                        file_backend_list.append(exec_line.strip('\n'))
                        continue
                else:
                    file_frontend_flag = False
                    file_backend_flag = False

        return [file_frontend_list, file_backend_list]

    def check(self, new_config_path):
        output = commands.getoutput('haproxy -f %s -c' % (new_config_path,))
        if output.strip() == 'Configuration file is valid':
            return True
        else:
            raise Exception('failed to check new config:%s' % (output.strip()))

    def reload(self):
        status, output = commands.getstatusoutput('/etc/init.d/haproxy reload')
        if status == 0:
            return 1
        else:
            return output

    def confirm(self, new_config):
        os.rename(self.config_path, self.config_path_bak)
        os.rename(new_config, self.config_path)

    def rollback(self):
        temp = self.config_path + '.error'
        os.rename(self.config_path, temp)
        os.rename(self.config_path_bak, self.config_path)


class GetController(BaseController):

    def execute(self, backend_title):
        response = BaseResponse()
        try:
            if not backend_title or not backend_title.strip():
                raise Exception('parameters can not be empty')
            #frontend_title = backend_title[backend_title.index('.')+1:]
            frontend_title = 'oldboy'
            file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title)
            response.data = {'frontend': file_frontend_list, 'backend': file_backend_list}
            response.status = True
        except Exception, e:
            response.message = str(e)
        return response


class AddController(BaseController):
    """ 新建配置功能类 """

    def process_frontend(self, backend_title, frontend_title, file_frontend_list):
        if not file_frontend_list:
            file_frontend_list.append('frontend %s' % (frontend_title,))
            file_frontend_list.append('%sbind 0.0.0.0:8000' % (" "*8,))
            file_frontend_list.append('%soption httplog' % (" "*8,))
            file_frontend_list.append('%soption httpclose' % (" "*8,))
            file_frontend_list.append('%soption  forwardfor' % (" "*8,))
            file_frontend_list.append('%slog global' % (" "*8,))

        #pre = backend_title[0:backend_title.index('.')]
        acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title)
        if not file_frontend_list.__contains__(acl):
            file_frontend_list.append(acl)
        use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title)
        if not file_frontend_list.__contains__(use_backend):
            file_frontend_list.append(use_backend)

    def process_backend(self, post_data, backend_title, file_backend_list):
        if not file_backend_list:
            file_backend_list.append('backend %s' % (backend_title,))
        for item in post_data['rules']:
            temp = "%sserver %s %s:%s weight %s maxconn %s" % (" "*8, item['server'], item['server'], item['port'], item['weight'], item['maxconn'], )
            if not file_backend_list.__contains__(temp):
                file_backend_list.append(temp)

    def write_config(self, file_frontend_list, file_backend_list, is_new_frontend, is_new_backend):
        in_file_path = self.config_path
        out_file_path = in_file_path + '.new'

        frontend_title = file_frontend_list[0]
        backend_title = file_backend_list[0]
        frontend_flag = False
        backend_flag = False
        temp_flag = True
        with open(in_file_path) as infile, open(out_file_path, 'w') as outfile:
            for line in infile:
                line = line.expandtabs(8)

                if frontend_flag:
                    if temp_flag:
                        for item in file_frontend_list:
                            outfile.write(item+'\n')
                        temp_flag = False
                    if line.startswith(' '*8):
                        continue
                    else:
                        frontend_flag = False
                        temp_flag = True

                if backend_flag:
                    if temp_flag:
                        for item in file_backend_list:
                            outfile.write(item+'\n')
                        temp_flag = False
                    if line.startswith(' '*8):
                        continue
                    else:
                        backend_flag = False
                        temp_flag = True

                if line.strip() == frontend_title.strip():
                    frontend_flag = True
                    continue

                if line.strip() == backend_title.strip():
                    backend_flag = True
                    continue

                outfile.write(line)

            if is_new_frontend:
                outfile.write('\n')
                for item in file_frontend_list:
                    outfile.write(item+'\n')
            if is_new_backend:
                outfile.write('\n')
                for item in file_backend_list:
                    outfile.write(item+'\n')
        return out_file_path

    def execute(self):
        response = BaseResponse()
        BaseController.rLock.acquire()
        try:
            post_data = self.get_request()

            self.check_post_data_format(post_data)

            backend_title = post_data['backend']
            #frontend_title = backend_title[backend_title.index('.')+1:]
            frontend_title = 'oldboy'

            file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title)

            is_new_frontend = False if file_frontend_list else True
            is_new_backend = False if file_backend_list else True

            self.process_frontend(backend_title, frontend_title, file_frontend_list)
            self.process_backend(post_data, backend_title, file_backend_list)
            new_config = self.write_config(file_frontend_list, file_backend_list, is_new_frontend, is_new_backend)

            if self.check(new_config):
                self.confirm(new_config)
                result = self.reload()
                if result == 1:
                    response.status = True
                    response.message = 'success'
                else:
                    self.rollback()
                    raise Exception(result)

        except Exception, e:
            response.message = str(e)
        finally:
            BaseController.rLock.release()
        return response


class DelController(BaseController):
    """ 删除配置功能类 """

    def process_backend(self, post_data, file_backend_list):
        for item in post_data['rules']:
            temp = "server %s %s:%s" % (item['server'], item['server'], item['port'],)
            '''
            if file_backend_list.__contains__(temp):
                del file_backend_list[file_backend_list.index(temp)]
            '''
            for rule in file_backend_list:
                if rule.strip().startswith(temp):
                    del file_backend_list[file_backend_list.index(rule)]


    def process_frontend(self, backend_title, file_frontend_list):
        #pre = backend_title[0:backend_title.index('.')]

        acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title)

        if file_frontend_list.__contains__(acl):
            del file_frontend_list[file_frontend_list.index(acl)]

        use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title)
        if file_frontend_list.__contains__(use_backend):
            del file_frontend_list[file_frontend_list.index(use_backend)]


    def del_config(self, file_frontend_list, file_backend_list, is_del_backend):
        in_file_path = self.config_path
        out_file_path = in_file_path + '.new'

        frontend_title = file_frontend_list[0]
        backend_title = file_backend_list[0]

        frontend_flag = False
        backend_flag = False
        temp_flag = True
        with open(in_file_path) as infile, open(out_file_path, 'w') as outfile:
            for line in infile:
                line = line.expandtabs(8)

                if frontend_flag:
                    if temp_flag:
                        for item in file_frontend_list:
                            outfile.write(item+'\n')
                        temp_flag = False
                    if line.startswith(' '*8):
                        continue
                    else:
                        frontend_flag = False
                        temp_flag = True

                if backend_flag:
                    if temp_flag:
                        if not is_del_backend:
                            for item in file_backend_list:
                                outfile.write(item+'\n')
                        temp_flag = False
                    if line.startswith(' '*8):
                        continue
                    else:
                        backend_flag = False
                        temp_flag = True

                if line.strip() == frontend_title.strip():
                    frontend_flag = True
                    continue

                if line.strip() == backend_title.strip():
                    backend_flag = True
                    continue

                outfile.write(line)
        return out_file_path

    def execute(self):
        response = BaseResponse()
        BaseController.rLock.acquire()
        try:
            post_data = self.get_request()

            self.check_post_data_format_part(post_data)

            backend_title = post_data['backend']
            #frontend_title = backend_title[backend_title.index('.')+1:]
            frontend_title = 'oldboy'

            file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title)

            if not file_frontend_list or not file_backend_list:
                raise Exception('config not exist')

            self.process_backend(post_data, file_backend_list)
            is_del_backend = False

            if len(file_backend_list) == 1:
                self.process_frontend(backend_title, file_frontend_list)
                is_del_backend = True

            new_config = self.del_config(file_frontend_list, file_backend_list, is_del_backend)
            if self.check(new_config):
                self.confirm(new_config)
                result = self.reload()
                if result == 1:
                    response.status = True
                    response.message = 'success'
                else:
                    self.rollback()
                    raise Exception(result)

        except Exception, e:
            response.message = e.message
        finally:
            BaseController.rLock.release()

        return response
lib/commons.py
settings = {
    'haproxy_config': '/etc/haproxy/haproxy.cfg',
    'haproxy_config_bak': '/etc/haproxy/haproxy.cfg.bak',
}
config.py

Tornodo源码剖析

- 路由内部机制
- 模板渲染原理
- epoll那里去了?
- ...

详细参见博客:http://www.cnblogs.com/wupeiqi/tag/Tornado/

 

posted @ 2015-11-27 09:29  武沛齐  阅读(...)  评论(... 编辑 收藏