s10期Tornao分享
Tornado介绍
Tornado 是 FriendFeed 使用的可扩展的非阻塞式 web 服务器及其相关工具的开源版本。这个 Web 框架看起来有些像web.py 或者 Google 的 webapp，不过为了能有效利用非阻塞式服务器环境，这个 Web 框架还包含了一些相关的有用工具 和优化。
Tornado 和现在的主流 Web 服务器框架（包括大多数 Python 的框架）有着明显的区别：它是非阻塞式服务器，而且速度相当快。得利于其 非阻塞的方式和对 epoll 的运用，Tornado 每秒可以处理数以千计的连接，这意味着对于实时 Web 服务来说，Tornado 是一个理想的 Web 框架。我们开发这个 Web 服务器的主要目的就是为了处理 FriendFeed 的实时功能 ——在 FriendFeed 的应用里每一个活动用户都会保持着一个服务器连接。（关于如何扩容 服务器，以处理数以千计的客户端的连接的问题，请参阅 C10K problem。）
Tornado快速上手
一、第一次
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
application = tornado.web.Application([
(r"/index", MainHandler),
])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
第一步：执行脚本，监听 8888 端口
第二步：浏览器客户端访问 /index --> http://127.0.0.1:8888/index
第三步：服务器接受请求，并交由对应的类处理该请求
第四步：类接受到请求之后，根据请求方式（post / get / delete ...）的不同调用并执行相应的方法
第五步：方法返回值的字符串内容发送浏览器
二、路由系统
路由系统其实就是 url 和 类 的对应关系
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
class StoryHandler(tornado.web.RequestHandler):
def get(self, story_id):
self.write("You requested the story " + story_id)
class BuyHandler(tornado.web.RequestHandler):
def get(self):
self.write("buy.wupeiqi.com/index")
application = tornado.web.Application([
(r"/index", MainHandler),
(r"/story/([0-9]+)", StoryHandler),
])
application.add_handlers('buy.wupeiqi.com$', [
(r'/index',BuyHandler),
])
if __name__ == "__main__":
application.listen(80)
tornado.ioloop.IOLoop.instance().start()
三、模板
Tornao中的模板语言和django中类似，模板引擎将模板文件载入内存，然后将数据嵌入其中，最终获取到一个完整的字符串，再将字符串返回给请求者。
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render('home/index.html')
settings = {
'template_path': 'template',
}
application = tornado.web.Application([
(r"/index", MainHandler),
], **settings)
if __name__ == "__main__":
application.listen(80)
tornado.ioloop.IOLoop.instance().start()
escape:tornado.escape.xhtml_escape的別名xhtml_escape:tornado.escape.xhtml_escape的別名url_escape:tornado.escape.url_escape的別名json_encode:tornado.escape.json_encode的別名squeeze:tornado.escape.squeeze的別名linkify:tornado.escape.linkify的別名datetime: Python 的datetime模组handler: 当前的RequestHandler对象request:handler.request的別名current_user:handler.current_user的別名locale:handler.locale的別名_:handler.locale.translate的別名static_url: forhandler.static_url的別名xsrf_form_html:handler.xsrf_form_html的別名reverse_url:Application.reverse_url的別名Application设置中ui_methods和ui_modules下面的所有项目
四、静态文件
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render('home/index.html')
settings = {
'template_path': 'template',
'static_path': 'static',
'static_url_prefix': '/static/',
}
application = tornado.web.Application([
(r"/index", MainHandler),
], **settings)
if __name__ == "__main__":
application.listen(80)
tornado.ioloop.IOLoop.instance().start()
<!DOCTYPE html> <html> <head lang="en"> <meta charset="UTF-8"> <title></title> <link href="{{static_url("commons.css")}}" rel="stylesheet" /> </head> <body> <h1>hello</h1> </body> </html>
def get_content_version(cls, abspath): """Returns a version string for the resource at the given path. This class method may be overridden by subclasses. The default implementation is a hash of the file's contents. .. versionadded:: 3.1 """ data = cls.get_content(abspath) hasher = hashlib.md5() if isinstance(data, bytes): hasher.update(data) else: for chunk in data: hasher.update(chunk) return hasher.hexdigest()
五、cookie
class MainHandler(tornado.web.RequestHandler):
def get(self):
if not self.get_cookie("mycookie"):
self.set_cookie("mycookie", "myvalue")
self.write("Your cookie was not set yet!")
else:
self.write("Your cookie was set!")
Cookie 很容易被恶意的客户端伪造。加入你想在 cookie 中保存当前登陆用户的 id 之类的信息，你需要对 cookie 作签名以防止伪造。Tornado 通过 set_secure_cookie 和 get_secure_cookie 方法直接支持了这种功能。 要使用这些方法，你需要在创建应用时提供一个密钥，名字为 cookie_secret。 你可以把它作为一个关键词参数传入应用的设置中：
class MainHandler(tornado.web.RequestHandler):
def get(self):
if not self.get_secure_cookie("mycookie"):
self.set_secure_cookie("mycookie", "myvalue")
self.write("Your cookie was not set yet!")
else:
self.write("Your cookie was set!")
application = tornado.web.Application([
(r"/", MainHandler),
], cookie_secret="61oETzKXQAGaYdkL5gEmGeJJFuYh7EQnp2XdTP1o/Vo=")
def _create_signature_v1(secret, *parts): hash = hmac.new(utf8(secret), digestmod=hashlib.sha1) for part in parts: hash.update(utf8(part)) return utf8(hash.hexdigest()) def _create_signature_v2(secret, s): hash = hmac.new(utf8(secret), digestmod=hashlib.sha256) hash.update(utf8(s)) return utf8(hash.hexdigest())
def decode_signed_value(secret, name, value, max_age_days=31, clock=None, min_version=None): if clock is None: clock = time.time if min_version is None: min_version = DEFAULT_SIGNED_VALUE_MIN_VERSION if min_version > 2: raise ValueError("Unsupported min_version %d" % min_version) if not value: return None # Figure out what version this is. Version 1 did not include an # explicit version field and started with arbitrary base64 data, # which makes this tricky. value = utf8(value) m = _signed_value_version_re.match(value) if m is None: version = 1 else: try: version = int(m.group(1)) if version > 999: # Certain payloads from the version-less v1 format may # be parsed as valid integers. Due to base64 padding # restrictions, this can only happen for numbers whose # length is a multiple of 4, so we can treat all # numbers up to 999 as versions, and for the rest we # fall back to v1 format. version = 1 except ValueError: version = 1 if version < min_version: return None if version == 1: return _decode_signed_value_v1(secret, name, value, max_age_days, clock) elif version == 2: return _decode_signed_value_v2(secret, name, value, max_age_days, clock) else: return None def _decode_signed_value_v1(secret, name, value, max_age_days, clock): parts = utf8(value).split(b"|") if len(parts) != 3: return None signature = _create_signature_v1(secret, name, parts[0], parts[1]) if not _time_independent_equals(parts[2], signature): gen_log.warning("Invalid cookie signature %r", value) return None timestamp = int(parts[1]) if timestamp < clock() - max_age_days * 86400: gen_log.warning("Expired cookie %r", value) return None if timestamp > clock() + 31 * 86400: # _cookie_signature does not hash a delimiter between the # parts of the cookie, so an attacker could transfer trailing # digits from the payload to the timestamp without altering the # signature. For backwards compatibility, sanity-check timestamp # here instead of modifying _cookie_signature. gen_log.warning("Cookie timestamp in future; possible tampering %r", value) return None if parts[1].startswith(b"0"): gen_log.warning("Tampered cookie %r", value) return None try: return base64.b64decode(parts[0]) except Exception: return None def _decode_signed_value_v2(secret, name, value, max_age_days, clock): def _consume_field(s): length, _, rest = s.partition(b':') n = int(length) field_value = rest[:n] # In python 3, indexing bytes returns small integers; we must # use a slice to get a byte string as in python 2. if rest[n:n + 1] != b'|': raise ValueError("malformed v2 signed value field") rest = rest[n + 1:] return field_value, rest rest = value[2:] # remove version number try: key_version, rest = _consume_field(rest) timestamp, rest = _consume_field(rest) name_field, rest = _consume_field(rest) value_field, rest = _consume_field(rest) except ValueError: return None passed_sig = rest signed_string = value[:-len(passed_sig)] expected_sig = _create_signature_v2(secret, signed_string) if not _time_independent_equals(passed_sig, expected_sig): return None if name_field != utf8(name): return None timestamp = int(timestamp) if timestamp < clock() - max_age_days * 86400: # The signature has expired. return None try: return base64.b64decode(value_field) except Exception: return None
写cookie过程：
- 将值进行base64加密
- 对除值意外的内容进行签名，哈希算法（无法逆向解析）
- 拼接 签名 + 加密值
读cookie过程：
- 读取 签名 + 加密值
- 对签名进行验证
- base64解密，获取值内容
注：许多API验证机制和安全cookie的实现机制相同
六、csrf
Tornado中的夸张请求伪造和Django中的相似，跨站伪造请求(Cross-site request forgery)
settings = {
"xsrf_cookies": True,
}
application = tornado.web.Application([
(r"/", MainHandler),
(r"/login", LoginHandler),
], **settings)
<form action="/new_message" method="post"> {{ xsrf_form_html() }} <input type="text" name="message"/> <input type="submit" value="Post"/> </form>
function getCookie(name) { var r = document.cookie.match("\\b" + name + "=([^;]*)\\b"); return r ? r[1] : undefined; } jQuery.postJSON = function(url, args, callback) { args._xsrf = getCookie("_xsrf"); $.ajax({url: url, data: $.param(args), dataType: "text", type: "POST", success: function(response) { callback(eval("(" + response + ")")); }}); };
注意：ajax需要使用引入jquery
七、UI方法
# uimethods.py
def tab(self):
return 'UIMethod'
# uimodules.py
from tornado.web import UIModule
class custom(UIModule):
def render(self, *args, **kwargs):
return 'UIModule'
settings = { 'template_path': 'template', 'static_path': 'static', 'static_url_prefix': '/static/', 'ui_methods': mt, 'ui_modules': md, }
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
<link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
<h1>hello</h1>
{% module custom(123) %}
{{ tab() }}
</body>
</html>
八、用户验证
1、通过cookie进行用户验证
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
login_user = self.get_secure_cookie("login_user", None)
if login_user:
self.write(login_user)
else:
self.redirect('/login')
class LoginHandler(tornado.web.RequestHandler):
def get(self):
self.current_user()
self.render('login.html', **{'status': ''})
def post(self, *args, **kwargs):
username = self.get_argument('name')
password = self.get_argument('pwd')
if username == 'wupeiqi' and password == '123':
self.set_secure_cookie('login_user', '武沛齐')
self.redirect('/')
else:
self.render('login.html', **{'status': '用户名或密码错误'})
settings = {
'template_path': 'template',
'static_path': 'static',
'static_url_prefix': '/static/',
'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh'
}
application = tornado.web.Application([
(r"/index", MainHandler),
(r"/login", LoginHandler),
], **settings)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
2、Tornado内部提供cookie验证的机制
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
return self.get_secure_cookie("login_user")
class MainHandler(BaseHandler):
@tornado.web.authenticated
def get(self):
login_user = self.current_user
self.write(login_user)
class LoginHandler(tornado.web.RequestHandler):
def get(self):
self.current_user()
self.render('login.html', **{'status': ''})
def post(self, *args, **kwargs):
username = self.get_argument('name')
password = self.get_argument('pwd')
if username == 'wupeiqi' and password == '123':
self.set_secure_cookie('login_user', '武沛齐')
self.redirect('/')
else:
self.render('login.html', **{'status': '用户名或密码错误'})
settings = {
'template_path': 'template',
'static_path': 'static',
'static_url_prefix': '/static/',
'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
'login_url': '/login'
}
application = tornado.web.Application([
(r"/index", MainHandler),
(r"/login", LoginHandler),
], **settings)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
def authenticated(method): """Decorate methods with this to require that the user be logged in. If the user is not logged in, they will be redirected to the configured `login url <RequestHandler.get_login_url>`. If you configure a login url with a query parameter, Tornado will assume you know what you're doing and use it as-is. If not, it will add a `next` parameter so the login page knows where to send you once you're logged in. """ @functools.wraps(method) def wrapper(self, *args, **kwargs): if not self.current_user: if self.request.method in ("GET", "HEAD"): url = self.get_login_url() if "?" not in url: if urlparse.urlsplit(url).scheme: # if login url is absolute, make next absolute too next_url = self.request.full_url() else: next_url = self.request.uri url += "?" + urlencode(dict(next=next_url)) self.redirect(url) return raise HTTPError(403) return method(self, *args, **kwargs) return wrapper
3、自定义session验证
由于在之前已经了解到，cookie是这样搞的：
写cookie过程：
- 将值进行base64加密
- 对除值意外的内容进行签名，哈希算法（无法逆向解析）
- 拼接 签名 + 加密值
读cookie过程：
- 读取 签名 + 加密值
- 对签名进行验证
- base64解密，获取值内容
所以，我们会将base64加密的值返回给用户。而对于session，他只会将签名返回给用户，然后根据签名获取redis或数据库中保存的其他值。即：在session中，签名和session值的集合是一一对应的。
uapodjfaksfuka;skdj ==> {'username': 'wupeiqi', 'age': 18}
uapodasdfasdasdfai ==> {'username': 'alex', 'age': 16}
a).知识储备
#!/usr/bin/env python
# -*- coding:utf-8 -*-
class Foo(object):
def __getitem__(self, key):
print '__getitem__',key
def __setitem__(self, key, value):
print '__setitem__',key,value
def __delitem__(self, key):
print '__delitem__',key
obj = Foo()
result = obj['k1']
#obj['k2'] = 'wupeiqi'
#del obj['k1']
b). session实现机制
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time
session_container = {}
create_session_id = lambda: sha1('%s%s' % (os.urandom(16), time.time())).hexdigest()
class Session(object):
session_id = "__sessionId__"
def __init__(self, request):
session_value = request.get_cookie(Session.session_id)
if not session_value:
self._id = create_session_id()
else:
self._id = session_value
request.set_cookie(Session.session_id, self._id)
def __getitem__(self, key):
return session_container[self._id][key]
def __setitem__(self, key, value):
if session_container.has_key(self._id):
session_container[self._id][key] = value
else:
session_container[self._id] = {key: value}
def __delitem__(self, key):
del session_container[self._id][key]
class BaseHandler(tornado.web.RequestHandler):
def initialize(self):
# my_session['k1']访问 __getitem__ 方法
self.my_session = Session(self)
class MainHandler(BaseHandler):
def get(self):
print self.my_session['c_user']
print self.my_session['c_card']
self.write('index')
class LoginHandler(BaseHandler):
def get(self):
self.render('login.html', **{'status': ''})
def post(self, *args, **kwargs):
username = self.get_argument('name')
password = self.get_argument('pwd')
if username == 'wupeiqi' and password == '123':
self.my_session['c_user'] = 'wupeiqi'
self.my_session['c_card'] = '12312312309823012'
self.redirect('/index')
else:
self.render('login.html', **{'status': '用户名或密码错误'})
settings = {
'template_path': 'template',
'static_path': 'static',
'static_url_prefix': '/static/',
'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
'login_url': '/login'
}
application = tornado.web.Application([
(r"/index", MainHandler),
(r"/login", LoginHandler),
], **settings)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
c). session框架？
#!/usr/bin/env python #coding:utf-8 import sys import math from bisect import bisect if sys.version_info >= (2, 5): import hashlib md5_constructor = hashlib.md5 else: import md5 md5_constructor = md5.new class HashRing(object): """一致性哈希""" def __init__(self,nodes): '''初始化 nodes : 初始化的节点，其中包含节点已经节点对应的权重 默认每一个节点有32个虚拟节点 对于权重，通过多创建虚拟节点来实现 如：nodes = [ {'host':'127.0.0.1:8000','weight':1}, {'host':'127.0.0.1:8001','weight':2}, {'host':'127.0.0.1:8002','weight':1}, ] ''' self.ring = dict() self._sorted_keys = [] self.total_weight = 0 self.__generate_circle(nodes) def __generate_circle(self,nodes): for node_info in nodes: self.total_weight += node_info.get('weight',1) for node_info in nodes: weight = node_info.get('weight',1) node = node_info.get('host',None) virtual_node_count = math.floor((32*len(nodes)*weight) / self.total_weight) for i in xrange(0,int(virtual_node_count)): key = self.gen_key_thirty_two( '%s-%s' % (node, i) ) if self._sorted_keys.__contains__(key): raise Exception('该节点已经存在.') self.ring[key] = node self._sorted_keys.append(key) def add_node(self,node): ''' 新建节点 node : 要添加的节点，格式为：{'host':'127.0.0.1:8002','weight':1}，其中第一个元素表示节点，第二个元素表示该节点的权重。 ''' node = node.get('host',None) if not node: raise Exception('节点的地址不能为空.') weight = node.get('weight',1) self.total_weight += weight nodes_count = len(self._sorted_keys) + 1 virtual_node_count = math.floor((32 * nodes_count * weight) / self.total_weight) for i in xrange(0,int(virtual_node_count)): key = self.gen_key_thirty_two( '%s-%s' % (node, i) ) if self._sorted_keys.__contains__(key): raise Exception('该节点已经存在.') self.ring[key] = node self._sorted_keys.append(key) def remove_node(self,node): ''' 移除节点 node : 要移除的节点 '127.0.0.1:8000' ''' for key,value in self.ring.items(): if value == node: del self.ring[key] self._sorted_keys.remove(key) def get_node(self,string_key): '''获取 string_key 所在的节点''' pos = self.get_node_pos(string_key) if pos is None: return None return self.ring[ self._sorted_keys[pos]].split(':') def get_node_pos(self,string_key): '''获取 string_key 所在的节点的索引''' if not self.ring: return None key = self.gen_key_thirty_two(string_key) nodes = self._sorted_keys pos = bisect(nodes, key) return pos def gen_key_thirty_two(self, key): m = md5_constructor() m.update(key) return long(m.hexdigest(), 16) def gen_key_sixteen(self,key): b_key = self.__hash_digest(key) return self.__hash_val(b_key, lambda x: x) def __hash_val(self, b_key, entry_fn): return (( b_key[entry_fn(3)] << 24)|(b_key[entry_fn(2)] << 16)|(b_key[entry_fn(1)] << 8)| b_key[entry_fn(0)] ) def __hash_digest(self, key): m = md5_constructor() m.update(key) return map(ord, m.digest()) """ nodes = [ {'host':'127.0.0.1:8000','weight':1}, {'host':'127.0.0.1:8001','weight':2}, {'host':'127.0.0.1:8002','weight':1}, ] ring = HashRing(nodes) result = ring.get_node('98708798709870987098709879087') print result """
from hashlib import sha1 import os, time session_container = {} create_session_id = lambda: sha1('%s%s' % (os.urandom(16), time.time())).hexdigest() class Session(object): session_id = "__sessionId__" def __init__(self, request): session_value = request.get_cookie(Session.session_id) if not session_value: self._id = create_session_id() else: self._id = session_value request.set_cookie(Session.session_id, self._id) def __getitem__(self, key): # 根据 self._id 找到相对应的redis服务器，如： r = redis.StrictRedis(host='localhost', port=6379, db=0) # 使用python redis api 链接 # 获取数据，即：return self._redis.hget(self._id, name) return session_container[self._id][key] def __setitem__(self, key, value): # 根据 self._id 找到相对应的redis服务器 # 使用python redis api 链接 # 设置session # self._redis.hset(self._id, name, value) if session_container.has_key(self._id): session_container[self._id][key] = value else: session_container[self._id] = {key: value} def __delitem__(self, key): # 根据 self._id 找到相对应的redis服务器 # 使用python redis api 链接 # 删除 # return self._redis.hdel(self._id, name) del session_container[self._id][key]
九、自定义模型绑定
模型绑定有两个主要功能：
- 自动生成html表单
- 用户输入验证
在之前学习的Django中为程序员提供了非常便捷的模型绑定功能，但是在Tornado中，一切需要自己动手！！！
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time
import re
class MainForm(object):
def __init__(self):
self.host = "(.*)"
self.ip = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$"
self.port = '(\d+)'
self.phone = '^1[3|4|5|8][0-9]\d{8}$'
def check_valid(self, request):
form_dict = self.__dict__
for key, regular in form_dict.items():
post_value = request.get_argument(key)
# 让提交的数据 和 定义的正则表达式进行匹配
ret = re.match(regular, post_value)
print key,ret,post_value
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
def post(self, *args, **kwargs):
obj = MainForm()
result = obj.check_valid(self)
self.write('ok')
settings = {
'template_path': 'template',
'static_path': 'static',
'static_url_prefix': '/static/',
'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh',
'login_url': '/login'
}
application = tornado.web.Application([
(r"/index", MainHandler),
], **settings)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
<!DOCTYPE html> <html> <head lang="en"> <meta charset="UTF-8"> <title></title> <link href="{{static_url("commons.css")}}" rel="stylesheet" /> </head> <body> <h1>hello</h1> <form action="/index" method="post"> <p>hostname: <input type="text" name="host" /> </p> <p>ip: <input type="text" name="ip" /> </p> <p>port: <input type="text" name="port" /> </p> <p>phone: <input type="text" name="phone" /> </p> <input type="submit" /> </form> </body> </html>
再次进行调整和优化：
#!/usr/bin/env python # -*- coding:utf-8 -*- import tornado.ioloop import tornado.web from hashlib import sha1 import os, time import re class IPField(object): regular = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$" def __init__(self, required=True): self._required = required def valid(self, name, value): if not self._required: return value else: ret = re.match(IPField.regular, value) if not ret: raise Exception('%s Illegal' % name) return ret.group() class MainForm(object): def __init__(self): self.ip = IPField(required=False) self.new_ip = IPField(required=True) def check_valid(self, request): is_success = True form_dict = self.__dict__ for key, regular in form_dict.items(): try: post_value = request.get_argument(key) # 让提交的数据 和 定义的正则表达式进行匹配 result = regular.valid(key, post_value) except Exception,e: print e is_success = False return is_success class MainHandler(tornado.web.RequestHandler): def get(self): self.render('index.html') def post(self, *args, **kwargs): obj = MainForm() result = obj.check_valid(self) print result self.write('ok') settings = { 'template_path': 'template', 'static_path': 'static', 'static_url_prefix': '/static/', 'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh', 'login_url': '/login' } application = tornado.web.Application([ (r"/index", MainHandler), ], **settings) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
Tornado应用
利用Tornado 的原生restfull，轻松的开发API
需求：Haproxy服务化，即：提供API可以对Ha配置文件进行获取，添加，删除
global log 127.0.0.1 local2 daemon maxconn 256 log 127.0.0.1 local2 info defaults log global mode http timeout connect 5000ms timeout client 50000ms timeout server 50000ms option dontlognull listen stats :8888 stats enable stats uri /admin stats auth admin:1234 frontend oldboy.org bind 0.0.0.0:80 option httplog option httpclose option forwardfor log global acl www hdr_reg(host) -i www.oldboy.org use_backend www.oldboy.net if www backend www.oldboy.org server 10.1.70.9 10.1.70.9 weight 20 maxconn 3000
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
from lib import commons
import json
class MainHandler(tornado.web.RequestHandler):
def get(self):
backend_title = self.get_argument('origin', None)
obj = commons.GetController(backend_title)
response = obj.execute(backend_title)
self.write(json.dumps(response.__dict__))
def post(self, *args, **kwargs):
data = self.get_argument('origin', None)
obj = commons.AddController(data)
response = obj.execute()
self.write(json.dumps(response.__dict__))
def delete(self, *args, **kwargs):
data = self.get_argument('origin', None)
obj = commons.DelController(data)
response = obj.execute()
self.write(json.dumps(response.__dict__))
application = tornado.web.Application([
(r"/haproxy/", MainHandler),
(r"/haproxy", MainHandler),
])
if __name__ == "__main__":
application.listen(port=9999, address='0.0.0.0')
tornado.ioloop.IOLoop.instance().start()
#!/usr/bin/env python # -*- coding:utf-8 -*- from __future__ import with_statement import threading import json import os import commands from config import settings class BaseResponse(object): """ 封装返回值""" def __init__(self): self.status = False self.message = '' self.data = '' class BaseController(object): """ 操作基础类 """ rLock = threading.RLock() def __init__(self, origin): self.config_path = settings['haproxy_config'] self.config_path_bak = settings['haproxy_config_bak'] self.origin = origin def get_request(self): try: post_data = self.origin if not post_data or not post_data.strip(): raise Exception('parameters can not be empty') data = json.loads(post_data) return data except Exception, e: raise Exception('parameter is not valid') def check_post_data_format(self, post_dict): if not post_dict.has_key('rules') or not post_dict.has_key('backend'): raise Exception('parameter is not valid, not contain rules and backend') for item in post_dict['rules']: if not item.has_key('server') or not item.has_key('weight') or not item.has_key('maxconn') or not item.has_key('port'): raise Exception('parameter is not valid, not contain server or weight or maxconn or port') def check_post_data_format_part(self, post_dict): if not post_dict.has_key('rules') or not post_dict.has_key('backend'): raise Exception('parameter is not valid, not contain rules and backend') for item in post_dict['rules']: if not item.has_key('server') or not item.has_key('port'): raise Exception('parameter is not valid, not contain server or port') def file_block_list(self, frontend_title, backend_title): file_backend_list = [] file_frontend_list = [] file_backend_flag = False file_frontend_flag = False with open(self.config_path, 'r') as f: for line in f: if line.strip() == 'frontend %s' % (frontend_title,): file_frontend_flag = True file_frontend_list.append('frontend %s' % (frontend_title,)) continue elif line.strip() == 'backend %s' % (backend_title,): file_backend_list.append('backend %s' % (backend_title,)) file_backend_flag = True continue exec_line = line.expandtabs(8) if exec_line.startswith('%s' % (' '*8,)): if file_frontend_flag: file_frontend_list.append(exec_line.strip('\n')) continue if file_backend_flag: file_backend_list.append(exec_line.strip('\n')) continue else: file_frontend_flag = False file_backend_flag = False return [file_frontend_list, file_backend_list] def check(self, new_config_path): output = commands.getoutput('haproxy -f %s -c' % (new_config_path,)) if output.strip() == 'Configuration file is valid': return True else: raise Exception('failed to check new config：%s' % (output.strip())) def reload(self): status, output = commands.getstatusoutput('/etc/init.d/haproxy reload') if status == 0: return 1 else: return output def confirm(self, new_config): os.rename(self.config_path, self.config_path_bak) os.rename(new_config, self.config_path) def rollback(self): temp = self.config_path + '.error' os.rename(self.config_path, temp) os.rename(self.config_path_bak, self.config_path) class GetController(BaseController): def execute(self, backend_title): response = BaseResponse() try: if not backend_title or not backend_title.strip(): raise Exception('parameters can not be empty') #frontend_title = backend_title[backend_title.index('.')+1:] frontend_title = 'oldboy' file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title) response.data = {'frontend': file_frontend_list, 'backend': file_backend_list} response.status = True except Exception, e: response.message = str(e) return response class AddController(BaseController): """ 新建配置功能类 """ def process_frontend(self, backend_title, frontend_title, file_frontend_list): if not file_frontend_list: file_frontend_list.append('frontend %s' % (frontend_title,)) file_frontend_list.append('%sbind 0.0.0.0:8000' % (" "*8,)) file_frontend_list.append('%soption httplog' % (" "*8,)) file_frontend_list.append('%soption httpclose' % (" "*8,)) file_frontend_list.append('%soption forwardfor' % (" "*8,)) file_frontend_list.append('%slog global' % (" "*8,)) #pre = backend_title[0:backend_title.index('.')] acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title) if not file_frontend_list.__contains__(acl): file_frontend_list.append(acl) use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title) if not file_frontend_list.__contains__(use_backend): file_frontend_list.append(use_backend) def process_backend(self, post_data, backend_title, file_backend_list): if not file_backend_list: file_backend_list.append('backend %s' % (backend_title,)) for item in post_data['rules']: temp = "%sserver %s %s:%s weight %s maxconn %s" % (" "*8, item['server'], item['server'], item['port'], item['weight'], item['maxconn'], ) if not file_backend_list.__contains__(temp): file_backend_list.append(temp) def write_config(self, file_frontend_list, file_backend_list, is_new_frontend, is_new_backend): in_file_path = self.config_path out_file_path = in_file_path + '.new' frontend_title = file_frontend_list[0] backend_title = file_backend_list[0] frontend_flag = False backend_flag = False temp_flag = True with open(in_file_path) as infile, open(out_file_path, 'w') as outfile: for line in infile: line = line.expandtabs(8) if frontend_flag: if temp_flag: for item in file_frontend_list: outfile.write(item+'\n') temp_flag = False if line.startswith(' '*8): continue else: frontend_flag = False temp_flag = True if backend_flag: if temp_flag: for item in file_backend_list: outfile.write(item+'\n') temp_flag = False if line.startswith(' '*8): continue else: backend_flag = False temp_flag = True if line.strip() == frontend_title.strip(): frontend_flag = True continue if line.strip() == backend_title.strip(): backend_flag = True continue outfile.write(line) if is_new_frontend: outfile.write('\n') for item in file_frontend_list: outfile.write(item+'\n') if is_new_backend: outfile.write('\n') for item in file_backend_list: outfile.write(item+'\n') return out_file_path def execute(self): response = BaseResponse() BaseController.rLock.acquire() try: post_data = self.get_request() self.check_post_data_format(post_data) backend_title = post_data['backend'] #frontend_title = backend_title[backend_title.index('.')+1:] frontend_title = 'oldboy' file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title) is_new_frontend = False if file_frontend_list else True is_new_backend = False if file_backend_list else True self.process_frontend(backend_title, frontend_title, file_frontend_list) self.process_backend(post_data, backend_title, file_backend_list) new_config = self.write_config(file_frontend_list, file_backend_list, is_new_frontend, is_new_backend) if self.check(new_config): self.confirm(new_config) result = self.reload() if result == 1: response.status = True response.message = 'success' else: self.rollback() raise Exception(result) except Exception, e: response.message = str(e) finally: BaseController.rLock.release() return response class DelController(BaseController): """ 删除配置功能类 """ def process_backend(self, post_data, file_backend_list): for item in post_data['rules']: temp = "server %s %s:%s" % (item['server'], item['server'], item['port'],) ''' if file_backend_list.__contains__(temp): del file_backend_list[file_backend_list.index(temp)] ''' for rule in file_backend_list: if rule.strip().startswith(temp): del file_backend_list[file_backend_list.index(rule)] def process_frontend(self, backend_title, file_frontend_list): #pre = backend_title[0:backend_title.index('.')] acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title) if file_frontend_list.__contains__(acl): del file_frontend_list[file_frontend_list.index(acl)] use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title) if file_frontend_list.__contains__(use_backend): del file_frontend_list[file_frontend_list.index(use_backend)] def del_config(self, file_frontend_list, file_backend_list, is_del_backend): in_file_path = self.config_path out_file_path = in_file_path + '.new' frontend_title = file_frontend_list[0] backend_title = file_backend_list[0] frontend_flag = False backend_flag = False temp_flag = True with open(in_file_path) as infile, open(out_file_path, 'w') as outfile: for line in infile: line = line.expandtabs(8) if frontend_flag: if temp_flag: for item in file_frontend_list: outfile.write(item+'\n') temp_flag = False if line.startswith(' '*8): continue else: frontend_flag = False temp_flag = True if backend_flag: if temp_flag: if not is_del_backend: for item in file_backend_list: outfile.write(item+'\n') temp_flag = False if line.startswith(' '*8): continue else: backend_flag = False temp_flag = True if line.strip() == frontend_title.strip(): frontend_flag = True continue if line.strip() == backend_title.strip(): backend_flag = True continue outfile.write(line) return out_file_path def execute(self): response = BaseResponse() BaseController.rLock.acquire() try: post_data = self.get_request() self.check_post_data_format_part(post_data) backend_title = post_data['backend'] #frontend_title = backend_title[backend_title.index('.')+1:] frontend_title = 'oldboy' file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title) if not file_frontend_list or not file_backend_list: raise Exception('config not exist') self.process_backend(post_data, file_backend_list) is_del_backend = False if len(file_backend_list) == 1: self.process_frontend(backend_title, file_frontend_list) is_del_backend = True new_config = self.del_config(file_frontend_list, file_backend_list, is_del_backend) if self.check(new_config): self.confirm(new_config) result = self.reload() if result == 1: response.status = True response.message = 'success' else: self.rollback() raise Exception(result) except Exception, e: response.message = e.message finally: BaseController.rLock.release() return response
settings = { 'haproxy_config': '/etc/haproxy/haproxy.cfg', 'haproxy_config_bak': '/etc/haproxy/haproxy.cfg.bak', }
Tornodo源码剖析
- 路由内部机制
- 模板渲染原理
- epoll那里去了？
- ...
详细参见博客：http://www.cnblogs.com/wupeiqi/tag/Tornado/
