css parsing中词法的RegEx(python)

W3C网站上的有点小bug,而且他们写的正则是FLEX一类的,和自己手写时不太一样。瞎改了一下,贴这,谁用得着谁拿去,有问题请通知作者更新。虽然是python的,其实现在这些RegEx都是perl风格的,应该能通用。

除了做词法,这些定义也可以这么用:如果是找找特定字串,从这里选取一部分改吧改吧就好了;或者在这堆天书里看看有啥用法是咱不知道的?

(?:..)的意思是不建立capturing group,不知道别的运行时是不是这么定义的;字串前面的r意思是不用转义字符,.NET里好像是@?已然忘了。另外,W3C的词法分析配置是忽略大小写,不忽略空白:别忘了后代选择器是基于空白的。另外,w3c的没有.class的定义,这是我自己加的,会让微软的filter变成:IDENT CLASS CLASS LPARENTHESE..这样(本来w3c版本也解析不了这个),其它副作用不明。

(版本:CSS 2.1 + CSS 3 Selector,问我为啥不80字符换行?不乐意。)

A = r"(?:a|\\0{0,4}(?:41|61)(?:\r\n|[ \t\r\n\f])?)"
C
= r"(?:c|\\0{0,4}(?:43|63)(?:\r\n|[ \t\r\n\f])?)"
D
= r"(?:d|\\0{0,4}(?:44|64)(?:\r\n|[ \t\r\n\f])?)"
E
= r"(?:e|\\0{0,4}(?:45|65)(?:\r\n|[ \t\r\n\f])?)"
G
= r"(?:g|\\0{0,4}(?:47|67)(?:\r\n|[ \t\r\n\f])?|\\g)"
H
= r"(?:h|\\0{0,4}(?:48|68)(?:\r\n|[ \t\r\n\f])?|\\h)"
I
= r"(?:i|\\0{0,4}(?:49|69)(?:\r\n|[ \t\r\n\f])?|\\i)"
K
= r"(?:k|\\0{0,4}(?:4b|6b)(?:\r\n|[ \t\r\n\f])?|\\k)"
L
= r"(?:l|\\0{0,4}(?:4c|6c)(?:\r\n|[ \t\r\n\f])?|\\l)"
M
= r"(?:m|\\0{0,4}(?:4d|6d)(?:\r\n|[ \t\r\n\f])?|\\m)"
N
= r"(?:n|\\0{0,4}(?:4e|6e)(?:\r\n|[ \t\r\n\f])?|\\n)"
O
= r"(?:o|\\0{0,4}(?:4f|6f)(?:\r\n|[ \t\r\n\f])?|\\o)"
P
= r"(?:p|\\0{0,4}(?:50|70)(?:\r\n|[ \t\r\n\f])?|\\p)"
R
= r"(?:r|\\0{0,4}(?:52|72)(?:\r\n|[ \t\r\n\f])?|\\r)"
S
= r"(?:s|\\0{0,4}(?:53|73)(?:\r\n|[ \t\r\n\f])?|\\s)"
T
= r"(?:t|\\0{0,4}(?:54|74)(?:\r\n|[ \t\r\n\f])?|\\t)"
U
= r"(?:u|\\0{0,4}(?:55|75)(?:\r\n|[ \t\r\n\f])?|\\u)"
X
= r"(?:x|\\0{0,4}(?:58|78)(?:\r\n|[ \t\r\n\f])?|\\x)"
Z
= r"(?:z|\\0{0,4}(?:5a|7a)(?:\r\n|[ \t\r\n\f])?|\\z)"

# from css 3 selector
#
D = r"d|\\0{0,4}(44|64)(\r\n|[ \t\r\n\f])?"
#
E = r"e|\\0{0,4}(45|65)(\r\n|[ \t\r\n\f])?"
#
N = r"n|\\0{0,4}(4e|6e)(\r\n|[ \t\r\n\f])?|\\n"
#
O = r"o|\\0{0,4}(4f|6f)(\r\n|[ \t\r\n\f])?|\\o"
#
T = r"t|\\0{0,4}(54|74)(\r\n|[ \t\r\n\f])?|\\t"
#
V = r"v|\\0{0,4}(58|78)(\r\n|[ \t\r\n\f])?|\\v"

s
= r"[ \t\r\n\f]+"
w
= r"[ \t\r\n\f]*"
nl
= r"(?:\n|\r\n|\r|\f)"
num
= "\-?(?:[0-9]+|[0-9]*\.[0-9]+)"
nonascii
= r"[^\0-\237]" # 2.1: [\200-\4177777]
unicode_ = r"\\[0-9a-f]{1,6}(?:\r\n|[ \n\r\t\f])?"
escape
= "(?:" + unicode_ + r"|\\[^\n\r\f0-9a-f])"

string1
= r'\"(?:[^\n\r\f\\"]|\\' + nl + r'|' + escape + ')*\"'
string2
= r"\'(?:[^\n\r\f\\']|\\" + nl + r"|" + escape + r")*\'"
string
= "(?:" + string1 + "|" + string2 + ")"
invalid1
= r'\"(?:[^\n\r\f\\"]|\\' + nl +'|' + nonascii + '|' + escape + ')*'
invalid2
= r"\'(?:[^\n\r\f\\']|\\" + nl + "|" + nonascii + "|" + escape + ")*"
invalid
= invalid1 + "|" + invalid2

comment
= r"\/\*[^*]*\*+(?:[^/*][^*]*\*+)*\/"
badcomment1
= r"\/\*[^*]*\*+(?:[^/*][^*]*\*+)*"
badcomment2
= r"\/\*[^*]*(?:\*+[^/*][^*]*)*"
badcomment
= r"(?:" + badcomment1 + "|" + badcomment2 + ")"

nmchar
= r"(?:[_a-z0-9\-]|" + nonascii + "|" + escape + ")"
nmstart
= r"(?:[_a-z]|" + nonascii + "|" + escape + ")"

ident
= r"[-]?" + nmstart + "(?:" + nmchar + ")*"
name
= nmchar + "+"

unicode_range
= r"u\+[0-9a-f?]{1,6}(?:-[0-9a-f]{1,6})?"

openbrace
= r"\{"
closingbrace
= r"\}"
openparenthese
= r"\("
closingparenthese
= r"\)"
openbracket
= r"\["
closingbracket
= r"\]"

url
= r"(?:[\!\#\$\%\&\*-\~]|" + nonascii + "|" + escape + ")*" # from http://www.w3.org/TR/CSS21/grammar.html

uri
= r"url\(" + w + string + w + r"\)" + r"|url\(" + w + r"(?:[\!\#\$\%\&\*-\[\]-\~]|" + nonascii + r"|" + escape + r")*" + w + r"\)"
baduri1
= r"url\(" + w + "(?:[\!\#\$\%\&\*-\[\]-\~]|" + nonascii + "|" + escape + ")*" + w
baduri2
= r"url\(" + w + string + w
baduri3
= r"url\(" + w + invalid
baduri
= "(?:" +baduri1 + "|" + baduri2 + "|" + baduri3 + ")"

NOT
= N+O+T
IMPORT
= I+M+P+O+R+T
PAGE
= P+A+G+E
MEDIA
= M+E+D+I+A
IMPORTANT
= I+M+P+O+R+T+A+N+T
URL
= U+R+L
words
= (
(
"S", s),
(
"COMMENT", comment),
(
"BADCOMMENT", badcomment),
(
"CDO", "<!--"),
(
"CDC", "-->"),
(
"INCLUDES", r"\~\="),
(
"DASHMATCH", r"\|\="),
(
"PREFIXMATCH", r"\^\="),
(
"SUFFIXMATCH", r"\$\="),
(
"SUBSTRINGMATCH", r"\*\="),
(
"STRING", string),
(
"BADSTRING", invalid),
(
"URI", "(?:" + URL + r"\(" + w + string + w + r"\)|" + URL + r"\(" + w + url + w + r"\))"),
(
"BAD_URI", baduri),
(
"FUNCTION", ident + r"\("),
(
"IDENT", ident),
(
"HASH", r"\#" + name),
(
"CLASS", r"\." + ident),
(
"IMPORT_SYM", r"\@" + IMPORT),
(
"PAGE_SYM", r"\@" + PAGE),
(
"MEDIA_SYM", r"\@" + MEDIA),
(
"CHARSET_SYM", r"\@charset "),
(
"IMPORTANT_SYM", r"\!(" + w + "|" + comment + ")*" + IMPORTANT),
(
"EMS", num + E + M),
(
"EXS", num + E + X),
(
"LENGTH", "(?:" + num + P + X + "|" + num + C + M + "|" + num + M + M + "|" + num + I + N + "|" + num + P + T + "|" + num + P + C + ")"),
(
"ANGLE", "(?:" + num + D + E + G + "|" + num + R + A + D + "|" + num + G + R + A + D + ")"),
(
"TIME", "(?:" + num + M + S + "|" + num + S + ")"),
(
"FREQ", "(?:" + num + H + Z + "|" + num + K + H + Z + ")"),
(
"DIMENSION", num + ident),
(
"PERCENTAGE", num + r"\%"),
(
"NUMER", num),
#("FUNCTION", ident + r"\("),
("PLUS", w + r"\+"),
(
"GREATER", w + r"\>"),
(
"COMMA", w + r"\,"),
(
"TILDE", w + r"\~"),
(
"NOT", ":" + NOT + r"\("),
(
"ATKEYWORD", r"\@" + ident),
(
"SEMICOLON", r"\;"),
(
"COLON", r"\:"),
(
"LBRACE", openbrace),
(
"RBRACE", closingbrace),
(
"RPARENTHESE", closingparenthese),
(
"LPARENTHESE", openparenthese),
(
"LBRACKET", openbracket),
(
"RBRACKET", closingbracket),
(
"EQUALSIGN", r"\="),
(
"PERIOD", r"\."),
(
"STAR", r"\*"),
(
"ANY", ".")
)

简单说明一下,如果用RegEx库做词法分析,我是把这些用“|”连起来,然后不停的match。现在RegEx都是回溯引擎,至少对于Python来说,放前面的会被优先test,所以具有相同“First集”的,越特殊越要放前面。比如num,LENGTH以它开头,NUMBER就是num,那么LENGTH的放前头。FUNCTION、URL之类同理。对于NFA/DFA引擎或等价(或严格的超集)的标准正则表达式库(!=RegEx),会leftmost longest,所以应该无需关心顺序。

最后那个ANY是干嘛?没法match的全都ANY。解析CSS很少需要严格要求用户的文件,不认识的不理就算了。词法上是这样,语法的时候更容易处理。

最后骂一句,我怎么干上程序员这行当了,弄这些繁琐的旁枝末节真TM无聊的紧。还有,早知道要处理整套页面应用相关问题,绝B用C/C++写了,从webkit里直接copy,还能顺便学点其它不相干的经验,日了。

posted on 2011-02-23 20:49  怪怪  阅读(808)  评论(1编辑  收藏  举报

导航