借助ltp 逐步程序化实现规则库 文本生成引擎基于规则库和业务词库 去生成文本

 

[
哪个地方做什么的哪家靠谱？
地名词库
行业、业务词库
]
苏州做网络推广的公司哪家靠谱？
苏州镭射机维修哪家最专业？
昆山做账的公司哪家比较好
广州称重灌装机生产厂家哪家口碑比较好



[
含有专家知识
]
郑州律师哪个好，如何判断合同是否有效？


[
哪个地方有做什么的？
]
广东哪里有专业的全铝书柜定制？
苏州吴中越溪哪里有通过率较高的会计培训班？

[
2-gram
]

行业 属性 通过 “2-gram”实现，“动词+名词”

昆山注册公司哪家专业？
    注册公司




{'words': '大型\t雕铣机\t哪个\t牌子\t好\t？', 'postags': 'b\tn\tr\tn\ta\twp', 'parser': '2:ATT\t4:ATT\t4:ATT\t5:SBV\t0:HED\t5:WP', 'netags': 'O\tO\tO\tO\tO\tO', 'role': [{4: 'A0:(0,3)'}]}
feature ATT SBV HED 相邻

{'words': '深圳市\t东荣\t纯水\t设备\t有限公司\t有\t什么\t产品\t，\t电话\t是\t多少\t？', 'postags': 'ns\tnz\tn\tn\tn\tv\tr\tn\twp\tn\tv\tr\twp', 'parser': '5:ATT\t3:ATT\t4:ATT\t5:ATT\t6:SBV\t0:HED\t8:ATT\t6:VOB\t6:WP\t11:SBV\t6:COO\t11:VOB\t6:WP', 'netags': 'B-Ni\tI-Ni\tI-Ni\tI-Ni\tE-Ni\tO\tO\tO\tO\tO\tO\tO\tO', 'role': [{5: 'A0:(0,4)\tA1:(6,7)'}, {10: 'A0:(9,9)\tA1:(11,11)'}]}

feature


[[正规|靠谱]|便宜|价格优惠|价格低]公司

含有ns
{'words': '武汉\t哪里\t有\t织发补发店\t？', 'postags': 'ns\tr\tv\tn\twp', 'parser': '3:SBV\t3:SBV\t0:HED\t3:VOB\t3:WP', 'netags': 'S-Ns\tO\tO\tO\tO', 'role': [{2: 'A0:(0,0)\tA0:(1,1)\tA1:(3,3)'}]}

地点-[哪里有做]-业务-[的][正规|靠谱]-公司？

{'words': '江西\t塑料\t厂家\t有\t哪些\t？', 'postags': 'ns\tn\tn\tv\tr\twp', 'parser': '3:ATT\t3:ATT\t4:SBV\t0:HED\t4:VOB\t4:WP', 'netags': 'S-Ns\tO\tO\tO\tO\tO', 'role': [{'3': 'A0:(0,2)\tA1:(4,4)'}]}
地点-[做]-业务-[的][正规|靠谱]-公司[有哪些]？

{'words': '龙江\t附近\t金色\t年华\t教育\t中心\t到底\t怎么样\t？', 'postags': 'ns\tnd\tn\tn\tv\tn\tv\tr\twp', 'parser': '2:ATT\t6:ATT\t4:ATT\t6:ATT\t6:ATT\t7:SBV\t0:HED\t7:VOB\t7:WP', 'netags': 'S-Ns\tO\tO\tO\tO\tO\tO\tO\tO', 'role': [{6: 'A0:(0,5)\tA1:(7,7)'}]}

地点-业务-[靠谱吗？到底怎样？]


{'words': '南充\t最\t好\t的\t化妆\t学校\t是\t哪家\t？', 'postags': 'ns\td\ta\tu\tv\tn\tv\tr\twp', 'parser': '6:ATT\t3:ADV\t6:ATT\t3:RAD\t6:ATT\t7:SBV\t0:HED\t7:VOB\t7:WP', 'netags': 'S-Ns\tO\tO\tO\tO\tO\tO\tO\tO', 'role': [{'2': 'ADV:(1,1)'}, {'6': 'A0:(0,5)\tA1:(7,7)'}]}

地点-[最好的|靠谱的]业务-[是哪家？]

{'words': '昆山\t铣刀\t厂家\t联系\t方式\t是\t多少\t？', 'postags': 'ns\tn\tn\tv\tn\tv\tr\twp', 'parser': '3:ATT\t3:ATT\t5:ATT\t5:ATT\t6:SBV\t0:HED\t6:VOB\t6:WP', 'netags': 'S-Ns\tO\tO\tO\tO\tO\tO\tO', 'role': [{'5': 'A1:(6,6)'}]}

地点-[做]-业务-[的][正规|靠谱]-公司[的联系方式是什么？|哪家口碑好值得信赖？]

{'words': '苏州\t装修\t别墅\t怎么\t能\t省\t钱\t？', 'postags': 'ns\tv\tn\tr\tv\tv\tn\twp', 'parser': '3:ATT\t3:ATT\t6:SBV\t6:ADV\t6:ADV\t0:HED\t6:VOB\t6:WP', 'netags': 'S-Ns\tO\tO\tO\tO\tO\tO\tO', 'role': [{'5': 'A0:(0,2)\tADV:(3,3)\tA1:(6,6)'}]}

地点-[做]-业务-[怎么能省钱？|费用是多少？|需要注意什么？|有哪些流程？]

from pyltp import *
import os
import re
import json

d_dir = '/usr/local/ltp_data_v3.4.0/'

segmentor = Segmentor()
s = '%s%s' % (d_dir, "cws.model")
segmentor.load(s)

postagger = Postagger()
s = '%s%s' % (d_dir, "pos.model")
postagger.load(s)

parser = Parser()
s = '%s%s' % (d_dir, "parser.model")
parser.load(s)

recognizer = NamedEntityRecognizer()
s = '%s%s' % (d_dir, "ner.model")
recognizer.load(s)

labeller = SementicRoleLabeller()
s = '%s%s' % ('/usr/local/ltp_data_v3.3.0/ltp_data/srl/', '')
labeller.load(s)


def gen_all(paragraph, split_join_tag='\t'):
    r = {}
    # 分词  其他分析依赖于该数据
    sentence = SentenceSplitter.split(paragraph)[0]
    # segmentor = Segmentor()
    # s = '%s%s' % (d_dir, "cws.model")
    # segmentor.load(s)
    words = segmentor.segment(sentence)
    r['words'] = split_join_tag.join(words)
    # print("\t".join(words))

    # 词性标注
    # postagger = Postagger()
    # s = '%s%s' % (d_dir, "pos.model")
    # postagger.load(s)
    postags = postagger.postag(words)
    r['postags'] = split_join_tag.join(postags)
    # print("\t".join(postags))

    # 依存句法关系
    # parser = Parser()
    # s = '%s%s' % (d_dir, "parser.model")
    # parser.load(s)
    arcs = parser.parse(words, postags)
    r['parser'] = split_join_tag.join("%d:%s" % (arc.head, arc.relation) for arc in arcs)
    # print("\t".join("%d:%s" % (arc.head, arc.relation) for arc in arcs))

    # 命名实体识别
    # recognizer = NamedEntityRecognizer()
    # s = '%s%s' % (d_dir, "ner.model")
    # recognizer.load(s)
    netags = recognizer.recognize(words, postags)
    r['netags'] = split_join_tag.join(netags)
    # print("\t".join(netags))

    # 语义角色类型

    # labeller = SementicRoleLabeller()
    # s = '%s%s' % ('/usr/local/ltp_data_v3.3.0/ltp_data/srl/', '')
    # labeller.load(s)
    roles = labeller.label(words, postags, netags, arcs)

    r['role'] = []
    for role in roles:
        d = {}
        d[role.index] = split_join_tag.join(
            ["%s:(%d,%d)" % (arg.name, arg.range.start, arg.range.end) for arg in role.arguments])
        # print(role.index, "".join(
        #     ["%s:(%d,%d)" % (arg.name, arg.range.start, arg.range.end) for arg in role.arguments]))
        r['role'].append(d)

    return r


ori_f = 'list_b_only_title.txt'
r_f = '%s%s' % (ori_f, '.run.txt')
with open(r_f, 'w', encoding='utf-8') as fow:
    with open(ori_f, 'r', encoding='utf8') as fo:
        for i in fo:
            p = i.replace('\n', '').replace('"', '')
            try:
                a = gen_all(p)
            except Exception as e:
                print(p, '  ', e)
                continue
            ws = '%s%s' % (json.dumps(a, ensure_ascii=False), '\n')
            fow.write(ws)
segmentor.release()
postagger.release()
parser.release()
recognizer.release()
labeller.release()


时间占比
特征提取（话术 句术  文本陈述方式  词组织/搭配方式 方式  文法 语义 ）  写正则表达式

import re
import json

ori_f = 'list_b_only_title.txt'
r_f = '%s%s' % (ori_f, '.run.txt')
# {'words': '昆山\t注册\t公司\t找\t哪家\t比较\t好\t？', 'postags': 'ns\tv\tn\tv\tr\td\ta\twp', 'parser': '3:ATT\t3:ATT\t4:SBV\t0:HED\t4:VOB\t7:ADV\t4:CMP\t4:WP', 'netags': 'B-Ni\tI-Ni\tE-Ni\tO\tO\tO\tO\tO', 'role': [{3: 'A0:(0,2)\tA1:(4,4)'}, {6: 'A0:(0,2)\tADV:(5,5)'}]}
select_l = []
reg_l = ['ATT\\t\d+:SBV\\t\d+:HED\\t\d+:VOB\\t\d+']
reg_l = ['HED\\t\d+:VOB\\t\d+:WP']
reg_l = ['HED\\t\d+:VOB\\t\d+:(WP|ADV\\t\d+:CMP)']
c = 0
with open(r_f, 'r', encoding='utf-8') as fowr:
    for iii in fowr:
        a = json.loads(iii)
        a_postags = a['postags']
        if 'ns' not in a_postags:
            continue
        for ii in reg_l:
            a_parser = a['parser']
            if re.compile(ii).search(a_parser) is not None:
                select_l.append(a)
        if c == 30:
            break
dd = 9

 

"""
地点-哪里有做-业务-的(正规|靠谱)-公司？
地点-做-业务-的(正规|靠谱)-公司(有哪些？|的联系方式是什么？|哪家口碑好值得信赖？)
地点-做-业务-(怎么能省钱？|费用是多少？|需要注意什么？|有哪些流程？)
地点-业务-(靠谱吗？到底怎样？)
地点-(最好的|靠谱的)业务-是哪家？

"""
p, b = '深圳市', ['广告设计', '网络推广']

ltp_model = ['地点-哪里有做-业务-的(正规|靠谱)-公司？', '地点-做-业务-的(正规|靠谱)-公司(有哪些|的联系方式是什么|哪家口碑好值得信赖)？',
             '地点-做-业务-(怎么能省钱|费用是多少|需要注意什么|有哪些流程|靠谱吗|到底怎样)？', '地点-(最好的|靠谱的)业务-是哪家？']
r_l = []
for s in ltp_model:
    s = s.replace('地点', p).replace('-', '')
    for i in b:
        r_l.append(s.replace('业务', i))

def deal_first_splittag_str(i):
    s_l_1 = []
    psl, psr = i.find('(', 0), i.find(')', 0)
    sl, sm, sr = i[0:psl], i[psl + 1:psr], i[psr + 1:]
    l = sm.split('|')
    for ii in l:
        s_l_1.append('%s%s%s' % (sl, ii, sr))
    return s_l_1


def deal_first_splittag(s_l_0):
    s_l_1 = []
    for i in s_l_0:
        psl, psr = i.find('(', 0), i.find(')', 0)
        if psl == -1:
            s_l_1.append(i)
        else:
            sl, sm, sr = i[0:psl], i[psl + 1:psr], i[psr + 1:]
            l = sm.split('|')
            for ii in l:
                s_l_1.append('%s%s%s' % (sl, ii, sr))
    return s_l_1


while True:
    f = 0
    for i in r_l:
        if '(' in i:
            f = 1
            del r_l[r_l.index(i)]
            l = deal_first_splittag_str(i)
            r_l += l
    if f == 0:
        break

d = 9