以前破解wanfang的protobuf还使用的标准的正向开发流程：手动编写 .proto + protoc。
今天试了下Blackboxprotobuf——无定义的黑盒解析工具：无需匹配 Protobuf 定义文件（.proto）即可处理编码后 Protobuf数据 的工具集。
本地需要先安装配置好以下软件或扩展：
- Charles：抓包代理工具，付费的，可免费试用30天，也可破解，或者使用fiddler抓包也可以。
- xxxxxxx-xxxxx（为什么不能展示出来：
）：免费的浏览器代理管理扩展，适配Chrome、Edge等主流浏览器。
之所以要用抓包工具，是因为charles可以直接将请求和响应保存为二进制文件，方便blackboxprotobuf后续去处理。
大概说一下blackboxprotobuf使用流程：
- chrome浏览器打开插件xxxxxxx-xxxxx，配置好charles的端口，默认8888。
- 打开charles，chrome浏览器访问网站，刷新所需的页面
- charles上找到目标接口，如：getDetailInFormation
- 点击响应，空白处右键Save Response
- 保存到桌面的getDetailInFormation.bin文件
- 编写程序使用blackboxprotobuf读取bin文件，解码得到响应内容
步骤截图如下：
以上是拿响应bin文件测试了下，能否解析到数据。下面只需将请求负载保存到bin得到明文的请求参数，每次修改请求参数中的值，转为proto识别的二进制去请求，得到的响应用上面的方式解析出来，一次请求就跑通了。
得到的数据如下：
{
"1": 1,
"2": "qid=khkqcyzt,",
"3": 21,
"4": [
{
"1": "Periodical",
"2": [
{
"1": 1,
"4": "ONLINE_READ"
},
{
"1": 2,
"4": "DOWNLOAD"
}
],
"3": "CiBQZXJpb2RpY2FsQ0hJU29scjkyMDI2MDkxNTE0MjIxORIRZ2R4eGh4eGIyMDI2MDkwMDEaCGtoa3FjeXp0",
"101": {
"1": "gdxxhxxb202609001",
"2": [
"《高等学校化学学报》第六届编辑委员会",
"Editorial Board of Chemical Journal of Chinese Universities"
],
"18": [
"高等学校化学学报",
"编辑委员会",
"第六届"
],
"20": "主 编(Editor-in-chief)于吉红 Jihong YU\n执行主编(Executive Editor-in-chief)王 丹 Dan WANG\n副主编(Associate Editor)\n卜显和 Xianhe BU 王树涛 Shutao WANG 陈春英 Chunying CHEN 龙亿涛 Yitao LONG",
"22": "gdxxhxxb",
"23": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"24": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"25": "WF",
"31": 1,
"32": 1,
"33": 2026,
"34": "9",
"36": "封2",
"37": "1",
"39": [
"ISTIC",
"PKU",
"EI",
"SCI",
"CSCD"
],
"40": "gdxxhxxb/gdxx2026/2609pdf/260901.pdf",
"44": "chi",
"45": "0251-0790",
"48": 11,
"50": 5,
"53": "Regular",
"54": "Periodical",
"55": "WF",
"74": "FULLTEXT",
"81": "QK_CHI"
}
},
{
"1": "Periodical",
"2": [
{
"1": 1,
"4": "ONLINE_READ"
},
{
"1": 2,
"4": "DOWNLOAD"
},
{
"1": 16,
"4": "HTML_READ"
}
],
"3": "CiBQZXJpb2RpY2FsQ0hJU29scjkyMDI2MDkxNTE0MjIxORIRZ2R4eGh4eGIyMDI2MDkwMDIaCGtoa3FjeXp0",
"101": {
"1": "gdxxhxxb202609002",
"2": [
"Jahn-Teller畸变对功能材料的调控",
"Regulation of Functional Materials by Jahn-Teller Distortion"
],
"3": [
"梁娜",
"黄科科"
],
"6": [
"LIANG Na",
"HUANG Keke"
],
"8": "吉林大学",
"10": "吉林大学化学学院,长春 130012",
"16": [
"Jahn-Teller畸变",
"轨道简并",
"对称性"
],
"17": [
"Jahn-Teller distortion",
"Orbital degeneracy",
"Symmetry"
],
"20": [
"由Jahn-Teller活性离子引发的本征晶格畸变可降低配位几何结构的对称性及其d轨道简并度,从而稳定催化剂表面的反应中间体,并有效调控电极材料在充放电过程中的电子与离子电导率,进而显著影响材料的宏观性能.本文系统阐述了Jahn-Teller畸变的起源与诱发机制,概述了其主要表征方法,并重点讨论了该效应在催化转化及能源存储等功能材料领域的应用,旨在为深入理解Jahn-Teller畸变与材料宏观性能之间的构效关系提供参考.",
"The intrinsic lattice distortion induced by Jahn-Teller-active ions reduces the symmetry of the coordina-tion geometry and the degeneracy of d-orbitals,thereby stabilizing reaction intermediates on the catalyst surface and effectively regulating the electronic and ionic conductivities of electrode materials during charge-discharge processes,which in turn significantly influence the macroscopic properties of the materials.This review systematically elucidates the origin and triggering mechanisms of the Jahn-Teller distortion,outlines its primary characterization methods,and highlights the applications of this effect in functional materials such as catalytic conversion and energy storage,aiming to provide a systematic reference for an in-depth understanding of the structure-property relationship between the Jahn-Teller distortion and macroscopic material performance."
],
"22": "gdxxhxxb",
"23": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"24": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"25": "WF",
"31": 1,
"32": 1,
"33": 2026,
"34": "9",
"36": "1-11",
"37": "11",
"38": [
"无机固体化学专栏",
"Inorganic Solid State Chemistry Special Issue"
],
"39": [
"ISTIC",
"PKU",
"EI",
"SCI",
"CSCD"
],
"40": "gdxxhxxb/gdxx2026/2609pdf/260902.pdf",
"41": "10.7503/cjcu20260202",
"42": [
"黄科科:吉林大学",
"梁娜:吉林大学"
],
"44": "chi",
"45": "0251-0790",
"48": 50,
"49": 1,
"50": 37,
"53": "Regular",
"54": "Periodical",
"55": "WF",
"56": 12,
"74": "FULLTEXT",
"81": "QK_CHI",
"85": 1
}
},
{
"1": "Periodical",
"2": [
{
"1": 1,
"4": "ONLINE_READ"
},
{
"1": 2,
"4": "DOWNLOAD"
},
{
"1": 16,
"4": "HTML_READ"
}
],
"3": "CiBQZXJpb2RpY2FsQ0hJU29scjkyMDI2MDkxNTE0MjIxORIRZ2R4eGh4eGIyMDI2MDkwMDMaCGtoa3FjeXp0",
"101": {
"1": "gdxxhxxb202609003",
"2": [
"磁性金属氧化物的新视角:非晶化的挑战",
"New Perspective on Magnetic Metal Oxides:the Challenges of Amorphization"
],
"3": [
"陈君杰",
"许翰坤",
"李强",
"邓金侠",
"林鲲"
],
"6": [
"CHEN Junjie",
"XU Hankun",
"LI Qiang",
"DENG Jinxia",
"LIN Kun"
],
"8": "北京科技大学",
"10": "北京科技大学化学与生物工程学院,北京 100083",
"16": [
"非晶金属氧化物",
"局域结构",
"磁性",
"制备方法"
],
"17": [
"Amorphous metal oxide",
"Local structure",
"Magnetism",
"Synthesis method"
],
"20": [
"非晶金属氧化物因其长程无序而短程有序的结构特征,在磁性研究中展现出区别于晶态体系的独特物理行为.本文综合评述了非晶金属氧化物的结构特点、主要磁性类型及其物理机制,并重点讨论了超顺磁、自旋玻璃、簇玻璃及随机交换磁态等典型磁行为的起源及其内在联系.同时,总结了熔融急冷、溶胶-凝胶、模板法及机械非晶化等主要制备方法,分析了不同制备策略对材料结构与性能的影响,旨在为理解非晶金属氧化物中结构无序与磁性之间的关系提供参考.",
"Amorphous metal oxides(AMOs),characterized by long-range disorder and short-range order,exhibit distinct magnetic behaviors compared to crystalline materials.This review provides an overview of their structural characteristics,magnetism,and underlying mechanisms,with emphasis on typical phenomena such as superpara-magnetism,spin glass,cluster glass and random exchange magnetic states,as well as their intrinsic correlations.Representative synthesis strategies,including melt-quenching,sol-gel,template,and mechanical amorphization,are also summarized.And their impacts on structure and properties are discussed.This review aims to elucidate the relationship between structural disorder and magnetism in AMOs."
],
"22": "gdxxhxxb",
"23": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"24": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"25": "WF",
"31": 1,
"32": 1,
"33": 2026,
"34": "9",
"36": "12-31",
"37": "20",
"38": [
"无机固体化学专栏",
"Inorganic Solid State Chemistry Special Issue"
],
"39": [
"ISTIC",
"PKU",
"EI",
"SCI",
"CSCD"
],
"40": "gdxxhxxb/gdxx2026/2609pdf/260903.pdf",
"41": "10.7503/cjcu20260142",
"42": [
"李强:北京科技大学",
"林鲲:北京科技大学",
"许翰坤:北京科技大学",
"邓金侠:北京科技大学",
"陈君杰:北京科技大学"
],
"44": "chi",
"45": "0251-0790",
"48": 21,
"50": 7,
"53": "Regular",
"54": "Periodical",
"55": "WF",
"56": 3,
"74": "FULLTEXT",
"81": "QK_CHI",
"85": 1
}
},
{
"1": "Periodical",
"2": [
{
"1": 1,
"4": "ONLINE_READ"
},
{
"1": 2,
"4": "DOWNLOAD"
},
{
"1": 16,
"4": "HTML_READ"
}
],
"3": "CiBQZXJpb2RpY2FsQ0hJU29scjkyMDI2MDkxNTE0MjIxORIRZ2R4eGh4eGIyMDI2MDkwMDQaCGtoa3FjeXp0",
"101": {
"1": "gdxxhxxb202609004",
"2": [
"原位高压表征技术与压致相变材料研究",
"In-Situ High-pressure Characterization Techniques and Research on Pressure-induced Phase Transition Materials"
],
"3": [
"裴天遥",
"王永刚",
"岳彬彬"
],
"6": [
"PEI Tianyao",
"WANG Yonggang",
"YUE Binbin"
],
"8": [
"北京高压科学研究中心",
"北京大学"
],
"10": [
"北京高压科学研究中心,北京 100094",
"北京大学材料科学与工程学院,北京 100871"
],
"16": [
"压致相变",
"原位表征",
"金刚石压砧",
"同步辐射"
],
"17": [
"Pressure-induced phase transition",
"In-situ characterization",
"Diamond anvil cell(DAC)",
"Synchrotron radiation"
],
"20": [
"高压诱导的结构相变是发现新物相、理解极端条件下物质行为的重要途径.然而,相变过程中的路径、中间态和动力学长期处于\"黑箱\"状态,需要原位探测技术予以揭示.本文综合评述了高压固体化学中结构相变的原位探测技术及机理研究进展.首先,按信息层次分类介绍了X射线衍射、中子衍射、X射线吸收精细结构、对分布函数、拉曼光谱、红外光谱、紫外-可见吸收光谱、荧光光谱及二次谐波等原位表征方法的原理与适用范围,并讨论了超快时间分辨技术和机器学习辅助数据分析等新兴方向.其次,以位移型相变(SrTiO3)、重构型相变(石墨-金刚石、Co3P2O8)、压致非晶化(冰)、等结构相变(Ce)及有序-无序相变(IMMHyPbBr4)为典型案例,展示了原位技术如何揭示压力诱导结构演化的微观机理.最后,总结了当前在时间分辨率和数据解释等方面面临的瓶颈,并对多技术联用、超快探测和机器学习等未来发展方向进行了展望.",
"High-pressure-induced structural phase transitions constitute a crucial pathway for discovering new phases and understanding material behaviors under extreme conditions.However,the transition pathways,intermediate states,and dynamics have long remained in a\"black box\",necessitation in situ probing techniques for elucidation.This article reviews the recent progress in in situ probing techniques and mechanistic studies of structural phase tran-sitions in high-pressure solid-state chemistry.First,the principles and applicable scopes of in situ characterization methods are introduced according to information levels,including X-ray diffraction,neutron diffraction,X-ray absorption fine structure,pair distribution function,Raman spectroscopy,infrared spectroscopy,ultraviolet-visible absorption spectroscopy,fluorescence spectroscopy,and second harmonic generation.Emerging directions such as ultrafast time-resolved techniques and machine-learning-assisted data analysis are also discussed.Second,using displacive phase transitions(SrTiO3),reconstructive phase transitions(graphite-diamond,Co3P2O8),pressure-induced amorphization(ice),isostructural phase transitions(Ce),and order-disorder phase transitions(IMMHyPbBr4)as rep-resentative examples,we demonstrate how in situ techniques reveal the microscopic mechanisms of pressure-driven structural evolution.Finally,current bottlenecks in time resolution and data interpretation are summarized,and future directions including multi-technique integration,ultrafast probing,and machine learning are prospected."
],
"22": "gdxxhxxb",
"23": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"24": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"25": "WF",
"31": 1,
"32": 1,
"33": 2026,
"34": "9",
"36": "32-49",
"37": "18",
"38": [
"无机固体化学专栏",
"Inorganic Solid State Chemistry Special Issue"
],
"39": [
"ISTIC",
"PKU",
"EI",
"SCI",
"CSCD"
],
"40": "gdxxhxxb/gdxx2026/2609pdf/260904.pdf",
"41": "10.7503/cjcu20260234",
"42": [
"王永刚:北京大学",
"岳彬彬:北京高压科学研究中心",
"裴天遥:北京高压科学研究中心"
],
"44": "chi",
"45": "0251-0790",
"48": 9,
"53": "Regular",
"54": "Periodical",
"55": "WF",
"56": 2,
"74": "FULLTEXT",
"81": "QK_CHI",
"85": 1
}
},
{
"1": "Periodical",
"2": [
{
"1": 1,
"4": "ONLINE_READ"
},
{
"1": 2,
"4": "DOWNLOAD"
},
{
"1": 16,
"4": "HTML_READ"
}
],
"3": "CiBQZXJpb2RpY2FsQ0hJU29scjkyMDI2MDkxNTE0MjIxORIRZ2R4eGh4eGIyMDI2MDkwMDUaCGtoa3FjeXp0",
"101": {
"1": "gdxxhxxb202609005",
"2": [
"宏观尺寸单晶多孔材料",
"Macroscopic Porous Single-crystalline Materials"
],
"3": [
"刘少芳",
"谢奎"
],
"6": [
"LIU Shaofang",
"XIE Kui"
],
"8": "上海交通大学",
"10": "上海交通大学机械与动力工程学院,上海 200240",
"16": [
"单晶-多孔材料",
"单晶结构",
"多孔结构",
"固-固相变"
],
"17": [
"Porous single-crystalline material",
"Single-crystalline structure",
"Porous architecture",
"Solid-solid phase transformation"
],
"20": [
"多孔材料和单晶材料在现代科学技术中均具有重要应用价值.若能在宏观尺寸单晶中引入孔隙,有望获得兼具开放孔结构和连续单晶骨架的新型材料,即宏观尺寸单晶多孔材料.其连续单晶骨架保持长程有序晶体结构,可有效减少晶界对传输过程和结构稳定性的影响,同时孔隙结构可提供传质通道和可接触内表面.然而,传统晶体生长过程通常趋向于致密化,孔隙往往被视为夹杂或缺陷,因此在单晶中可控引入孔结构仍面临本征挑战.本文围绕基于\"固-固相变\"的晶格重构策略展开综述,重点讨论高密度单晶母相经特定组分脱除、晶格重排和相变界面迁移后,原位转化为低密度目标相单晶骨架并形成孔隙结构的基本机制.进一步结合二氧化钛和二氧化铈等典型氧化物体系,分析了母相晶向、组成、相对密度变化和热处理条件对晶面取向、孔隙率、孔径及表面结构的调控作用.最后,以光电化学能源转化为例,探讨了连续单晶骨架、三维连通孔道和可调缺陷结构之间的协同效应,并展望了该类材料在电子-声子输运解耦、同位素分离等结构敏感型功能过程中的潜在应用前景.",
"Porous materials and single crystals both hold significant application value in modern science and technology.Introducing porosity into macroscopic single crystals offers a promising route to a new class of materials that combine an open porous architecture with a continuous single-crystalline skeleton,namely macroscopic porous single crystal(PSC).In PSC,the continuous single-crystalline skeleton maintains long-range crystallographic order and effectively reduces grain-boundary effects on transport processes and structural stability,while the porous architecture provides mass-transport channels and accessible internal surfaces.However,conventional crystal growth processes generally tend toward densification,and pores are usually regarded as inclusions or defects.Therefore,the controllable introduction of porous structures into single crystals remains a fundamental challenge.This review focuses on lattice reconstruction strategies driven by solid-solid phase transformations.We discuss the basic mechanism by which a high-density single-crystalline mother phase is transformed in situ into a low-density target single-crystalline skeleton through selective component removal,lattice rearrangement,and phase-boundary migration,accompanied by pore formation.Using typical oxide systems such as TiO2 and CeO2 as examples,we further analyze how the crystallographic orientation,composition,relative density variation,and heat-treatment conditions of the mother phase regulate the facet orientation,porosity,pore size,and surface structure of the resulting PSC.Finally,with photoelectrochemical energy conversion as a representative application,we discuss the synergistic effects of the continuous single-crystalline skeleton,three-dimensionally interconnected pore channels,and tunable defect structures.We also outline future opportunities for these materials in structure-sensitive functional processes,including electron-phonon transport decoupling and isotope separation."
],
"22": "gdxxhxxb",
"23": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"24": [
"高等学校化学学报",
"Chemical Journal of Chinese Universities"
],
"25": "WF",
"31": 1,
"32": 1,
"33": 2026,
"34": "9",
"36": "50-64",
"37": "15",
"38": [
"无机固体化学专栏",
"Inorganic Solid State Chemistry Special Issue"
],
"39": [
"ISTIC",
"PKU",
"EI",
"SCI",
"CSCD"
],
"40": "gdxxhxxb/gdxx2026/2609pdf/260905.pdf",
"41": "10.7503/cjcu20260187",
"42": [
"刘少芳:上海交通大学",
"谢奎:上海交通大学"
],
"44": "chi",
"45": "0251-0790",
"48": 7,
"50": 1,
"53": "Regular",
"54": "Periodical",
"55": "WF",
"56": 1,
"74": "FULLTEXT",
"81": "QK_CHI",
"85": 1
}
}
]
}
上面返回数据只有5条，是在这里限制了下。
核心代码（python）如下：
def convert_int_to_bytes(d): if d < 256: return bytes([0,0,0,0, d]) b1 = bytearray(5) # 创建一个长度为5的字节数组，初始值为0 # 下面将d的每一位进行处理，这里假设Convert.ToByte是转换为无符号字节，并且d是一个整数 # 原始代码中d被右移8位，并在循环中取余256，这里我们模拟这个过程 # 由于Python的int是动态长度的，我们需要模拟固定长度的处理，假设d是一个32位整数 # 但根据原始代码，我们只关心低32位，因为每次右移8位并取余256 # 原始代码处理了4次，所以这里我们也处理4次 for wy in range(4, 0, -1): b1[wy] = d & 0xFF # 取d的低8位 d >>= 8 # 右移8位 # 第一个字节已经初始化为0，所以不需要再设置，但如果d的高8位需要存储（根据原始注释，第一位是0，后四位是根据计算出来的，但这里我们只处理了4位） # 由于我们创建了一个长度为5的字节数组，并且第一个字节初始化为0，所以不需要额外操作 # 但是，如果d的高8位在多次右移后仍然有值（即d原本超过32位），这里会丢失，因为原始代码只处理了4次 # 如果需要处理更大的数，可以增加循环次数和字节数组长度 return b1 def decode_bytes_to_str(data): """ 递归地将数据结构中的所有bytes对象解码为UTF-8字符串。 """ if isinstance(data, dict): # 如果数据是字典，递归处理每个值 return {key: decode_bytes_to_str(value) for key, value in data.items()} elif isinstance(data, list): # 如果数据是列表，递归处理每个元素 return [decode_bytes_to_str(element) for element in data] elif isinstance(data, bytes): # 如果数据是bytes对象，尝试解码为UTF-8字符串 try: return data.decode('utf-8') except UnicodeDecodeError: # 如果UTF-8解码失败，保留原始bytes对象或根据需要处理 return data # 或者你可以选择返回一个错误消息、占位符等 else: # 如果数据不是字典、列表或bytes对象，直接返回 return data encoded_data = bytes(blackboxprotobuf.encode_message(decoded_message, type_definition)) encoded_data = convert_int_to_bytes(len(encoded_data)) + encoded_data response = requests.post(url, data=encoded_data, headers=headers, proxies=proxies, timeout=35) resp_data, resp_type = blackboxprotobuf.decode_message(response.content[5:]) resp_data_with_str = decode_bytes_to_str(resp_data) print(f'resp_data: {resp_data}') print(f'resp_type: {resp_type}') print(f'resp_data_with_str: {resp_data_with_str}')
问题：
- 为什么读取bin文件要截取？
- 解码后的内容仍然有bytes？
- 解码后内容只有字段编号和类型，语义缺失？
回答：
- 剥离外层包装：很多Web接口的Protobuf请求会额外包裹5字节的gRPC帧头，前1字节是标记位，后4字节是大端存储的消息长度，直接把前5字节切掉再喂给
decode_message，就能避免解析卡死或乱码。 - 嵌套的内容，写个方法递归decode为utf-8即可
- 不是问题
经过测试，大部分时间返回如下空数据，尝试了protobuf_to_json这个方法也是如此，对于bytes特别大的时候protobuf_to_json巨卡，半天没有返回。返回空数据问题，暂时没有解决，因为响应是200，将响应条数改为5条也没有解决，所以需要多次尝试请求。
如有看到博客的大佬，知道这是什么问题，烦请给我留言说明下，🦀🦀
blackboxprotobuf相恨见晚~
实践出真知~
浙公网安备 33010602011771号