如何把Nginx日志POST请求中 \x22yymkMc\x22:\x22\xE7\x22 字符转换成utf-8

背景：nginx的版本低于1.15.1以下，无法正常打印post请求里面参数。

Ng日志格式如下

log_format  main  '[$remote_addr] [$remote_port] [$remote_user] [$http_cookie] [$time_local] [$request] [$http_host] '
                  '[$request_method] [$http_x_forwarded_for] [$status] [$upstream_status] [$body_bytes_sent] [$http_referer] '
                  '[$http_user_agent][$upstream_addr] [$request_time] [$upstream_response_time] '
                  '[$content_type]  [$request_body]' ; 

参数说明

upstream_response_time    指从Nginx向后端建立连接开始到接受完数据然后关闭连接为止的时间。
request_time              指的就是从接受用户请求的第一个字节到发送完响应数据的时间，即$request_time包括接收客户端请求数据的时间、后端程序响应的时间、发送响应数据给客户端的时间(不包含写日志的时间)。

 

生成日志如下

[111.12.120.78] [54424] [-] [JSESSIONID=YqHZ17U1WIop3IyfxyDme8cMTOTT4nM36Obfz46DA3eN4d_pQrG3!-307907846] [19/May/2022:09:37:26 +0800] [POST /user-manager-platform/api/base/code/getGjhdqDm HTTP/1.1] [135.12.120.80] [POST] [-] [200] [200] [347] [-] [Apache-HttpClient/4.5 (Java/1.8.0_161)][135.12.120.78:8101] [0.007] [0.007] [application/json] [201]
[111.12.120.80] [38996] [-] [-] [19/May/2022:09:37:26 +0800] [POST /app-nw-web/api/zzsb/getGjhdqDm HTTP/1.1] [135.12.120.80] [POST] [-] [200] [200] [347] [-] [curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2][135.12.120.78:8201] [0.015] [0.015] [application/json;charset=utf-8] [{\x22nsrxhWtr\x22:null,\x22djxhWtr\x22:null,\x22xmWtr\x22:null,\x22cbsfMc\x22:null,\x22cbsfDm\x22:null,\x22sftsrq\x22:null,\x22shhm\x22:null,\x22sbbm\x22:null,\x22zsxzDm\x22:null,\x22xzMc\x22:null,\x22zgswjgDm\x22:null,\x22zgswjgMc\x22:null,\x22jfjs\x22:null,\x22zsxmDm\x22:null,\x22djxh\x22:null,\x22sbjbjgDm\x22:null,\x22zspmDm\x22:null,\x22zszmDm\x22:null,\x22cxjmCbxzList\x22:[],\x22cxjmCbxxDTOList\x22:[],\x22lhjyryAndCxjmCbxzDTOList\x22:[],\x22lhjyryAndCxjmCbxzDTOListStr\x22:null,\x22selectedCbxzDTOListStr\x22:null,\x22sfzjlxDmWtr\x22:\x22201\x22,\x22sfzjlxMcWtr\x22:null,\x22sfzjhmWtr\x22:null,\x22nsrsbhWtr\x22:null,\x22zsms\x22:null,\x22lhjyryZsms\x22:null,\x22cxjmZsms\x22:null,\x22ksjfNd\x22:null,\x22gjdmWtr\x22:null,\x22jflx\x22:null,\x22zgswskfjDm\x22:null,\x22zgswskfjMc\x22:null,\x22jkly\x22:null,\x22ssxzqhDm\x22:null,\x22preData\x22:null,\x22cbrytslbDm\x22:null,\x22sjhmWtr\x22:null,\x22multiInsureSwitch\x22:null,\x22multiInsuredMsg\x22:null,\x22multiInsured\x22:null,\x22to\x22:0,\x22jfrlxDm\x22:null,\x22from\x22:0,\x22jfrlxXz\x22:null,\x22isDb\x22:null}]

通过python 进行解码

>>> value = "{\x22czr\x22:\x22\xE8\xB5\xB5\xE5\xBB\xBA\xE9\xBE\x99\x22,\x22nsrxh\x22:\x2221072116445510002425\x22,\x22sjDm\x22:\x22002\x22,\x22sjMc\x22:\x22\xE8\xB4\xA6\xE6\x88\xB7\xE7\x99\xBB\xE5\xBD\x95\x22,\x22yymkDm\x22:\x22001\x22,\x22yymkMc\x22:\x22\xE7\xBD\x91\xE5\x8E\x85\xE9\x97\xA8\xE6\x88\xB7\x22,\x22czsj\x22:1652921868089,\x22qddm\x22:\x2201\x22,\x22sjlx\x22:\x22G\x22,\x22clientIp\x22:\x2260.31.197.106\x22,\x22swjgDm\x22:null}"
>>> print (value.decode('string_escape'))
{"czr":"赵龙","nsrxh":"21072116445510002425","sjDm":"002","sjMc":"登录","yymkDm":"001","yymkMc":"网厅门户","czsj":1652921868089,"qddm":"01","sjlx":"G","clientIp":"60.31.197.100","swjgDm":null}

扩展：

拿到post里面的参数可以通过curl进行请求模拟，定位问题

curl -H "Content-Type:application/json;charset=utf-8"  http://111.12.120.80:80/app-nw-web/api/zzsb/getGjhdqDm -X POST -d '{"nsrxhWtr":null,"djxhWtr":null,"xmWtr":null,"cbsfMc":null,"cbsfDm":null,"sftsrq":null,"shhm":null,"sbbm":null,"zsxzDm":null,"xzMc":null,"zgswjgDm":null,"zgswjgMc":null,"jfjs":null,"zsxmDm":null,"djxh":null,"sbjbjgDm":null,"zspmDm":null,"zszmDm":null,"cxjmCbxzList":[],"cxjmCbxxDTOList":[],"lhjyryAndCxjmCbxzDTOList":[],"lhjyryAndCxjmCbxzDTOListStr":null,"selectedCbxzDTOListStr":null,"sfzjlxDmWtr":"201","sfzjlxMcWtr":null,"sfzjhmWtr":null,"nsrsbhWtr":null,"zsms":null,"lhjyryZsms":null,"cxjmZsms":null,"ksjfNd":null,"gjdmWtr":null,"jflx":null,"zgswskfjDm":null,"zgswskfjMc":null,"jkly":null,"ssxzqhDm":null,"preData":null,"cbrytslbDm":null,"sjhmWtr":null,"multiInsureSwitch":null,"multiInsuredMsg":null,"multiInsured":null,"to":0,"jfrlxDm":null,"from":0,"jfrlxXz":null,"isDb":null}'

常用参数说明

-H参数添加 HTTP 请求的标头。
    Content-Type，用于定义网络文件的类型和网页的编码。
-X参数指定 HTTP 请求的方法。
-d参数用于发送 POST 请求的数据体。