mitmproxy使用指南
前言
这是一款抓包软件比起其他的抓包软件（如：fiddler、burpsuite），它的优点在于做二次开发加脚本处理文件。
一、python环境安装
本人用的是Python3.7.3去官网下载安装即可。
二、安装mitmproxy
- 使用
pip install mitmproxy或pip install pipx和pipx install mitmproxy。 - 如果你需要抓https的包你还需要再运行后访问http://mitm.it/来安装安装证书。
三、使用命令
mitmproxy 工具有以下三部分组成：
- mitmproxy -> 命令行工具（win不支持）
- mitmdump -> 加载 python 脚本
- mitmweb -> web 界面工具
常用参数
-h 帮助信息
-p 修改监听端口
-s 加载 python 脚本（例如：-s "script.py --bar" # 执行脚本，通过双引号来添加参数）
默认监听端口8080
四、脚本样例
#!/usr/bin/python
# -*- coding: UTF-8 -*-
from mitmproxy import http
def request(flow: http.HTTPFlow):
# 增加请求的头信息
flow.request.headers["myheader"] = "shifeng"
print(flow.request.headers)
#!/usr/bin/python
# -*- coding: UTF-8 -*-
from mitmproxy import http
def request(flow: http.HTTPFlow):
# 修改判断条件
if "quote.json" in flow.request.pretty_url and "x=" in flow.request.pretty_url:
# 打开保存在本地的数据文件
with open("/Users/chenshifeng/MyCode/PythonCode/SFDSZL/interface/quote.json") as f:
# 创造一个 response
flow.response = http.HTTPResponse.make(
200, # (optional) status code
# 读取文件中数据作为返回内容
f.read(),
# 指定返回数据的类型
{"Content-Type": "application/json"} # (optional) headers
)
#!/usr/bin/python
# -*- coding: UTF-8 -*-
import json
from mitmproxy import http
def response(flow: http.HTTPFlow):
# 加上过滤条件
if "quote.json" in flow.request.pretty_url and "x=" in flow.request.pretty_url:
# 把响应数据转化成python对象，保存到data中
data = json.loads(flow.response.content)
# 对第一个股票保持原样
data['data']['items'][0] = data['data']['items'][0]
# 对第二个股票名字加长一倍
data['data']['items'][1]['quote']['name'] = data['data']['items'][1]['quote']['name'] * 2
# 对第三个股票名字变成空
data['data']['items'][2]['quote']['name'] = None
# 把修改后的内容赋值给 response 原始数据格式
flow.response.text = json.dumps(data)
参考：https://www.lmlphp.com/user/23204/article/item/573492/
五、一些报错
现象：安装以后无法使用报错显示 ImportError: cannot import name ‘soft_unicode‘ from ‘markupsafe‘
解决：命令行运行 python -m pip install markupsafe==2.0.1
现象：正常启动比且代理服务器已开启无反应或没有想要的数据
解决：可以是端口被占用了，换个端口试试
现象：抓不到心怡的数据
解决：这里发现window上的设置也有影响关闭如下图设置就出现数据了
现象： << Cannot establish TLS with client (sni: browser.events.data.msn.cn): TlsException("SSL handshake error: Error([('SSL routines', 'ssl3_read_bytes', 'sslv3 alert certificate unknown')])")
解决：用-s带上如下脚本
"""
This inline script allows conditional TLS Interception based
on a user-defined strategy.
Example:
> mitmdump -s tls_passthrough.py
1. curl --proxy http://localhost:8080 https://example.com --insecure
// works - we'll also see the contents in mitmproxy
2. curl --proxy http://localhost:8080 https://example.com --insecure
// still works - we'll also see the contents in mitmproxy
3. curl --proxy http://localhost:8080 https://example.com
// fails with a certificate error, which we will also see in mitmproxy
4. curl --proxy http://localhost:8080 https://example.com
// works again, but mitmproxy does not intercept and we do *not* see the contents
Authors: Maximilian Hils, Matthew Tuusberg
"""
import collections
import random
from enum import Enum
import mitmproxy
from mitmproxy import ctx
from mitmproxy.exceptions import TlsProtocolException
from mitmproxy.proxy.protocol import TlsLayer, RawTCPLayer
class InterceptionResult(Enum):
success = True
failure = False
skipped = None
class _TlsStrategy:
"""
Abstract base class for interception strategies.
"""
def __init__(self):
# A server_address -> interception results mapping
self.history = collections.defaultdict(lambda: collections.deque(maxlen=200))
def should_intercept(self, server_address):
"""
Returns:
True, if we should attempt to intercept the connection.
False, if we want to employ pass-through instead.
"""
raise NotImplementedError()
def record_success(self, server_address):
self.history[server_address].append(InterceptionResult.success)
def record_failure(self, server_address):
self.history[server_address].append(InterceptionResult.failure)
def record_skipped(self, server_address):
self.history[server_address].append(InterceptionResult.skipped)
class ConservativeStrategy(_TlsStrategy):
"""
Conservative Interception Strategy - only intercept if there haven't been any failed attempts
in the history.
"""
def should_intercept(self, server_address):
if InterceptionResult.failure in self.history[server_address]:
return False
return True
class ProbabilisticStrategy(_TlsStrategy):
"""
Fixed probability that we intercept a given connection.
"""
def __init__(self, p):
self.p = p
super(ProbabilisticStrategy, self).__init__()
def should_intercept(self, server_address):
return random.uniform(0, 1) < self.p
class TlsFeedback(TlsLayer):
"""
Monkey-patch _establish_tls_with_client to get feedback if TLS could be established
successfully on the client connection (which may fail due to cert pinning).
"""
def _establish_tls_with_client(self):
server_address = self.server_conn.address
try:
super(TlsFeedback, self)._establish_tls_with_client()
except TlsProtocolException as e:
tls_strategy.record_failure(server_address)
raise e
else:
tls_strategy.record_success(server_address)
# inline script hooks below.
tls_strategy = None
def load(l):
l.add_option(
"tlsstrat", int, 0, "TLS passthrough strategy (0-100)",
)
def configure(updated):
global tls_strategy
if ctx.options.tlsstrat > 0:
tls_strategy = ProbabilisticStrategy(float(ctx.options.tlsstrat) / 100.0)
else:
tls_strategy = ConservativeStrategy()
def next_layer(next_layer):
"""
This hook does the actual magic - if the next layer is planned to be a TLS layer,
we check if we want to enter pass-through mode instead.
"""
if isinstance(next_layer, TlsLayer) and next_layer._client_tls:
server_address = next_layer.server_conn.address
if tls_strategy.should_intercept(server_address):
# We try to intercept.
# Monkey-Patch the layer to get feedback from the TLSLayer if interception worked.
next_layer.__class__ = TlsFeedback
else:
# We don't intercept - reply with a pass-through layer and add a "skipped" entry.
mitmproxy.ctx.log("TLS passthrough for %s" % repr(next_layer.server_conn.address), "info")
next_layer_replacement = RawTCPLayer(next_layer.ctx, ignore=True)
next_layer.reply.send(next_layer_replacement)
tls_strategy.record_skipped(server_address)
