随笔分类 -  爬虫

boss
摘要：import subprocess import re import requests from urllib.parse import urlparse, parse_qs from functools import partial subprocess.Popen = partial(subpr 阅读全文
posted @ 2024-03-17 20:02 牧羊人の冬天 阅读(374) 评论(0) 推荐(0)
学易云rsa
摘要：import requests import subprocess from functools import partial subprocess.Popen = partial(subprocess.Popen, encoding="utf-8") import execjs cookies = 阅读全文
posted @ 2024-01-24 18:11 牧羊人の冬天 阅读(19) 评论(0) 推荐(0)
惠金所-MD5
摘要：import base64 import json import hashlib import requests cookies = { 'sajssdk_2015_cross_new_user': '1', 'sensorsdata2015jssdkcross': '%7B%22distinct_ 阅读全文
posted @ 2024-01-24 17:07 牧羊人の冬天 阅读(67) 评论(0) 推荐(1)
看准网
摘要：import base64 import json from Crypto.Cipher import AES import random from Crypto.Util.Padding import pad, unpad key = 'G$$QawckGfaLB97r' iv = 'qqqwww 阅读全文
posted @ 2023-12-08 15:13 牧羊人の冬天 阅读(178) 评论(0) 推荐(0)
selenium 反爬
摘要：本地浏览器执行 import time from selenium import webdriver chrome_option = webdriver.ChromeOptions() chrome_option.add_experimental_option('excludeSwitches', 阅读全文
posted @ 2023-11-08 14:08 牧羊人の冬天 阅读(67) 评论(0) 推荐(0)
链家二手房
摘要：import pandas as pd import requests from bs4 import BeautifulSoup # 获取数据的函数 def get_data(page): url = f"https://sz.lianjia.com/ershoufang/pg{page}/" r 阅读全文
posted @ 2023-10-18 18:33 牧羊人の冬天 阅读(45) 评论(0) 推荐(0)
12306
摘要：import re import requests def keys_values(d, value): return list(d.keys())[list(d.values()).index(value)] headers = { "Cookie": "_uab_collina=16969283 阅读全文
posted @ 2023-10-10 18:44 牧羊人の冬天 阅读(82) 评论(0) 推荐(0)
中国观鸟记录中心
摘要：import requests # 调用js报错时，修改默认编码格式 import subprocess from functools import partial subprocess.Popen = partial(subprocess.Popen, encoding="utf-8") impo 阅读全文
posted @ 2023-10-09 11:38 牧羊人の冬天 阅读(333) 评论(0) 推荐(0)