Crawler （一）

检查网站构建的技术类型 —— builtwith模块   ：此模块将URL作为参数，下载该URL并对其进行分析，然后返回该网站使用的技术。

安装方法：pip install builtwith

例子：

import builtwith

ret = builtwith.parse("http://www.suchdo.com")

for k, v in ret.items():
    print("{} is {}".format(k, v))

结果：

web-frameworks is ['Microsoft ASP.NET']
web-servers is ['IIS', 'Tengine']
operating-systems is ['Windows Server']
javascript-frameworks is ['jQuery']

 

 查询域名的注册者 —— WHOIS库

安装：pip install python-whois

例子：

import whois

print(whois.whois("suchdo.com"))

结果：

{
  "domain_name": [
    "SUCHDO.COM",
    "suchdo.com"
  ],
  "registrar": "Alibaba Cloud Computing (Beijing) Co., Ltd.",
  "whois_server": "grs-whois.hichina.com",
  "referral_url": null,
  "updated_date": "2018-05-09 03:12:01",
  "creation_date": "2018-04-12 13:16:26",
  "expiration_date": "2023-04-12 13:16:26",
  "name_servers": [
    "DNS31.HICHINA.COM",
    "DNS32.HICHINA.COM"
  ],
  "status": "ok https://icann.org/epp#ok",
  "emails": "DomainAbuse@service.aliyun.com",
  "dnssec": "unsigned",
  "name": null,
  "org": null,
  "address": null,
  "city": null,
  "state": "guang dong",
  "zipcode": null,
  "country": null
}