分享一个比较全面的robots.txt写法

Sitemap: http://www.sdfangyuan.cn/sitemap.xml

User-agent: *
Disallow: /*?*
Disallow: /*#*
Disallow: /*%*
Disallow: /*^*
Disallow: /caches/
Disallow: /phpcms/
Disallow: /install/
Disallow: /phpsso_server/
Disallow: /statics/
Disallow: /api/
Disallow: /2018/
Disallow: /*.php
Disallow: /*.asp
Disallow: /m/
Disallow: /404.html
Disallow: /portal.php
Disallow: /index.php
Disallow: /*-typeid-*
Disallow: /*-areaid-0*
Disallow: /*-list-*
Disallow: /*-maxprice-*
Disallow: /*-minprice-*
Disallow: /*-order-*
Disallow: /*-day-*
Disallow: /*-kw--*

User-agent: Baiduspider-image
Allow: /uploads
posted @ 2020-01-14 15:51  hepeacer  阅读(170)  评论(0)    收藏  举报