nginx的，我也禁止了很多，如果上了CF的话，也可以在CF的防火墙规则里面添加
- if ($http_user_agent ~ "Neevabot|TTD-Content|FeedDemon|ThinkBot|MTRobot|SMTBot|LieBaoFast|Punkspider|MauiBot|Barkrowler|MegaIndex.ru|JikeSpider|TkBot|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|SemrushBot|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|EasouSpider|LinkpadBot|Ezooms|bsalsa|DotBot|newspaper|DataXu|Daum|BLEXBot|Scrapy|PetalBot|proximic|GrapeshotCrawler|Mail.RU_Bot|Nimbostratus-Bot|DuckDuckGo-Favicons-Bot|ias-sg|AdsTxtCrawler|ias-va|ias-or|SeznamBot|ias-va|ias|ias-ir|evc-batch|AspiegelBot|Re-re Studio|ias-or|^$" )
- {
- return 403;
- }
复制代码
|
嘟嘟社区