How To Block Bad Bots

Ever since Google and other major search engines rolled out crawlers and other bots for the sake of indexing a couple of decades ago, visits by these automated scripts has represented more than half of global web traffic. Search engine crawlers are hardly the only bots visiting your websites these days; scrapers, machine learning constructs, artificial intelligence programs, specific algorithms, and cloud computing scripts are among the most common automated scripts making the rounds of the internet. The problem with bots is that quite a few are deployed for malicious purposes. Information security specialists estimate that about a quarter of all internet bots can be classified as malware; while this means that most bots are actually helpful, it is unsettling to realize that many of them are targeting servers for the purpose of spreading denial of service attacks, distributing spam or inflating third-party web traffic statistics. Bots can be handled by means of blocking them with blacklists or allowing them through access lists, which are also known as whitelists. Blacklisting can be done right from the .htaccess file or robots.txt; however, some of the most effective methods involve third-party solutions offered by the likes of Cloudflare or through specific plugins in case you use WordPress. As to which bots you should be blocking, your best bet is to let a third-party service handle it or else build a list from the real-time information provided by reputable sources such as MyIP.ms. As of mid-March 2019, some of the most pernicious bots include: * seizing.ironicalt.com - an American spam bot cleverly coded to pass itself off as an actual visitor. * ip-10-29.dataclub.eu - similar to the above, but originating from Sweden. * static.vnpt-hanoi.com.vn - a very active spam bot from Vietnam. * ec2-3-80-68-245.compute-1.amazonaws.com - not much is known about what this cloud computing scraper actually looks for; nonetheless, it is considered to be a bandwidth nuisance. * ic-images-2-nuova.crawler.istella.it - an Italian scraper bot that actively looks for images to display on an experimental search engine. * 117.41.187.57 - this is one of many Chinese bots that masquerade as Google search engine crawlers, but their purpose is unknown. For more information click here https://www.reddit.com/r/SEO/comments/b087so/whataresomebotsthateverywebsitemustblock/.