How To Block Bad Bots

Ever since Google and other major search engines rolled out crawlers and other bots for the sake of indexing a couple of decades ago, visits by these automated scripts has represented more than half of global web traffic. Search engine crawlers are hardly the only bots visiting your websites these days; scrapers, machine learning constructs, artificial intelligence programs, specific algorithms, and cloud computing scripts are among the most common automated scripts making the rounds of the internet. The problem with bots is that quite a few are deployed for malicious purposes. Information security specialists estimate that about a quarter of all internet bots can be classified as malware; while this means that most bots are actually helpful, it is unsettling to realize that many of them are targeting servers for the purpose of spreading denial of service attacks, distributing spam or inflating third-party web traffic statistics. Bots can be handled by means of blocking them with blacklists or allowing them through access lists, which are also known as whitelists. Blacklisting can be done right from the .htaccess file or robots.txt; however, some of the most effective methods involve third-party solutions offered by the likes of Cloudflare or through specific plugins in case you use WordPress. As to which bots you should be blocking, your best bet is to let a third-party service handle it or else build a list from the real-time information provided by reputable sources such as As of mid-March 2019, some of the most pernicious bots include: * - an American spam bot cleverly coded to pass itself off as an actual visitor. * - similar to the above, but originating from Sweden. * - a very active spam bot from Vietnam. * - not much is known about what this cloud computing scraper actually looks for; nonetheless, it is considered to be a bandwidth nuisance. * - an Italian scraper bot that actively looks for images to display on an experimental search engine. * - this is one of many Chinese bots that masquerade as Google search engine crawlers, but their purpose is unknown. For more information click here

10 Best SEO is the internets best rankings directory for SEO Agencies. Our experience in the industry ensures an accurate list. Best SEO Agencies & SEO Agency Awards are fastly becoming the largest and most well known SEO awards

SEO Consulting

We won't optimize your site, but we will help you find the right online marketer for your needs.


Sign up below to get updates from our site!