This crawler is used for a service called "Babbar" which describes itself as "a SEO authority measurement tool". Their bot used to identify itself as Attentio/Nutch-0.9-dev Barkrowler They are still around but we have not seen their bot since 2010 so blocking them may not be very important. Their bot used to be hostile and annoying. There is no benefit in having this waste bandwidth unless you are willing to pay for their services - in which case you need to allow it to get the data they collect about your site.Īttentio from Belgium describes themselves as "a corporate intelligence service". This belongs to a company offering SEO analytic services to paying customers. Allowing them may actually hurt you since many use them to setup sites with garbage content carrying the keywords similar to those used on your site in order to gain search engine traffic. These services are not at all useful if you're not one of their customers. They may be useful if you are one paying them and using them. Basically, you can register at these companies and pay them to tell you what web pages are on your website (along with other data). They are all run by different companies who all provide the same class of service: "Research" and "Analysis" to paying clients. The most well-known ones are AhrefsBot, BLEXBot, mj12bot and SemrushBot. These robots are used by closed services which are only available to paying customers. "SEO", advertisement other "research" robots 4.2 htaccess: For Less Friendly Yet Identifiable Bots.4.1 robots.txt: For Worthless But Conforming Bots.1 "SEO", advertisement other "research" robots.
0 Comments
Leave a Reply. |