Anubis is a new proof-of-work system inspired by Hashcash that aims to reduce aggressive web scraping by AI companies, website operators say [1, 2, 3]. It works by increasing the computational load for scrapers operating at scale, making it more costly for AI systems to harvest data without permission [1, 2, 3].
Unlike traditional anti-scraping techniques, Anubis forces scrapers to solve cryptographic puzzles before accessing site content, applying a cost that impacts automated bots more than human users [1, 2, 3]. Site administrators report that this approach helps reduce server strain and website downtime caused by excessive scraping requests [1, 2, 3].
The system requires modern JavaScript capabilities to execute. However, some browser plugins, such as JShelter, can disable these features. Operators recommend users disable such plugins temporarily when visiting sites using Anubis to avoid access issues [1, 2, 3].
Anubis serves as a temporary layer of defense while more sophisticated fingerprinting tools that can detect headless browsers and other advanced scraping techniques are under development [1, 2, 3]. These future methods aim to provide more precise identification of scraping bots without impacting legitimate traffic.
The deployment of Anubis is part of a growing effort by websites worldwide to protect their content from high-volume AI data harvesting that can disrupt user experience and server operation [1, 2, 3].
