$ luarocks install robotstxt
Robotstxt provides a Lua/LPEG based robots.txt file parser and
rule evaluator. Basic workflow is to load via a file or IO
handle, then set a user agent, then check if given URLs are
allowed or denied.
Versions
Dependencies
lua >= 5.4, < 5.5