robotstxt

Robots.txt parser / rule evaluator

$ luarocks install robotstxt

Robotstxt provides a Lua/LPEG based robots.txt file parser and
rule evaluator. Basic workflow is to load via a file or IO
handle, then set a user agent, then check if given URLs are
allowed or denied.

Versions

1.3-1141 days ago5 downloads
1.2-1141 days ago0 downloads
1.1-1159 days ago7 downloads
1.0-2302 days ago17 downloads
1.0-1302 days ago0 downloads

Dependencies

lua >= 5.4, < 5.5

Manifests