dwsmart, 24 days ago @methode and co added a new library to the awesome open source Google robots txt parser We've added it to @jammer_volts & mine's robots.txt tool here: https://tamethebots.com/tools/robotstxt-checker The last column now shows robots.txt status, lines parsed, How many valid directives that robots.txt file has, and any ones that are not understood, for each URL tested. #seo #robotstxt
@methode and co added a new library to the awesome open source Google robots txt parser We've added it to @jammer_volts & mine's robots.txt tool here: https://tamethebots.com/tools/robotstxt-checker
The last column now shows robots.txt status, lines parsed, How many valid directives that robots.txt file has, and any ones that are not understood, for each URL tested. #seo #robotstxt
Add comment