Obeying Robots.txt , much easily
I just pushed a gem , obey_robots_dot_txt it helps people who do crawling and needs to obey robots.txt
×
The Ruby and Rails community linklog
Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!
Post a comment