on software development & the web

messie - a web crawler written in Ruby

03 Feb 2012

A few weeks ago I wrote my first two gems and released them on The first one about I will talk today is messie. I often want to crawl web pages in tiny little projects I do, so I thought: "Why not write a gem that I can use for every project and that handles crawling of web pages in an abstract manner".


By now, messie is still < 1.0 (currently 0.3.2) so it shurely isn't perfect yet, but it already has a lot of features that might be very useful to you.


Try it!

To install it, just use gem:

[code lang="bash"]

gem install messie


Read the at to get on from here. If you need help just file an issue on github or fork the repository and send me a pull request, I'd be happy about it :)