thewebdev.de

home

messie - a web crawler written in Ruby

03 Feb 2012

A few weeks ago I wrote my first two gems and released them on rubygems.org. The first one about I will talk today is messie. I often want to crawl web pages in tiny little projects I do, so I thought: "Why not write a gem that I can use for every project and that handles crawling of web pages in an abstract manner".

Features

By now, messie is still < 1.0 (currently 0.3.2) so it shurely isn't perfect yet, but it already has a lot of features that might be very useful to you.

Roadmap

Try it!

To install it, just use gem:

[code lang="bash"]

gem install messie

[/code]

Read the README.md at github.com to get on from here. If you need help just file an issue on github or fork the repository and send me a pull request, I'd be happy about it :)