web-crawler

This is a Web Crawler implementation written in GOLANG.

Getting Started

Once you have GOLANG on your system, you'll need to install a few dependencies:

~$ go get github.com/llimllib/loglevel
~$ go get golang.org/x/net/html
~$ go get golang.org/x/net/html/atom

After you're done with that, you can start crawling wherever you want:

~$ go run main.go <URL>