Skip to content
This repository has been archived by the owner on Apr 14, 2019. It is now read-only.
/ CoWaPS Public archive

CodeWarsProfileScraper. Only meant for practicing scraping.

License

Notifications You must be signed in to change notification settings

palash25/CoWaPS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Logo

Code Wars Profile Scraper

This is a web scraper written in python to scrape the profile of a user on code wars(a competitive programming website). Although the CodeWars API is publicly available I made this scraper just for the love of scraping.

See it in action

cowaps in action

Requirements

You need requests, BeautifulSoup and texttable to run this script.

Installation

Run the following commands on your terminal

1. System-wide installation

1. Requests

Using apt-get: $ sudo apt-get install python3-requests

Using pip: $ pip3 install requests

2. BeautifulSoup

Using apt-get: $ sudo apt-get install python3-bs4

Using pip: $ pip3 install beautifulsoup4

3. TextTable

Using apt-get: $ sudo apt-get install python3-texttable

Using pip: $ pip3 install texttable

2. Installation using virtualenv

Fisrt you need to install virtualenv, if you don't already have one. Install it using the following command:

$ python3 -m pip3 install --user virtualenv

Then you need to create a virtual environment. To create a virtual environment, go to your project’s directory and run virtualenv using the following command:

$ python3 -m virtualenv env

Before you can start installing or using packages in your virtualenv you’ll need to activate it. Run the following command to activate virtualenv:

$ source env/bin/activate

Now, to install the required packages follow the instructions as mentioned under the section System-wide installation.

Note:

  1. It is recommended to run sudo apt-get update before running the above commands.
  2. If you don't have python3/pip3 installed, replace python3/pip3 with python/pip and run the command.
  3. If you like what you see give it a ⭐ .

About

CodeWarsProfileScraper. Only meant for practicing scraping.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages