Skip to content

Web Scraping GSMarena to extract smartphone specs with their ratings

Notifications You must be signed in to change notification settings

zehan-alam/Web-Scraping--GSMarena

Repository files navigation

Smartphone Specifications Scraping from GSM Arena

This project utilizes Python's BeautifulSoup library to extract smartphone specifications from GSM Arena. The step-by-step process is as follows:

  1. scrap_brand_links.py: Extracts all brand links from the website and saves them in brand_links.txt.

  2. scrap_phone_links.py: Iterates through brand links and finds phone links, including paginations, saving them in phone_links.txt.

  3. ip_generator.py: Collects IP addresses from https://sslproxies.org/ and stores them in ip_addresses.txt for potential proxy usage in the scraping process to avoid blocks.

  4. csv_generator.py: Creates specs.csv file for storing the scraped data. Run this if you need to reset the CSV file and start fresh.

  5. scrap_specs.py: Goes through phone links, collects specifications data, and ratings. Utilizes proxies to avoid potential IP blocks. Resumes from where it left off if a proxy is blocked.

Note: You can customize the brands to scrape in scrap_specs.py to select specific brands rather than scraping all.

For the detailed project code and implementation, refer to the relevant scripts in the repository. Please ensure proper usage of proxies and adhere to the website's terms of use while scraping data.

About

Web Scraping GSMarena to extract smartphone specs with their ratings

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages