crawler service

crawl articles from popular news publishers in Vietnam:

Tech stack

Prerequisites:

docker-compose build

docker-compose up -d

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
crawled_data		crawled_data
scrapy_crawler		scrapy_crawler
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
__init__.py		__init__.py
docker-compose.yaml		docker-compose.yaml
main.py		main.py
main_local.py		main_local.py
models.py		models.py
provider.py		provider.py
requirements.txt		requirements.txt
scrapy.cfg		scrapy.cfg