Download the archives of a webcomic for offline reading.
This is my first Python project, so things may be a little rough for a while, particularly the install process, but I'm learning fast!
One nice thing about this program is it does all of the scraping with CSS selectors, so it's really easy to add support for a new comic!
All info on this page is written only with Linux in mind. The code itself should be fairly portable, and the installation should be relatively generic for a Python project, but I cannot offer any guidance on non-linux platforms. Feel free to share your experiences though!
This was written specifically for Python 3, so make sure you have python3
installed. At this point, I would recommend installing in a virtualenv
so
make sure that's installed as well.
I may submit this to the Python Package Archive if I radically clean this up. But for now:
git clone 'https://github.com/dn3s/webcomic-dl.git'
cd webcomic-dl
./setup.py install
I'm still learning about packaing and distribution, so for now you may want to
do this in a virtualenv
. There's a .gitignore
entry for a directory called
virtualenv
, so you can set it up there for painless updates:
git clone 'https://github.com/dn3s/webcomic-dl.git'
cd webcomic-dl
virtualenv -p python3 virtualenv
. virtualenv/bin/activate
./setup.py develop
Then when you want to use it, make sure you activate the virtualenv with
. <path_to_webcomic-dl>/virtualenv/bin/activate
.
If anyone has advice on overall design and architecture, I'd love to hear it!
If you want to add support for your favorite webcomic, take a look at
webcomic_dl/comics
; I'll document it more thoroughly soon.
If you want to contribute directly, it's probably wise to setup a virtualenv
,
and please install the pre-commit hook (cp pre_commit.sh .git/hooks/pre-commit
)
One other note, if you want to edit this README, edit README.pre
, as
README.md
is overwritten by the pre-commit hook.
Pull requests welcome!
The program is perfectly usable now, but still needs some work for to improve maintainability:
- Main control logic needs refactoring
- Module layout is a little wonky.
Comic
should not be inwebcomic_dl/__init__.py
- HTML viewer needs a heavy refactor/total rewrite; the code is very much a quick hack
- Clean up existing comments, docstrings, thouroughly document how to add comic support, format existing comics consistently, create a starting template for people to copy
- Prettier progress display
usage: webcomic-dl [-h] [-n <n>] [-f] [-d <dir>] [-o] [--no-resume]
[-m <file>] [--metadata-only] [-c] [-i] [-v]
comic
Download a webcomic archive
positional arguments:
comic the name or URL of the webcomic to comic download
optional arguments:
-h, --help show this help message and exit
-n <n>, --max <n> Maximum number of comics to download. 0 means
unlimited
-f, --from-url By default, webcomic-dl will download from the first
webcomic in the series, regardless of the URL. This
flag overrides it. For some comics that don't have
numbers associated with their comics, this may cause
naming collisions if you later decide to go back and
download older comics. Thus, only use this if you are
either certain you will never want earlier comics, or
are willing to re-download the newer ones.
-d <dir>, --dir <dir>
Specify an output directory
-o, --overwrite Force overwriting of downloaded comics. By default,
webcomic-dl will not overwrite already-downloaded
comics.
--no-resume By default, webcomic-dl will resume downloads from
where it last left off. This flag overrides it. Does
not download already-downloaded comics unless -o is
also used, but still needs to download the webpage of
each comic
-m <file>, --metadata-file <file>
Specify where to save metadata. By default it is saved
in <output_dir>/info.json
--metadata-only Don't download comics, just metadata
-c, --compact Disable pretty-printing the output JSON, if you're
really trying to save some bytes
-i, --index Places an HTML-based viewer called index.html in the
output directory
-v, --verbose Be verbose. Mostly for testing purposes