Skip to content

3.0.0 - All New and Improved
Compare
Choose a tag to compare
@MatteoCampinoti94 MatteoCampinoti94 released this 09 Aug 12:01
v3.0.0
This tag was signed with the committer’s verified signature.
MatteoCampinoti94 Matteo Campinoti
GPG key ID: 2242D33B26C86E10
Learn about vigilant mode.
0508e18
This commit was signed with the committer’s verified signature.
MatteoCampinoti94 Matteo Campinoti
GPG key ID: 2242D33B26C86E10
Learn about vigilant mode.

Release 3.0.0 marks a complete change in how the program is run, its capabilities and future development.

Following the change of interface on Fur Affinity in January 2020, version 2 stopped working, but with this new release the tool can once again get content from FurAffinity and is much simpler to update to support future changes to FA's web interface.

This change was achieved thanks to the FAAPI package (from the same author of FALocalRepo [email protected]). All scraping functions are now independent of the interface, allowing for much quicker development and response to changes in the Fur Affinity website.

The interface of the program was changed from an interactive menu to a command line tool. This allows for much quicker execution of commands, simpler code and automation via shell scripts.

The database has been updated and is now over 50% lighter for large numbers of downloaded submissions. Furthermore, it now holds the cookies used by the scraper, reducing the program footprint.

All database functions have been completely overhauled and are now considerably faster, especially searching. Using a 500k submissions table and a modern SSD drive, searching for a specific tag takes 0,90s on average, and searching for a string in the descriptions takes only 1,30s. Time may vary depending on search parameters and drive speed.

The last big change regards the packaging and distribution of the program. falocalrepo is now a PyPi package, easily installed with a single pip command. All dependencies have also been packaged and distributed on PyPi and are handled without the need for git submodules. The new distribution method allows running falocalrepo in any folder, without the need to have the program itself stored with the database.

⚙️ Changes

  • Use FAAPI package to separate scraper development from interface
  • Support for new FA's interface
  • Menu interface replaced with a command line tool
  • Database cleanup
  • Cookies stored in database
  • Database queries improvements
  • Command line help
  • Distribution via PyPi

📦 Dependencies

  • Set FAAPI dependency to 2.5.0

🚚 Distribution

🔗 Links

Assets 2
Loading