[RAGFlow] Interest in better project dependency management

[http403]

Hi community and members of infiniflow,

Will the community be interested in pausing the development of RAGFlow a bit, and restructure the project using better dependency management tool? I'm trying to make the Docker image smaller by cutting out the GPU specific dependencies, which I use Poetry to aid me, and I discover few dependencies conflicts:

• volcengine need pycryptodome==3.9.9 got pycryptodome=3.20.0
• volcengine need pytz==2020.5 got pytz==2024.1
• bcembedding need transformers>=4.35.0,<4.37.0 got transformer==4.38.1

Note: bcembedding and volcengine aren't version pinned

It will be nice to use some form of dependency management tools like Poetry or Pipenv to avoid such issues. Not to mention PyCryptodome 3.9.9 and pytz both released in 2020, which are very old which PyCryptodome have CVE-2023-52323 vulnerability before version 3.19.1.

Again, I don't mind chime in my time to make it happen.

[KevinHuSh]

Thanks very much!
I'm not sure how long you need to make this amazing refinement?
What about RAGFlow new a branch named 'poetry'?

[http403]

No specific timeline yet, and although you suggested a poetry branch, which dependency management tool would you prefer? They all have some pros and cons, and I don't want this drag the whole development team later on.

• Pipenv
  • I used it quite frequently
  • Lower learning curve IMO
  • Non standard PEP
• Poetry
  • I used it less than Pipenv
  • Slightly higher learning curve as it also a packaging manager
  • Uses standard pyproject.toml

There are other dependency management tools out there like pdm. It is best for your team to discuss and decide what's the easiest to work with.

P.S. Please enable the Discussion board on ragflow repo so I can create a issue from this discussion. Thanks.

[yingfeng]

discussion is enabled

[http403]

Unable to transfer it to ragflow. Would you do it or I just create an brand new issue?

[yingfeng]

Hi, issue is created as a feature request. #1052

[http403]

Thanks