-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the option to not use a cache #53
Conversation
Use case 3: |
@felixfbecker, I added it to the doc. |
Thanks! I would like to point out that |
Indeed cache introduces a lot of issues and it would be nice to have an option avoid it. Here is a use case: babel-runtime can be used transitively 5 times in one project. If you try to avoid cache then would it download and unzip the package 5 times? |
@bestander If babel-runtime is depended on 5 times on the same version, the package should not have to be duplicated, it would be flat in node_modules (what you are describing sounds like old NPM v2 style). If the versions are different, you need to download a different archive anyway. |
node_modules is never guaranteed to be flat, it's just (in npm 3 and later) as flat as possible. The dependency graph might require the duplication. |
@ljharb in what cases does it require duplication if the versions are the same? |
@felixfbecker if A and B require X v1, and C and D require X v2, you are virtually guaranteed to get 2 copies of one version of X, and the other version would be deduped at the top level - totaling 3 copies. |
True. yarn could be made smart enough in that case to copy Xv1 from A to B and Xv2 from C to D (or if the option is set, hardlink them). |
Yeah, from implementation point of view it might require too many code structure compromises. |
@bestander what would you propose? |
Each of those cases can be addressed with a workaround.
|
Unless I'm missing something, a long running server can't clean caches between runs, because many installations for different folders could run at any time in parallel. It doesn't seem like a good idea to clean the cache while a different installation is in the middle of the fetch or linking phase. |
Every run can create a cache folder in a random folder in /tmp |
That is correct and the workaround we are currently using. But this is a performance-critical application (which is also why we use yarn, not npm), and the linking phase takes up a considerate amount of time. If I understand it correctly, that is the time spent copying/linking packages from the cache to node_modules? |
Yeah, linking phase is copying files. |
Wouldn't move be subject to the same concurrency issues as cleaning the cache? |
If you use a unique cache folder for each install then no |
Oh, right. I wonder if move is significantly faster than hardlinking though? |
If you move a folder it is super fast.
Moving files individually might not
…On 23 March 2017 at 18:18, Felix Becker ***@***.***> wrote:
Oh, right. I wonder if move is significantly faster than hardlinking
though?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#53 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACBdWHy-QXjS79mma7j8quexrjCD2nN4ks5roreEgaJpZM4Mc-6z>
.
|
Then adding an option to move would be great. It's unfortunate |
Any chances to have an options, which disables cache? I have a docker, and I have to run |
@Diokuz, it may be hard to implement at this moment. E.g. use a custom cache folder and remove it after install. |
My use case here is to make local development easier when working with a project which uses a local folder as a package. eg: // package.json
dependencies: {
"my-dependency": "file:../my-dependency"
} If I make a change in Sometimes I get an older version of Maybe there's a better way to deal with this that I'm not aware of, but it would be great to be able to run Just my two cents! |
@brandonsturgeon, your case is better handled by |
+1 for this feature. Same reason as @brandonsturgeon |
+1 for the feature One more case: I have a package.json
yarn.lock contains the only one record with git+ssh:
Dockerfile contains:
But due to caching everything yarn requires |
+1 for this feature. I would prefer to disable cache completely, it can improve speed of our CI builds in docker and this cache is irrelevant because we don't need it at all. |
Being able to do this via config (and therefore an env var) would be very beneficial for CI, not only for performance but also because of yarnpkg/yarn#683 (it's hard to mandate/educate that all the users of our CI server should run yarn with |
We also use Yarn in a Docker CI environment where cache is only overhead, since it is discarded as soon as the given build step is done. Being able to somehow disable cache would be greatly beneficial. |
I think there is enough interest in this feature.
Anyone wants to give it a try?
…On Fri, Jan 5, 2018 at 1:30 AM Christopher Svanefalk < ***@***.***> wrote:
We also use Yarn in a Docker CI environment where cache is only overhead,
since it is discarded as soon as the given build step is done. Being able
to somehow disable cache would be greatly beneficial.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#53 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACBdWEJtlgjDWAS2FgHoEvIaM0UYJrUyks5tHeumgaJpZM4Mc-6z>
.
|
This is such a basic crucial feature. This makes me want to switch to npm again. |
Here is how approach this in Dockerfile meanwhile (see example): RUN YARN_CACHE_FOLDER=/dev/shm/yarn_cache yarn --production
The size of the Docker's
That said, turning off yarn's cache completely would be a much better option inside CI/CD. |
@bestander, do you have any high level suggestions for how someone might get started on implementing this? |
@dguo, it might be tricky, yarn's cache paths are deeply ingrained in fetching and linking phases.
|
why it is not merged? it is does not have any collision with master |
Because it's an rfc and it likely won't be implemented due to the internal design of the v1. From the v2 onward there are no node_modules, the cache is the only artifact needed to run the project, so this RFC doesn't make sense there either. |
@arcanis would you mind linking to some docs/discussion describing the v2 differences for those of us casual users popping in here interested in having the no-cache feature? Just curious to read up a bit more so i understand where yarn is going -- thanks! |
The v2 leverages PnP and zip loading to directly load the vendor files from within their archives (similar to phar archives in PHP, or asar in Electron). In this context the cache, albeit necessary, is the only artifact we generate. If you configure it to be stored within your project and don't version it, it will essentially be a "no cache Yarn". |
I have a simple case why I would also need this 'no cache' option. Here is my react native project
React Native does not support symlink in Note: I have not checked why but calling multiple times An alternative would be a mechanism like |
Here is a rendered version.
This RFC is related to yarnpkg/yarn#986.