Fixing React Native nightly

[lunaleaps]

Problem

We currently run nightlies of react-native but we don't have nightlies for dependencies that we ship from the monorepo. This slightly invalidates the intention of running a nightly of react-native -- to provide a daily functional snapshot of the codebase to ideally test and catch regressions earlier.

Context:

Note: I use the term "monorepo packages" to refer to packages under the packages folder excluding packages/react-native.

About monorepo releases

• When do we ship a new minor?
  • When starting a new pre-release react-native, we ship a new "main" version of monorepo packages.
    • This is accomplished via the bump-oss-packages and align-package-versions which increment and update the versions within all the packages. (monorepo packages may depend on one another)
  • ex. When we cut RC 0.72, we shipped 0.73.0 versions of all monorepo packages
• Ship a patch
  • On main, if someone updates the version of a monorepo package, we have CircleCI job to publish
  • On a release branch, picks and bumps like we do for react-native

Note:

• monorepo packages have published versions that are "ahead" of react-native versions
  • ex. Today, we have 0.72 RC in react-native , but we can already publish 0.73.2 for a monorepo package
• Changes to monorepo packages can be made on main without bumping the version (and publishing).
  • There's no enforcement of this because really there aren't any consumers of this package. It's just prep-work for when we cut 0.73 RC

Potential Solutions

1. We could enforce that whenever you make a change in a monorepo package, it requires a version bump and publish.

This ensures we're "flushing" any changes and so our nightly react-native can specify in its package.json the latest "main" version (which it already does).

2. Ship nightlies of react-native dependencies from our monorepo.

This option lends to some more questions...

What version should we use for the monorepo nightlies?

Options:

1. We could use 0.0.0-... naming schema which we use for react-native.
   • As discussed in another discussion, this naming scheme loses some context about the versioning. Esp. given that these monorepo package versions have dependencies on versions of react-native.
2. Use "main" version naming scheme, ex. 0.73.0-nightly... -- similar to proposed version for react-native nightly
   • This approach would require some changes to how/when we publish monorepo packages today.
   • Essentially, we should only publish monorepo packages when we cut the react-native RC. All work on main for a monorepo package is "in-progress"

My recommendation

I think option 2 feels like the more consistent approach but takes a bit more work.
Re: 1 -- I'm not sure how reasonable it is to publish a patch everytime we make a change to a monorepo package -- but no one would really be expected to use it except for nightlies.

Other musings

I think there's a conflict here of what "version" we have committed to code and what "version" we have published. For react-native package we have the concept that our package.json committed in main is a non-package, re: 1000.0.0.

But our monorepo packages imply that the version they're committed are the latest published version -- which allows for this "limbo" of changes -- how are those represented? I think that's the essential issue with the question of nightlies. I think option 2 moves us to a consistent approach where we'd also set our monorepo packages to a non-package version on main.

Looking for

Most importantly, does solution 2 miss something? That's the approach I'm leaning towards.

[hoxyq]

I am leaning more towards the second solution with using 0.73.0-nightly... naming scheme. Its descriptive.

Few thoughts:

1. Using 0.73.0-nightly... naming scheme should be followed with more strict rules on bumping all packages on main branch right after the release branch cut. To be honest, I don't see serious problems with publishing some changes from main branch inside nightlies for previous version. If we make a branch cut for 0.73, then 0.72 is successfully released, why would you want to depend on 0.72.0-nightly?

2. This approach might be expensive to maintain. Are we aiming to make it work from the box for any possible next new package?

[lunaleaps]

Re: 1, hmm I'm not sure I'm completely following. In the case of cutting 0.73 release, we would be publishing all our monorepo dependencies at version 0.73 (instead of doing this when we cut 0.72). Not sure what you mean by depending on 0.72-nightly, does that make sense?

[lunaleaps]

Re 2. Could you elaborate what is expensive to maintain?

[hoxyq]

Re: 1, hmm I'm not sure I'm completely following. In the case of cutting 0.73 release, we would be publishing all our monorepo dependencies at version 0.73 (instead of doing this when we cut 0.72). Not sure what you mean by depending on 0.72-nightly, does that make sense?

It was a hypothetical question. Trying to figure out consequences of a situation where we made a branch cut, but have not yet bumped packages inside main branch to next minor version.

[hoxyq]

Re 2. Could you elaborate what is expensive to maintain?

Added more in #638 (reply in thread), lets continue there

[lunaleaps]

I see, we could change main's /packages/react-native/package.json to refer to nightly versions so it'd always be current

[kelset]

It sounds to me like you are mixing two different concepts: releases and versioning on the monorepo packages, and releasing nightlies.

I think that we should aim for these two processes to be clearly and properly separated - let me try to bring some further context for folks that are not in the release crew so they might not be fully aware of everything going on.

"normal" releases

First thing, to make sure we're fully clear 'cause I can see some room for confusion:

When we cut RC 0.72, we shipped 0.73.0 versions of all monorepo packages

there are two separate things going on here:

in -stable branch

When we have cut 0.72, we also released new patch versions 0.72.x for all monorepo packages that have unreleased changes. And in ´0.72-stable´, we do new patch releases for these packages through automation when new changes happen, ahead of new 0.72.x minor for react-native itself (example commits, 1 and 2, ahead of 72 RC1). This to say: we also keep doing 0.72.x releases patches for the other monorepo packages, from 0.72-stable.

in main branch

The reason why we shipped 0.73 of all monorepo packages from main AFTER the branch cut is because we had to bump everything in main to 0.73.x so that new releases from main for any of those packages wouldn't polluted the 0.72 releases stream.

When to do new patches for packages in main is one of the problems that you are raising. Currently that's mostly done on an ad-hoc basis when ex. there's someone outside the Meta monorepo that needs to test a new version of a package, or similar. (= these bumps are committed in GH history in main branch)

nightlies

A separate problem is releasing nightlies - both for react-native and these monorepo packages.

Currently a nightly is a completely automated process that only releases a snapshot of react-native core every 24hrs, taking latest from main at the time of the automation triggering. There is nothing committed in the GH history for this "bump". There are two alternatives for the monorepo packages: either we also publish a nightly with the same version at the same time on the same commit for all the monorepo packages, or we do nothing (which is the current situation).

Doing nothing means that the latest nightly will consume the latest released from main version of said monorepo packages (keeping up with the example above, it'd mean latest 0.73.x release of the packages), which, as you mentioned, might lead to a slightly inconsistent nightly.

my 2c$

Like @hoxyq, I am very concerned with how noisy and expensive releasing a nightly for all packages every 24hrs might be.

I think that the first thing to ask before picking one of the two options you are proposing is: how frequent are changes in the other (!=react-native) folder packages? If the frequency is high, then I think it'd make sense to go for option 2 and release nightlies for everything every time. This would ensure that nightly are consistent without requiring constant "manual/semi-automated" bumps of packages in main.

But if the frequency is low (let's say a change every couple weeks on avg), I think it might make more sense to go for option 1 and just make so that when a new change lands in one of these other packages, it's released. There's probably room for automation here.

[kelset]

maybe we could have some sort of in-between? Like a manually maintained list of "high frequency" packages that would get released as nightlies via automation (like literally having a "release nightlies" script that has an array of strings with the name of the packages it will have to release)? That would allow for a more flexible approach.

[hoxyq]

Where do you see the complexity in second approach coming from? I mean, its definitely more expensive than the first one, but I think its worth it.

I might not have the same context for it, but let me share how I see it:

1. Downsides of first or in-between solutions.
   Risk of publishing broken patch version of some package. It is not that critical though, since we do this branch cut in main, so releases should be stable.

2. On complexity of the second approach
   If we can commit to some detailed workflow of how we version all new packages, whether we publish nightlies versions of them or not, then there should be no problems.
   What I meant by expensive to maintain is the fact of how scalable this solution is, this is the only potential problem I see. I am not sure that this workflow won't be redesigned once again, when there will be like ~20-30 packages?

Overall, I am thinking its still worth it

[kelset]

I explained here #638 (reply in thread) what I mean about the expensive side of things

[cortinico]

It sounds to me like you are mixing two different concepts: releases and versioning on the monorepo packages, and releasing nightlies.

Echoing what @kelset said, to me fixing the React Native nightlies == make sure react-native depends on @nightly version of the monorepo packages.

The versioning schema of nightlies can be discussed separately, it can be 0.0.0- or 0.73.0-x, as long as it's consistent between react-native and the monorepo packages.

Like @hoxyq, I am very concerned with how noisy and expensive releasing a nightly for all packages every 24hrs might be.

Can you elaborate on how "expensive" it is publishing a 0.0.0- version tagged as @nightly for every monorepo package? I believe is as expensive as publishing a @nightly for React Native (i.e. either we do nightlies for all the packages or we don't do them at all).

[kelset]

Can you elaborate on how "expensive" it is publishing a 0.0.0- version tagged as @nightly for every monorepo package? I believe is as expensive as publishing a @nightly for React Native (i.e. either we do nightlies for all the packages or we don't do them at all).

yeah I'm not talking strictly about the "amount of work" that needs to be done (once an automation is setup, publishing one or publishing all is basically the same), it's more actually the computational side of having CI have to do all the releases and going through npm and polluting the releases histories and stuff... like, if most days the nightly release are the same as the standard released code, we're wasting electricity to do something useless. Maybe I'm a bit over sensitive because Saturday was Earth Day ;)

[cortinico]

Practically those packages are all just .js files so there is not much computation involved.
Agree that we could be smarter and not re-publish a new version for @nightly if the previous one is unchanged.

[lunaleaps]

To summarize:

• It sounds like people want option 2 where we ship nightlies of monorepo packages
• Ideally we only publish nightly if there have been changes
  • Note: we lose out on version alignment here between all the nighties -- I don't know if that's a big deal.
• Naming scheme depends on what decision we make from Renaming our nightly version #632

Does that sound right to folks?

[lunaleaps]

Closing this as this work has been completed!