Build manuscript from a branch different than master

[jaybee84]

Is it possible to power the pdf/html output builds using a branch different than the master? If yes, how should one set that up?

[agitter]

We haven't done that, but it seems feasible with some customization. Could you please tell us more about the workflow you'd like? Then we can point you in the right direction in regards to editing the continuous integration workflow and/or deploy script.

I also plan to edit the issue name, if you don't mind. This is a good question. I could see others search for it in the issues.

[jaybee84]

@agitter Thanks for your response and for renaming the issue.

The workflow I am thinking of is PR > draft-branch > master. Most PRs will be against draft-branch. When a considerable section is built after consolidating work through individual PRs, the section is pushed from draft-branch to master branch. Somewhat analogous to "dev" branch and "prod" branch in case of an app (dev == draft-branch, prod == master).

As the sections are being built, it would be great to visualize the layout of the draft-branch and make any necessary changes without the need to push it to the master branch. It may also help the readability of the draft-branch to facilitate the intermediate review processes.

[agitter]

Thanks for describing your goals. That workflow makes sense.

Several Manubot projects use continuous integration artifacts to accomplish similar things. Artifacts are available in GitHub Actions and AppVeyor, but not Travis CI. With either of those services, the continuous integration build on a non-master branch will store PDF and HTML outputs so that you can preview the layout, new references, figures, etc. AppVeyor can also push comments into a pull request with direct links to the updated manuscript artifacts.

• Where to find the GitHub Actions artifacts
• Example AppVeyor comment with links to artifacts

That is my recommended option if it accomplishes what you're looking for because the setup is easy. It would also keep the "official" versions of your manuscript that are hosted on GitHub pages clean. They would only contain the master branch updates, not the draft-branch updates that may need further revisions.

If you instead want to deploy manuscripts after merging pull requests to master or draft-branch, that should also be possible. However, it would require updating and testing the workflow. The deploy script would probably need to be modified as well.

[jaybee84]

Thanks @agitter for the solutions... We are going to test out the GH actions artifacts for now.

Is there documentation that I could follow to update the build script in case we want to go ahead with deploying the manuscript after PR merge to draft-branch (instead of master)?

[agitter]

Is there documentation that I could follow to update the build script

No, there isn't existing documentation because that hasn't been attempted before. We could help you understand the current build process so that you can modify it to meet your needs.

If this is for your https://github.com/jaybee84/ml-in-rd/ manuscript, it looks like you are using the GitHub Actions workflow for continuous integration. In that case, you would modify this line of manubot.yaml:

rootstock/.github/workflows/manubot.yaml

Line 57 in 8b9b5ce

if: github.ref == 'refs/heads/master' && github.event_name == 'push' && !github.event.repository.fork

My first attempt would be to replace
github.ref == 'refs/heads/master'
with
(github.ref == 'refs/heads/master' || github.ref == 'refs/heads/draft-branch')
However, I haven't tested it to know whether || is supported syntax. That assumes you want to deploy from both branches. Switching to deploy from refs/heads/master to refs/heads/draft-branch in the above line (instead of either one) should be straightforward.

I don't see any immediate changes that would be required in the ci/deploy.sh script that runs in that workflow step. If it doesn't behave as expected, you could let us know and we can inspect the GitHub Actions build log.

[jaybee84]

Thanks! I will try that and report back .. :)

[jaybee84]

It seems that adding "draft-branch" to the manubot yaml did not reflect in the gh-pages updating from the draft-branch (see new yaml)

I also added "draft-branch" to the push and PR branch sources in the yaml. That also did not update the gh-pages to reflect the latest merges in the draft-branch. The gh-pages still show the content of the master branch.

[agitter]

The branches: patterns in the yaml file are expected to be a list. See the example here for listing multiple branches. I don't expect (master || draft-branch) is valid syntax, which may be why the workflow didn't run on jaybee84/ml-in-rd@6324fd1.

It may be that the (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/draft-branch') syntax is also invalid. You could see if it runs with only github.ref == 'refs/heads/draft-branch'. You could also implement your own logic in the "Deploy Manuscript" step. That would mean removing the if line entirely and using an if statement in the run section to decide whether to deploy or not. It would look like this

rootstock/.github/workflows/manubot.yaml

Lines 44 to 47 in 8b9b5ce

	run: |
	if [ "${SPELLCHECK:-}" = "true" ]; then
	bash ci/install-spellcheck.sh
	fi

which checks the state of an environment variable instead of the branch.

[agitter]

I'm closing this per jaybee84/ml-in-rd#34 (comment) It is possible to deploy a manuscript from a different default branch besides master. However, see that discussion for more nuances about trying to deploy both a master and draft version of the manuscript.

@jaybee84 if you want to discuss further customization, you can tag me in jaybee84/ml-in-rd#34 or a new issue in that repo.

[lubianat]

Small tutorial on how to build manuscript from a different branch than the master branch.

Goal:

Render html and pdf for pushes in other branches than master branch.
Keep the GitHub Pages html for the version currently in the master branch.

Discussion goes on at #341.
The issue #341 motivated : jaybee84/ml-in-rd#34 (comment)

I am using Github Actions for building the manuscript, so I have to make changes to the manubot.yaml file.

The yaml file of my project currently mentions the master branch in a few parts:

@agitter `s fix was to remove the conditional to branch in Deploy Manuscript, add GITHUB_REF: ${{ github.ref }} add to Deploy Manuscript’s env and change the run to:

     run: |

        # log for debugging

         echo Current ref: $GITHUB_REF

         if [ "$GITHUB_REF" = "refs/heads/draft-branch" ]; then
           bash ci/deploy.sh
         fi

And add the other branch to the header. Let’s try that.

You can see the changes I made in my commit here

The build was triggered (hooray) and is running now here.

In the logs, it mentions:

This is a good sign.
There seems to be no cache. I cannot say if this is a problem, an optimization task or irrelevant:

It may be the case that it will just take longer now. I’ll check the next builds too.

It did not, however, push to the output branch, where I had compiled a .docx before, as I was expecting.

It did, nevertheless produced the right artifact, so it was a success!

(the end)