pkgsubmission.Rmd

# Guide for Editors  {#pkgsubmission}


## Editor-in-Chief

As for rOpenSci's current peer-review system, packages are submitted directly
to the [`ropensci/software-review`
repository](https://github.com/ropensci/software-review) on GitHub, with
submissions handled initially by the rotating Editor-in-Chief (EiC). All
statistical software submissions should begin with a pre-submission enquiry,
although direct submissions may be accepted at the EiC's discretion. The EiC
should perform the following duties on initial pre-submission enquiries of
statistical software:

- Check that the appropriate submission template has been used, and ask authors
  to edit and update data if not.
- Ask authors to respond to any comments issued by the `ropensci-review-bot`
  about template format and content.
- Call `@ropensci-review-bot check srr` to generate a summary of compliance
  with our "srr" (Software Review Roclets) system for documenting standards
  compliance.
- Confirm from the report generated by `check srr` that the software complies
  with at least half of all applicable standards, 
- Perform other EiC responsibilities described in our [main *Dev
  Guide*](https://devguide.ropensci.org/editorguide.html#eicchecklist).

Full submissions generate a detailed report from our `ropensci-review-bot`, as
described below. Examples can be seen by looking at [any recent
submissions](https://github.com/ropensci/software-review/issues). The EiC need
only purvey the summary checks within the initial section of the report to
confirm that all the submission passes all checks, and that the report
concludes with a statement that,

> *This package may be submitted*

In response to that statement, the sole tasks for an EiC prior to delegating
a handling editor are to check the following single item:

- [ ] The categories nominated by the submitting authors are appropriate for
  this package

And to choose one the following two items:

- [ ] The package does not fit within any additional categories of statistical
  software.
- [ ] The package could potentially be described by the following additional
  categories of statistical software:
    - \<list categories here\>


Additional effort by the EiC will only be required in "edge cases" where
a package may be unable to pass one of the checks, as in [this sample automated
check](https://github.com/ropensci/statistical-software-review-book/blob/main/scripts/pkgcheck-ex.md),
which concludes with the statement that,

> *All failing checks above must be addressed prior to proceeding*

In these cases, submitting authors must explain why these checks may fail, and
the EiC must then determine whether these failures are acceptable.
Such cases ought nevertheless be rare, and it may be expected in the majority
of cases that the sole tasks of the EiC are to confirm a positive bot response,
to complete the two checklist items given above, and to allocate a handling
editor. This latter step is done by calling `@ropensci-review-bot assign <name>
as editor`.


## Handling Editor {#pkgsub-handling-editor}

The Handling Editor should use the summary report generated by the opening of
the issue to perform an initial assessment and to guide assignment of
reviewers. The contents of these automated checks are described at the
end of this chapter. The EiC need only consider the initial summary
checklist, but handling editors should consider all details contained within
the automated report. General procedures for Handling Editors, including
explanations of relevant `ropensci-review-bot` commands, are described in the
[Editor's Chapter of the general Dev
Guide](https://devguide.ropensci.org/editorguide.html). This section describes
specific editorial requirements for statistical submissions.

### Handling Editor Checklist

Having considered the automated package report, and addressed all
issues raised within that report, Handling Editors should check the following items 
before assigning reviewers.

- [ ] Any issues raised during initial processing by EiC have been resolved (or
  there were none).
- [ ] Issues raised in the `goodpractice` checks, statistical anomalies, and other
  details of the report do not appear to be major problems and can be directed to
  reviewers for further scrutiny.
- [ ] **Either** (i) this package is aiming for a bronze badge, **or**, (ii)
  for packages aiming for silver or gold badges, the authors have clarified
  which of the four aspects listed in the "*Guide for Authors*" section on
  [silver badges](#pkgdev-silver) they intend to fulfil.

Issues flagged by the handling editor may
require iteration with submitting authors. As stated in the 
[*Dev Guide*](https://devguide.ropensci.org/editorguide.html#upon-submission):

- If authors believe changes might take time, [apply the holding
  label](#policiesreviewprocess) to the submission.
- If the package raises a new issue for rOpenSci policy, start a conversation
  in Slack or open a discussion on the [rOpenSci
  forum](https://discuss.ropensci.org/) to discuss it with other editors
  ([example of policy
  discussion](https://discuss.ropensci.org/t/overlap-policy-for-package-onboarding/368)).

Once all items have been checked, Handling Editors may proceed to find and
assign two reviewers. As for [general software
reviews](https://devguide.ropensci.org/editorguide.html#look-for-and-assign-two-reviewers),
start by commenting `@ropensci-review-bot seeking reviewers`. A email template
to invite prospective reviewers of statistical packages is [in the
appendix](#reviewrequesttemplate). Once reviewers have agreed, use 
the command `@ropensci-review-bot add <@GITHUB_USERNAME> to reviewers`, and
generally follow the procedure given in the [*Dev
Guide*](https://devguide.ropensci.org/editorguide.html#look-for-and-assign-reviewers).
Handling editors may provide guidance to reviewers on issues that they think may be worth
looking into based on their initial review of the package and automated checks.

[Note that the *Dev Guide* is rapidly iterating as new capabilities are added to the review bot,
and you may wish to refer to the draft version at <devdevguide.netlify.app/>.]

### Re-generating package check results

The handling editor may update the initial package check results at any time
with the following command:

```
@ropensci-review-bot check package
```

This is likely to be necessary following each review, to ensure any issues
identified from the initial checks have been satisfactorily addressed.


### Disagreements on Badge Grades

Handling editors are responsible for resolving any disagreements between
authors' stated or desired grade of badge and reviewers' recommendations.
[Click here to jump to the corresponding recommendations for
reviews](#disagreement-with-authors-intentions). The views of reviewers should
generally be prioritized in such cases. Grades as declared by authors are
contained in the opening comment of the issue. These may be extracted by
calling:

```
@ropensci-review-bot stats grade
```

Editors may modify these grades by editing the opening comment, and changing
the value of the `"statsgrade"` variable.

### Approval

After having completed a checklist and ensuring agreement on badge grade, the
handling editor may approve a submission with the following command:

```
@ropensci-review-bot approve
```

The bot will identify that this is a statistical software issue, extract the
appropriate grade, and attach a corresponding badge which will also label the
latest version of our statistical standards.

## Automated Checks

Upon initial submission, the `ropensci-review-bot` performs a suite of tests
and checks, and will upload a report into the GitHub issue thread. This report
is the primary source of information used to inform initial editorial
decisions. All editors must familiarise themselves with the structure and
contents of these automated reports, an example of which can be seen by
[clicking
here](https://github.com/ropensci/statistical-software-review-book/blob/main/scripts/pkgcheck-ex.md).
A similar report can be reproduced locally by running the code within the
following sub-section.

<details>
<summary>Package report code here (click to see).</summary>
<p>

```{r pkgcheck-code}
#| eval: false
#| echo: true
library (srr)
library (pkgcheck)
path <- srr_stats_pkg_skeleton ()
srr_stats_roxygen (category = "regression",
                   filename = file.path (path, "R", "srr-standards.R"))
check <- pkgcheck (path)
md <- checks_to_markdown (check, render = TRUE)
```

</p>
</details>


The crucial section for statistical packages is the first, which describes
checks conducted by the [`srr` (**S**oftware **R**eview **R**oclets)
package](https://github.com/ropensci-review-tools/srr). This check confirms
that all statistical standards have been documented within the code, and all
packages must pass this check. The report linked to in that section is
primarily intended to aid reviewers, and may be ignored by handling editors.

The second section describes the "*Statistical Properties*" of the package
being submitted, and should be considered by handling editors. In particular,
this section contains information which identifies any statistically noteworthy
properties of the package. The example report illustrates how this report
immediately identifies that the package has very little code, very few
functions, and very few tests. Handling editors should consider these
statistical details, and particularly any noteworthy aspects (defined by
default as lying within upper or lower fifth percentiles in comparison with all
current CRAN packages). Any aspects which seem concerning should be explicitly
raised with submitting authors prior to proceeding. The measures currently
considered include various metrics for:

- Size of code base, both overall and in sub-directories
- Numbers of files in various sub-directories
- Numbers of functions
- Numbers of documentation lines per function
- Numbers of parameters per function
- Numbers of blank lines

A final metric, `fn_call_network_size`, quantifies the number of
inter-relationships between different functions. In `R` directories, these are
function calls, while relationships may be more complex within `src` or `inst`
directories. Small network sizes indicate packages which either construct few
objects (functions), or in which internal objects have no direct relationships.

The third and final section of the automated report contains details of
[`goodpractice`](https://github.com/MangoTheCat/goodpractice) checks including:

- Code coverage estimates for each file (from the [`covr`
  package](https://github.com/r-lib/covr)).
- Code style reports from the [`lintr`
  package](https://github.com/jimhester/lintr).
- Cyclomatic complexity reports from the [`cyclocomp`
  package](https://github.com/MangoTheCat/cyclocomp).
- Any errors, warnings, or notes raised by running `R CMD check` (from the
  [`rcmdcheck` package](https://github.com/r-lib/rcmdcheck).

Any aspects of these
[`goodpractice`](https://github.com/MangoTheCat/goodpractice) reports which do
not pass the initial checklist (such as warnings or errors from `R CMD check`,
or test coverage < 75%) should be clarified with authors prior to proceeding
with review.

Finally, the initial *Statistical Description* includes details of computer
languages used in a package, and should be used to ensure reviewers have
appropriate experience and abilities with the language(s) in which a package is
written.