@ropensci-review-bot help rOpenSci!

Maëlle Salmon 🏠 https://masalmon.eu 🐦 ma_salmon

Licence CC-BY-SA

🔗 https://ropensci-review-bot.netlify.app/

What is buffy?

✨ Editorial bots generator coded in Ruby ✨

rOpenSci (Statistical) Software Peer-Review

✨ Transparent, constructive, non adversarial and open review process of R packages✨

GitHub Issue Thread

Diagram representing GitHub issue thread with emojis as avatars, and wobbly lines as text. Under the GitHub issue thread, a legend indicating who among the emojis is Author /Editor / Reviewer / ropensci-review-bot. An editor is assigned to the issue, and there are issue labels.

Why is rOpenSci software review special?

❄️ Specific R package checks;

❄️ Repositories transferred after approval;

❄️ Levels of minting in statistical software review (gold/silver/bronze);

❄️ Airtable database;

❄️ …

Every system is probably special, buffy is extensible! 🎉

What’s an adaptation of buffy?

In the “ropensci” branch of our buffy fork at https://github.com/ropensci-org/buffy/tree/ropensci :

  • Configuration;

  • Specific responders!

Responders documented at ReadTheDocs.

The big picture

Diagram representing automation for rOpenSci software peer review. On the left, a GitHub issue thread with emojis as avatars, and wobbly lines as text. Under the GitHub issue thread, a legend indicating who among the emojis is Author /Editor / Reviewer / ropensci-review-bot. At the center of the diagram is an Heroku app using the buffy Ruby tool, that receives information from GitHub via webhooks. The app digests messages received and depending on the command pings an external service represented on the right (with a plumber logo); fills the Airtable-based software review database; manages ropensci GitHub organization via GitHub API; posts back or labels in the GitHub issue thread.

Package submission

Screenshot of a GitHub issue corresponding to a package submission. The first comment contains information posted by authors.

Welcome

Screenshot of a GitHub issue corresponding to a package submission. The account ropensci-review-bot writes 'Thanks for submitting to rOpenSci, our editors and @ropensci-review-bot will reply soon. Type @ropensci-review-bot help for help.'.

Package checks at submission

Screenshot of a GitHub issue corresponding to a package submission. Package checks results have been posted.

Package checks later

  • @ropensci-review-bot check package by editors in the issue;

  • pkgcheck::pkgcheck() locally.

Screenshot of a GitHub issue corresponding to a package submission. Editor requires new checks via '@ropensci-review-bot check package', then results are posted.

Assigning a reviewer, before 🤖

😅 Something like…

  • Calculate due date, three weeks from now.

  • Assign reviewers by tagging in the issue with the following format:

    Reviewer: @githubname1 
Reviewer: @githubname2
Due date: YYYY-MM-DD
  • Remind reviewers to fill the volunteering form if relevant.
  • Store reviewer info in software review database (Airtable).
  • Switch numbered label to 3/reviewers-assigned once 2 reviewers are assigned.

Assigning a reviewer, with 🤖

@ropensci-review-bot assign @username as reviewer

Screenshot of a GitHub issue corresponding to a package submission. The editor posts '@ropensci-review-bot add @beatrizmilz to'.

Assigning a reviewer, with 🤖

😌 @ropensci-review-bot assign @username as reviewer

  • Easier to remember;

  • We can tweak what happens in the background!

How we adopted
and adapted buffy

Robot posing next to a black board where it is written 'My favorite robot' with a drawing of the same robot

Picture by Pavel Danilyuk on Pexels.

Who are “we”:

Apart from Karthik Ram and Arfon Smith,

  • Juanjo Bazán, buffy developer 💎

  • 👋

  • Mark Padgham, rOpenSci Software Research Scientist.

  • All users = patient beta-testers of buffy! Package authors, reviewers, editors, rOpenSci community manager Stefanie Butland…

How we assessed our needs

Reading…

How we tested features

  • In a test repository:
Screenshot of a GitHub issue thread where Juanjo and Mark are writing messages mentioning the bot, as part of tests. 'hey @ropensci-review-bot do something'; '@ropensci-review-bot what about this'; 'Thanks @ropensci-review-bot'
  • In real life. 😉

Commands

@ropensci-review-bot <do-something>

  • Meta (help, code of conduct);

  • Editorial management;

  • Review(er) management;

  • Package checking;

  • Approval.

Still a humane process 🤗

  • The reviews are written by humans!

  • Editors write comments personally thanking e.g. reviewers.

  • Out-of-scope decisions are also justified with a non automatic comment.

How did we introduce commands?

  • Direct communication with editors;

  • Release 0.7.0 of our dev guide!

High angle picture of a robot holding a laptop

Picture by Alex Knight on Pexels.

https://ropensci-review-bot.netlify.app/

@ropensci-review-bot thank developers & testers & today's audience

  • A very useful tool, less mental overhead and tedium!

  • More transparency!

Black miniature robot on laptop keyboard

Picture by Kindel Media on Pexels.