← Back

Probot App or GitHub Action? (Updated)

This is a second post on the same topic - the first was published before GitHub Actions v2 was introduced.

Spoiler: it (still) depends.

GitHub Actions and Probot are two totally separate projects with the same end goal: to enable and empower developers to extend their workflows, and make GitHub work exactly how they want it to. I get questions about when I’d use either one, and while a lot of the original version of this post is still true, Actions has evolved since then. A quick primer on each tool:

GitHub Actions

I won’t go too deep into what Actions are - you can check out the official docs to learn more. The important notes for right now are:

  1. Run code to respond to an event on GitHub
  2. GitHub will run anything in a virtual machine

The key point is that GitHub runs your Actions in an ephemeral virtual machine - there’s no hosting, server costs or deployment to worry about. It’s sort of like a scoped serverless function that’s triggered by events in a GitHub repository. These can be either Container actions (code that runs in a Docker container) or a JavaScript action, where the JS code is run immediately inside the virtual machine.

Probot

Probot is an open-source framework for building GitHub Apps in Node.js. It handles boilerplate things like authentication, and provides a straightforward EventEmitter-like API.

It’s been enabling developers to build automation and workflow tools for a while - I like to think of it as an inspiration for Actions!

What’s best for you?

Let’s start by talking about GitHub Actions’ sweet spot. It’s the kind of automation tool that I’d never want to build using Probot: something long-running.

At the time of writing, Actions has a timeout of 6 hours. That means that (and this isn’t an exaggeration) you can run whatever code you want in a virtual machine so long as it doesn’t take longer than that time - and GitHub will run it for you 😍.

Actions are intended to be reused across workflows - you can “import” an action and use it, even if the code lives outside of your repository, sort of like a dependency. One vitally important Action is actions/checkout - it clones the repository that the workflow is running in, and allows your workflow to interact directly with the repo’s code.

So to me, the best use for Actions is something that makes use of your codebase. Things like deployment or publishing tools, formatters, CLI tools - things that need access to your code. One concrete example - let’s say you want to simply log all instances of a string in your codebase, maybe for auditing reasons:

- uses: actions/checkout@v2
- runs: grep "GitHub Actions" -r .

This will log all instance of GitHub Actions - no muss, no fuss, that’s all you’d need to do (plus the other workflow file boilerplate).

Speed

In v1 of this post, I cited performance as one of Actions’ major downsides. In the time since, Actions have become way faster - and while they won’t match the immediacy of a Probot app, they’re more than fast enough for most automation tools.

This is still a point of interest though - if you need your automation tool to act immediately after something happens on GitHub, Actions will need to spin up a virtual machine and start running code. A Probot app will always be running, and will immediately receive a webhook payload and begin executing the relevant handler.

Actions are scoped to repositories

Currently, GitHub Actions only act within repositories, and only on repository-focused events. For example, if you create an issue within a repository, you can kick off an Actions workflow from that trigger. You won’t be able to trigger an Actions workflow when something happens at the organization level - like a new member being added, or a repository being created. I’ve seen a ton of feature requests around this functionality, so I’m optimistic that there’s more coming down the line.

Actions’ logs show up in the repository

This is a really great feature - your automation tool can write logs that are accessible directly in the GitHub UI. Compared to a separate service with its own logging system, this can really bring the context of workflow activity to the right people, without a ton of effort or context-switching.