CircleCI Workflows for any GitHub Event

CircleCI Workflows in multiple files, triggered by any GitHub event

Photo by Daniel von Appen on Unsplash

Running a CircleCI workflow from any GitHub event is surprisingly easy and unlocks additional possibilities with your existing CircleCI workflows.

I’ll touch on how we can use GitHub Actions and CircleCI together before demonstrating a possible use case. This post assumes you have a basic knowledge of CircleCI, GitHub Workflows/Actions (which I’ll likely just refer to as “Actions” henceforth), and YAML.

Now, before you go implementing, you should know that this is definitely something I would qualify as a “hack”, and I’ll describe why at the end.

Background

Currently, CircleCI does not support multiple workflow files and only supports the GitHub events push and pull_request. With many teams and organizations moving to a “GitOps” mentality as a part of their DevOps journey, supporting other events is crucial for faster feedback and automation. Separating individual workflows into smaller consumable pieces can also enable CI changes at a faster pace.

YAML for days

In my day job, I see tons of CI config files (not all are CircleCI, either) and they often share one striking similarity — they are long and complicated. They become this unmanageable nightmare, akin to the horror stories developers tell of 10000 line-long functions…

PyTorch’s CircleCI config file (don't worry its autogenerated)

GitHub Actions/Workflows vs CircleCI Workflows

GitHub Actions has really simplified the experience for developers responsible for their CI pipelines. Rather than pouring through a large config file to find a workflow for some random service deployment in your monolith, Actions enables developers to manage a single workflow file for each piece of their CI pipeline.

Homebrew GitHub Workflow files

CircleCI cant do that… or can it…?

GitHub Workflow -> CircleCI Workflow

A CircleCI pipeline runs whatever the .circleci/config.yml contains when you push. This is a core feature of any configuration-as-code, since it enables you to test CI changes as a part of the pull request.

So, what if we…

1. Replace the CircleCI config file in a GitHub workflow?
2. Push those changes to a remote branch?
3. Verify the CircleCI workflow completes successfully?
4. Cleanup the branch?

GitHub Workflow

If you're familiar with Actions, see the workflow file below. If you’re not, I’ll try to describe what’s happening below the snippet.

GitHub Workflow to commit and push a branch

The on key defines the GitHub events and event types that will trigger the workflow.

Any job in the jobs key will be executed. In this case, there is only one job.

The job runs-on an Ubuntu VM and runs the steps inside itself.

The first step checks out the code on the main (master) branch.

Next, a step replaces the .circleci/config.yml with a config file that matches the event name and type.

Finally, we use a prebuilt action to commit that change, push the change to a remote branch, verify the CircleCI pipeline completes (via the GitHub Checks API), and then delete the branch.

CircleCI Config Files

Any GitHub event you want to trigger on CircleCI should match a config file in the .circleci/ directory as <event-name>-<type>.yml. For example, the GitHubissues event with theopened type will run the config file named issues-opened.yml. The following directory example maps to events in the workflow YML example above.

Multiple CircleCI Config File Example

Fill in the new CircleCI config files with whatever you want — send a text message, deploy a staging environment, or perform a final security scan before a GitHub release. I bet you already have a CircleCI workflow you can re-use!

The default config.yml will still run on any push like a normal CircleCI pipeline. You can set up a workflow in the default config file to never run any pipelines if you want.

Try it out

To see this in real-time:

1. Comment on this issue.
2. Watch the workflow start on GitHub.
3. While that runs, wait here to watch the pipeline on CircleCI.

Remember, the GitHub Workflow will wait for CircleCI to complete.

GitHub Actions Workflow filtered by issue comment events

CircleCI pipelines filtered by the branch matching the GitHub event

Why would you not do this?

As I said in the beginning, I’d definitely call this a “hack”. I generally don’t like using that term, but sometimes it feels appropriate when you describe using one CI tool to take advantage of another. Especially when this “advantage” comes down to spending more money, since you’ll be spending credits on both CI platforms.

There’s also the context in the GitHub events that you won't get in the CircleCI workflow without additional work, such as: the user that created the issue, what the issue comment is, who closed the issue, etc.

I have also ignored any race conditions, like trying to delete a branch that has already been deleted by another workflow.

So, with that said, take this as an opportunity to expand your CircleCI workflows in your GitHub projects, especially ones already using CircleCI. Just remember, “hacks” are only temporary solutions.

Here’s the link to the repo directly for reference or forking:

dsayling/gitops-poc