Skip to main content

· One min read
Charles Korn

The 0.79.0 release added publication of checksums for the wrapper scripts. These checksums make it easy to verify the integrity of the wrapper scripts used in your project.

Verifying the integrity of the wrapper scripts in your project is important, as they could be maliciously modified and these modifications may not be immediately apparent.

To make verifying your project's wrapper scripts even easier, I'm pleased to announce the release of the new batect-wrapper-validation-action action, which validates the integrity of the wrapper scripts as part of a GitHub Actions workflow.

The readme has more details on how the action works and how to use it. For most projects, it's as simple as adding the following job to an existing workflow:

jobs:
validate-batect-wrapper:
name: Validate Batect wrapper scripts
runs-on: ubuntu-20.04

steps:
- name: Check out code
uses: actions/[email protected]

- name: Validate Batect wrapper scripts
uses: batect/batect-wrapper-validation-[email protected]

You can also use the action in an existing workflow or existing job.

warning

This action must run before any invocations of Batect.

If the action runs after an invocation of Batect and the wrapper script has been modified maliciously, the malicious version may be able to modify itself to appear genuine.

· 6 min read
Charles Korn
Cross-post

This is a cross-post from my personal blog.

A fairly frequent question I get is: why is Batect written in Kotlin? And often the question behind the question is: why isn't it written in Golang?

To answer those questions we need to go back in time to 2017, when I first started working on Batect.

I had the idea for what would become Batect after working with a couple of different teams who were all trying to use Dockerised development environments. And after going through the pain of trying to once again reinvent the same less-than-ideal setup we'd had on a previous team, I decided to build a proof of concept for my idea.

At the time, all I wanted to do was quickly build a proof of concept. I was more concerned about building it quickly and having fun and learning while I built it than anything else: I wasn't expecting the proof of concept to be anything more than some throwaway code. (Famous last words.) I had been dabbling in Kotlin for a while and thought this would be a great opportunity to play with a language I really liked.

Fast forward a bit and we come to late 2017. I'd finished the proof of concept in Kotlin and demoed it to my team for feedback. They gave some really positive feedback and that encouraged me to seriously consider turning Batect into something more than a proof of concept.

At this point, I had a codebase that could run a sample project, a working interface to Docker that invoked the docker command to pull images, create containers etc., and some basic tests. I had two choices: continue with this largely working app, or throw it away and start afresh.

As part of considering whether or not to start afresh, I thought for a while about whether to continue in Kotlin or switch to Golang. There were four things that were going through my mind:

I had something that worked and a codebase that was in relatively good shape.

Switching to Golang would require starting from scratch. Given I was doing this entirely on my own time, that didn't seem like a great use of time, especially for something that still didn't have any active users.

The main argument in favour of Golang was the fact that the vast majority of the Docker ecosystem is written in Golang.

If Batect was written in Golang, I could take advantage of things such as the Docker client library for Golang, rather than write my own for Kotlin. I was using the docker CLI to communicate with the Docker daemon from Kotlin and changing to use the HTTP API didn't seem that difficult if needed later.

Another argument in favour of Golang was the ability to build a single self-contained binary for distribution, rather than requiring users to install a JVM.

However, earlier that year, JetBrains had announced the first preview of Kotlin/Native, which would allow the same thing for Kotlin code. Younger, naïve-r me assumed that would be good enough and easy enough to adopt in the future should the use of a JVM really turn out to be a problem.

The last thing was simply personal preference and how much I enjoyed working with the language.

Again, this was something I was doing on my own time and had no active users yet, so personal enjoyment was one of the main priorities for me. At the time, I was working on a production Golang system and was finding the language somewhat lacking in comparison to Kotlin. Dependency management in Golang was a pain (this would be fixed with the introduction of modules in Go 1.11 in August the next year), and I found Kotlin's syntax and type system enabled me to write expressive, safe code.

So I chose to continue in Kotlin.

Looking back on that decision, there are definitely some things that remain true today, and others where the situation turned out to be a bit different:

Not rewriting Batect from scratch meant I was able to continue adding new features and incorporating feedback. This meant that Batect was ready to introduce to a new team in late 2017. This team chose not only to take a risk and adopt a completely unproven tool, but continue to this day to be some of its strongest advocates. If I'd stopped to rewrite Batect in Golang, I would have missed that opportunity.

Using the docker CLI worked reasonably well for quite some time, but eventually the performance hit of spawning new processes to interact with the Docker daemon was starting to have a noticeable impact.

Switching to use the API directly was, sadly, not as straightforward as I hoped. In particular, I had failed to consider the client-side complexities of some of Docker's features, such as connection configuration management, image registry credentials and managing the terminal while streaming I/O to and from the daemon.

This has played out over and over again, and has been a significant drain on my time, especially when it came to adding support for BuildKit, which is largely undocumented, requires extensive client-side logic to implement correctly and relies on a number of Golang idiosyncrasies.

Kotlin/Native sadly hasn't matured as quickly as I expected. So Batect still requires a JVM, and this adds a small barrier to entry for some people. Having said that, JetBrains is still actively developing Kotlin/Native and significant progress has been made in the last 12 months or so, so I remain hopeful that removing the need for a JVM is still an achievable goal. This will not be painless -- Batect has dependencies on some JVM-only libraries at the moment -- but it certainly seems within reach.

The last point will always be a matter of personal opinion, but Kotlin remains my favourite language to this day.

Every now and then I question my choice and whether it was the right decision to continue building Batect in Kotlin. While some things may well have been easier had I chosen to use Golang, Batect has hundreds of active users who love using it, and I still enjoy working on it after all this time, and those are the two things that matter most to me.

Thank you to Andy Marks, Inny So and Jo Piechota for providing feedback on a draft version of this post.

· One min read
Charles Korn

It's been a big few days for Batect, with two significant milestones achieved:

  • Batect has once again been featured on the Thoughtworks Technology Radar. It has been placed in "trial", the ring one step from "adopt".

  • Batect's GitHub repository broke through 500 stars.

I'd like to thank everyone who's helped get Batect to this point - whether you've simply tried "hello, world" with Batect, submitted some feedback or advocated for its adoption on your teams, Batect exists for its users, and I'm incredibly grateful for all the support and encouragement I've received from every one of you.

There's still much more to do, and many ways Batect could be further improved - watch this space!

Disclosure: while I am employed by Thoughtworks, I am not involved in the selection process for entries on the Radar, nor did I nominate Batect for inclusion. I did provide a review of the entry description prior to its publication. See "how do we build the Radar?" on the frequently asked questions page for more information on the process behind the Technology Radar.

· One min read
Charles Korn

If you've ever wanted to hear the story of how Batect got to where it is today, and some of the lessons learnt along the way, here's your chance: I'm presenting a talk titled Applying lessons from open source to your own platform on Tuesday, June 1 at 6pm (Melbourne time).

For those of you in Melbourne, you can attend in person, and it will also be live-streamed on Zoom for anyone to watch anywhere.

Registration is required at https://www.thoughtworks.com/by-thoughtworks/events.

· One min read
Charles Korn

I'm once again running a survey to collect feedback on Batect, and this year I'm offering an AU$50 voucher (or equivalent in your local currency) as a prize to encourage responses.

The survey is available at: https://forms.gle/1S7a7C7zfPzi39vN6

If you're using Batect on your team at the moment, I would be super grateful if you could also share this within your team - the more responses I get, the better feedback I get, and the better I can make Batect.

· 2 min read
Charles Korn
tip

tl;dr: run ./batect --upgrade to upgrade to Batect v0.69.0 or later to ensure Batect continues to work after May 1

Bintray announced on February 3 that it will be shutting down on May 1, 2021.

Batect's wrapper script (batect on macOS and Linux, and batect.cmd on Windows) downloads Batect from Bintray if it has not already been cached on your machine.

After Bintray's shutdown on May 1, downloading Batect from Bintray will no longer work. Batect v0.69.0 introduces a new download server (updates.batect.dev) which does not rely on Bintray.

In order to continue using Batect without any issues after May 1, you must do one of the following:

Option 1: upgrade to v0.69.0 or later

To switch to the new server, the easiest thing to do is to upgrade with ./batect --upgrade. This will automatically update your wrapper script to the new version that uses the new download server.

Option 2: remain on an old version and switch to new server

The new download server also supports old versions. If you need to remain on an older version, make the following changes in your project's copy of the wrapper script:

batect
-DOWNLOAD_URL_ROOT=${BATECT_DOWNLOAD_URL_ROOT:-"https://dl.bintray.com/batect/batect"}
-DOWNLOAD_URL=${BATECT_DOWNLOAD_URL:-"$DOWNLOAD_URL_ROOT/$VERSION/bin/batect-$VERSION.jar"}
+DOWNLOAD_URL_ROOT=${BATECT_DOWNLOAD_URL_ROOT:-"https://updates.batect.dev/v1/files"}
+DOWNLOAD_URL=${BATECT_DOWNLOAD_URL:-"$DOWNLOAD_URL_ROOT/$VERSION/batect-$VERSION.jar"}
batect.cmd
-$DownloadUrlRoot = getValueOrDefault $env:BATECT_DOWNLOAD_URL_ROOT "https://dl.bintray.com/batect/batect"^
+$DownloadUrlRoot = getValueOrDefault $env:BATECT_DOWNLOAD_URL_ROOT "https://updates.batect.dev/v1/files"^


$UrlEncodedVersion = [Uri]::EscapeDataString($Version)^


-$DownloadUrl = getValueOrDefault $env:BATECT_DOWNLOAD_URL "$DownloadUrlRoot/$UrlEncodedVersion/bin/batect-$UrlEncodedVersion.jar"^
+$DownloadUrl = getValueOrDefault $env:BATECT_DOWNLOAD_URL "$DownloadUrlRoot/$UrlEncodedVersion/batect-$UrlEncodedVersion.jar"^

· 2 min read
Charles Korn

tl;dr: Renovate now supports Batect, which means it can automatically send you PRs to update Batect itself as well as any images or bundles you reference in your configuration files.

Keeping dependencies and tools up-to-date is a necessary but tedious task for any codebase. Tools like Dependabot and Renovate help to automate this process by scanning your repository for dependencies, identifying which ones are out of date and then sending you PRs to update them.

I'm pleased to announce that Renovate has accepted a series of PRs that add support for Batect to Renovate. These cover all three scenarios required to keep Batect and your configuration up-to-date:

  • updating Batect itself (equivalent to running ./batect --upgrade)
  • updating any Docker image references in your configuration files
  • updating any bundle references in your configuration files

If you use Dockerfiles to build images for use with Batect, Renovate already has you covered as well - updating base image references in Dockerfiles has been supported by Renovate for a while.

No configuration is required to activate this. If you're already using Renovate's hosted service, you'll start receiving update PRs automatically. If you host Renovate yourself, update to 24.2.0 or later.

By default, Renovate will only update image or bundle references in files named batect.yml or batect-bundle.yml. If you use file includes to split your configuration over multiple files, you'll need to configure which files to search with fileMatch in your renovate.json. Update 22/12: if your configuration is split over multiple files with file includes, no further configuration is required - Renovate will now automatically detect image and bundle references in files included in your main configuration file.

Take a look at the documentation for more information.

Subscribe to the Batect newsletter

Get news and announcements direct to your inbox.