Running Browsertrix Cloud on Minikube or k3s on NixOS

ops nix archive

published 2023-05-04 23:17

updated 2023-05-05 14:01


Clone the repo

{{{bash git clone }}}

Install k3s on Nix

{{{nix {


networking.firewall.allowedTCPPorts = [ 6443 ]; services.k3s.enable = true; services.k3s.clusterInit = true; services.k3s.role = "server";

services.k3s.extraFlags = toString [

"--kubelet-arg=v=4" # Optionally add additional args to k3s


environment.systemPackages = [ pkgs.k3s pkgs.kubernetes-helm ];

} }}}

Install the Helm Charts

The docs say to run export the KUBECONFIG env-var and then run helm upgrade --install -f ./chart/values.yaml -f ./chart/examples/local-config.yaml btrix ./chart/

Because Nix runs k3s under the root user, we also need to run this command as root

{{{bash sudo su export KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm upgrade --install -f ./chart/values.yaml -f ./chart/examples/local-config.yaml btrix ./chart/ }}}

I had a problem with my backend starting up before mongodb was ready, and it being unable to apply the db migrations. Deleting the pod and restarting resolved this issue. The chart should probably be set up to wait for mongodb to be configured before running the backend.

The logs are available here

Using Minikube with a DevShell

NixOS doesn't have a service for minikube configured, so let's use a devshell Remember to put your flake.nix file in the browsertrix folder and add it to the git repo!



{ description = "Simple minikube dev shell";

inputs = { nixpkgs.url = "github:nixos/nixpkgs"; };

outputs = { self, nixpkgs }: let pkgs = nixpkgs.legacyPackages.x86_64-linux; in { devShell.x86_64-linux = pkgs.mkShell { buildInputs = with pkgs; [ minikube kubernetes-helm jq ]; shellHook = '' alias kubectl='minikube kubectl' . <(minikube completion bash) . <(helm completion bash)

      # kubectl and docker completion require the control plane to be running
      if [ $(minikube status -o json | jq -r .Host) = "Running" ]; then
              . <(kubectl completion bash)
              . <(minikube -p minikube docker-env)

} }}}

You'll also need to ensure you have at least one backend configured for minikube to use. You'll need to re-login for the group changes to take effect.

{{{nix {

# Either docker
virtualisation.docker.enable = true;

virtualisation.docker.storageDriver = "btrfs"; # only if you're using btrfs

or use kvm

virtualisation.libvert.enable = true; users.users.anish.extraGroups = [ "libvert" "docker" ]; } }}}

Enter the devshell and start up minikube, and run the helm command from the docs. This time we don't need sudo because minikube is running under our own user.

{{{bash minikube start

Will block the terminal, will need to open a new one

minikube dashboard

Creates "default-http-backend"

minikube addons enable ingress helm upgrade --install -f ./chart/values.yaml -f ./chart/examples/local-config.yaml btrix ./chart/ }}}