monok8s

Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit

https://docs.mono.si/gateway-development-kit/getting-started

DISCLAIMER

USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this. I am testing them all by myself right now.

IMPORTANT NOTES

• The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
  • So ip addr eth0 is your right most port
• If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.

Build

Find the latest package versions and update build.env

• kernel
• busybox
• CRI-O
• Kubelet

Then run

make release

Then run, follow the instructions

make cluster-config

Flashing

USB Drive

1. make release
2. Format the USB in vfat
3. Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
4. Run

usb start
usb tree
fatls usb 0:1   # For fat
ext4ls usb 0:1  # For ext4 if you insist
fatload usb 0 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000

1. Inside initramfs, run

flash-emmc.sh

1. If it boots, create the A/B deployment scheme

• (WORK IN PROGRESS)

tftp (network is required)

1. Put out/[RELEASE].img.gz and out/board.itb into your ftp server

setenv ipaddr 10.0.0.153
setenv serverip 10.0.0.129
tftp 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000

1. Inside initramfs, run

flash-emmc.sh

Making sub stages

make build-base # The image the builds kernel and everything
make kernel     # Builds our kernel from NXP
make initramfs
make itb        # Builds out/board.itb (contains the kernel and the initramfs)

Architecture

• A/B deployment (So we can just do kubectl apply upgrade.yaml with version jumps. Agent will walk through it automatically)
• Read-only OS

Upgrade process

We use a CRD with an agent to handle this. Our versions follows upstream's.

kubectl apply -f upgrade.yaml

apiVersion: k8s.mono.si/v1alpha1
kind: OSUpgrade
metadata:
  name: "my-ugrade"
spec:
  version: "v1.35.1" # Or just "latest"
  imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
  checksum: "sha256:..."

kubectl get osugrades

NAME           TARGET     STATUS      AGE
my-upgrade-1   latest     pending     1m
my-upgrade-2   v1.35.3    accepted    1m # latest gets realized into a version number
their-upgrade  v1.33.2    succeeded   1m

kubectl get osupgradeprogress

NODE        SOURCE         STATUS
my-node     my-upgrade-2   active
their-node  my-upgrade-2   completed

NOTES

The device's dts files are located at here

https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale

• We need both mono-gateway-dk-sdk.dts and mono-gateway-dk.dts since the sdk one includes the non-sdk one.
• The actual dts being used is the mono-gateway-dk-sdk.dts