monok8s

Alpine-based Kubernetes cluster image for Mono's Gateway Development Kit

https://docs.mono.si/gateway-development-kit/getting-started

DISCLAIMER

USE AT YOUR OWN RISKS. I leverage ChatGPT heavily for this. I am testing them all by myself right now.

IMPORTANT NOTES

The 3 RJ45 ports are label in eth1, eth2, eth0 respectively by the kernel (left to right)
- So ip addr eth0 is your right most port
If the fan stopped spinning. Unplug ASAP! Otherwise CPU temp goes to the moon.

Build

Find the latest package versions and update build.env

Then run

make release

Then run, follow the instructions

make cluster-config

Flashing

USB Drive

make release
Format the USB in vfat
Copy out/[RELEASE].img.gz and out/board.itb to your usb drive's root
Run

usb start
usb tree
fatls usb 0:1   # For fat
ext4ls usb 0:1  # For ext4 if you insist
fatload usb 0 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000

Inside initramfs, run

flash-emmc.sh

If it boots, create the A/B deployment scheme

(WORK IN PROGRESS)

tftp (network is required)

Put out/[RELEASE].img.gz and out/board.itb into your ftp server

setenv ipaddr 10.0.0.153
setenv serverip 10.0.0.129
tftp 0x80000000 board.itb

setenv bootargs "console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 root=/dev/ram0 rootwait rw"
bootm 0x80000000

Inside initramfs, run

flash-emmc.sh

Making sub stages

make build-base # The image the builds kernel and everything
make kernel     # Builds our kernel from NXP
make initramfs
make itb        # Builds out/board.itb (contains the kernel and the initramfs)

Architecture

A/B deployment (So we can just do kubectl apply upgrade.yaml with version jumps. Agent will walk through it automatically)
Read-only OS

Upgrade process

We use a CRD with an agent to handle this. Our versions follows upstream's.

kubectl apply -f upgrade.yaml

apiVersion: monok8s.io/v1alpha1
kind: OSUpgrade
metadata:
  name: "my-ugrade-2"
spec:
  version: "v1.35.3"
  imageURL: "https://updates.example.com/monok8s-1.2.3.img.zst"
  checksum: "sha256:..."
  nodeSelector: {}
status:
  phase: Pending
  resolvedVersion: "v1.35.3"
  observedGeneration: 1
  summary:
    targetedNodes: 0
    pendingNodes: 0
    runningNodes: 0
    succeededNodes: 0
    failedNodes: 0

kubectl get osugrades

NAME            DESIRED    RESOLVED   PHASE       TARGETS   OK   FAIL   AGE
my-upgrade-3    stable     v1.35.4    RollingOut  3         1    0      1m
my-upgrade-2    v1.35.3    v1.35.3    Accepted    2         0    0      1m
my-downgrade-1  v1.33.2    v1.33.2    Rejected    2         0    2      1m

kubectl get osupgradeprogress

NAME                NODE        SOURCE        CURRENT  TARGET   STATUS
osupgrade-abc123f   node-1      my-upgrade-2  v1.34.1  v1.35.3  downloading
osupgrade-cde456g   node-2      my-upgrade-2  v1.35.3  v1.35.3  completed

apiVersion: monok8s.io/v1alpha1
kind: OSUpgradeProgress
metadata:
  name: "osupgrade-abc123f"
spec:
  sourceRef:
    name: my-upgrade-2
  nodeName: node-1
status:
  currentVersion: "v1.34.1"
  targetVersion: "v1.35.3"
  phase: Downloading
  startedAt: null
  completedAt: null
  lastUpdatedAt: null
  retryCount: 0
  inactivePartition: "B"
  failureReason: ""
  message: ""

NOTES

The device's dts files are located at here

https://github.com/we-are-mono/OpenWRT-ASK/tree/mono-25.12.0-rc3/target/linux/layerscape/files/arch/arm64/boot/dts/freescale

We need both mono-gateway-dk-sdk.dts and mono-gateway-dk.dts since the sdk one includes the non-sdk one.
The actual dts being used is the mono-gateway-dk-sdk.dts

penguin/monok8s

README.md