# Docker RO image # Pre-boot ## Create base image, simpel wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-01-12/2021-01-11-raspios-buster-armhf-lite.zip [temelin ro_raspi3_fs]# unzip 2021-01-11-raspios-buster-armhf-lite.zip Archive: 2021-01-11-raspios-buster-armhf-lite.zip inflating: 2021-01-11-raspios-buster-armhf-lite.img [temelin ro_raspi3_fs]# ls -l total 2267512 -rw-r--r-- 1 root root 1862270976 11. led 14.08 2021-01-11-raspios-buster-armhf-lite.img -rw-r--r-- 1 root root 459635953 11. led 14.11 2021-01-11-raspios-buster-armhf-lite.zip ## Mount image partition locally [temelin ro_raspi3_fs]# fdisk -l 2021-01-11-raspios-buster-armhf-lite.img Disk 2021-01-11-raspios-buster-armhf-lite.img: 1,73 GiB, 1862270976 bytes, 3637248 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xe8af6eb2 Device Boot Start End Sectors Size Id Type 2021-01-11-raspios-buster-armhf-lite.img1 8192 532479 524288 256M c W95 FAT32 (LBA) 2021-01-11-raspios-buster-armhf-lite.img2 532480 3637247 3104768 1,5G 83 Linux Disk offset to be mounted at loopback: part start (**8192**) * sector size (**512**) = **4194304** [temelin ro_raspi3_fs]# losetup -o 4194304 /dev/loop0 2021-01-11-raspios-buster-armhf-lite.img [temelin ro_raspi3_fs]# losetup -o 272629760 /dev/loop1 2021-01-11-raspios-buster-armhf-lite.img [temelin ro_raspi3_fs]# mkdir root boot [temelin ro_raspi3_fs]# mount /dev/loop0 boot [temelin ro_raspi3_fs]# mount /dev/loop1 root Enable ssh server and permit login via ssh by default [temelin ro_raspi3_fs]# touch ./boot/ssh Disable serial console, change root device to generic mmcblk0p2, init a resize when inserted [temelin ro_raspi3_fs]# echo "console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh" > ./boot/cmdline.txt [temelin ro_raspi3_fs]# echo "dtoverlay=disable-bt" >> ./boot/config.txt [temelin ro_raspi3_fs]# echo "enable_uart=1" >> ./boot/config.txt [temelin ro_raspi3_fs]# rm ./root/etc/systemd/system/multi-user.target.wants/hciuart.service [temelin ro_raspi3_fs]# mv ./root/etc/fstab ./root/etc/fstab.bkp [temelin ro_raspi3_fs]# cat ./root/etc/fstab | sed 's/PARTUUID=[0-9a-z\-]\+\([0-9]\)\ /dev\/mmcblk0p\1/g' > ./root/etc/fstab Sync / umount [temelin ro_raspi3_fs]# umount boot root # Additional steps and remarks 1. UART > HW (already done at copy phase) 2. GSM - generic - ip config / reboot / setup (netctl, ipspace, network-manager) - recommend netctl / ipspace, net manager has dependencies on Gnome. 3. wahtchdog mod (configuration not known, just installed) 4. BalenaOS, Hypriot, RTOS, alternatives - BalenaOS: Not looking all that bad, customisation not necessary. Need to test kernel availability and modules for HW devices (CAN / Modbus) Has option for a RO file system. Would recommend a Raspberry PI4 - Hypriot: Dedicated docker image. Does not seem to be actively developed anymore. Non ROFS support out of the box - RTOS non-linux standards RTOS would require a complete rewrite. No Java. - Arch Linux / ubuntu ... Arch: highly customisable, but might be challenging to maintain. Good community support, great tutorials and documentation. Rolling release, no full system upgrades. 64-bit available Ubuntu: based on debian, bloated more than others, has dedicated streams to implementthird party / paid / non-opensource code on the fly FreeBSD: Similar to arch, pain to get non-FBSD/GPL code signed, verified and implemented. Very lightweight, stable. 64-bit available # Prepare SD card ## dd to SDcard [temelin ro_raspi3_fs]# dd if=./2021-01-11-raspios-buster-armhf-lite.img of=/dev/sdb status=progress 1854296576 bytes (1,9 GB, 1,7 GiB) copied, 377 s, 4,9 MB/s 3637248+0 records in 3637248+0 records out 1862270976 bytes (1,9 GB, 1,7 GiB) copied, 380,898 s, 4,9 MB/s # Post-boot [raspi /]# sudo apt-get update [raspi /]# sudo apt-get upgrade [raspi /]# sudo reboot ## Set up country? Wifi needed? [raspi /]# sudo apt install network-manager #(to get nmcli, 23MB) [raspi /]# sudo apt remove python3.7 --purge #python3.5 compatible, subject to testing [raspi /]# sudo apt install watchdog python3.5 openjdk-8-jre #(800 MB additional) ## Disable swap [raspi /]# sudo sync [raspi /]# sudo swapoff -a [raspi /]# sudo apt-get purge -y dphys-swapfile [raspi /]# sudo rm /var/swap [raspi /]# sudo sync ## making a RO overlay root fs => enable it in raspi-config This only adds entries to /boot/cmdline.txt and a different overlay kernel to config.txt By removing it, we can get to RW system manually. [raspi /]# sudo raspi-config nonint do_overlayfs 0 ## Mounting root as RW? [raspi /]# sudo losetup -f /dev/mmcblk0p1 [raspi /]# sudo losetup -f /dev/mmcblk0p2 [raspi /]# sudo mount -o rw /dev/loop1 /mnt [raspi /]# sudo mount -o rw /dev/loop0 /mnt/boot ## If you need to mount / update [raspi /]# sudo chroot /mnt [raspi /]# sudo mount -t proc /proc proc/ [raspi /]# sudo mount --rbind /sys sys/ [raspi /]# sudo mount --rbind /dev dev/ ## Docker's overlay2 needs to be a tmpfs - Skipped for now Docker wants to do some mambo-jumbo inside and does not want overlay, need tmpfs sudo mount -t tmpfs tmpfs /var/lib/docker need to copy contents of overlay2? sudo mount -t tmpfs tmpfs /var/lib/docker sudo cp -r /mnt/var/lib/docker /var/lib/ cp -r /var/lib/docker/* /mnt/var/lib/docker # Results: RO FS: pi@raspberrypi:~ $ df -h Filesystem Size Used Avail Use% Mounted on /dev/root 15G 1.5G 13G 11% / devtmpfs 430M 0 430M 0% /dev tmpfs 463M 0 463M 0% /dev/shm tmpfs 463M 6.2M 456M 2% /run tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 463M 0 463M 0% /sys/fs/cgroup /dev/mmcblk0p1 253M 48M 205M 19% /boot tmpfs 93M 0 93M 0% /run/user/1000 HCI / UART: root@raspberrypi:~# ls -l /dev/serial0 lrwxrwxrwx 1 root root 7 Mar 18 16:05 /dev/serial0 -> ttyAMA0 root@raspberrypi:~# ls -l /dev/serial1 lrwxrwxrwx 1 root root 5 Mar 18 16:05 /dev/serial1 -> ttyS0 # re-create SDcard image ## make image, shrink it ( [temelin ro_raspi3_fs]# e2fsck -f -y -v -C 0 /dev/sdb2 [temelin ro_raspi3_fs]# resize2fs -p /dev/sdb2 2764800K [temelin ro_raspi3_fs]# parted -s /dev/sdb 'print' Model: Multi Flash Reader (scsi) Disk /dev/sdb: 32,0GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 4194kB 273MB 268MB primary fat32 lba 2 273MB 3200MB 2927MB primary ext4 [temelin ro_raspi3_fs]# parted -s /dev/sdb 'resizepart 2 3105M' ## Create an empty RAW device + partitions [temelin ro_raspi3_fs]# dd if=/dev/zero of=./ro_img.dd status=progress bs=1M count=2900 [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mktable msdos' [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mkpart primary fat32 2048s 257MiB' [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mkpart primary ext4 257MiB 100%' [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'print' Model: (file) Disk /mnt/ssd2/ro_img.dd: 3041MB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 1049kB 269MB 268MB primary lba 2 269MB 3041MB 2771MB primary ## Create filesystems. Need to mount with offset, again 2048 * 512 = 1048576, 526336 * 512 = 269484032 [temelin ro_raspi3_fs]# fdisk -l ./ro_img.dd Disk ./ro_img.dd: 2,83 GiB, 3040870400 bytes, 5939200 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xded84696 Device Boot Start End Sectors Size Id Type ./ro_img.dd1 2048 526335 524288 256M c W95 FAT32 (LBA) ./ro_img.dd2 526336 5939199 5412864 2,6G 83 Linux [temelin ro_raspi3_fs]# losetup -o 1048576 /dev/loop0 ./ro_img.dd [temelin ro_raspi3_fs]# losetup -o 269484032 /dev/loop1 ./ro_img.dd [temelin ro_raspi3_fs]# mkfs.vfat -F 32 -n BOOT /dev/loop0 [temelin ro_raspi3_fs]# mkfs.ext4 -L rootfs /dev/loop1 [temelin ro_raspi3_fs]# mount /dev/loop0 ./boot [temelin ro_raspi3_fs]# mount /dev/loop1 ./root [temelin ro_raspi3_fs]# ls -ld boot_sd || mkdir boot_sd [temelin ro_raspi3_fs]# ls -ld root_sd || mkdir root_sd [temelin ro_raspi3_fs]# mount /dev/sdb1 ./boot_sd [temelin ro_raspi3_fs]# mount /dev/sdb2 ./root_sd [temelin ro_raspi3_fs]# rsync -auP ./boot_sd/ ./boot/ [temelin ro_raspi3_fs]# rsync -auP ./root_sd/ ./root/ ## Don't forget to re-do the resize at initial boot [temelin ro_raspi3_fs]# echo "console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh" > ./boot/cmdline.txt