ro_raspi3_fs/README.md

# Docker RO image
# Pre-boot
## Create base image, simpel
    wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-01-12/2021-01-11-raspios-buster-armhf-lite.zip

    [temelin ro_raspi3_fs]# unzip 2021-01-11-raspios-buster-armhf-lite.zip
    Archive:  2021-01-11-raspios-buster-armhf-lite.zip
    inflating: 2021-01-11-raspios-buster-armhf-lite.img

    [temelin ro_raspi3_fs]# ls -l
    total 2267512
    -rw-r--r-- 1 root   root   1862270976 11. led 14.08 2021-01-11-raspios-buster-armhf-lite.img
    -rw-r--r-- 1 root   root    459635953 11. led 14.11 2021-01-11-raspios-buster-armhf-lite.zip

## Mount image partition locally

    [temelin ro_raspi3_fs]# fdisk -l 2021-01-11-raspios-buster-armhf-lite.img
    Disk 2021-01-11-raspios-buster-armhf-lite.img: 1,73 GiB, 1862270976 bytes, 3637248 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0xe8af6eb2

    Device                                    Boot  Start     End Sectors  Size Id Type
    2021-01-11-raspios-buster-armhf-lite.img1        8192  532479  524288  256M  c W95 FAT32 (LBA)
    2021-01-11-raspios-buster-armhf-lite.img2      532480 3637247 3104768  1,5G 83 Linux

Disk offset to be mounted at loopback: part start (**8192**) * sector size (**512**) = **4194304**

    [temelin ro_raspi3_fs]# losetup -o 4194304 /dev/loop0 2021-01-11-raspios-buster-armhf-lite.img
    [temelin ro_raspi3_fs]# losetup -o 272629760 /dev/loop1 2021-01-11-raspios-buster-armhf-lite.img

    [temelin ro_raspi3_fs]# mkdir root boot
    [temelin ro_raspi3_fs]# mount /dev/loop0 boot
    [temelin ro_raspi3_fs]# mount /dev/loop1 root

Enable ssh server and permit login via ssh by default

    [temelin ro_raspi3_fs]# touch ./boot/ssh

Disable serial console, change root device to generic mmcblk0p2, init a resize when inserted

    [temelin ro_raspi3_fs]# echo "console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh" > ./boot/cmdline.txt
    [temelin ro_raspi3_fs]# echo "dtoverlay=disable-bt" >> ./boot/config.txt
    [temelin ro_raspi3_fs]# echo "enable_uart=1" >> ./boot/config.txt
    [temelin ro_raspi3_fs]# rm ./root/etc/systemd/system/multi-user.target.wants/hciuart.service
    [temelin ro_raspi3_fs]# mv ./root/etc/fstab ./root/etc/fstab.bkp
    [temelin ro_raspi3_fs]# cat ./root/etc/fstab | sed 's/PARTUUID=[0-9a-z\-]\+\([0-9]\)\ /dev\/mmcblk0p\1/g' > ./root/etc/fstab


Sync / umount

    [temelin ro_raspi3_fs]# umount boot root

# Additional steps and remarks

1. UART > HW (already done at copy phase)
2. GSM - generic - ip config  / reboot / setup (netctl, ipspace, network-manager)
    - recommend netctl / ipspace, net manager has dependencies on Gnome.
3. wahtchdog mod (configuration not known, just installed)
4. BalenaOS, Hypriot, RTOS, alternatives
    - BalenaOS:
        Not looking all that bad, customisation not necessary. Need to test
        kernel availability and modules for HW devices (CAN / Modbus)
        Has option for a RO file system. Would recommend a Raspberry PI4

    - Hypriot:
        Dedicated docker image. Does not seem to be actively developed anymore.
        Non ROFS support out of the box

    - RTOS non-linux standards
        RTOS would require a complete rewrite. No Java.

    -  Arch Linux / ubuntu ...

        Arch: highly customisable, but might be challenging to maintain. Good
        community support, great tutorials and documentation. Rolling release,
        no full system upgrades. 64-bit available

        Ubuntu: based on debian, bloated more than others, has dedicated streams
        to implementthird party / paid / non-opensource code on the fly

        FreeBSD:
        Similar to arch, pain to get non-FBSD/GPL code signed, verified and
        implemented. Very lightweight, stable. 64-bit available


# Prepare SD card
## dd to SDcard

    [temelin ro_raspi3_fs]# dd if=./2021-01-11-raspios-buster-armhf-lite.img of=/dev/sdb status=progress
    1854296576 bytes (1,9 GB, 1,7 GiB) copied, 377 s, 4,9 MB/s
    3637248+0 records in
    3637248+0 records out
    1862270976 bytes (1,9 GB, 1,7 GiB) copied, 380,898 s, 4,9 MB/s


# Post-boot

    [raspi /]# sudo apt-get update
    [raspi /]# sudo apt-get upgrade
    [raspi /]# sudo reboot

## Set up country? Wifi needed?

    [raspi /]# sudo apt install network-manager #(to get nmcli, 23MB)
    [raspi /]# sudo apt remove python3.7 --purge #python3.5 compatible, subject to testing
    [raspi /]# sudo apt install watchdog python3.5 openjdk-8-jre #(800 MB additional)

## Disable swap

    [raspi /]# sudo sync
    [raspi /]# sudo swapoff -a
    [raspi /]# sudo apt-get purge -y dphys-swapfile
    [raspi /]# sudo rm /var/swap
    [raspi /]# sudo sync

## making a RO overlay root fs => enable it in raspi-config
This only adds entries to /boot/cmdline.txt and a different overlay kernel to config.txt
By removing it, we can get to RW system manually.

    [raspi /]# sudo raspi-config nonint do_overlayfs 0

## Mounting root as RW?

    [raspi /]# sudo losetup -f /dev/mmcblk0p1
    [raspi /]# sudo losetup -f /dev/mmcblk0p2
    [raspi /]# sudo mount -o rw /dev/loop1 /mnt
    [raspi /]# sudo mount -o rw /dev/loop0 /mnt/boot

## If you need to mount / update

    [raspi /]# sudo chroot /mnt
    [raspi /]# sudo mount -t proc /proc proc/
    [raspi /]# sudo mount --rbind /sys sys/
    [raspi /]# sudo mount --rbind /dev dev/

## Docker's overlay2 needs to be a tmpfs - Skipped for now
Docker wants to do some mambo-jumbo inside and does not want overlay, need tmpfs

    sudo mount -t tmpfs tmpfs /var/lib/docker

need to copy contents of overlay2?

    sudo mount -t tmpfs tmpfs /var/lib/docker
    sudo cp -r /mnt/var/lib/docker /var/lib/
    cp -r /var/lib/docker/* /mnt/var/lib/docker


# Results:
RO FS:

    pi@raspberrypi:~ $ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/root        15G  1.5G   13G  11% /
    devtmpfs        430M     0  430M   0% /dev
    tmpfs           463M     0  463M   0% /dev/shm
    tmpfs           463M  6.2M  456M   2% /run
    tmpfs           5.0M  4.0K  5.0M   1% /run/lock
    tmpfs           463M     0  463M   0% /sys/fs/cgroup
    /dev/mmcblk0p1  253M   48M  205M  19% /boot
    tmpfs            93M     0   93M   0% /run/user/1000

HCI / UART:

    root@raspberrypi:~# ls -l /dev/serial0
    lrwxrwxrwx 1 root root 7 Mar 18 16:05 /dev/serial0 -> ttyAMA0
    root@raspberrypi:~# ls -l /dev/serial1
    lrwxrwxrwx 1 root root 5 Mar 18 16:05 /dev/serial1 -> ttyS0

# re-create SDcard image
## make image, shrink it (
    [temelin ro_raspi3_fs]# e2fsck -f -y -v -C 0 /dev/sdb2
    [temelin ro_raspi3_fs]# resize2fs -p /dev/sdb2 2764800K
    [temelin ro_raspi3_fs]# parted -s /dev/sdb 'print'
        Model: Multi Flash Reader (scsi)
        Disk /dev/sdb: 32,0GB
        Sector size (logical/physical): 512B/512B
        Partition Table: msdos
        Disk Flags:

        Number  Start   End     Size    Type     File system  Flags
        1      4194kB  273MB   268MB   primary  fat32        lba
        2      273MB   3200MB  2927MB  primary  ext4
    [temelin ro_raspi3_fs]# parted -s /dev/sdb 'resizepart 2 3105M'

## Create an empty RAW device + partitions

    [temelin ro_raspi3_fs]# dd if=/dev/zero of=./ro_img.dd status=progress bs=1M count=2900
    [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mktable msdos'
    [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mkpart primary fat32 2048s 257MiB'
    [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'mkpart primary ext4 257MiB 100%'
    [temelin ro_raspi3_fs]# parted -s ./ro_img.dd 'print'
        Model:  (file)
        Disk /mnt/ssd2/ro_img.dd: 3041MB
        Sector size (logical/physical): 512B/512B
        Partition Table: msdos
        Disk Flags:

        Number  Start   End     Size    Type     File system  Flags
        1      1049kB  269MB   268MB   primary               lba
        2      269MB   3041MB  2771MB  primary

## Create filesystems. Need to mount with offset, again
2048 * 512 = 1048576, 526336 * 512 = 269484032

    [temelin ro_raspi3_fs]# fdisk -l ./ro_img.dd
        Disk ./ro_img.dd: 2,83 GiB, 3040870400 bytes, 5939200 sectors
        Units: sectors of 1 * 512 = 512 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disklabel type: dos
        Disk identifier: 0xded84696

        Device       Boot  Start     End Sectors  Size Id Type
        ./ro_img.dd1        2048  526335  524288  256M  c W95 FAT32 (LBA)
        ./ro_img.dd2      526336 5939199 5412864  2,6G 83 Linux


    [temelin ro_raspi3_fs]# losetup -o 1048576 /dev/loop0 ./ro_img.dd
    [temelin ro_raspi3_fs]# losetup -o 269484032 /dev/loop1 ./ro_img.dd

    [temelin ro_raspi3_fs]# mkfs.vfat -F 32 -n BOOT /dev/loop0
    [temelin ro_raspi3_fs]# mkfs.ext4 -L rootfs /dev/loop1

    [temelin ro_raspi3_fs]# mount /dev/loop0 ./boot
    [temelin ro_raspi3_fs]# mount /dev/loop1 ./root
    [temelin ro_raspi3_fs]# ls -ld boot_sd || mkdir boot_sd
    [temelin ro_raspi3_fs]# ls -ld root_sd || mkdir root_sd
    [temelin ro_raspi3_fs]# mount /dev/sdb1 ./boot_sd
    [temelin ro_raspi3_fs]# mount /dev/sdb2 ./root_sd
    [temelin ro_raspi3_fs]# rsync -auP ./boot_sd/ ./boot/
    [temelin ro_raspi3_fs]# rsync -auP ./root_sd/ ./root/

## Don't forget to re-do the resize at initial boot

    [temelin ro_raspi3_fs]# echo "console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh" > ./boot/cmdline.txt