- [Sitara AM64x U-Boot](#sitara-am64x-u-boot)
  - [Get the latest version](#get-the-latest-version)
  - [Build from source for testing](#build-from-source-for-testing)
    - [Get dependencies](#get-dependencies)
    - [Build](#build)
    - [Deploy](#deploy)
  - [Build and package for deployment](#build-and-package-for-deployment)
  - [Upgrade to new upstream release](#upgrade-to-new-upstream-release)
  - [Problems and adaptions](#problems-and-adaptions)
  - [Resources](#resources)

# Sitara AM64x U-Boot

The following documentation aims to document everything you need to know to build U-Boot for the Sitara AM64x platform.

## Get the latest version

Each image build automatically installs the latest version of the bootloader.

But if you just want to get the latest version of the compiled artifacts, the easiest way is to get them from our APT server.

[apt.balluff.systems - u-boot-balluff-sitara](http://apt.balluff.systems/balluff-linux/pool/main/u/u-boot-balluff-sitara/)

Simply download the `.deb` file to your system and unpack the contents to obtain the binary artifacts.

## Build from source for testing

There are scenarios where you want to test a new feature or you want to adapt the bootloader. To do this, you can build the bootloader from source.

Understanding this process also helps to understand what happens during the package build.

The following steps describe this process.

### Get dependencies

There are some dependencies needed to build U-Boot for the Sitara AM64x platform.   
The following repositories are required:

```sh
# TI Firmware
git clone https://git.ti.com/git/processor-firmware/ti-linux-firmware.git
cd ti-linux-firmware
git checkout 09.02.00.010 # or ti-linux-firmware (which should be the latest released version)
cd ..

# OP-TEE
git clone https://github.com/OP-TEE/optee_os.git
cd optee_os
git checkout 4.2.0 # check if there is a newer version
CROSS_COMPILE=arm-linux-gnueabihf- CROSS_COMPILE64=aarch64-linux-gnu- PLATFORM=k3-am64x CFG_ARM64_core=y make
cd ..

# TF-A
git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
cd trusted-firmware-a
git checkout lts-v2.10.3 # check if there is a newer lts version
CROSS_COMPILE=aarch64-linux-gnu- ARCH=aarch64 SPD=opteed TARGET_BOARD=lite make PLAT=k3
cd ..
```

### Build

**Tipp:** Set the `REPOS_PATH` environment variable to the path where all repositories are cloned to save time when referencing on the different repositories later.

```sh
export REPOS_PATH=<full-path-to-cloned-repos>
```

```sh
# Build r5 image
cd cel-bootloader-sitara/
git checkout development # currently checkout development until a release is tagged
ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make am64x_evm_r5_defconfig O=../sitara-build/r5 # O= is used to specify the path for out-of-tree build
cd ../sitara-build/r5
ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make menuconfig # in case of config changes
ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make savedefconfig # in case of exporting config changes
ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- BINMAN_INDIRS=${REPOS_PATH}/ti-linux-firmware/ make -j$(nproc)
cd ../..
```

```sh
# Build a53 image
cd cel-bootloader-sitara/
git checkout development # currently checkout development until a release is tagged
ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- make am64x_evm_a53_defconfig O=../sitara-build/a53 # O= is used to specify the path for out-of-tree build
cd ../sitara-build/a53
ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- make menuconfig # in case of config changes
ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- make savedefconfig # in case of exporting config changes
ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- BINMAN_INDIRS=${REPOS_PATH}/ti-linux-firmware/ BL31=${REPOS_PATH}/trusted-firmware-a/build/k3/lite/release/bl31.bin TEE=${REPOS_PATH}/optee_os/out/arm-plat-k3/core/tee-raw.bin make -j$(nproc)
cd ../..
```

You should now see at least the following artifacts in the sitara-build directories.

```sh
tree -L 1 sitara-build/r5
sitara-build/r5
├── arch
...
├── tiboot3-am64x-gp-evm.bin
├── tiboot3-am64x-gp-evm.bin.map
├── tiboot3-am64x_sr2-hs-evm.bin
├── tiboot3-am64x_sr2-hs-evm.bin.map
├── tiboot3-am64x_sr2-hs-fs-evm.bin
├── tiboot3-am64x_sr2-hs-fs-evm.bin.map
├── tiboot3.bin -> ./tiboot3-am64x_sr2-hs-fs-evm.bin
...

19 directories, 60 files


tree -L 1 sitara-build/a53/
sitara-build/a53/
├── arch
...
├── tispl.bin
├── tispl.bin_unsigned
...
├── u-boot.img
├── u-boot.img_unsigned
...

19 directories, 61 files
```

In the case of the latest EVM board the artifact `tiboot3-am64x_sr2-hs-fs-evm.bin` is used as MLO, `tispl.bin` as SPL  and `u-boot.img` as bootloader image.

The other artifacts with `*_unsigned` and artifacts like `tiboot3-am64x-gp-evm.bin` are only work on old versions of the EVM board where the SoC does not require the validation of the images.

### Deploy

To use the artifacts, you can copy them to the boot partition of the prepared SD card.

```sh
cp sitara-build/r5/tiboot3-am64x_sr2-hs-fs-evm.bin /media/<username>/boot/tiboot3.bin # ROM code will only load tiboot3.bin so please make sure you rename it properly
cp sitara-build/a53/tispl.bin /media/<username>/boot/
cp sitara-build/a53/u-boot.img /media/<username>/boot/

sync /media/<username>/boot/

sudo umount /media/<username>/boot/
sudo umount /media/<username>/rfsa/
sudo umount /media/<username>/data/
```

## Build and package for deployment

If you want to adapt to the bootloader and publish the changes, you can follow the steps below.

1. Clone the `cel-bootloader-sitara` repository, if not already done, and checkout a working branch.

2. Make the necessary adaptions of the upstream sources or inside the debian folder.

3. If necessary, do a manual testbuild to check that everything is working as expected.

   This is described in an earlier section.

4. Commit your changes as normal, but note the following points which are important for the patch handling:

   - include only changes of the upstream sources or changes inside the debian folder per commit
   - include only one logical change per commit
   - make the commit description as meaningful as possible
     (ideally add a detailed description and not just a short message)

5. Create a patch file for each commit that changes the upstream sources.

    Upstream is basically anything outside the debian folder.
    If you are only editing things inside the debian folder you can skip this and the following steps regarding the patch files.

    Make sure you are in the root directory of this repository before creating the patch files.

    Below you can see an example for creating a patch file for the commit `5f6d...0c8e`.

    Here it's also important to not forget to use the `-1` flag to create a patch file only for the single commit. Otherwise every diff between the commit and `HEAD` will be used.

    ```sh
    git format-patch --start-number 9 -1 5f6d41aa254f61b71cc258b1b3fc474ee2440c8e
    ```

    **Note:** For the value of `--start-number`, just increment the number of the last patch file in `debian/patches/`, this helps with keep the patch files in order and we don't lose track.

    ```sh
    ls -l debian/patches/
     total 40
     -rw-rw-r-- 1 1000 1000 1189 Aug  7 10:34 0001-Set-bootcmd_ti_mmc-as-default-bootcmd-instead-of-boo.patch
     #...
     -rw-rw-r-- 1 1000 1000  836 Aug  7 10:34 0008-Edit-.gitignore-to-include-quilt-files.patch
     -rw-rw-r-- 1 1000 1000  456 Aug  7 10:34 series
    ```

6. Integrate the created patch properly into the Debian scope and remove the patch file you just created.

    ```sh
    quilt import 0009-description-of-the-patch.patch
    rm 0009-description-of-the-patch.patch
    ```

7. Fix behaviour of the used patch system `quilt`.

    Now the problem is that we have integrated an already applied patch into `quilt`, so we have to manually unapply the patch once.   
    This is simply because `quilt` thinks the patch hasn't been applied.   
    Of course it would be also possible to make manual adjustments within the `.pc/` folder, but this is not recommended.

    ```sh
    # Revert the patch with patch
    patch -p1 --reverse < debian/patches/0009-description-of-the-patch.patch
    # Apply it again with quilt
    quilt push -a
    # This will update the .pc/ folder properly
    ```

    You can use the following two commands to check that everything has gone well:

    ```sh
    git status # should show no changes on upstream files because each patch is applied
    quilt applied # should show all patches as applied
    ```

8. Update the Debian changelog.

    ```sh
    dch -i -D stable
    ```

    It is important to distinguish between an upgrade to a higher upstream version and changes inside the upstream sources.   
    Depending on the change, you should update the upstream version number (and later create a new upstream tarball) or a new patch level.   
    The change description should ideally include the reference to the newly applied and removed patches, and perhaps a short description of the changes.

9. Check-in and commit the files inside `.pc/` and `debian/`

    It's important that each patch is applied, which can be checked again with "git status" and "quilt applied".

    ```sh
    git add debian/ .pc/
    git commit # Message can be something like "Bump Debian package version to 2024.04-3"
    ```

10. Tag the last commit with the Debian package version and push it to origin.

    ```sh
    git tag debian/2024.04-3
    git push origin debian/2024.04-3
    ```

11. Check that the upstream tarball is available in the parent directory.

    ```sh
    ls ../u-boot-balluff-sitara_2024.04.orig.tar.xz
    ```

12. Get the upstream tarball if it is missing.

    There are two approaches to get the upstream tarball:

    The first and easiest approach is to download the tarball from our APT server.

    ```sh
    # Switch to the parent directory
    cd ..
    # Download the upstream tarball
    wget http://apt.balluff.systems/balluff-linux/pool/main/u/u-boot-balluff-sitara/u-boot-balluff-sitara_2024.04.orig.tar.xz
    ```

    **Note:** The version inside the filename changes after an upgrade to a higher upstream version.

    Alternatively open the following link in your browser, download the upstream tarball and place it in the parent directory of the source tree.

    [http://apt.balluff.systems/balluff-linux/pool/main/u/u-boot-balluff-sitara/](http://apt.balluff.systems/balluff-linux/pool/main/u/u-boot-balluff-sitara/)

    This first approach only works if you are not upgrading to a higher upstream version with this package update, then use the second approach.

    The second approach is to manually build the upstream tarball from source.

    Unapply each patch, as we only want the upstream sources in the tarball.

    ```sh
    quilt pop -a
    ```

    Prepare to build the upstream tarball.

    ```sh
    # Switch to the parent directory
    cd ..
    # Copy the U-Boot sources to a new temporary directory with a proper name
    # This is recommended so that we have the name and version of the source package also in the extracted archive
    cp -r cel-bootloader-sitara/ u-boot-balluff-sitara_2024.04/
    ```

    Build the upstream tarball and remove the temporary directory.

    ```sh
    tar --exclude="debian" --exclude=".pc" --exclude=".git" -cJf u-boot-balluff-sitara_2024.04.orig.tar.xz u-boot-balluff-sitara_2024.04/
    rm -rf u-boot-balluff-sitara_2024.04/
    ```

    Go back to the source tree and apply all patches again.

    ```sh
    cd cel-bootloader-sitara/
    quilt push -a
    ```

13. Build the new package version.

    ```sh
    CC=aarch64-linux-gnu- dpkg-buildpackage --no-sign --host-arch arm64 --post-clean
    ```

14. Get the new artifacts for integration into our APT server.

    ```sh
    ls -l ../
    drwxrwxr-x 28 1000 1000      4096 Aug  7 13:46 cel-bootloader-sitara
    -rw-rw-r--  1 1000 1000      6706 Aug  7 13:50 u-boot-balluff-sitara_2024.04-3_arm64.buildinfo
    -rw-rw-r--  1 1000 1000      1797 Aug  7 13:50 u-boot-balluff-sitara_2024.04-3_arm64.changes
    -rw-rw-r--  1 1000 1000     14264 Aug  7 13:47 u-boot-balluff-sitara_2024.04-3.debian.tar.xz
    -rw-rw-r--  1 1000 1000      1001 Aug  7 13:47 u-boot-balluff-sitara_2024.04-3.dsc
    -rw-rw-r--  1 1000 1000  17269496 Aug  7 13:45 u-boot-balluff-sitara_2024.04.orig.tar.xz
    -rw-r--r--  1 1000 1000   1002096 Aug  7 13:50 u-boot-image-balluff-sitara-am64x_2024.04-3_arm64.deb
    ```

## Upgrade to new upstream release

If there is a new upstream release of U-Boot and we want to upgrade to that version we can use the following steps.

```sh
# Clone the cel-bootloader-sitara repository if not already done
git clone git@ssh.dev.azure.com:v3/BalluffAzDevOpsProd/Common%20Embedded%20Linux/cel-bootloader-sitara

# Catch the latest changes our repository
git pull

# Now check if you have already added the upstream U-Boot tree
git remote -v
origin	git@ssh.dev.azure.com:v3/BalluffAzDevOpsProd/Common%20Embedded%20Linux/cel-bootloader-sitara (fetch)
origin	git@ssh.dev.azure.com:v3/BalluffAzDevOpsProd/Common%20Embedded%20Linux/cel-bootloader-sitara (push)
upstream	https://source.denx.de/u-boot/u-boot.git (fetch)
upstream	https://source.denx.de/u-boot/u-boot.git (push)

# If the two line starting with "upstream" are missing you can add the upstream repository with the following command
git remote add --mirror=fetch upstream https://source.denx.de/u-boot/u-boot.git

# Fetch the changes from the upstream U-Boot tree
git fetch upstream --tags

# Push the latest tags to our repository
git push origin --all

# Now you can merge the state you want to have into you working branch
# For example:
git checkout development
git checkout -b feature/xyz-update_to_v2024.07
quilt pop -a # unapply all patches before merging the new upstream version
git merge v2024.07

# Now you should try to apply the patches again
quilt push -a

# When this doesn't work you need to manually resolve the merge conflicts,
# which can also make it necessary to remove or replace a patch.
```

## Problems and adaptions

- Warning in r5 SPL `Loading Environment from MMC... *** Warning - No MMC card found, using default environment`

  Fixed by config change (environment is not stored at all):

  ```diff
  -CONFIG_ENV_IS_IN_MMC=y
   CONFIG_SYS_RELOC_GD_ENV_ADDR=y
  -CONFIG_SYS_MMC_ENV_PART=1
  +# CONFIG_SPL_ENV_IS_NOWHERE is not set
  ```

- Warning when running bootcmd `WARN: fdtfile already set. Stop using findfdt in script`

  Handling of fdt has changed, now a function `ti_set_fdt_env` (in `board/ti/common/fdt_ops.c`) search for a compatible device tree based on the `board_name` variable. Therefor the `findfdt` command is not needed anymore and was replaced by that warning. Upstream `findfdt` is still invoked inside `bootcmd_ti_mmc` but can be removed. You see the adjustments in the diff below.

  ```diff
  diff --git a/include/env/ti/ti_common.env b/include/env/ti/ti_common.env
  index 02b410c3ad..17eff76346 100644
  --- a/include/env/ti/ti_common.env
  +++ b/include/env/ti/ti_common.env
  @@ -23,8 +23,7 @@ get_fit_overlaystring=
   get_fit_config=setexpr name_fit_config gsub / _ conf-${fdtfile}
   run_fit=run get_fit_config; bootm ${addr_fit}#${name_fit_config}${overlaystring}
   do_main_cpsw0_qsgmii_phyinit=0
  -bootcmd_ti_mmc=
  -       run findfdt; run init_${boot};
  +bootcmd_ti_mmc=run init_${boot};
   #if CONFIG_CMD_REMOTEPROC
          if test ${do_main_cpsw0_qsgmii_phyinit} -eq 1;
                  then run main_cpsw0_qsgmii_phyinit;
  ```

- Use `bootcmd_ti_mmc` directly in `bootcmd` instead of `bootflow`:

  Bootflow is the standardized way to boot in U-Boot and should make it possiple to define in a standard way inside the distro how to boot the system.

  You can create for example the file `extlinux/extlinux.conf` inside boot.

  ```conf
  label CEL (3.0.0_dev)
      kernel /vmlinuz-6.1.26
      append rw
      fdtdir /dtb/ti/k3-am642-evm.dtb
  ```

  When triggering bootflow -lb the bootloader will scan for bootflows and you can choose the one you want to boot. With that you get a similar workflow like with grub where you can select different kernels or even different distros in a multi-boot setup.

  But in a quick test that doesn't work as expected:

  ```sh
  => bootflow scan -lb
  Scanning bootdev 'mmc@fa00000.bootdev':
    1  extlinux     ready   mmc          1  mmc@fa00000.bootdev.part_ /extlinux/extlinux.conf
  ** Booting bootflow 'mmc@fa00000.bootdev.part_1' with extlinux
  Ignoring malformed menu command:  autoboot
  Ignoring malformed menu command:  hidden
  CEL Boot Options.
  1:      CEL 3.0.0
  Enter choice: 1
  1:      CEL 3.0.0
  Retrieving file: /vmlinuz-6.1.26
  append: console=ttyS2,115200n8 earlycon=ns16550a,mmio32,0x02800000 root=/dev/mmcblk1p2 rw rootfstype=ext4 rootwait
  Retrieving file: /dtb/ti/k3-am642-evm.dtb
  kernel_comp_addr_r or kernel_comp_size is not provided!
  Boot failed (err=-14)
  ```

  Ignoring the warning regarding the menu command it fails when trying to boot the kernel because `kernel_comp_addr_r` or `kernel_comp_size` is not provided (which is not provided).

  Maybe in the next U-Boot release this will be adapted, but for now it's enough to use the `bootcmd_ti_mmc` directly in `bootcmd`.

  ```diff
  -CONFIG_BOOTCOMMAND="run envboot; bootflow scan -lb"
  +CONFIG_BOOTCOMMAND="run envboot; run bootcmd_ti_mmc"
  ```

  The bootflow feature makes with our use case probably no sense anyway.

  In case this feature because relevant in the future we should also set `bootmeths` to `extlinux` to block the search after any efi partition.

- Appended localversion

  By the simple config change for r5 and a53 build we can append cel behind the U-Boot version string.

  ```diff
  +CONFIG_LOCALVERSION="-cel"
  ```

- Warning in a53 `## Error: "boot_rprocs" not defined`

  That happened because of the config option `CONFIG_CMD_REMOTEPROC`, this should include the environment file which defines `boot_rprocs` and some other variables.

  Below you can see the adjustements.

  ```diff
  diff --git a/board/ti/am64x/am64x.env b/board/ti/am64x/am64x.env
  index 9a8812d4ee..438da8d616 100644
  --- a/board/ti/am64x/am64x.env
  +++ b/board/ti/am64x/am64x.env
  @@ -1,6 +1,10 @@
   #include <env/ti/ti_common.env>
   #include <env/ti/mmc.env>
   #include <env/ti/k3_dfu.env>
  +#include <env/ti/nand.env>
  +#if CONFIG_CMD_REMOTEPROC
  +#include <env/ti/k3_rproc.env>
  +#endif
  
   name_kern=Image
   console=ttyS2,115200n8
  @@ -14,6 +18,8 @@ bootpart=1:2
   bootdir=/boot
   rd_spec=-
  
  +rproc_fw_binaries= 0 /lib/firmware/am64-main-r5f0_0-fw 1 /lib/firmware/am64-main-r5f0_1-fw 2 /lib/firmware/am64-main-r5f1_0-fw 3 /lib/firmware/am64-main-r5f1_1-fw
  +
   args_usb=run finduuid;setenv bootargs console=${console}
          ${optargs}
          root=PARTUUID=${uuid} rw

  ```

  For a quick test the firmware binaries from ti where copied to `/lib/firmware` and boot_rprocs was executed.

  ```sh
  => setenv dorprocboot 1 # when this variable is set boot_rprocs is executed before starting the kernel
  => printenv rproc_fw_binaries
  rproc_fw_binaries= 0 /lib/firmware/am64-main-r5f0_0-fw 1 /lib/firmware/am64-main-r5f0_1-fw 2 /lib/firmware/am64-main-r5f1_0-fw 3 /lib/firmware/am64-main-r5f1_1-fw
  => run boot_rprocs
          Remote Processors are already initialized
  92288 bytes read in 40 ms (2.2 MiB/s)
  Load Remote Processor 0 with data@addr=0x82000000 92288 bytes: Failed!
  92416 bytes read in 40 ms (2.2 MiB/s)
  Load Remote Processor 1 with data@addr=0x82000000 92416 bytes: Failed!
  92416 bytes read in 40 ms (2.2 MiB/s)
  Load Remote Processor 2 with data@addr=0x82000000 92416 bytes: Failed!
  92416 bytes read in 40 ms (2.2 MiB/s)
  Load Remote Processor 3 with data@addr=0x82000000 92416 bytes: Failed!
  ```

  Unsure if it is a problem with U-Boot or the firmware binaries.
  For now we can ignore it because feature is not required yet in U-Boot.

- Fix eMMC Boot

  ```sh
  U-Boot SPL 2024.04-cel-00009-g52990ec1c4 (Jun 18 2024 - 17:16:10 +0200)
  Resetting on cold boot to workaround ErrataID:i2331
  Please resend tiboot3.bin in case of UART/DFU boot
  resetting ...

  U-Boot SPL 2024.04-cel-00009-g52990ec1c4 (Jun 18 2024 - 17:16:10 +0200)
  SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
  SPL initial stack usage: 13384 bytes
  Trying to boot from MMC1
  mmc_load_image_raw_sector: mmc block read error
  Partition 1 invalid on device 0
  spl_register_fat_device: fat register err - -1
  spl_load_image_fat: error reading image tispl.bin, err - -1
  SPL: failed to boot from all boot devices
  ### ERROR ### Please RESET the board ###
  ```

  `mmc_load_image_raw_sector: mmc block read error`

  `CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR` in r5 config
  --> set to 0x400 but 0x800 would be correct

  `CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR` in a53 config
  --> set to 0x1400 but 0x1800 would be correct

  The helpful information can be found here [software-dl.ti.com - AM64x Booting from eMMC boot partition for K3 class of SoCs](https://software-dl.ti.com/processor-sdk-linux-rt/esd/AM64X/latest/exports/docs/linux/Foundational_Components/U-Boot/UG-Memory.html#booting-tiboot3-bin-tispl-bin-and-u-boot-img-from-emmc-boot-partition-for-k3-class-of-socs)

  ```sh
  => mmc dev 0 1
  => fatload mmc 1 ${loadaddr} tiboot3.bin
  => mmc write ${loadaddr} 0x0 0x800
  => fatload mmc 1 ${loadaddr} tispl.bin
  => mmc write ${loadaddr} 0x800 0x1000 # 0x800 is the correct offset
  => fatload mmc 1 ${loadaddr} u-boot.img
  => mmc write ${loadaddr} 0x1800 0x2000 # 0x1800 is the correct offset
  ```

  ```diff
  diff --git a/configs/am64x_evm_a53_defconfig b/configs/am64x_evm_a53_defconfig
  index 1a6206c843..f88f0eddc3 100644
  --- a/configs/am64x_evm_a53_defconfig
  +++ b/configs/am64x_evm_a53_defconfig
  @@ -45,7 +45,7 @@ CONFIG_SPL_STACK_R=y
   CONFIG_SPL_SYS_MALLOC=y
   CONFIG_SPL_SYS_MALLOC_SIZE=0x800000
   CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_SECTOR=y
  -CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR=0x1400
  +CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR=0x1800
   CONFIG_SPL_DMA=y
   CONFIG_SPL_ENV_SUPPORT=y
   CONFIG_SPL_ETH=y
  diff --git a/configs/am64x_evm_r5_defconfig b/configs/am64x_evm_r5_defconfig
  index c177150d56..d9c43c8a46 100644
  --- a/configs/am64x_evm_r5_defconfig
  +++ b/configs/am64x_evm_r5_defconfig
  @@ -52,7 +52,7 @@ CONFIG_SPL_CUSTOM_SYS_MALLOC_ADDR=0x84000000
   CONFIG_SPL_SYS_MALLOC_SIZE=0x1000000
   CONFIG_SPL_EARLY_BSS=y
   CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_SECTOR=y
  -CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR=0x400
  +CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR=0x800
   CONFIG_SPL_DMA=y
   CONFIG_SPL_ENV_SUPPORT=y
   CONFIG_SPL_ETH=y
  @@ -70,7 +70,7 @@ CONFIG_SPL_RAM_SUPPORT=y
   CONFIG_SPL_RAM_DEVICE=y
   CONFIG_SPL_REMOTEPROC=y
   CONFIG_SPL_SPI_LOAD=y
  -CONFIG_SYS_SPI_U_BOOT_OFFS=0x80000
  +CONFIG_SYS_SPI_U_BOOT_OFFS=0x100000
   CONFIG_SPL_THERMAL=y
   CONFIG_SPL_YMODEM_SUPPORT=y
   CONFIG_HUSH_PARSER=y
  ```

  With the adapted offsets the eMMC boot works as expected.

  At least till the point where the kernel should be loaded from a partition which doesn't exist yet.

  ```sh
  U-Boot SPL 2024.04-cel-00010-g160db9a2a9-dirty (Jun 19 2024 - 07:35:37 +0200)
  Resetting on cold boot to workaround ErrataID:i2331
  Please resend tiboot3.bin in case of UART/DFU boot
  resetting ...

  U-Boot SPL 2024.04-cel-00010-g160db9a2a9-dirty (Jun 19 2024 - 07:35:37 +0200)
  SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
  SPL initial stack usage: 13384 bytes
  Trying to boot from MMC1
  Authentication passed
  Authentication passed
  init_env from device 9 not supported!
  Authentication passed
  Authentication passed
  Starting ATF on ARM64 core...

  NOTICE:  BL31: v2.10.3(release):lts-v2.10.3
  NOTICE:  BL31: Built : 10:59:18, Jun 17 2024

  U-Boot SPL 2024.04-cel-00010-g160db9a2a9-dirty (Jun 19 2024 - 07:39:37 +0200)
  SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
  Trying to boot from MMC1
  Authentication passed
  Authentication passed


  U-Boot 2024.04-cel-00010-g160db9a2a9-dirty (Jun 19 2024 - 07:39:37 +0200)

  SoC:   AM64X SR2.0 HS-FS
  Model: Texas Instruments AM642 EVM
  Board: AM64-EVM rev C
  DRAM:  2 GiB
  Core:  62 devices, 30 uclasses, devicetree: separate
  MMC:   mmc@fa10000: 0, mmc@fa00000: 1
  Loading Environment from nowhere... OK
  In:    serial@2800000
  Out:   serial@2800000
  Err:   serial@2800000
  Net:   eth0: ethernet@8000000port@1
  Hit any key to stop autoboot:  0
  MMC: no card present
  MMC: no card present
  ** Bad device specification mmc 1 **
  MMC: no card present
  ** Bad device specification mmc 1 **
  Couldn't find partition mmc 1:2
  Can't set block device
  MMC: no card present
  ** Bad device specification mmc 1 **
  Couldn't find partition mmc 1:2
  Can't set block device
  libfdt fdt_check_header(): FDT_ERR_BADMAGIC
  No FDT memory address configured. Please configure
  the FDT address via "fdt addr <address>" command.
  Aborting!
  Bad Linux ARM64 Image magic!
  =>
  ```

- Environment is not persistent by default config

  Regarding commit `949e9cf709391136aa9a7a7f2c2215a9855261ef` from TI they disable the persistent storage of the environment to make the development process easier. This makes partly sense, because you may change something you forgot which can create problems. But for our use case we at least need the persistent environment for Mender. Here we need some variables which can keep their state between a reboot.

  TI added support of loading and interpreting `uEnv.txt` in their bootcmd but this helps probably only for development and not for the use case with Mender. It is also not really suitable for a production use case because of security implications.

  Link to commit: [git.ti.com - ti-u-boot / commit=949e9cf709391136aa9a7a7f2c2215a9855261ef](https://git.ti.com/cgit/ti-u-boot/ti-u-boot/commit/?h=ti-u-boot-2023.04-next&id=949e9cf709391136aa9a7a7f2c2215a9855261ef)

  Reversing the commit for the a53 defconfig.

  ```diff
  diff --git a/configs/am64x_evm_a53_defconfig b/configs/am64x_evm_a53_defconfig
  index f88f0eddc3..cc9c67f6a5 100644
  --- a/configs/am64x_evm_a53_defconfig
  +++ b/configs/am64x_evm_a53_defconfig
  @@ -14,6 +14,7 @@ CONFIG_HAS_CUSTOM_SYS_INIT_SP_ADDR=y
   CONFIG_CUSTOM_SYS_INIT_SP_ADDR=0x80480000
   CONFIG_SF_DEFAULT_SPEED=25000000
   CONFIG_ENV_SIZE=0x20000
  +CONFIG_ENV_OFFSET=0x680000
   CONFIG_DM_GPIO=y
   CONFIG_SPL_DM_SPI=y
   CONFIG_DEFAULT_DEVICE_TREE="k3-am642-evm"
  @@ -89,6 +90,9 @@ CONFIG_OF_LIST="k3-am642-evm k3-am642-sk"
   CONFIG_MULTI_DTB_FIT=y
   CONFIG_SPL_MULTI_DTB_FIT=y
   CONFIG_SPL_MULTI_DTB_FIT_NO_COMPRESSION=y
  +CONFIG_ENV_IS_IN_MMC=y
  +CONFIG_SYS_MMC_ENV_DEV=1
  +CONFIG_SPL_ENV_IS_NOWHERE=y
   CONFIG_NET_RANDOM_ETHADDR=y
  ```

  For testing the default environment was printed out with `printenv` and copied to a plain text file on the host system.

  Here it's important to remove unwanted line breaks before creating the environment out of the file.

  Below you can see the process of creating and flashing of the environment binary.

  ```sh
  mkenvimage -s 0x20000 -o sitara-uboot-env.bin sitara_default_env.txt 

  scp -i ~/.ssh/id_rsa_cel_balluff sitara-uboot-env.bin   balluff@sitara.local:

  # On the target system
  dd if=sitara-uboot-env.bin of=/dev/mmcblk1 bs=1024 seek=6656 conv=fsync

  # The offset 6656 is calculated by the following steps
  # 0x680000 = 6815744
  # 6815744 / 1024 = 6656
  ```

  Now we can test, if we can store the environment persistent.

  ```sh
  => printenv bootcmd_ti_mmc
  bootcmd_ti_mmc=run init_${boot}; if test ${do_main_cpsw0_qsgmii_phyinit} ... 
  => setenv bootcmd_ti_mmc "echo perstistent-env"
  => saveenv
  Saving Environment to MMC... Writing to MMC(1)... OK
  => reset
  resetting ...

  U-Boot SPL 2024.04-cel-00010-g160db9a2a9-dirty (Jun 19 2024 - 07:35:37 +0200)
  SYSFW ABI: 3.1 (firmware rev 0x0009 '9.2.8--v09.02.08 (Kool Koala)')
  SPL initial stack usage: 13384 bytes
  ...
  Hit any key to stop autoboot:  0
  => printenv bootcmd_ti_mmc
  bootcmd_ti_mmc=echo perstistent-env
  ```

  We can alos reset the environment to the default values.

  ```sh
  => env default -a 
  ## Resetting to default environment
  => printenv bootcmd_ti_mmc
  bootcmd_ti_mmc=run init_${boot}; if test ${do_main_cpsw0_qsgmii_phyinit} -eq 1; then run main_cpsw0_qsgmii_phyinit; fi; run boot_rprocs; if test ${boot_fit} -eq 1; then run get_fit_${boot}; run get_fit_overlaystring; run run_fit; else; run get_kern_${boot}; run get_fdt_${boot}; run get_overlay_${boot}; run run_kern; fi;
  ```

  Now as a final test check if we can work with the environment from userspace.

  ```sh
  root@sitara:~# nano /etc/fw_env.config 
  # Set the following entry 
  # /dev/mmcblk1            0x680000                0x20000

  # With that change we can readout and alter the environment with fw_printenv and fw_setenv which is the requirement for Mender

  root@sitara:~# fw_printenv 
  addr_fit=0x90000000
  arch=arm
  args_all=setenv optargs earlycon=ns16550a,mmio32,0x02800000 ${mtdparts}
  args_mmc=run finduuid;setenv bootargs console=${console} ${optargs} root=PARTUUID=${uuid} rw rootfstype=${mmcrootfstype}
  args_usb=run finduuid;setenv bootargs console=${console} ${optargs} root=PARTUUID=${uuid} rw rootfstype=${mmcrootfstype}
  baudrate=115200
  ...
  serial#=0000000000000487
  soc=k3
  stderr=serial@2800000
  stdin=serial@2800000
  stdout=serial@2800000
  update_to_fit=setenv loadaddr ${addr_fit}; setenv bootfile ${name_fit}
  usbboot=setenv boot usb; setenv bootpart 0:2; usb start; run init_usb; run get_kern_usb; run get_fdt_usb; run run_kern;
  vendor=ti
  ```

- Package upgrades on device doesn't work

  If you try to upgrade the package on the target device using `apt`, you will get error messages like the following.

  ```
  unable to make backup link of './boot/u-boot/tiboot3.bin' before installing new version: Operation not permitted
  ```

  This is simply because `/boot/u-boot` is a directory on the `boot` partition which has a `vfat` filesystem.

  The package manager tries to make a backup of the files, before installing the new version, but this is not possible on a `vfat` filesystem. So it refuses to upgrade the package at all.

  If you need to upgrade the package on the device, which may be the case if you want to test the package manually, you can use the workaround described below.

  ```sh
  # After executing the following command you should assure that the device is not rebooted until the new package ist installed
  apt purge u-boot-image-balluff-sitara-am64x
  apt update
  apt install u-boot-image-balluff-sitara-am64x
  reboot
  ```

  This is not a real problem for CEL, because we provide updates via Mender and not via `apt`.

  One possible solution would be to install the artifacts not directly in `/boot/u-boot` but in another directory on the rfs partition such as `/boot` or `/usr`.

  Then we would need a copy command in the `postinst` script to copy the files to the boot partition and a remove command in the `postrm` script to clean up the boot partition.

  The downside of this solution is that we need twice as much disk space for the U-Boot artifacts.


## Resources

- [docs.u-boot.org TI/K3](https://docs.u-boot.org/en/latest/board/ti/k3.html)
- [docs.u-boot.org - TI/AM64x EVM](https://docs.u-boot.org/en/latest/board/ti/am64x_evm.html)
- [docs.u-boot.org - Environment](https://docs.u-boot.org/en/latest/usage/environment.html)
- [docs.u-boot.org - Bootstd](https://docs.u-boot.org/en/stable/develop/bootstd.html)
- [hub.digi.com - U-Boot reference-manual](https://hub.digi.com/dp/path=/support/asset/u-boot-reference-manual/)
- [trustedfirmware-a.readthedocs.io - Initial Build](https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/initial-build.html)
- [software-dl.ti.com - OPTEE](https://software-dl.ti.com/processor-sdk-linux/esd/AM64X/09_00_00_03/exports/docs/linux/Foundational_Components_OPTEE.html)
- [software-dl.ti.com - ATF](https://software-dl.ti.com/processor-sdk-linux/esd/AM64X/09_00_00_03/exports/docs/linux/Foundational_Components_ATF.html)
- [software-dl.ti.com - Migration_Guide](https://software-dl.ti.com/processor-sdk-linux/esd/AM64X/09_00_00_03/exports/docs/linux/Foundational_Components_Migration_Guide.html)
- [dev.ti.com - Bootflow](https://dev.ti.com/tirex/explore/content/am64x_academy_9_00_00_00_v2/_build_am64x_academy_9_00_00_00_v2/source/linux/ch-develop/dev-bootflow.html)
- [dev.ti.com - Build U-Boot](https://dev.ti.com/tirex/explore/content/am64x_academy_9_00_00_00_v2/_build_am64x_academy_9_00_00_00_v2/source/linux/ch-develop/dev-build-uboot.html)
