I am working on OTA updates for an Orin AGX and trying to understand how rollbacks work if the update fails.
According to the doc, the bootloader and rootfs A/B slots are always linked.
I verified this using the ‘set-active-boot-slot’ command of nvbootctrl. Regardless of what I pass via the -t option (rootfs or bootloader), this command switches both the Active bootloader lost and Active rootfs slot.
The doc talks about the following two services used to verify an update:
l4t-rootfs-validation-config.serviceruns customer-defined logic to assess if the update was a success.
What are we meant to do if we decide the update was unsuccessful? Call nvbootctrl to change the boot slot?nv-l4tbootloader-config.service just calls nvbootctrl verify. The doc is a bit sparse regarding this command as it just states “Verify the bootloader and rootfs boot.”.
What does “nvbootctrl verify” do exactly? Is the source code for the tool available somewhere? How does it decide if the update was successful? Does it do anything if it decides the update was unsuccessful (e.g. swap slots and reboot)? The service itself takes no action after calling it… Any suggestion to put the Orin in a state where verify would fail so we can test the behaviour?
The doc also states:
By default, if the system failed to boot to rootfs, the nv-l4t-bootloader-config.service background service does not have a chance to validate the rootfs boot status. When the failure happens continuously for three times:
If the rootfs A/B is enabled, the system switches to the unused rootfs slot.
Does it means that it may swap the rootfs slots but not the bootloader
slots, leaving a potentially inconsistent set of bootloader/roots (this would contradict the earlier statement about keeping bootloader and rootfs slots in sync)? Or would it simply swap the bootloader/rootfs slots, effectively reverting to the other slot?
What is the reasoning about making l4t-rootfs-validation-config.service run before nv-l4tbootloader-config.service? As the later seems to do a low-level check, it seems back to front to me…
hello jeanphilippe.arnaud,
you should refer to developer guide for Rollback Protection.
in general, each time you update rollback level in the fuse, you may update one or more than one binary levels.
you have to update the rollback fuse, we only need to update the rollback.cfg and then rebuilt/re-flash the target.
Hello Jerry,
Thanks, but I don’t think this answers my questions. I copy/paste them again in this reply…
The doc talks about the following two services used to verify an update:
- l4t-rootfs-validation-config.service: runs customer-defined logic to assess if the update was a success.
What are we meant to do if we decide the update was unsuccessful? Call nvbootctrl to change the boot slot?
- nv-l4tbootloader-config.service: just calls nvbootctrl verify. The doc is a bit sparse regarding this command as it just states “Verify the bootloader and rootfs boot.”.
What does “nvbootctrl verify” do exactly? Is the source code for the tool available somewhere? How does it decide if the update was successful? Does it do anything if it decides the update was unsuccessful (e.g. swap slots and reboot)?
The service itself takes no action after calling it… Any suggestion to put the Orin in a state where verify would fail so we can test the behaviour?
What is the reasoning about making l4t-rootfs-validation-config.service run before nv-l4tbootloader-config.service? As the later seems to do a low-level check, it seems back to front to me…
hello jeanphilippe.arnaud,
the command, nvbootctrl verify is used to verify and reset the boot status, and L4TLauncher checks this status during every boot.
for instance, If the device is unable to boot the filesystem properly, it will fail over to rootfs B or the recovery kernel after three attempts.
Hello Jerry,
Any chance you could give me a bit more details about what it actually does? I have read the doc already and am asking for more details…
Which variable(s) does it look at? How does it decide if the “boot status” is Ok or not?
What variable(s) does it reset exactly?
Is the source code available somewhere?
nv-l4tbootloader-config.service: just calls “nvbootctrl verify” and does not do anything with the result as far as I can see. So what’s the purpose of the service? Or would it automatically reboot the Orin if if failed?
Regarding your example where the device fails to boot the filesystem properly, I have corrupted the APP partition where the rootfs is stored; all that happens is that Linux boots in the initramfs and stays there. This is not unexpected as it’s unable to mount the partition where nvbootctrl is located. Should there be a watchdog to reset the CPU so the L4T has a chance to switch to another partition?
Any chance you could answer my questions below?
4t-rootfs-validation-config.service: runs customer-defined logic to assess if the update was a success.
What are we meant to do if we decide the update was unsuccessful? Call nvbootctrl to change the boot slot?
What is the reasoning about making l4t-rootfs-validation-config.service run before nv-l4tbootloader-config.service? As the later seems to do a low-level check, it seems back to front?
hello jeanphilippe.arnaud,
nvbootctrl verify it will do the following two things.
(1) Verify bootloader status: delete the UEFI variable BootChainFwStatus if it exists, do nothing if not.
(2) Verify RootFS status: clear the rootfs scratch register.
Hello Jerry,
Thanks for the clarification.
Two more questions:
- can
nvbootctrl verify fail? If so, what are we meant to do?
If not, it sounds like this command is only clearing state information, and the term ‘verify’ + the documentation is misleading.
- What is the “rootfs scratch register”?
hello jeanphilippe.arnaud,
it’s boot control function to check the BootChainFwStatus UEFI variable, which is created by UEFI when UEFI updates the bootloader partition.
you may see-also UEFI sources for details, i.e. InitializeRootfsStatusReg
system
Closed
12
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
