OpenBSD/arm64 on QEMU with networking

With the increasing popularity of ARM64/AArch64 systems, from the Raspberry Pi 3 and PINE64 to Fujitsu’s move away from SPARC64 supercomputers, there hasn’t been a better time to get started with learning this architecture. I wanted to make a start to an Aarch64 assembly language tutorial but didn’t have access to my RPi3, so I looked into the state of QEMU’s emulation. I didn’t need RPi3-specific hardware – which is just as well as I can’t remember off-hand how the bootcode and start.elf crap would work with QEMU – anyway, I opted for a generic device using Linaro’s EDK2 UEFI firmware. The first pre-built EDK2 binary I downloaded wouldn’t play nicely with the OpenBSD kernel so I grabbed a release mentioned by the FreeBSD team – which worked.[…]


ARM64JSON: AArch64 instructions encoded in JSON


The repository contains ARM64 (AArch64) instruction encoding in a machine-readable JSON:

* ISA_v83A_A64_xml_00bet6_instructions.json contains encoding of every instruction, including ARM64v2/v3 extensions.

* ISA_v83A_A64_xml_00bet6_group_class.json contains hierarchical encoding ARM64 top level -> Instruction group (e.g. “Data Processing — Immediate”) -> Instruction class (e.g. “Add/subtract (immediate)”). No instruction encodings in this file.

The simple and easyly-organised JSON data was extracted from a machine-readable ARM64 specs. A64 ISA XML for Armv8.3 ver. 00bet6.1 released by ARM.



khwasan: kernel hardware assisted address sanitizer

“Kernel HardWare-assisted AddressSANitiser (KHWASAN). It’s a much less RAM-hungry ASAN for your kernel, AArch64-only”

This patchset adds a new mode to KASAN, which is called KHWASAN (Kernel
HardWare assisted Address SANitizer). There’s still some work to do and
there are a few TODOs in the code, so I’m publishing this as a RFC to
collect some initial feedback.

The plan is to implement HWASan [1] for the kernel with the incentive,
that it’s going to have comparable performance, but in the same time
consume much less memory, trading that off for somewhat imprecise bug
detection and being supported only for arm64.[…]!topic/kasan-dev/XXflR8Qy6g0

vTZ: Virtualizing ARM TrustZone

vTZ: Virtualizing ARM TrustZone
Zhichao Hua, Jinyu Gu, Yubin Xia, Haibo Chen, Binyu Zang, Haibing Guan

ARM TrustZone, a security extension that provides a secure world, a trusted execution environment (TEE), to run security-sensitive code, has been widely adopted in mobile platforms. With the increasing momentum of ARM64 being adopted in server markets like cloud, it is likely to see TrustZone being adopted as a key pillar for cloud security. Unfortunately, TrustZone is not designed to be virtualizable as there is only one TEE provided by the hardware, which prevents it from being securely shared by multiple virtual machines (VMs). This paper conducts a study on variable approaches to virtualizing TrustZone in virtualized environments and then presents vTZ, a solution that securely provides each guest VM with a virtualized guest TEE using existing hardware. vTZ leverages the idea of separating functionality from protection by maintaining a secure co-running VM to serve as a guest TEE, while using the hardware TrustZone to enforce strong isolation among guest TEEs and the untrusted hypervisor. Specifically, vTZ uses a tiny monitor running within the physical TrustZone that securely interposes and virtualizes memory mapping and world switching. vTZ further leverages a few pieces of protected, self-contained code running in a Constrained Isolated Execution Environment (CIEE) to provide secure virtualization and isolation among multiple guest TEEs. We have implemented vTZ on Xen 4.8 on both ARMv7 and ARMv8 development boards. Evaluation using two common TEE-kernels (secure kernel running in TEE) such as seL4 1 and OP-TEE shows that vTZ provides strong security with small performance overhead.

Click to access fetch.php

EBC adds AArch64 support!

UEFI has a bytecode, the uEfi ByteCode (EBC). It has traditionally been a bytecode used to consolidate all 3 Intel platforms (x86, x64, Itanic), into a single bytecode, so there only needs to be a single driver on the flash, saving flash memory. Unfortunately, it only supported Intel platforms, not ARM, so it was not a universal bytecode for EFI, only a bytecode for Intel systems. Now, someone has ported AArch64 to ARM, so now EBC may now be more interesting!

Import the AArch64 EBC implementation from

Tested with MdeModulePkg/Application/HelloWorld built for EBC.
Would appreciate some reviewing and testing.

Jeff Brasen (1):
  MdeModulePkg/EbcDxe: Add AARCH64 EBC VM support

Leif Lindholm (1):
  ArmVirtPkg: enable EBC interpreter for AArch64 QEMU

More info:

ARM Technical Learning Resources

Tom Stevens of ARM posted a blog with a new list of links for developers new to ARM. About 40 links, mostly on ARM assembly and C, Cortex-M, NEON, ARMv8/AArch64. Here’s a list of the initial 10 or so links’s titles:

Getting Started with ARM Microcontroller and Assembly Programming
How to Load Constants in Assembly for ARM Architecture
Branch and Call Sequences Explained
Building Customized Debug and Trace Solutions for Multicore SoCs
Caches and Self-Modifying Code
Code Size: A Comprehensive Comparison of MicroMIPS32 and Thumb Code Size Using Many Megabytes of Customer Code
How to Call a Function from ARM Assembler
Designing in ARM: Part One- Switching Microcontrollers
Getting Started with ARM Assembly and C Programming
Detecting Overflow from MUL

Full article:

If this were done outside, it’d probably github-hosted and called ‘’. (BTW, I’m slowly working on an ‘awesome firmware’ link collection, if someone doesn’t do one first…)

AArch64 Xen gets Dom0 ACPI support

Shannon Zhao of Huawei submitted a 17-part patch to the Linux-ARM-kernel, Linux-EFI, Linux-Kernel, and Xen-devel lists, which adds ACPI support for Xen Dom0 on AArch64 systems.

This patch set adds ACPI support for Xen Dom0 on ARM64. The relevant Xen ACPI on ARM64 design document could be found from [1]. The corresponding Xen patches can be fetched from [2].

  Xen: ACPI: Hide UART used by Xen
  xen/grant-table: Move xlated_setup_gnttab_pages to common place
  Xen: xlate: Use page_to_xen_pfn instead of page_to_pfn
  arm/xen: Use xen_xlate_map_ballooned_pages to setup grant table
  xen: memory : Add new XENMAPSPACE type XENMAPSPACE_dev_mmio
  Xen: ARM: Add support for mapping platform device mmio
  Xen: ARM: Add support for mapping AMBA device mmio
  Xen: public/hvm: sync changes of HVM_PARAM_CALLBACK_VIA ABI from Xen
  xen/hvm/params: Add a new delivery type for event-channel in HVM_PARAM_CALLBACK_IRQ
  arm/xen: Get event-channel irq through HVM_PARAM when booting with ACPI
  ARM: XEN: Move xen_early_init() before efi_init()
  ARM64: ACPI: Check if it runs on Xen to enable or disable ACPI
  ARM: Xen: Document UEFI support on Xen ARM virtual platforms
  XEN: EFI: Move x86 specific codes to architecture directory
  ARM64: XEN: Add a function to initialize Xen specific UEFI runtime services
  FDT: Add a helper to get specified name subnode
  Xen: EFI: Parse DT parameters for Xen specific UEFI


AArch64 gets ACPI NUMA support

Hanjun Guo of Huawei submitted a 12-part patch to the Linux-ACPI, Linux-ARM-Kernel, and Linux-Kernel lists, which adds ACPU NUMA (Non-Uniform Memory Access) support for AArch64 systems.

Based on Ganapat’s v9 dt based NUMA patch set [1] for ARM64, this patch set introduce the ACPI based configuration to provide NUMA information. ACPI 5.1 already introduced NUMA support for ARM64, which can get the NUMA domain information from SRAT and SLIT table, so parse those two tables to get mappings from cpu/mem to numa node configration and system locality.

  acpi, numa: Use pr_fmt() instead of printk
  acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
  acpi, numa: remove duplicate NULL check
  acpi, numa: introduce ACPI_HAS_NUMA_ARCH_FIXUP
  arm64, acpi, numa: NUMA support based on SRAT and SLIT
  acpi, numa: Enable ACPI based NUMA on ARM64
  acpi, numa: move acpi_numa_slit_init() to common place
  arm64, numa: rework numa_add_memblk()
  x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
  acpi, numa: move bad_srat() and srat_disabled() to common place
  acpi, numa: remove unneeded acpi_numa=1
  acpi, numa: reuse acpi_numa_memory_affinity_init()


Olimex ARM64 OSH laptop update

Olimex is working on an Open Source Hardware-based AArch64-based laptop, based on their Open Source Hardware-based AArch64 dev board. They have a update on the system. Including some prototype pictures:

“needless to mention this window button will become Tux. :-)”

I wonder about what firmware they’ll use, and if the use will be able to update it themselves, from source….

Open Estuary: ecosystem for AArch64 systems

Quoting their documentation:

Estuary provides a total solution for general-purpose computer based on aarch64 architecture. Anyone can quickly bring up an ARM64 platform – both software and hardware with Estuary. The major goals for Estuary project are:
* A stable\available quick start solution based on ARM64 computer, so that anyone can easily get and bring up the complete development environment, so that you can just focus on your particular field quickly.
* A common basic platform, which should be sustainable and will be continuously improved, can attract\absorb and integrate all 3rd part achievement ceaselessly.
* Sufficient technological documentation, so that any user or partner can easily get useful help from it.
* Simple and effective communication forum for ARM64 ecosystem, so that users can interact and share each other to improve ARM64 ecosystem.
* A bug tracking system, so that anyone can raise, track and help to resolve the issues, this will help to mature the solution and enhance the ecosystem.
* Multiple ARM64-based hardware platforms, and related software to speed up the ARM64 ecosystem development.
* The most key goal of Estuary project is to support, enable and speed up the maturity of ARM64 ecosystem.

For hardware, they currently support two AArch64 SOC boards, D01, D02 boards. For OS, they support Linux or Windows. For, firmware they say they support “Multi-kinds of boot loader will be enabled for Estuary, include but not limited to UEFI\Grub, uboot.” But currently they appear to support UEFI and GRUB. Amongst the other tools included in Estuary is a LAVA system, Linaro’s Automated Validation system, which performs continuous integration on QEMU- and multiple ARM-based devices, letting you reinstall firmware, OS, run pre-OS tests, and gather results.

They include their UEFI changes in their project changelogs, eg their 2.0 release from last month included two entries on UEFI:
9. Pubished the latest UEFI source code for D02 and eanbled building UEFI from source code directly.
11. Upgraded UEFI to add PCIE scaning.
    a. Remove GE4 and keep only GE5 on UEFI, to avoid confusion with 2 NICs.
    b. Re-enable PCI Express link up for port 1 and 2 (According to PCI Express slots on the board).
    c. Fix PXE timeout issue.
    d. Re-enable showbrdnum and setbrdnum commands on EBL.
    e. Fix several other bugs.

They are still using Ubuntu 12.04.
More information:

Open Source Hardware-based AArch64 board from Olimex

U-Boot AArch64 and ARM Trusted Boot support

This week Linus Walleij of Linaro posted a long blog article on U-Boot this week, with good background on U-Boot on ARM, as well as current AArch64 support, including integration with ARM Trusted Firmware (ARM TF). Excerpting the concluding paragraphs of the blog:

We now have pieced together a system that will start U-Boot from ARM Trusted Firmware and then have U-Boot load the Linux kernel and a device tree and start it. Are there problems remaining?
* One of the big outstanding issues are those where things are fragile because memory references need be hard-coded in U-Boot or ARM Trusted Firmware. For example U-Boot currently assumes that ARM TF will use 16MB of the DRAM memory. If the ARM TF change things around and use more or less memory, U-Boot needs to be reconfigured and recompiled. U-Boot on the other hand, will then pass whatever knowledge it has about the memory to the Linux kernel by augmenting the device tree. So if ARM TF could communicate the memory available to U-Boot and the OS this would be great.
* U-Boot relies on prior boot stages such as ARM Trusted Firmware to install PSCI handlers, while on ARMv7 this was usually done by augmenting U-Boot to do the same. Letting U-Boot install PSCI handlers is a bit bogus, since it is a piece of resident code left in memory after U-Boot has executed and not really “boot loader” code. U-Boot was augmented to compile these into a special memory area, copy them there and leave them around for the operating system to use later. Still there are people who might like to do this on ARMv8 U-Boot, especially those not using ARM Trusted Firmware.
* People apparently toy with the idea of booting U-Boot on bare metal, using a very small or no ROM nor ARM Trusted Firmware, letting U-Boot just execute immediately on the system. As U-Boot relies on something else to set up main memory and providing PSCI, this currently does not work. Doing this would require U-Boot to initialize memory and install PSCI handlers. It would also need to be small enough to execute from on-chip RAM.
* Chain of trust booting with signed boot levels, signed U-Boot and a signed kernel image and a signed device tree, making an example of a totally locked-down system. The Flattened Image Tree (FIT) supported by U-Boot is likely the best way forward here, but requires U-Boot to access public key infrastructure to verify images unless you want to compile the public key directly into U-Boot, which is often not a good idea.
* Fastboot – the Android boot protocol used by the Little Kernel, exists in U-Boot but has not been tested or verified. It can use USB or Ethernet alike.
* More hardware support – such as booting from the USB stick or MMC/SD card found in the Juno board. This was not covered by the experimental port.

Read the full article here:

AArch64 support for ACPI BERT in Linux

ACPI is not only in the firmware of Intel-based systems, but it’s also now inside ARM-based systems. Today on the Linaro-ACPI mailing list, there was a patch for ACPI BERT support for AArch64. I think support had already been in for Intel already, unclear how long.

APEI Boot Error Record Table (BERT) support

Under normal circumstances, when a hardware error occurs, kernel will be notified via NMI, MCE or some other method, then kernel will process the error condition, report it, and recover it if possible. But sometime, the situation is so bad, so that firmware may choose to reset directly without notifying Linux kernel. Linux kernel can use the Boot Error Record Table (BERT) to get the un-notified hardware errors that occurred in a previous boot. In this patch, the error information is reported via printk. Once error log is printed out clear error status so it would not be print during next boot again.

ACPI/APEI is designed to verifiy/report H/W errors, like Corrected Error(CE) and Uncorrected Error(UC). It contains four tables: HEST, ERST, EINJ and BERT. The first three tables have been merged for a long time, but because of lacking BIOS support for BERT, the support for BERT is pending until now. Recently on ARM 64 platform it is has been supported. So here we come. The following log is a BERT record after system reboot because of hitting a fatal error.

BERT: Obtained BERT iomem region <00000000fe801000-00000000fe802000> for BERT.
[Hardware Error]: Error record from previous boot:
[Hardware Error]: event severity: fatal
[Hardware Error]:  Error 0, type: fatal
[Hardware Error]:   section_type: memory error
[Hardware Error]:   physical_address: 0x00000000fe800000
[Hardware Error]:   physical_address_mask: 0x0000000000000fff
[Hardware Error]:   card: 0 module: 1 bank: 0 device: 1 row: 1 column: 1 bit_pos

For more information about BERT, please refer to ACPI Specification version 6.0, section 18.3.1.

For more information about this patch, see this thread:

For more on BERT, also see /src/acpi/bert/bert.c in the FirmWareTestSuite (FWTS).

Click to access ACPI_6.0.pdf

Absolute Joins the RSA Ready Technology Partnership Program

Yesterday, Absolute announced that they’ve joined the RSA Ready Technology Partnership Program.

“Absolute announced today a new collaboration with RSA to offer enhanced endpoint data collection and remediation. As part of the RSA Ready Technology Partnership program, this effort is designed to deliver seamless interoperability between Absolute and RSA Security Analytics, an industry leading advanced threat detection and forensics platform. Using the Absolute SIEM connector, mutual customers can now get deeper visibility into their endpoint deployments by feeding vital Absolute endpoint data directly into the RSA Security Analytics monitoring platform.  If an endpoint security alert is triggered, customers will be able to promptly investigate and respond to potential threats within the broader context of the RSA Security Analytics environment. Customers will also be able to correlate logs, packets, NetFlow, and endpoint data, all within the same platform.”

“Absolute’s Persistence(R) technology is embedded into the core of most devices at the factory. Once activated, it provides organizations with comprehensive visibility into all of their devices so they can confidently manage mobility, investigate potential threats, and take action if a security incident occurs. Most importantly, they can apply remote security measures to protect each device and the data it contains.”

See the full announcement for more details:

Absolute’s CompuTrace is a unique security tool for firmware, it’s device is embedded into many (most?) modern systems, and the device checks if software support is disabled in the firmware, and re-enables it.

“Absolute Data & Device Security (DDS), formerly Absolute Computrace, is an adaptive endpoint security solution. It provides you with a persistent connection to all of your endpoints and the data they contain.”

“Our OEM partners embed Persistence technology into the BIOS or firmware of computers, netbooks, tablets, and smartphones during the manufacturing process. Once activated, customers who purchase these devices benefit from an extra level of security, persistence, and support.”

“Persistence technology from Absolute provides you with visibility and control over all of your devices, regardless of user or location. If an Absolute software client is removed from an endpoint, it will automatically reinstall so you can secure each device and the sensitive data it contains. No other technology can do this. Persistence technology is built into tens of millions of devices around the world and provides organizations with a trusted lifeline to each device in their deployment, regardless of user or location.”

You can use UEFITool to see if Absolute is in your firmware by searching for “computrace” Unicode string.

‘Silicon autopsy’ tools for ARM

Earlier this week Eoin McCann posted a new article in the ARM Processors blog about ‘silicon autopsy’: dealing with silicon errors on ARM systems. Of course it’s a bit of a product pitch for ARM’s CoreSight and related products, but product pitch aside the blog give a good description of the various problems and tools available for those performing ARM-based silicon autopsies. ARM has a free Community Edition as well as an expensive Ultimate Edition of DS-5, ARM’s Eclipse-based Developer Studio IDE for firmware.

Silicon autopsy: Understanding when chips fail: In a lot of ways debug is similar to being a medical doctor. A patient comes in with some complaints and lists their symptoms, but you need to run tests in order to properly diagnose the issue before focusing the mind on how to fix it. A lot has been written and discussed in the past about debugging hardware, but most of the attention is dedicated to the pre-silicon stage when issues can be identified close to the source and rectified before it is too late. These bugs are similar to performing an autopsy on a body, sifting through all of the potential clues to narrow down what has gone wrong, and how it can be rectified. Bugs that are found in the silicon itself are typically much more difficult to identify, and can drain an enormous amount of time and resources to fix properly. Today I will speak about silicon debug, the challenges associated with it and what can be done to improve it.

More Information:

coreboot 4.1 released

It has been 5 years since coreboot 4.0, so coreboot 4.1 is a big deal! Excerpting the coreboot blog:

“There have been as many significant original developments, such as support for many new architectures (ARM, ARM64, MIPS, RISC-V), and related architectural changes like access to non-memory mapped SPI flash, or better insight about the internals of coreboot at runtime through the cbmem console, timestamp collection, or code coverage support.”

In addition to new features, it appears coreboot will get on a more regular release cycle:

“Future releases will happen more frequently, and with more guarantees about the state of the release, like having a cool down phase where boards can be tested and so on. I plan to create a release every three months, so the changes between any two release don’t become too overwhelming. Starting with coreboot 4.1, we will maintain a high level changelog and ‘flag days’ document. The latter will provide a concise list of changes which went into coreboot that require chipset or mainboard code to change to keep it working with the latest upstream coreboot.”

I am looking forward to seeing how this release works with the UEFI CoreBootPkg…

More Information:

Spring Plugfest presentations uploaded

The PDFs of the presentations from last months’ UEFI Forum plugfest have been uploaded to
(scroll about half-way through the page, after the Youtube videos…)

* System Prep Applications – Powerful New Feature in UEFI 2.5 – Kevin Davis (Insyde Software)
* Filling UEFI/FW Gaps in the Cloud – Mallik Bulusu (Microsoft) and Vincent Zimmer (Intel)
* PreBoot Provisioning Solutions with UEFI – Zachary Bobroff (AMI)
* An Overview of ACPICA Userspace Tools – David Box (Intel)
* UEFI Firmware – Securing SMM – Dick Wilkins (Phoenix Technologies)
* Overview of Windows 10 Requirements for TPM, HVCI and SecureBoot – Gabe Stocco, Scott Anderson and Suhas Manangi (Microsoft)
* Porting a PCI Driver to ARM AArch64 Platforms – Olivier Martin (ARM)
* Firmware in the Data Center: Goodbye PXE and IPMI. Welcome HTTP Boot and Redfish! – Samer El-Haj-Mahmoud (Hewlett Packard)
* A Common Platforms Tree – Leif Lindholm (Linaro)

This’ll be a very short blog, as I’m busy reading 9 new PDFs… 🙂 I’ll do blogs on some these specific presentations in the coming days.



ARM Trusted Firmware

Starting around 2013, ARM started to release “ARM Trusted Firmware” as a BSD-licensed Github-hosted open source project.  ARM Trusted Firmware is the trusted execution environment that runs behinds the scenes of the OS on AArch64 platforms. It works in conjunction with UEFI, including Secure Boot.

In upcoming blog posts, I’ll be writing some articles with more details about this project. For now, I’ll suggest reading their Firmware Design Guide and watching the below Youtube-hosted Linaro intro video.

More Information: