Uncategorized

Intel releases LUV (Linux UEFI Validation) v2.1

Today Ricardo Neri of Intel announced the 2.1 release of LUV. In additon to updating Linux to v4.11, FWTS to V17.06.00, CHIPSEC to v1.3.1, BITS to v2079, and NDCTL v56, they also started doing nightly builds. Here are some of the other highlights of this release, from the announcement:

Gayatri Kammela won the prize of the most active contributor with many bug fixes and a new feature. She fixed our netboot image, which was missing the ramdisk(!). She added support for debugging and logging of BITS output via network. Likewise, she reworked the LUV configuration file to make more sense to both humans and computers by making clear when parameters are not used. She also investigated and fixed dependencies in systemd that caused delays in the execution of tests. Lastly, she fixed a couple of build-time bugs.

Naresh Bhat updated our Linux kernel recipe to retrieve the kernel configuration directly from the source tree rather than manually updating it. This helped us to remove those eyesore patches for updating our configuration that needed to be sent every time we bumped to a new kernel version. The overall result looks great and is closer to the intended use of the kernel and Yocto Projects’s scripts to merge multiple configuration fragments. I took this opportunity to sanitize the configuration for x86 to add missing configurations and reorganize them.

Sai Praneeth Prakhya added functionality to dump relevant and useful dumps as part of the testing results. Now LUV is capable of dumping the kernel’s boot log, the contents of the ACPI tables as well as the properties of the CPUs in the system. Very useful! Also, he helped us to bump to Linux v4.11. He also took burden of rebasing our implementation to detect firmware’s illegal memory access in this new version of Linux.

Matt Hart updated our GRUB configuration to automate boots across all CPU architectures by not waiting for human intervention to complete boots.

See the full announcement for lists of Known and Fixed Issues:
https://lists.01.org/mailman/listinfo/luv

In addition to stuff mentioned in LUV announcement, LUV also did some updates to how LUV calls CHIPSEC, see these posts:
https://lists.01.org/pipermail/chipsec/2017-July/thread.html

These days, LUV-live ships with BIOS MBR or UEFI GPT partition types, local or network boot types, and x86 or x64 architecture type, multiple choices for the image:
https://download.01.org/linux-uefi-validation/v2.1/
https://download.01.org/linux-uefi-validation/v2.1/sha256sums.asc

 

Standard
Uncategorized

LUV announces v2.1-rc2 release

Ricardo Neri of Intel posted a LONG announcement about LUV V2.1-rc2, most of which included here. There are a LOT of new features in this LUV release!

This is to announce the release of LUV v2.1-rc2. It has been a while since the last time of our last release. This is not the ideal release cadence are working to make changes. We will now release more frequently. We aim to release a new version every 4-5 weeks with the content we accumulate over that period of time. Given the large number of new features and changes in this release, it made sense to release it as rc2 of v2.1 to allow for issues to arise and stabilize towards the next release cycle.

This release include the client side of our telemetrics solution. This solution is based on the implementation done for Clear Linux[1]; abiding Intel privacy policies[2]. Please note that telemetrics is an opt-in feature and is disabled by default and only works for systems within Intel networks. We will work now on the server side of the solution.

In this release we have migrated from systemV to systemd, which is inline with most Linux distributions. Also, our telemetrics client needed it to function. Megha Dey did all the heavy lifting to migrate to systemd; which was not an easy task (kudos to her!). She worked on stabilizing network and revamping our splash screen, which now uses plymouth.

Sai Praneeth Prakhya extended our existing implementation to detect illegal access to UEFI Boot Services memory regions after boot. His extension now allows to detect such access to also conventional memory. Likewise, it now detects these acceses at runtime and long after UEFI SetVirtualAddressMap. This has been quite useful recently to detect bugs related to UEFI capsules in certain firmware implementations.

Gayatri Kammela worked on providing tools to make the netboot images more useful. She completed a reference implementation of an HTTP server to collect test results in a test farm. The documentation of this implementation can be found here[2]; we don’t provide collection services. Of course, the client-side implementation of this solution is part of this release. Along with this solution, she wrote a script to customize a netboot binary (an EFI application) to work with her reference implementation[4].

Naresh Bhat updated the kernel configuration for aarch64. He also worked on providing a more clean, unified and structured kernel command line for all the supported CPU architectures. He also enabled support of netboot images for aarch64.

Fathi Boudra kindly reworked the kernel configuration fragments to avoid unnecessary duplications.

Matt Hart added a new luv.poweroff option.

Configuration of LUV has been simplified by moving all the parameters that the user might configure a LUV.cfg file found in the boot partition of the disk image. No more meddling with the grub.cfg configuration file.

We now provide images built for both GPT and MBR partition schemes.

Updated test suites: We include FWTS V17.03.00, CHIPSEC v1.2.5 plus all the changes available as of this week towards the release of v.1.2.6, which should be coming soon. BITS was bumped to v2079. We use Linux v4.10. This release is based on the Morty version of the Yocto Project.

meta-oe and updates to the build process: Our build process changed a bit. We now include certain components from the  meta-oe layer[5]. Such layer has been added to our repository, but it still need to be added locally to the build/conf/bblayers.conf file when building.

Binary images for x86: A announcement to download binary images for x86 will be sent this week.

See rest of announcement for list of Known Issues, and Fixed Issues.

[1] https://clearlinux.org/features/telemetry
[2] http://www.intel.com/content/www/us/en/privacy/intel-privacy.html
[3] https://github.com/01org/luv-yocto/wiki/Send–LUV-test-results-to-an-HTTP-server
[4] https://github.com/01org/luv-yocto/wiki/Using-LUV-Script-modify_luv_netboot_efi.py
[5] https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/

Full announcement:
https://lists.01.org/mailman/listinfo/luv

Standard
Uncategorized

LUV adds EFI_WARN_ON_ILLEGAL_ACCESSES

Sai Praneeth Prakhya of Intel has posted a patch to the LUV project list, with new clever new abilities to increase LUV’s ability to detect bad UEFI firmware.

Presently, LUV detects illegal accesses by firmware to EFI_BOOT_SERVICES_* regions only during “SetVirtualAddressMap()”. According to UEFI spec, this function will be called only once; by kernel during boot. Hence, LUV cannot detect any other illegal accesses that firmware might do after boot. Moreover, LUV can detect illegal accesses *only* to EFI_BOOT_SERVICES_CODE/DATA regions. This patch set tries to address the above mentioned two issues:
1. Detect illegal accesses to other EFI regions (like EFI_LOADER_CODE/DATA, EFI_CONVENTIONAL_MEMORY)
2. Detect illegal accesses to these regions even after kernel has booted
Recently, we came across machines with buggy firmware that access EFI memory regions like EFI_CONVENTIONAL_MEMORY, EFI_BOOT_SERVICES_CODE/DATA and EFI_LOADER_CODE/DATA even after kernel has booted. Firmware accesses these regions when some efi_runtime_service() is invoked by test cases like FWTS. These illegal accesses can potentially cause kernel hang. Hence, it’s good to have a test case in LUV which can detect these illegal accesses and hence report them to user. This requires making changes to kernel and searching dmesg for relative warnings. As there are 9 patches to linux kernel to enable this feature and putting all these 9 kernel patches in a single LUV patch makes the LUV patch gigantic; hence I have split them into smaller ones (as suggested by Ricardo). The first patch in this series (“linux-yocto-efi-test: Do not support EFI_BOOT_SERVICES_WARN”) removes support to “EFI_BOOT_SERVICES_WARN” and the later patches add all the bits and pieces together and the 10th patch (“linux-yocto-efi-test: Introduce EFI_WARN_ON_ILLEGAL_ACCESSES”) enables the (new) feature.

Full patch:
https://lists.01.org/mailman/listinfo/luv
.

Standard
Uncategorized

LUV gets telemetrics

Megha Dey of Intel just submitted a 4-part patch to LUV, adding telemetrics. Below is slightly-edited comments from patch, some build instructions omitted. For full text see email, URL at end.

[Luv] [PATCH V1 0/4] Introduce telemetrics feature in LUV

This patchset consists of all the patches needed to enable the telemetrics feature in LUV. LUV brings together multiple separate upstream test suites into a cohesive and easy-to-use product and validates UEFI firmware at critical levels of the software stack. It may be possible that one of the test suites crashes while running. It may be even possible that a kernel panic is observed. Under these scenarios, it would be useful for LUV to call home and submit forensic data to analyze and address the problem. The telemetrics feature will do just this.  Of course, this will be an opt-in feature(command line argument telemetrics.opt-in) and users will get clear indication that data is being collected. We have used the telemetrics-client code developed by the clear-linux team and tried to incorporate it in LUV. It has support for crashprobe (invoked whenever a core dump is created), oopsprobe(invoked when there is a kernel oops observed) and pstore-probe(invoked when there is a kernel panic and system reboots). In any of these scenarios, telemetrics records will be sent to the server, currently residing at(one used by clear linux):
 http://rnesius-tmdev.jf.intel.com/telemetryui/
The build ID 122122 can be used to filter the LUV telemetrics records which can be further analysed. In due course, we will have to implement a server of our own to handle this. For telemetrics to work in LUV, the following changes were needed:

1. Migrate to SystemD: LUV currently uses the SystemV init manager but since telemetrics-client repo and the latest yocto have updated on to SystemD, LUV also needs to migrate to SystemD. Since Systemd will not work with the existing psplash graphical manager, we have disabled the splash screen

2.    Migrate to Plymouth: LUV currently uses the psplash graphical manager, but since SystemD is compatible with only Plymouth graphical manager, we have migrated to Plymouth. PLEASE NOTE: Before migrating to plymouth, we have to merge the morty branch of the meta-oe layer provided by open embedded into the LUV repo and add the meta-oe layer to the build/conf/bblayers.conf file. Here are the steps to do this: <omitted> The loglevel has been set to 0 else there are lots of kernel messages overwriting the plymouth screen. Hence, details about the individual tests in the testsuites will not appear when the splash screen is set to false when using plymouth. If the user wants the test details to be shown, they would have to remove the ‘quiet’ and ‘loglevel=0’ kernel command line parameters.

3. Enable networking: After shifting to systemD, the LUV image is not being assigned an IP on boot. This is because it is still using a systemV startup script to do the same. Since systemD names its interfaces differently, we could not see any interfaces with a valid IP. This patch adds the networkd package, introduces a network config file which starts dhcp by default for all interfaces whose names start with en(pci devices which get renamed by udev) or eth(backward compatible) and a service file (networking.service) which will bring up the network and make sure an IP is assigned during boot. It refers:
    https://wiki.archlinux.org/index.php/systemd-networkd

4. Enable telemetrics in LUV: A yocto recipe which fetches the clear-linux telemetrics-client repo, builds it and installs all the necessary service files, daemons and probes has been added. Also, Add a kernel line parameter which lets the user opt-in to the telemetrics feature (telemetrics.opt-in). By default, this feature is disabled. Currently, the telemetrics records can be found here: http://rnesius-tmdev.jf.intel.com/telemetryui/

Full announcement and patch:
https://lists.01.org/mailman/listinfo/luv

Standard
Uncategorized

LUV 2.0 released!

The Intel LUV team, at least including: Gayatri Kammela (12), Megha Dey (12), Naresh Bhat (1), and Ricardo Neri (46) have released 2.0 of LUV, the Linux UEFI Validation Project.

These are the highlights of the release:

*Different types of image available (i386 and x86_x64)
*Logging and debugging via network (or serial)
*Tests for persistent memory (NVDIMM)
*Support for i386
*Booting LUV via network (PXE, HTTP boot later)
*Miscellaneous updates (BITS perf improvements, Linux 4.4 kernel, …)
*Dropped support for fido (focus is on Jethro)
*Known issues and limitations (Debugging works only over Ethernet, not WiFi, …)

Read the full announcement, there are pages of details not included here.

One new feature is i386 support. LUV 1.x was x64-centric, now we hopefully also use LUV 2.0 for testing x86 systems! But signed shim is still only available for 64-bit, so Secure Boot is not enabled for 32-bit support [yet?]. Quoting the release notes:  “At the last minute we faced a kernel issue when booting on a i386-based system. We are debugging. Once this is cleared, a bootable image will be uploaded (issue #76 on [3])”

Full announcement:
https://lists.01.org/pipermail/luv/2016-April/001035.html
https://download.01.org/linux-uefi-validation/v2.0
https://download.01.org/linux-uefi-validation/v2.0/sha256_sums.asc
[1]. https://github.com/01org/luv-yocto/tree/master/meta-luv
[2]. https://github.com/pmem/ndctl
[3]. https://github.com/01org/luv-yocto

Standard
Uncategorized

NTCTL (NFIT Defined Control) tests added to LUV

Megha Dey of Intel just checked in a 5-part patch to the LUV project, adding a new NDCTL (NFIT Defined Control) test suite to LUV.

This patchset adds the NDCTL(NFIT Defined Control) test suite to LUV. Apart from the recipe, it updates the Linux kernel headers, adds the related binaries and a parser to the final LUV image.It addresses issue 58. We also compile and install the required kernel modules for running the  NDCTL test suite and add the configurations needed by the NDCTL testsuite as config fragments to the default config values from v4.4 kernel. A Non-Volatile DIMM (NVDIMM), is a module that can be integrated into the main memory of a compute platform, perform workloads at DRAM speeds, yet be persistent & provide data retention in the event of a power failure or system crash. The LIBNVDIMM subsystem provides block device drivers for three types of NVDIMMs namely nd_pmem (NFIT enabled version of existing ‘pmem’ driver), nd_blk (mmio aperture method for accessing persistent storage) and nd_btt(give persistent memory disk semantics)that can simultaneously support both PMEM and BLK mode access. The NVDIMM Firmware Interface Table (NFIT) numerates persistent memory ranges, memory-mapped-I/O apertures, physical memory devices (DIMMs), and their associated properties. Prior to the arrival of the NFIT, non-volatile memory was described to a system only using a single system-physical-address range where writes are expected to be durable after a system power loss. Now, the NFIT specification standardizes not only the description of PMEM, but also BLK and platform message-passing entry points for control and configuration. The NDCTL test suite has 5 tests in total divided into 2 sets of tests: One uses the manufactured NFIT (NVDIMM Firmware Interface Table) to build the nfit_test module as an external module and arrange for the external module replacements of nfit, libnvdimm, nd_pmem, and nd_blk and the other which has the actual *destructive* tests that create namespaces and perform I/O tests on them.

  luv: NDCTL:  Update the linux kernel headers
  core-image-efi-initramfs: Add NDCTL binaries
  luv-test-manager: Add stderr output to LUV parser
  luv : NDCTL: Add NDCTL test suite
  linux-efi-yocto-test: build NDCTL test suite

More info:
https://github.com/01org/luv-yocto/issues/58
https://www.kernel.org/doc/Documentation/nvdimm/nvdimm.txt
http://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
https://github.com/pmem/ndctl
http://permalink.gmane.org/gmane.linux.kernel.commits.head/535671
https://lists.01.org/mailman/listinfo/luv
https://lwn.net/Articles/640891/

Standard
Uncategorized

LUV/BITS/CHIPSEC ported from x64 to x86!!

Get ready to test your Intel x86 systems!

Megha Dey of Intel submitted an 8-part patch to LUV that enables it to build on x86.

LUV has been useful for 64-bit x64 systems, and now is getting useful for 32-bit x86 systems!

Including 32-bit versions of BITS and CHIPSEC!

Is this the first time that pre-compiled binaries of CHIPSEC for x86 systems have been available? Not sure. Anyway, if you build from source you can start now, otherwise, look for the LUV-live binary download site to start having 32- and 64-bit versions, hopefully

Excerpt from part 0 of the patch:

[PATCH 0/8] Build and run LUV on 32 bit platforms

Currently LUV can be built only for 64 bit target platforms. This patchset contains patches which make sure that LUV can be compiled and run on both 32 as well as 64 bit target platforms. This required reworking of the PE header checks, adding call wrappers used by the shim bootloader to store and restore context, making sure chainloader.c compiled for 32 bit systems, adding support to ensure correct direct directory structure for 32 bit case and removing bugs in chipsec so that it could build without any erros on 32 bit systems. Also, the bits recipe is updated to build the grub EFI image only for target builds.This patchset addresses the following issue:
https://github.com/01org/luv-yocto/issues/57

grub: chainloader: shim: rework PE header checks
grub: shim: Add call wrappers for 32 bit systems
grub: shim: compile chainloader.c for 32bit system
luv : Correct directory structure for 32 bit case
luv: Add the ARCH parameter to chipsec Makefile
luv: chipsec : compile for 32 bit systems
bits: only build grub EFI image for target builds
bits: grub: specify location of images and modules for mkimage

More information:

https://lists.01.org/mailman/listinfo/luv

Standard