RackHD is a technology stack created for enabling hardware management and orchestration, to provide cohesive APIs to enable automated infrastructure. In a Converged Infrastructure Platform (CIP) architecture, RackHD software provides hardware management and orchestration (M&O). It serves as an abstraction layer between other M&O layers and the underlying physical hardware. Developers can use the RackHD API to create a user interface that serves as single point of access for managing hardware services regardless of the specific hardware in place.
DMTF Releases Updated MCTP SMBus/I2C Transport Binding Specification
The DMTF’s Platform Management Components Intercommunication (PMCI) Working Group defines standards to address “inside the box” communication and functional interfaces between the components of the platform management subsystem (e.g., management controllers, managed devices, etc.). PMCI’s Management Component Transport Protocol (MCTP) over SMBus/I2C Transport Binding Specification is now available in version 1.1.0 . This specification addresses how MCTP packets are delivered over a physical SMBus or I2C medium using SMBus transactions. It defines how physical addresses are used, how fixed addresses are accommodated, how physical address assignment is accomplished for hot-plug or other devices that require dynamic physical address assignment, and how MCTP support is discovered. In addition, timing specifications for bus and MCTP control operations are included, and a “fairness” protocol is defined for the purpose of avoiding deadlock and starvation/lockout situations among MCTP endpoints. The binding has been designed to be able to share the same bus as devices communicating using earlier SMBus/I2C management protocols, such as Alert Standard Format (ASF) and Intelligent Platform Management (IPMI), and with vendor-specific devices using SMBus/I2C protocols. The specification also allows a given device to incorporate non-MCTP SMBus functions alongside MCTP.
Senior Software Development Engineer – BIOS Firmware
The AWS Hardware Engineering team creates server designs for Amazon’s innovative web services. Our designs are industry-leading in frugality and operational excellence, and are critical to the success of the AWS business and the more than one million customers who use AWS today. Our Firmware Engineers solve challenging technology problems, and build architecturally sound, high-quality components to enable AWS to realize critical business strategies. The ideal candidate for this role will be an innovative self-starter. You will be a BIOS firmware expert, gain a strong understanding of our firmware stack, and analyze it in its current and future context. You will use comprehensive knowledge of the system in your projects to find the best solutions to multi-factor problems. You will work with engineers across the company as well as external companies and lead firmware development efforts. You will collaborate with internal and external development engineers (architecture, hardware, validation, software services). AWS Engineers are shaping the way people use computers and designing the future of cloud computing technology – come help us make history! What you will do: You will be a member of a team designing AWS-specific hardware, firmware and software. You will be a part of the firmware effort from conception, through validation and into production. You will explore emerging technologies and their impact on AWS. You will work closely with AWS software engineers to tailor devices for the AWS environment.[…]
Software Development Engineer – Server Manageability Firmware
The AWS Hardware Engineering team creates server designs for Amazon’s innovative web services. Our designs are industry-leading in frugality and operational excellence, and are critical to the success of the AWS business and the more than one million customers who use AWS today. Our Firmware Engineers solve challenging technology problems, and build architecturally sound, high-quality components to enable AWS to realize critical business strategies. The ideal candidate for this role will be an innovative self-starter. You will be a Baseboard Management Controller (BMC) firmware expert, gain a strong understanding of our firmware stack, and analyze it in its current and future context. You will use comprehensive knowledge of the system in your projects to find the best solutions to multi-factor problems. You will work with engineers across the company as well as external companies and lead firmware development efforts. You will collaborate with internal and external development engineers (architecture, hardware, validation, software services). AWS Engineers are shaping the way people use computers and designing the future of cloud computing technology – come help us make history! What you will do: You will be a member of a team designing AWS-specific hardware, firmware and software. You will be a part of the firmware effort from conception, through validation and into production. You will explore emerging technologies and their impact on AWS. You will work closely with AWS software engineers to tailor devices for the AWS environment.[…]
Here’s advice from a few months ago by SuperMicro on how to use IPMI in a network environment:
“If you are utilizing Supermicro in your lab environment, there is a great feature that comes with Supermicro boards that allows BMC IPMI management of the server. It is basically an out of band management of the server much like a switch OOB management interface. I wanted to post some screenshots of most of the various areas of control you have with the IPMI console of a Supermicro box. It is fairly comprehensive. Let’s take a look at Supermicro IPMI management walkthrough.”
New potential product on CrowdSupply with a NICE set of features (…and I wonder how secure it will be):
* Blob-free operation
* Fully libre (open-source) IBM OPAL primary firmware w/ PetitBoot interface
* Fully libre (open-source) OpenBMC secondary (IPMI / OoBM) firmware
* NO signing keys preventing firmware modification
DMTF SMASH and DASH are pre-os technologies, somewhat like IPMI and Redfish. SMASH is for servers, DASH is for desktops. AMI and Realtek have DASH working over WiFi now. The new risk brought with this feature is that, if attacker can find exploit in WiFi DASH implementation, they can attack system remotely. Before, they needed an Ethernet connection, now they can use WiFi. IPMI and Redfish have similar risks. I wonder if servers are also available via WiFi with SMASH? Excerpt from press release:
American Megatrends Inc. (AMI), in collaboration with Realtek Semiconductor, an AMI Technology Partner, is pleased to introduce RealManage™ 2.0, a WiFi DASH solution integrated with the RTL8111FP-CG NIC controller chip from Realtek.
DASH (Desktop and mobile Architecture for System Hardware) is a client management standard released by the DMTF (Distributed Management Task Force) and is a web services-based standard for secure out-of-band and remote management of desktops and mobile systems. Realtek has long been an Ethernet NIC market leader and with the RTL8111FP-based next-generation DASH remote management solution called RealManage 2.0, Realtek aims to keep its market position and remain a force for technology innovation.
“With the rising popularity of the GUI BIOS, enterprise customers required out-of-band KVM (Keyboard, Video, and Mouse) functions beyond the standard ‘Text Console Redirection’ feature. Realtek’s RealManage 2.0 is our answer; a powerful DASH solution that supports Wi-Fi and Ethernet DASH, and is compliant with a GUI BIOS,” said Realtek’s Vice President and Spokesman, Yee-Wei Huang. “It brings a whole new application methodology and experience to commercial customers, providing a wealth of data and tools for remote out-of-band client management tasks.”
DMTF released Redfish 1.0 a while ago, and now they’ve done their first revision to this IPMI replacement technology. Excerpting DMTF’s press release:
The latest specification and schemas for the DMTF’s Redfish standard are now available. Now available for download, the 2016.1 publication includes new Redfish schemas for AttributeRegistry, Bios, Drive, Memory, MemoryCollection, MemoryMetrics, SecureBoot, Storage, StorageCollection and Volume. In addition, this release includes minor updates to the Chassis, ComputerSystem, Event, Manager, Power, Resource, SimpleStorage and Thermal schemas, along with all previously released schemas using updated file naming conventions. Released separately as a Work in Progress (WIP) for public comment, the DSP8010-WIP-2016.0.9a () publication includes new Redfish schemas for providing firmware update services (UpdateService, FirmwareInventory) and PCIe switch and device management (PCIeDevice, PCIeFunction, PCIePort, PCIeSwitch, and PCIeZone, and respective Collection schemas). In addition, DMTF has released version 1.0.2 of the Redfish Scalable Platforms Management API Specification, which defines the protocols, data model, and behaviors for Redfish.
Alex Hung of Canonical has announced the 16.05.01 release of FWTS, the FirmWare Test Suite.
There are new ACPI and IPMI and TCG OVAL and Linux device-tree tests. In addition to new features, there’s an even larger list of bugfixes for most classes (UEFI, BIOS, ACPI, etc.) of tools, not excerpted below, see the full announcement for those.
* acpi: add MSCT table sanity check
* acpi: add EINJ table sanity check
* ACPICA: Update to version 20160318 (LP: #1559312)
* Introduce olog scan, to check OPAL msglog.
* Introduce IPMI BMC Info
* devicetree: add infrastructure for device-tree tests
* devicetree/dt_sysinfo: Add device tree system information tests
* devicetree/dt_base: Add base device-tree validity checks
* debian/control: change depends on libjson0-dev to libjson-c-dev
* auto-packager: mkpackage.sh: add yakkety and remove vivid
* debian/control: add back libjson0-dev for precise
IPMIPWN came out 2 months ago and I missed it. 😦 IPMIPWN’s readme excerpt:
IPMI cipher 0 attack tool
There are a few good tools out there (Metasploit) to help you find and identify the IPMI cipher 0 vulnerability, but because its relatively trivial to exploit I have seen nothing that helps you pwn it. While it is easy to exploit, I have found I keep having to brush up on commands and junk every time I come across it which is where my tools comes in. My IPMIPWN tool does all the real work for you, it will attempt to exploit the cipher 0 vulnerability using a list of predefined default user accounts and setup an backdoor account with a semi-random username and random password. All successful backdoors are logged in loot.log. This tool works best on Kali, it does require you to have ipmiutils “apt-get install ipmitool” and NMAP installed. Enjoy.
Besides IPMIPWN, and Metasploit, the tools FreeIPMI and IPMItool are also worth checking out. There is an IPMI Util tool for Windows, and an Intel IPMI tool for MS-DOS, both of which I have not tried out.
If you are new to IPMI security, start here:
Deb McLemore of IBM has submitted a BMC Info, new IPMI tool to FirmWare Test Suite (FWTS).
Introduce IPMI BMC Info
This feature adds the foundation to perform an IPMI BMC Info check that will determine if a Host is capable of IPMI messaging and if so will perform a basic IPMI message exchange to determine the version of IPMI running on the hardware. In the future the IPMI infrastructure can be used to further interrogate the FRU Inventory and other Sensors to help correlate data and surface any discrepancies on inventory or hardware characteristics.
For more information, see the patch sent to the fwts-devel mailing list:
OpenBSD 5.9 has been released. There are a few firmware-related improvements in this release, such as:
* New efifb(4) driver for EFI frame buffer.
* amd64 can now boot from 32 bit and 64 bit EFI.
* Initial support for hardware reduced ACPI added to acpi(4).
* New asmc(4) driver for the Apple System Management Controller.
* New dwiic(4) driver for the Synopsys DesignWare I2C controller.
* Support for ACPI configured SD host controllers has been added to sdhc(4).
* The sdmmc(4) driver now supports sector mode for eMMC devices, such as those found on some BeagleBone Black boards.
* The ipmi(4) driver now supports OpenIPMI compatible character device.
Daocheng Bu of Intel added new IPMI libraries to UEFI’s public EDK-I dev kit. Earlier, IPMI 2.0 definitions were added, now a library to support a generic PIMI submitt command has been added:
Update Ipmi2.0 definitions header file and MdeModulePkg.dsc
file for Ipmi libraries. Add Ipmi realted libraries to support
generic Ipmi submit command. Also add Ppi/Protocol definitions
that will be produced by Ipmi Peim and drivers.
MdePkg: Update Ipmi2.0 definitions header file.
MdeModulePkg: Add IpmiLib and Ppi/Protocol header file.
MdeModulePkg: Add BaseIpmiLib Null Library Instance.
MdeModulePkg: Add PeiIpmiLibIpmiPpi Library Instance.
MdeModulePkg: Add DxeIpmiLibIpmiProtocol Library Instance.
MdeModulePkg: Add SmmIpmiLibSmmIpmiProtocol Library Instance.
MdeModulePkg: Update MdeModulePkg.dsc file for IpmiLib.
18 files changed, 803 insertions(+), 135 deletions(-)
More info: see the patch on the edk2-devel list:
Daocheng Bu of Intel has added IPMI 2.0 definitions to the EDK2 project.
MdePkg/Include/IndustryStandard/Ipmi.h | 26 +
…/IndustryStandard/IpmiNetFnAppDefinitions.h | 614 +++++++++++++++++++++
…/IndustryStandard/IpmiNetFnBridgeDefinitions.h | 238 ++++++++
…/IndustryStandard/IpmiNetFnChassisDefinitions.h | 294 ++++++++++
…/IpmiNetFnFirmwareDefinitions.h | 26 +
…/IpmiNetFnGroupExtDefinitions.h | 26 +
…/IpmiNetFnSensorEventDefinitions.h | 44 ++
…/IndustryStandard/IpmiNetFnStorageDefinitions.h | 514 +++++++++++++++++
…/IpmiNetFnTransportDefinitions.h | 531 ++++++++++++++++++
9 files changed, 2313 insertions(+)
For more information see the edk2-devel posts (or look at the current EDK2 trunk in the above files):
I just learned about Facebook’s OpenBMC, thanks to Sai Dasari of Facebook, who just posted a message to the Open Compute Project’s hardware management list, talking about DMTF Redfish and Facebook’s OpenBMC.
OpenBMC is an open software framework to build a complete Linux image for a Board Management Controller (BMC).
When we were developing Facebook’s top-of-rack “Wedge” switch, we followed our usual process in the beginning; our partner was responsible for developing the BMC software. However, in the first months of the project, many requirements for the BMC software emerged, introducing extra complexity, coordination, and delays into the BMC software-development process. To address these challenges, at one of Facebook’s hackathon events, four engineers worked to create our own BMC software. Within 24 hours, we were able to build a minimum BMC software image, including an SSH server and the ability to change fan speed, power-on the host CPU, and blink some LEDs. It was far from a production image, but it gave us a strong confidence that we could eventually develop our own BMC software for “Wedge.” Fast-forward eight months, and we’ve deployed our solution — code-named “OpenBMC” — into production along with Wedge. And today we’re sharing OpenBMC with the open source community in the hope that we can collaborate based on this open software framework for next-generation system management.
[[ UPDATE: WordPress mangles the below URL to Pauolo’s SMM talk. Download the PDF from the linux-kvm.org link below instead. ]]
The KVM Forum recently finished, and their post-conference materials are available, including videos of some of the presentations. There are multiple interesting talks on QEMU and KVM for security researchers. Two talks that jump out to me are:
Securing secure boot: system management mode in KVM and Tiano Core
by Paolo Bonzini
Using IPMI in QEMU
by Corey Minyard
Lee Calcote of Seagate wrote an article on the recent DMTF Redfish 1.0 release, and about Seagate’s support of this new API, and IPMI. Excerpts:
Like most systems manufacturers, Seagate supports IPMI and will continue to support it as a critical standard in the data center in lieu of broad adoption of Redfish. Where IPMI strains to meet the requirements of today’s massive multiscale environments, Redfish addresses IPMI inadequacies of interoperability, security, simplicity and scalability.
Redfish 1.0 is only the beginning. Seagate and other industry leaders are already engaging within the DMTF Scalable Platform Management Forum on enhancements beyond Redfish 1.0 standard.
What does Redfish mean for Seagate partners and customers? It means a new level of control, management and monitoring for the data center, using a modern, secure RESTful API that is commonly understood and will be widely supported.
Read the full post here:
This week at Intel Developer Forum (IDF), AMI showcased their MegaRAC manageability solutions. MegaRAC is AMI’s Remote Management Firmware family of products for both in-band and out-of-band management, including supporting IPMI, Intel AMT, AMD systems with DMTF DASH. Amongst the new features of MegaRAC SP-X are DMTF Redfish support, and Intel(R) Innovation Engine support.
I don’t know much about Intel’s new “Innovation Engine” is yet, so I’ll excerpt one paragraph from the AMI press release:
“The Innovation Engine is a small, embedded, Intel-architecture processor and I/O subsystem built into future Intel data center platforms,” said Lisa Spelman, General Manager of Data Center Marketing at Intel. “Firmware such as MegaRAC PM-X running on the IE can improve or differentiate the system-builders’ platforms in a wide range of ways, including manageability, cost reduction or security.”
Maybe this means that AMI is the second vendor to support Redfish, after HP?
Read AMI’s full press release here:
The Embedded Linux Conference Europe (ELCE) is happening in October. There’s a set of UEFI talks happening at the event:
UEFI Forum Update and Open Source Community Benefits, Mark Doran
Learn about the recent UEFI Forum activities and the continued adoption of UEFI technology. To ensure greater transparency and participation from the open source community, the Forum has decided to allow for public review of all specification drafts. Find out more about this new offering and other benefits to being involved in firmware standards development by attending this session.
What Linux Developers Need to Know About Recent UEFI Spec Advances, Jeff Bobzin
Users of modern client and server systems are demanding strong security and enhanced reliability. Many large distros have asked for automated installation of a local secure boot profile. The UEFI Forum has responded with the new Audit Mode specified in the UEFI specification, v2.5, offering new capabilities, enhanced system integrity, OS recovery and firmware update processes. Attend this session to find out more about the current plans and testing schedules of the new sample code and features.
LUV Shack: An automated Linux kernel and UEFI firmware testing infrastructure, Matt Fleming
The Linux UEFI Validation (LUV) Project was created out of necessity. Prior to it, there was no way to validate the interaction of the Linux kernel and UEFI firmware at all stages of the boot process and all levels of the software stack. At Intel, the LUV project is used to check for regressions and bugs in both eh Linux kernel and EDK2-based firmware. They affectionately refer to this testing farm as the LUV shack. This talk will cover the LUV shack architecture and validation processes.
The Move from iPXE to Boot from HTTP, Dong Wei
iPXE relies on Legacy BIOS which is currently is deployed by most of the world’s ISPs. As a result, the majority of x86 servers are unable to update and move to a more secure firmware platform using UEFI. Fortunately, there is a solution. Replacing iPXE with the new BOOT from HTTP mechanism will help us get there. Attend this session to learn more.
UEFI Development in an Open Source Ecosystem, Michael Krau, Vincent Zimmer
Open source development around UEFI technology continues to progress with improved community hosting, communications and source control methodologies. These community efforts create valuable opportunities to integrate firmware functions into distros. Most prevalent UEFI tools available today center on chain of trust security via Secure Boot and Intel® Platform Trust Technology (PTT) tools. This session will address the status of these and other tools. Attendees will have the opportunity to share feedback as well as recommendations for future open UEFI development resources and processes.
UEFI aside, there’s many other presentations that look interesting, for example:
Isn’t it Ironic? The Bare Metal Cloud – Devananda van der Veen, HP
Developing Electronics Using OSS Tools – Attila Kinali
How to Boot Linux in One Second – Jan Altenberg, linutronix GmbH
Reprogrammable Hardware Support for Linux – Alan Tull, Altera
Measuring and Reducing Crosstalk Between Virtual Machines – Alexander Komarov, Intel
Introducing the Industrial IO Subsystem: The Home of Sensor Drivers – Daniel Baluta, Intel
Order at Last: The New U-Boot Driver Model Architecture – Simon Glass, Google
Suspend/Resume at the Speed of Light – Len Brown, Intel
The Shiny New l2C Slave Framework – Wolfram Sang
Using seccomp to Limit the Kernel Attack Surface – Michael Kerrisk
Tracing Virtual Machines From the Host with trace-cmd virt-server – Steven Rostedt, Red Hat
Are today’s FOSS Security Practices Robust Enough in the Cloud Era – Lars Kurth, Citrix
Security within Iotivity – Sachin Agrawal, Intel
Creating Open Hardware Tools – David Anders, Intel
The Devil Wears RPM: Continuous Security Integration – Ikey Doherty, Intel
Building the J-Core CPU as Open Hardware: Disruptive Open Source Principles Applied to Hardware and Software – Jeff Dionne, Smart Energy Instruments
How Do Debuggers (Really) Work – Pawel Moll, ARM
Make your Own USB device and Driver with Ease! – Krzysztof Opasiak, Samsung
Debugging the Linux Kernel with GDB – Peter Griffin, Linaro
Redfish, an IPMI replacement, has shipped the first release of their spec. Quoting the press release:
DMTF Helps Enable Multi-Vendor Data Center Management with New Redfish 1.0 Standard
DMTF has announced the release of Redfish 1.0, a standard for data center and systems management that delivers improved performance, functionality, scalability and security. Designed to meet the expectations of end users for simple and interoperable management of modern scalable platform hardware, Redfish takes advantage of widely-used technologies to speed implementation and help system administrators be more effective. Redfish is developed by the DMTF’s Scalable Platforms Management Forum (SPMF), which is led by Broadcom, Dell, Emerson, HP, Intel, Lenovo, Microsoft, Supermicro and VMware with additional support from AMI, Oracle, Fujitsu, Huawei, Mellanox and Seagate. The release of the Redfish 1.0 standard by the DMTF demonstrates the broad industry support of the full organization.
Don’t forget to grab the Redfish “Mockup” as well as the specs and schema.
UEFI 2.5 has a JSON API to enable accessing Redfish. HP was first vendor with systems that supported UEFI 2.5’s new HTTP Boot, a PXE replacement. Intel checked in HTTP Boot support into TianoCore, so it’s just a matter of time until other vendors have similar products. JSON-based Redfish and HTTP-based booting makes UEFI much more of a “web app”, w/r/t security research, and the need for system administrators to more closely examine how firmware is updated on their systems, to best protect them.
Dmigtry Tantsur of Red Hat announced version 2.0 of OpenStack’s hardware introspection service was released today on the openstack-announce list.
“This is an auxiliary service for discovering hardware properties for a node managed by OpenStack Ironic. Hardware introspection or hardware properties discovery is a process of getting hardware parameters required for scheduling from a bare metal node, given it’s power management credentials (e.g. IPMI address, user name and password). A special ramdisk is required to collect the information on a node. The default one can be built using diskimage-builder and ironic-discoverd-ramdisk element. Highlights of this release:
* Main Python module was renamed to ironic_inspector
* Client library was split away to a separate project
* edeploy plugin was removed in favor of more generic one called ‘extra_hardware’
* Processing hooks interface was changed
* The way we return API errors was changed to better match Ironic one
* Removed deprecated /v1/discover endpoint
* All options (except for ‘database’) were moved to sections instead of using ‘discoverd’ for everything
* oslo.db configuration should be used instead of ‘discoverd.database’ option
* keystonemiddleware options should be used instead of reusing ‘ironic’ credentials for checking authentication
* Deprecated ‘authenticate’ opt in favor of ‘auth_strategy’
* Explicit green thread pool is used instead of just launching new threads
* NodeInfo class became more helpful for hooks
* Now it’s possible to hook into processing chain when node is not found
* Inspector no longer checks for Ironic presence on start up as it was causing problems in real life
* SSL/TLS Support”