MicroPython for UEFI and Intel MicroPython-based UEFI test framework released

Re: https://firmwaresecurity.com/2018/03/20/intel-implementing-micropython-as-a-uefi-test-framework/

MicroPython for UEFI systems is available, see Brian’s edk2-devel list posting and the Tianocore wiki for more details:

https://lists.01.org/pipermail/edk2-devel/2018-August/028339.html

https://github.com/tianocore/edk2-staging/tree/MicroPythonTestFramework

https://github.com/tianocore/edk2-staging/tree/MicroPythonTestFramework/MicroPythonPkg

https://github.com/tianocore/edk2-staging/tree/MicroPythonTestFramework/MpyTestFrameworkPkg

https://micropython.org/

Intel seeks BIOS/UEFI Tools Developer

BIOS-UEFI Firmware Tools Engineer

As BIOS-UEFI Firmware Tools Engineer you will develop tools and scripts needed for build and test automation infrastructure that is the backbone of the the Continuous Integration process in Intel’s Data Center UEFI firmware BIOS team.[…]

https://jobs.intel.com/ShowJob/Id/1573600/BIOS%20UEFI%20Firmware%20Tools%20Engineer

PS: I need to figure out a way to get some swag/etc from jobs that’re filled via this blog. ;-(

PS: Intel HR: spaces in URLs is generally frowned upon.

 

Intel: Implementing MicroPython as a UEFI test framework

https://software.intel.com/en-us/blogs/2018/03/08/implementing-micropython-as-a-uefi-test-framework

MicroPython for UEFI - Stack Overview

MicroPython for UEFI

[…]my team investigated several options to support Python 3.x in UEFI boot services. Because firmware is a constrained environment compared to OS runtime, we evaluated porting MicroPython to UEFI. MicroPython is a Python 3 variant designed for microcontrollers, with memory and size optimizations that make it ideal for pre-OS applications. […] Supporting Python 3.x & MicroPython in open source creates new opportunities for the TianoCore community to validate robust firmware solutions.[…]

https://software.intel.com/en-us/blogs/2018/03/08/implementing-micropython-as-a-uefi-test-framework

MicroPython for UEFI - Stack Overview

UEFI Python: CPython 3.x or MicroPython 3.4?

In the beginning, Intel maintained CPython 2.x for UEFI. Current version is 2.7x. It looks like they may have stopped porting it, I am unclear.

In addition to Intel’s CPython 2.7x port, recently Google has proposed updating CPython 3.x to support UEFI, see:

https://firmwaresecurity.com/2017/11/01/proposal-for-cpython-v3-x-for-uefi/

I may’ve missed it, but I not sure Intel is involved with this effort.

At the UEFI Plugfest later this month, Intel is giving a talk about MicroPython for UEFI, “Implementing MicroPython in UEFI”.  So I am *guessing* that Intel has chosen MicroPython over CPython. I presume this means that CPython 2.7x port is now abandonware. I hope the CPython 3.x proposal is not dead, because of this MicroPython effort. Hoping both get traction, but will not hold my breath…

https://micropython.org/
https://github.com/micropython/micropython

Differences between CPython and MicroPython:
http://docs.micropython.org/en/latest/pyboard/genrst/index.html
https://pypi.python.org/pypi?%3Aaction=search&term=micropython

Once once of these Python 3.x implementations works, and the CHIPSEC project ports from 2.x to 3.c, I can finally stop using v2!!

MicroPython Logo

 

EternalRed0’s Unicorn Engine tutorial

Unicorn Engine tutorial

Jan 14, 2018

In this tutorial you will learn how to use Unicorn Engine by solving practical exercises. There are 4 exercises and I will solve the first exercise for you. For the others I am providing hints and solutions, which you can obtain by clicking the spoiler buttons.[…]

http://eternal.red/2018/unicorn-engine-tutorial/

http://www.unicorn-engine.org/

https://github.com/unicorn-engine/unicorn

Duo Labs releases: IDAPython, Coretex M Firmware and Amnesia modules

Examining Personal Protection Devices: Hardware and Firmware Research Methodology in Action
Todd Manning, Duo Labs
[…]This paper describes a methodology for retrieving device firmware, and for loading firmware into IDA Pro, a common disassembler. This paper focuses on the disassembly of this firmware, and the discussion of a novel approach to defeating readback protection discovered in one ARM Cortex M implementation. During the course of this research, I developed code for IDA Pro to assist in loading and grooming Cortex M firmware images. The IDAPython code is comprised of two modules. The first module annotates Cortex M vector tables, which gives IDA Pro hints about where code exists in the firmware image. The Cortex M module can automatically annotate firmware with a vector table located at the start of a firmware image, or can be tailored by the end user to work with relocated tables. The second module, called Amnesia, uses byte-level heuristics to detect ARM instructions in the firmware. Amnesia also contains heuristics that operate at the ARM instruction level to determine function boundaries based on common ARM function prologues and epilogues.[…]

Cortex M Firmware (cortex_m_firmware.py): This Cortex M Firmware module grooms an IDA Pro database containing firmware from an ARM Cortex M microcontroller. This module will annotate the firmware vector table, which contains a number of function pointers. This vector table annotation will cause IDA Pro to perform auto analysis against the functions these pointers point to. The Cortex M Firmware module also calls into the Amnesia module to automate discovery of additional code in the firmware image using the Amnesia heuristics.

Amnesia (amnesia.py): Amnesia is an IDAPython module designed to use byte level heuristics to find ARM thumb instructions in undefined bytes in an IDA Pro database. Currently, the heuristics in this module find code in a few different ways. Some instructions identify and define new code by looking for comon byte sequences that correspond to particular ARM opcodes. Other functions in this module define new functions based on sequences of defined instructions.

https://duo.com/blog/examining-personal-protection-devices-hardware-and-firmware-research-methodology-in-action

https://github.com/duo-labs/idapython

Proposal for CPython v3.x for UEFI

Thiebaud Weksteen of Google posted a message on the UEFI Forum EDK2 and Python development lists, asking for help with an official port of Python for UEFI. Please help out if you can!! Excerpt of his message below, full message (and thread):
https://lists.01.org/pipermail/edk2-devel/2017-November/016805.html

UEFI has become the standard for firmware (BIOS) interface. Intel has provided an open source implementation under the name EDK2 (part of the TianoCore initiative) [1] for some time. This implementation has evolved significantly and now provides the functionalities of a small OS with a standard library similar to POSIX. In 2011, a port of Python 2.7.1 was added to the EDK2 repository [2]. This port then evolved to 2.7.2 which is still defined as the reference port [3]. In 2015, another port was added of Python 2.7.10 in parallel of 2.7.2 [4]. Since then, both implementations have diverged from upstream and know vulnerabilities have not been fixed. I would like to bring support for edk2 in the official Python repository to remediate this situation, that is officially support edk2 as a platform. Technically, there would be three main aspects for the on-boarding work:

1) Fix headers and source to resolve definition conflicts, similarly to ABS definition in [5];
2) Add the edk2module.c [6] to handle platform-specific functionalities, similarly to the posixmodule.c;
3) Add the build configuration file [7] and necessary modifications within Python to handle the edk2 toolchain;

This work would target the master branch (that is Python 3). I would be interested in hearing your thoughts on this idea.

[1] https://github.com/tianocore/edk2
[2] https://github.com/tianocore/edk2/commit/006fecd5a177b4b7b6b36fab6690bf2b2fa11829
[3] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/PythonReadMe.txt
[4] https://github.com/tianocore/edk2/commit/c8042e10763bca064df257547d04ae3dfcdfaf91
[5] https://gist.github.com/tweksteen/ed516ca7ab7dfa8d18428f59d9c22a3e
[6] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/Efi/edk2module.c
[7] https://github.com/tianocore/edk2/blob/master/AppPkg/Applications/Python/PythonCore.inf

 

new book on Apple reversing/debugging

Advanced Apple Debugging & Reverse Engineering
Explore code through LLDB, Python and DTrace, to discover more about any program than you ever thought possible.

https://store.raywenderlich.com/products/advanced-apple-debugging-and-reverse-engineering?_ga=2.129698885.852507492.1502412840-255700375.1502412840

 

microparse

“Microparse: Microcode update parser for AMD, Intel, and VIA processors written in Python 3.x.”

Security Analysis of x86 Processor Microcode
Daming D. Chen, Gail-Joon Ahn
December 11, 2014

Modern computer processors contain an embedded firmware known as microcode that controls decode and execution of x86 instructions. Despite being proprietary and relatively obscure, this microcode can be updated using binaries released by hardware manufacturers to correct processor logic faws (errata). In this paper, we show that a malicious microcode update can potentially implement a new malicious instructions or alter the functionality of existing instructions, including processor-accelerated virtualization or cryptographic primitives. Not only is this attack vector capable of subverting all software-enforced security policies and access controls, but it also leaves behind no postmortem forensic evidence due to the volatile nature of write-only patch memory embedded within the processor. Although supervisor privileges (ring zero) are required to update processor microcode, this attack cannot be easily mitigated due to the implementation of microcode update functionality within processor silicon. Additionally, we reveal the microarchitecture and mechanism of microcode updates, present a security analysis of this attack vector, and provide some mitigation suggestions. A tool for parsing microcode updates has been made open source, in conjunction with a listing of our dataset.

https://github.com/ddcc/microparse

 

https://www.dcddcc.com/docs/2014_paper_microcode.pdf

 

Intel Distribution for Python

Intel has a new Anaconda Python distribution, built with Intel libs to enhance speed/concurrency to help.

It is registration-required freeware if you download it from Intel. It mentions below that  it might also be installable via Anaconda, unclear if registration is required for Anaconda. I’ve not found the source to this project yet, but I could’ve missed it.

https://software.intel.com/en-us/blogs/2016/09/08/intel-distribution-for-python

A few excerpts from their home page below:

Get up to 97 times faster numerical processing on SciPy stack: NumPy, SciPy, and scikit-learn boosted by the Intel® Math Kernel Library. Speed up data analytics with pyDAAL and parallelize Python workloads with enhanced Intel® Threading Building Blocks (Intel® TBB).

The all-included, out-of-the box distribution accelerates core Python packages including NumPy, SciPy, pandas, scikit-learn, Jupyter, matplotlib, and mpi4py. It integrates the powerful Intel® Math Kernel Library (Intel® MKL), Intel® Data Analytics Acceleration Library (Intel® DAAL) and pyDAAL, Intel® MPI Library, and Intel® Threading Building Blocks (Intel® TBB).

Multithreaded Python workloads can take advantage of multicore architectures from Intel using Intel TBB optimized thread scheduling and efficient communication with the Intel MPI Library.

Installing and managing your Python environment is made easy with the all-in-one installer from Intel or by installing packages from Anaconda Cloud using conda. Third-party packages can be installed with both conda and PIP.

Use the Intel® Distribution for Python powered by Anaconda as a drop-in replacement for your current Python environment to get high performance out of the box. Your Python applications immediately gain significant performance and can further be tuned to extract every last bit of performance using the Intel® VTune™ Amplifier.

The Intel® Distribution for Python is available for free for Python 2.7.x and 3.5.x on OS X*, Windows* 7 and later, and Linux*.

PS: Alas, this does not to appear to be integrated with Intel’s port of CPython to UEFI. 🙂

IPyIDA

 

Marc-Etienne M.Lévei has an IDA shell in IPython! (I wish more security tool  projects would integrate with IPython.)

IPyIDA is a python-only solution to use a IPython console in the context of IDA Pro. It spawns an IPython kernel that you can connect to with ipython console –existing in your shell or by opening a QT Console window in IDA Pro with <Shift-.>. You can then benefit from IPython’s autocompletion, online help, monospaced font input field, graphs, and so on.

https://github.com/eset/ipyida

UEFI Firmware Parser now in Cheese Shop

The other day I noticed some Github activity for Teddy Reed’s UEFI Firmware Parser, but didn’t notice any formal new announcement. It appears I was not looking in the right place. The parser is now in the official Python Cheese Shop! And it is named “uefi_firmware”, not UEFI Firmware Parser, that explains that comment in the comment log. 🙂 It’ll be nice to have this tool more easily-available in Python. I hope the next time the UEFI Forum updates it’s UEFI port of CPython, they add this module to the UEFI port.

https://pypi.python.org/pypi/uefi_firmware

https://firmwaresecurity.com/2015/12/28/uefi-firmware-parser-updated-2/

GDB Dashboard: Python visual UI

Modular visual interface for GDB in Python. This comes as a standalone single-file .gdbinit which, among the other things, enables a configurable dashboard showing the most relevant information during the program execution. Its main goal is to reduce the number of GDB commands issued to inspect the current program status allowing the programmer to focus on the control flow instead.

https://github.com/cyrus-and/gdb-dashboard

tool: ThunderGate

I just learned about ThunderGate, by Saul St John, The current version is 0.8.499, initial release was 4 months ago. It is a Python RE tool for Apple Thunderbolt Ethernet (and other) controllers, with PCI Option ROM, and UEFI support! I’m excerpting the readme and usage output below, see the URLs for full details, including omitted scary warning disclaimers:

ThunderGate is a collection of tools for the manipulation of Tigon3 Gigabit Ethernet controllers, with special emphasis on the Broadcom NetLink 57762, such as is found in Apple Thunderbolt Gigabit Ethernet adapters. Tigon3 controllers contain a variety of architectural blocks, including a PCI endpoint, an 802.3 media access controller, on-chip ram, DMA read and write engines, nonvolatile storage, and one or more MIPS processors. These features are exposed by ThunderGate through an easy-to-use Python interface, allowing for reverse engineering, development, and deployment of custom firmware and applications. Examples provided include a userspace VFIO tap driver, a firmware application capable of monitoring and manipulating network traffic and host memory, and a PCI option rom containing an EFI boot services driver which can either inhibit the employ or compromise the effectivity of Intel I/O MMU address translation (VT-d). The ThunderGate firmware implements a network protocol allowing for remote control of the device and host system by an Ethernet-connected peer. Currently supported actions include reading and writing from device and host memory, forging network traffic, sending host interrupts, and manipulation of PCI capabilities configuration.

$ py/main.py -h
usage: main.py [-h] [-v] [-d] [-t] [-s] device
  device        BDF of tg3 PCI device
  -h, –help     show this help message and exit
  -i, –install  install thundergate firmware
  -u, –uio      use uio pci generic interface
  -v, –vfio     use vfio interface
  -d, –driver   load userspace tap driver
  -t, –tests    run tests
  -s, –shell    ipython cli

More Information:
https://github.com/sstjohn/thundergate
http://thundergate.io/