VFIO ARM resource

• [2014] VOS - Platform Device Assignment to KVM-on-ARM Virtual Machines via VFIO

• [2014] KVM Platform Device Passthrough

• [2013] (freescale) VFIO for Platform Devices

• Linaro 找到的 add new device to VFIO PLATFORM 教學

Terminology

• VFIO_PLATFORM: VFIO support of device under the platform bus.

  Platform bus means platform dependent bus. In Linux, platform devices are registered at /sys/bus/platform.

  Compared to it, there are some common platform independent buses like pci, usb, and amba. They are also at /sys/bus/ directory.

  see sysfs for more information.

VFIO_PLATFORM for DMA Controller (PL330)

• Testing VFIO_PLATFORM with the PL330 DMA Controller

  • VOS 寫了 ARM platform 的 VFIO support, 支援 PL330 DMA Controller 跟 IOMMU.

  • 測試環境是 emulator (ARM Foundation Model)

  • patch - VFIO support for platform devices

    • find VFIO_PLATFORM code in mainline linux (linux 4.7 is ok): grep -R drivers/vfio -e 'Antonios Motakis'

  • VFIO_PLATFORM userspace test driver

• VFIO_PLATFORM paper

VFIO_PLATFORM for Calxeda xgmac ethernet driver

• [2014] KVM Platform Device Passthrough

  • Calxeda xgmac ethernet driver

  • KVM VFIO irqfd

• xgmac device pass through source code 疑似有進 QEMU 跟 Linux kernel mainline.

  • 原 driver source code: drivers/net/ethernet/calxeda/xgmac.c

  • QEMU patch?:

    • 1. [PATCH v15 00/10] KVM platform device passthrough: https://www.spinics.net/lists/kvm-arm/thrd15.html#14541

    • 2. https://patches.linaro.org/patch/43179/

  • [Question] Where is linux kernel patch? is it exist?

  • vfio platform xgmac reset module (KVM) https://patchwork.kernel.org/patch/6358861/

    • drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c

Some kernel patches

• Merge tag ‘vfio-v4.2-rc1’ of git://github.com/awilliam/linux-vfio

  • add reset hooks and Calxeda xgmac reset for vfio-platform (Eric Auger)

  • enable vfio-platform for ARM64 (Eric Auger)

  • tag Baptiste Reynal as vfio-platform sub-maintainer (Alex Williamson)

  • …

• [linux-v4.2] vfio: platform: add the VFIO PLATFORM module to Kconfig

• [linux-v4.1] vfio: amba: VFIO support for AMBA devices

• Merge tag ‘vfio-v3.19-rc1’ of git://github.com/awilliam/linux-vfio

  • Enable iommu-type1 for ARM SMMU (Will Deacon)

KVM PCIe/MSI Passthrough on ARM/ARM64

Eric Auger, KVM PCIe/MSI Passthrough on ARM/ARM64

• ARM-based system require special support for MSI in the context of VFIO passthrough (ARM server, PCI MSI)

• chapters

  • ARM MSI Controllers: GICv2m and GICv3 ITS

  • KVM PCI/MSI passthrough, x86/ARM Differences

    • On x86, MSI write transactions hit special 1MB phyaddr. It’s APIC config space and not DRAM, so it bypass IOMMU. It doesn’t need IOMMU mapping.

    • On ARM, however, MSI write transactions is normal MMIO region. Therefore an IOMMU mapping must exist.

    • The goal: create the needed IOMMU mappings for MSI write transactions. eventually reach hardware MSI frame.

  • Assigned device MSI Setup: VFIO Legacy Implementation for x86, and Requested adaptation for ARM

  • Interrupt Safety: guest cannot trigger MSIs that correspond to interrupt IDs of devices belonging to host or other guest.

  • Conclusions

    • GICv2m MSI controllers will require users to load VFIO with the allow_unsafe_interrupts parameter.

    • GICv3 ITS platforms will work with VFIO without any additional parameters.

• patches: Linux kernel, QEMU

Other Resource

• Direct Device Assignment for Untrusted Fully-Virtualized Virtual Machines, IBM Research, 2008

• Introduction on performance analysis and profiling methodologies for KVM on ARM virtualization