【.com速译】LinuxPCI子系统是Linux内核中最重要的子系统之一。本文介绍如何使用QEMU模拟不同的PCI/PCIe配置，以帮助研究LinuxPCI子系统。这种能力方便了Linux管理员或开发人员研究、调试和开发Linux内核，因为使用QEMU自定义PCI/PCIe配置要容易得多。例如有了SeaBIOS源码，研究PCI的初始化和检测过程就容易多了。此外，重启QEMU/KVM虚拟机比重启裸机服务器要快得多。本文所有示例，KVM虚拟机运行OracleLinux8，虚拟机内核版本为5.10.0，QEMU版本为5.2.0。所有示例都以启动盘(ol8.qcow2)作为默认IDE运行。由于本文的目的是研究PCI/PCIe，因此我们将以virtio-scsi-pciHBA为例，并且不会将任何SCSILUN连接到HBA。请参阅之前关于如何将SCSILUN连接到virtio-scsi-pciHBA的博客文章。本文重点介绍QEMU和PCI/PCIe的使用，不涉及任何PCI/PCIe规范的基础知识。PCI桥本节演示如何通过PCI-2-PCI桥创建PCI二级总线。辅助总线是通过“pci-bridge”创建的。qemu-system-x86_64-machinepc,accel=kvm-vnc:8-smp4-m4096M\-netnic-netuser,hostfwd=tcp::5028-:22\-hdaol8.qcow2-serialstdio\-devicepci-bridge,id=bridge0,chassis_nr=1\-devicevirtio-scsi-pci,id=scsi0,bus=bridge0,addr=0x3\-devicepci-bridge,id=bridge1,chassis_nr=2\-devicevirtio-scsi-pci,id=scsi1,bus=bridge1,addr=0x3\-devicevirtio-scsi-pci,id=scsi2,bus=bridge1,addr=0x4上面的QEMU命令行创建了两条PCI辅助总线。一条辅助总线(01:00.0)有一个virtio-scsi-pciHBA(01:03.0)，第二条辅助总线(02:00.0)有两个virtio-scsi-pciHBA(02:03.0和02:04.0)。[root@vm~]#lspci00:00.0Hostbridge:IntelCorporation440FX-82441FXPMC[Natoma](rev02)00:01.0ISAbridge:IntelCorporation82371SBPIIX3ISA[Natoma/TritonII]00:01.1IDE接口:IntelCorporation82371SBPIIX3IDE:IntelCorporation82371SBPIIX3IDE[1Natoma/Titon3IDEB02/Titon0B]MBPIIX4ACPI(rev03)00:02.0VGAcompatiblecontroller:Device1234:1111(rev02)00:03.0Ethernetcontroller:IntelCorporation82540EMGigabitEthernetController(rev03)00:04.0PCIbridge:RedHat,Inc.QEMUPCI-PCIQEMHPCIbridge00:05.0atUCIbridge00:05.0atUCIbridge00:05.0atUCIbridge0storageInc.RedHat0控制器Hat,Inc.VirtioSCSI02:03.0SCSI存储控制器：RedHat,Inc.VirtioSCSI02:04.0SCSI存储控制器：RedHat,Inc.VirtioSCSI以下lspci输出和数字描述了此示例的PCI总线拓扑。[root@vm~]#lspci-t[0000:00]-+-00.0+-01.0+-01.1+-01.3+-02.0+-03.0+-04.0-[01]----03.0\-05.0-[02]--+-03.0\-04.0图1PCIRootBus这部分演示了如何通过“轻量级”PXB（PCIExtensionBridge）主桥创建一个额外的PCIRootBus。它在QEMU命令行上是“pxb”。它仅针对i440fx实现，并且只能放在总线0上。qemu-system-x86_64-machinepc,accel=kvm-vnc:8-smp4-m4096M\-netnic-netuser,hostfwd=tcp::5028-:22\-hdaol8.qcow2-serialstdio\-devicepxb,id=bridge1,bus=pci.0,bus_nr=3\-devicevirtio-scsi-pci,bus=bridge1,addr=0x3\-devicepxb,id=bridge2,bus=pci。0,bus_nr=8\-devicevirtio-scsi-pci,bus=bridge2,addr=0x3\-devicevirtio-scsi-pci,bus=bridge2,addr=0x4上面的QEMU命令行创建了两个额外的PCI根总线。第一个根总线(04:00.0)有一个virtio-scsi-pciHBA(04:03.0)，第二个根总线(09:00.0)有两个virtio-scsi-pciHBA(09:03.0和09:04.0)。[root@vm~]#lspci00:00.0Hostbridge:IntelCorporation440FX-82441FXPMC[Natoma](rev02)00:01.0ISAbridge:IntelCorporation82371SBPIIX3ISA[Natoma/TritonII]00:01.1IDE接口:IntelCorporation82371SBPIIX3IDE:IntelCorporation82371SBPIIX3IDE[1Natoma/Titon3IDEB02/Titon0B]MBPIIX4ACPI(rev03)00:02.0VGAcompatiblecontroller:Device1234:1111(rev02)00:03.0Ethernetcontroller:IntelCorporation82540EMGigabitEthernetController(rev03)00:04.0Hostbridge:RedHat,Inc.QEMUPCIExpanderbridge00:05.0Hostbridge:RedHat,Inc.QEMUPCIExpanderbridge03:00.0PCIbridge:RedHat,Inc.QEMUPCI-PCIbridge04:03.0SCSIstoragecontroller:RedHat,Inc.VirtioSCSI08:00.0PCIbridge:RedHat,Inc.QEMUPCI-PCIbridge09:03.0SCSIstoragecontroller:RedHat,Inc.VirtioSCSI09:04.0SCSIstoragecontroller:RedHat,Inc.VirtioSCSI输出和数字描述如下显示了此示例的PCI总线拓扑。[root@vm~]#lspci-t+-[0000:08]---00.0-[09]--+-03.0|\-04.0+-[0000:03]---00.0-[04]----03.0\-[0000:00]-+-00.0+-01.0+-01.1+-01.3+-02.0+-03.0+-04.0\-05.0图2PCIe根联合体这部分演示了如何通过额外的根联合体（RootComplex）创建一个额外的PCIe根总线。根据QEMU源代码，PCIe特性仅受x86架构上的“q35”机器类型和AArch64上的“virt”机器类型支持。根联合体是通过在QEMU命令行上使用“pxb-pcie”创建的。qemu-system-x86_64-machineq35,accel=kvm-vnc:8-smp4-m4096M\-netnic-netuser,hostfwd=tcp::5028-:22\-hdaol8.qcow2-serialstdio\-devicepxb-pcie,id=pcie.1,bus_nr=2,bus=pcie.0\-deviceioh3420,id=pcie_port1,bus=pcie.1,chassis=1\-devicevirtio-scsi-pci,bus=pcie_port1\-deviceioh3420,id=pcie_port2,bus=pcie.1,chassis=2\-devicevirtio-scsi-pci,bus=pcie_port2\-devicepxb-pcie,id=pcie.2,bus_nr=8,bus=pcie.0\-deviceioh3420,id=pcie_port3,bus=pcie.2,chassis=3\-devicevirtio-scsi-pci,bus=pcie_port3上面的QEMU命令行创建了两个额外的PCIe根联合体。第一个根复合体有一个virtio-scsi-pciHBA(09:00.0)，第二个根复合体有两个virtio-scsi-pciHBA（03:00.0和04:00.0）。[root@vm~]#lspci00:00.0Hostbridge:IntelCorporation82G33/G31/P35/P31ExpressDRAMController00:01.0VGAcompatiblecontroller:Device1234:1111(rev02)00:02.0Ethernetcontroller:IntelCorporation82574LGigabitNetworkConnection00:03.0Hostbridge:RedHat,Inc.QEMUPCIeExpanderbridge00:04.0Hostbridge:RedHat,Inc.QEMUPCIeExpanderbridge00:1f.0ISAbridge:IntelCorporation82801IB(ICH9)LPCInterfaceController(rev02)00:1f.2SATAcontroller:IntelCorporation82801IR/IO/IH(ICH9R/DO/DH)6portSATAController[AHCImode](rev02)00:1f.3SMBus:IntelCorporation901mi(82ICorporation))SMBusController(rev02)02:00.0PCIbridge:IntelCorporation7500/5520/5500/X58I/OHubPCIExpressRootPort0(rev02)02:01.0PCIbridge:IntelCorporation7500/5520/5500/X58I/OHubPCIExpressRootPort0(rev02)03:0HubPCIExpressRootPort0(rev02)03:01.0PCIbridge:IntelCorporation7500/5520/5500/X58I/OHubPCIExpressRootPort0(rev02)03:SI.0SCtorageSSI.0SCrev01)04:00.0SCSIstoragecontroller:RedHat,Inc.VirtioSCSI(rev01)08:00.0PCIbridge:IntelCorporation7500/5520/5500/X58I/OHubPCIExpressRootPort0(rev02)09:00.0SCSIstoragecontroller:RedHat,InC。VirtioSCSI(rev01)以下lspci输出和数字描述了此示例的PCIe拓扑[root@vm~]#lspci-t-+-[0000:08]---00.0-[09]----00.0+-[0000:02]-+-00.0-[03]----00.0|\-01.0-[04]----00.0\-[0000:00]-+-00.0+-01.0+-02.0+-03.0+-04.0+-1f.0+-1f.2\-1f.3图3PCISwitch本节演示如何创建PCIeswitch。qemu-system-x86_64-machineq35,accel=kvm-vnc:8-smp4-m4096M\-netnic-netuser,hostfwd=tcp::5028-:22\-hdaol8.qcow2-serialstdio\-deviceioh3420,id=root_port1,总线=pcie.0\-devicex3130-upstream,id=upstream1,bus=root_port1\-devicexio3130-downstream,id=downstream1,bus=upstream1,chassis=9\-devicevirtio-scsi-pci,bus=downstream1\-devicexio3130-downstream,id=downstream2,bus=upstream1,chassis=10\-devicevitio-scsi-pci,bus=downstream2上面的QEMU命令行创建了一个PCIe交换机，并连接了两个virtio-scsi-pciHBA。上游端口连接到根总线，每个下游端口连接到virtio-scsi-pciHBA（03:00.0和04:00.0）。[root@vm~]#lspci00:00.0Hostbridge:IntelCorporation82G33/G31/P35/P31ExpressDRAMController00:01.0VGAcompatiblecontroller:Device1234:1111(rev02)00:02.0Ethernetcontroller:IntelCorporation82574LGigabitNetworkConnection00:03.0PCIbridge:IntelCorporation7500/5520/5500/X58I/OHubPCIExpressRootPort0(rev02)00:1f.0ISAbridge:IntelCorporation82801IB(ICH9)LPCInterfaceController(rev02)00:1f.2SATAcontroller:IntelCorporation82801IR/IO/IH(ICH9R/DO/DH)6portSATAController[AHCImode](rev02)00:1f.3SMBus:IntelCorporation82801FamilyI(ICH9Family)SMBusController(rev02)01:00.0PCIbridge:TexasInstrumentsXIO3130PCIExpressSwitch(Upstream)(rev02)02:00.0PCIbridge:TexasInstrumentsXIO3130PCIExpressSwitch(Downstream)(rev01)02:01.0PCIbridge:TexasInstrumentsXIO3130PCIExpressSwitch(Downstream)(rev01)03:00.0SCSIstoragecontroller:RedHat,Inc.VirtioSCSI(rev01)04:00.0SCSI存储控制器：RedHat,Inc.VirtioSCSI(rev01)以下lspci输出和数字描述了此示例的PCIe拓扑。[root@vm~]#lspci-t-[0000:00]-+-00.0+-01.0+-02.0+-03.0-[01-04]----00.0-[02-04]--+-00.0-[03]----00.0|\-01.0-[04]----00.0+-1f.0+-1f.2\-1f.3图4IOMMU现在，IOMMU一直被裸机使用.QEMU可以模拟IOMMU，帮助开发者调试和研究Linux内核IOMMU的源代码以及DMA重映射和中断重映射的工作原理。以下QEMU命令行演示了如何为虚拟机创建IntelIOMMU（启用中断重新映射）。除QEMU命令行外，还应在虚拟机的Linux内核命令行末尾添加“intel_iommu=on”。qemu-system-x86_64-machineq35,accel=kvm,kernel-irqchip=split-vnc:8-smp4-m4096M\-netnic-netuser,hostfwd=tcp::5028-:22\-hdaol8.qcow2-serialstdio\-devicenvme,drive=nvme0,serial=deadbeaf1,max_ioqpairs=4\-drivefile=disk1.qcow2,if=none,id=nvme0\-deviceintel-iommu,intremap=on根据虚拟机系统日志，IOMMU可用并使用由Linux内核启用。[root@vm~]#dmesg|egrep"iommu|IOMMU"...[0.019828]DMAR:IOMMUenabled[0.203209]DMAR-IR:IOAPICid0underDRHDbase0xfed90000IOMMU0[0.628348]iommu:Defaultdomaintype:Passthrough[1.078994]pci0:0000:grouping00.0[1.079892]PCI0000：00：01.0：AddingToiOmmugRoup1[1.080775]PCI0000：00：00：02.0：AddingToiOmmugOup2[1.081654]pci0000：00：00：00：00：03.0：AddingToimugugup3[1.08：03[1.08：08：08.10825545]：08.1.082545].2:Addingtoiommugroup4[1.084315]pci0000:00:1f.3:Addingtoiommugroup4原标题：AstudyoftheLinuxkernelPCIsubsystemwithQEMU,作者：DongliZhang且来源为.com]
