Hi Furkan, thank you for your question! Let me try to clarify a couple of points before addressing your question directly. OPAE is targeted at supporting a wide range of programmable accelerator platforms, and its fundamental concepts are kept intentionally broad to account for differences in platform implementation. As such, it may include features that are not available on all supported platforms or may differ in realization. I believe you're referring to the FPGA platform that's part of the current Intel Hardware Accelerator Research Program (that was the target platform of the presentation at FPL2017). That platform does not include an IOMMU, so any addresses returned by fpgaGetIOAddress() will indeed be physical addresses. (Trivia: fpgaGetIOAddress() was originally called fpgaGetIOVA(), but we changed it precisely because of the reasons you're outlining.) It appears that not all variable names carrying this IO address have been renamed to account for this, though; I'm sorry for any confusion this may have caused. If you look at other potential platforms that OPAE could support (or even at actual platforms OPAE supports right now, such as http://altera.com/pac), you will find examples of configurations that do have an IOMMU (the PAC, for example, is a PCIe card, so it lives behind any IOMMU of the platform it's plugged into). An IOMMU does not necessarily provide the same virtual-to-phyiscal translations that are created for software applications by the MMU -- IO devices in virtualized systems may have their own IO virtual address space (hence the name "IO Virtual Address"). In many virtualized systems, the IOMMU will only provide a single mapping for each virtual machine. To allow AFUs to use true "process virtual addresses" instead of "IO addresses" (which may be virtual or physical depending on IOMMU availability) you'll need additional logic in the FPGA to do the translation in sync with the OS page tables. As it happens, the "intel-fpga-bbb" repository https://github.com/OPAE/intel-fpga-bbb includes an IP block (we call then "basic building blocks" or BBBs) for this named MPF -- you may recall a section of the FPL 2017 workshop that explained it. MPF includes a feature for virtual-to-physical address translations (VTP) for AFUs. If you instantiate this block in your AFU design, your AFU can use the virtual addresses of its owning process (you can share pointers) -- limited to explicitly shared buffers, of course --, and there is no need to use fpgaGetIOAddress() at all. On Fri, Feb 16, 2018 at 04:12:22PM +0100, Furkan Turan wrote: It depends on the target platform. The integrated Xeon+FPGA system deployed as part of HARP does not; a deployment of the PAC card may very well do (as may other platforms). The IOMMU translations are managed by the operating system. The OPAE driver uses the Linux kernel DMA subsystem, which is aware of (and manages) these mappings. Yes, using MPF (see above). Other (future?) platforms may also provide other solutions to this -- it has been done before for other accelerator technologies. That's an interesting question. I believe you could program an IOMMU to mirror the page mappings of an MMU and thus allow page-level translations. I don't know how that would be implemented using the Linux kernel, though (I believe it currently only creates mappings for explicitly requested DMA buffers, which have to be contiguous (duh)). You may also run into issues where the IOMMU is already used for translating between guest physical and host physical addresses for virtual machines -- you'll need something like a second-level address translation mechanism for that (similar to what MMUs do with EPT/RVI). I hope this answers your questions. Thank you - Enno

On 02/16/2018 12:02 PM, Luebbers, Enno wrote: Would you be able to share any information about platforms you plan to support with us? Do you support or plan to support Xeon/FPGA platforms officially? - kaz