Skip to content
  • Gurchetan Singh's avatar
    gfxstream: mega-change to support guest Linux WSI with gfxstream · c4444b82
    Gurchetan Singh authored
    This is a mega-change to support Linux guest WSI with gfxstream.
    We tried to do a branch where every commit was buildable and
    runnable, but that quickly proved unworkable. So we squashed
    the branch into a mega-change.
    
    Zink provides the GL implementation for Linux guests, so we just
    needed to implement the proper Vulkan Wayland/X11 WSI
    entrypoints.
    
    The overall strategy to support this is to use Mesa's WSI
    functions.  The Vulkan WSI layer was also considered:
    
    https://gitlab.freedesktop.org/mesa/vulkan-wsi-layer
    
    But it was less maintained compared to Mesa.  The way Mesa common
    layers communicate with drivers is the through base objects
    embedded in driver and a common dispatch layer:
    
    https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/docs/vulkan/dispatch.rst
    https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/docs/vulkan/base-objs.rst
    
    Our objects are defined in gfxstream_vk_private.h.  Currently,
    Mesa-derived Vulkan objects just serve as shim to gfxstream
    Vulkan’s internal handle mapping. Long-term, we can use
    Mesa-derived objects inside gfxstream guest Vulkan exclusively.
    
    The flow is typically inside a Vulkan entrypoint is:
    
    - VK_FROM_HANDLE(vk-object) to convert to a gfxstream_vk_obj
      object
    - Call ResourceTracker::func(gfxstream_vk_obj->internal) or
      VkEncoder::func(gfxstream_vk_obj>internal)
    - Return result
    
    A good follow-up cleanup would be to delete older gfxstream
    objects.  For example, we now have struct gfxstream_vk_device
    and info_VkDevice in ResourceTracker.
    
    Most of this logic was auto-generated and included in
    func_table.cpp. Some vulkan functions were too difficult to
    auto-generate or required special logic, and these are included
    in gfxstream_vk_device.cpp.  For example, anything that needs to
    setup the HostConnection requires special handling.
    
    Android Blueprint support is added to the parts of Mesa needed
    to build the Vulkan runtime.  One thing to call out it's
    required to build the guest/vulkan_enc and guest/vulkan files
    in the same shared library now, when previously have
    libvulkan_enc.so and libvulkan_ranchu.so was sufficient
    [otherwise, some weak pointer logic wouldn't work].
    
    A side effect of this is libOpenglSystem must also be a static
    lib, and so should libandroid_aemu too.  That conceptually makes
    sense and the Meson build had been doing this all a long.  We
    can probably transition everything besides libGLESv1_emulation.so,
    libGLESv2_emulation.so and libvulkan_ranchu.so to be static.
    
    This requires changes in the end2end tests, because since each
    HostConnection is separate and internal to it's constituent
    library. Lifetimes need to be managed separately: for example
    the HostConnection instance created by the end2end tests would
    not be visible inside libvulkan_ranchu.so anymore. Probably the
    best solution would be improve the testing facade so a
    HostConnection represents one virtio-gpu context, while some
    other entity represents a virtio-gpu device (client-server
    would work).
    
    vk.xml was modified, but change sent to Khronos:
    
    https://gitlab.khronos.org/vulkan/vulkan/-/merge_requests/6325
    
    Fuchsia builds still need to be migrated, but they already have
    Fuchsia Mesa with all the build rules so that shouldn't be too
    bad. Just need to copy them over the gfxstream/Mesa hybrid.
    
    The new command for building Linux guests is:
    
    meson amd64-build/ -Dvulkan-drivers="gfxstream" -Dgallium-drivers="" -Dvk-no-nir=true -Dopengl=false
    
    Big shout-out to Aaron Ruby, who did most of the gnarly codegen
    needed to get the function table logic to work.
    
    BUG=313466265
    BUG=298679844
    TEST=
         * Run Weston/vkcube on Linux and automotive platform
         * launch_cvd --gpu_mode=gfxstream vkcube
         * launch_cvd --gpu_mod=gfxstream_guest_angle
         * vkcube + 3D Mark Slingshot extreme work with guest ANGLE and
           GL-VK interop
         * GfxstreamEnd2EndTests
         * Some select dEQP tests
    
    Aaron Ruby (46):
      gfxstream: function table: remove entry points that are hand-written.
      gfxstream: function table: more changes
      gfxstream: function table: scope internal_arrays to encoder
      gfxstream: function table: autogenerate compoundType params
      gfxstream: add handwritten EnumeratePhysicalDeviceGroup entrypoint.
      gfxstream: function table: handle nested handle arrays
      gfxstream: function table: adding some handwritten implementations
      gfxstream: revert some unnecessary changes
      gfxstream: use vk_object_zalloc/free instead of vk_zalloc/free.
      gfxstream: revert most gfxstream objects to use vk_object_base
      gfxstream: function table: handwritten commmand-buffers/pools
      gfxstream: codegen functionality to handle special param
      gfxstream: function table: random fixes
      gfxstream: add vk_command_buffer_ops handlers
      gfxstream: func_table.py: Codegen support for nested compound type
      gfxstream: remove handwritten/add autogen entry points
      gfxstream: add gfxstream_vk_device.cpp
      gfxstream: query device and instance extensions early
      gfxstream: func_table: explicit allocation for nested arrays/compound
        types
      gfxstream: goldfish_vulkan: fix commandBuffer allocation.
      gfxstream: meson: Raise api_version in ICD config to 1.1.
      gfxstream: function table: add more handwritten entries
      gfxstream: goldfish_vulkan: update VkDescriptorSetAllocateInfo logic
      gfxstream: function table: NULL check on internal_object dereference
      gfxstream: function table: Remove POSTPROCESSES handling from
        functable
      gfxstream: mesa: Add 'gfxstream' as a -Dvulkan-drivers
      gfxstream: ResourceTracker: add some allowedExtensions
      gfxstream: gfxstream_vk_device: add wsi_common_entrypoints
      gfxstream: Move instance handling into gfxstream_vk_device.cpp
      gfxstream: ResourceTracker: Enable Linux WSI-related extensions
      gfxstream: wsi: add wsi_device initialization
      gfxstream: gfxstream_vk_device: use Mesa common physical device
        management
      gfxstream: ResourceTracker: translate mesa objects in user buffer
      gfxstream: exclude VkSampler and VkDescriptorSet objects from
        translation
      gfxstream: Add guest-side external memory support with colorBuffers.
      gfxstream: function table: Modify semaphoreList inputs to no-op
        semaphores
      gfxstream: function table: Allow VK_NULL_HANDLE for free/destroy APIs.
      gfxstream: cereal: Add VK_EXT_depth_clip_enable as supported feature.
      gfxstream: vulkan_enc: un-namespace vk_util.h and vk_struct_id.h
      gfxstream: gfxstream_vk_device.cpp: Support VK_KHR_surface and
        VK_*_surface
      gfxstream: vulkan_enc: Add support for Mesa-only extensions.
      gfxstream: ResourceTracker: Use DEVICE_TYPE_VIRTUAL_GPU always
      gfxstream: platform: add dma-buf export support with dedicatedBuffer.
      gfxstream: ResourceTracker: add VK_EXT_depth_clip_enable allowed
        extension
      gfxstream: ResourceTracker: external memory via QNX_screen_buffer
        extension
      gfxstream: Add VK_QNX_external_memory_screen_buffer to VulkanDispatch
    
    Gurchetan Singh (18):
      gfxstream: mesa: write Android.bp files
      gfxstream: generate gfxstream_vk_entrypoints.{c, h}
      gfxstream: vulkan_enc: add gfxstream_vk_private.h (objects)
      gfxstream: function table: modify function table to use gfxstream_vk_*
      gfxstream: compiles
      gfxstream: build system improvements
      gfxstream: ResourceTracker: don't crash without
        VkBindImageMemorySwapchainInfoKHR
      gfxstream: vk.xml: make some vkAcquireImageANDROID params optional
      gfxstream_vk_device: filter out swapchain maintenance guest side
      gfxstream: end2end: fixes for End2End tests
      gfxstream: func_table: custom vkEnumerateInstanceLayerProperties
      gfxstream: add VK_EXT_DEBUG_UTILS_EXTENSION_NAME into Mesa list
      gfxstream: clang-format guest code
      gfxstream: libandroid AEMU static
      gfxstream: vkEnumerateInstanceVersion
      gfxstream: vkCreateComputePipeLines
      gfxstream: make end2end tests happy
      gfxstream: delete prior vk.xml, vk_icd_gen.py
    
    Change-Id: Ic4cc50d5098dddd77c9265afa8f432a2fdea8845
    c4444b82