Metal. (MoltenVK) support for iOS an Mac devices


#1

HI all.
As most of you know , Apple announced about deprecation of OpenGL & OpenGL-ES on its devices.
There is still a grace period that allows to use them , but its matter of time until they will pull the plug.
As a result Urho3D won’t run on Apple devices anymore in the future.

The last couple of weeks I was working on a solution by using 2 open source libraries :

1 - ANGLE ( Almost Native Graphics Layer Engine ) , an open source BSD licensed abstraction developed by Google , basically an OpenGL-ES abstraction running on top of DirectX/Vulkan backends.

2- MoltenVK , its an Vulkan abstraction running on top of Metal on iOS and Mac , Appache 2 license , developed by The Brenwill Workshop Ltd .

Both of these libraries are actively developed by big cooperates and there are daily fixes and improvements.

I completed the work successfully .
So basically it’s an OpenGL-ES running on top of Metal , using liberal open source libraries.
You can find the code in : https://github.com/elix22/Urho3D/tree/angle-vulkan
It’s on the angle-vulkan branch.
You can clone it using : git clone -b angle-vulkan https://github.com/elix22/Urho3D.git

There are 4 scripts that will create for you the Xcode projects :
cmake_ios-metal.sh - will create an iOS Xcode project using the metal backend (my work)
cmake_ios-gles.sh - will create an iOS Xcode project using the default OpenGL-ES backend
cmake_xcode-metal.sh - will create an Mac Xcode project using the metal backend (my work)
cmake_xcode-gl.sh - will create an Mac Xcode project using the default OpenGL-ES backend

These are all debug builds .
In order to get the best performance you will have to modify the scheme to release build configuration and also disable metal API validation .

All the samples seem to work on my iPad .
On my Mac all are working except 10_RenderToTexture.
The performance is great , getting stable 60 FPS on my iPAD in release configuration.

Known issues :

  • On my Mac , once exiting from the app , it crashes , I will try to provide a fix for that in the coming weeks.
  • Shadows are disabled due to missing functionality on Angle side, needs some investigation
  • I disabled HDPI for now , not working properly , it needs more investigation.

I plan to write a blog in the coming weeks describing the inner workings of the implementation to help other developers continue improving this solution and to incorporate it into other game engines.

You are encouraged to try it ( a specially on iOS devices) and provide some feedback .

Enjoy !


#2

I don’t have any Apple devices (which is strange because I worked there for a while) but the value of a vulkan wrapper extends to other devices. This is interesting.

[I quit working at That Fruit Company due to a four hour commute, my car breaking down and taking three weeks to remedy, and finally, a nasty accident involving facial reconstruction - it was not my best year to be honest]


#3

Thanks, I’ll give them a round of test on the first free time i find.


#4

First test on Mac Os:

02_HelloGUI
[Wed Jan 23 20:05:42 2019] INFO: Opened log file /Users/max/Library/Application Support/urho3d/logs/HelloGUI.log
[Wed Jan 23 20:05:42 2019] INFO: Created 1 worker thread
[Wed Jan 23 20:05:42 2019] INFO: Added resource path /usr/local/Urho/anglevk/Urho3D/build-xcode-metal/bin/Data/
[Wed Jan 23 20:05:42 2019] INFO: Added resource path /usr/local/Urho/anglevk/Urho3D/build-xcode-metal/bin/CoreData/
[Wed Jan 23 20:05:42 2019] INFO: Added resource path /usr/local/Urho/anglevk/Urho3D/build-xcode-metal/bin/Autoload/LargeData/
[mvk-info] MoltenVK version 1.0.28. Vulkan version 1.0.95.
The following Vulkan extensions are supported:
VK_KHR_16bit_storage v1
VK_KHR_8bit_storage v1
VK_KHR_bind_memory2 v1
VK_KHR_dedicated_allocation v3
VK_KHR_descriptor_update_template v1
VK_KHR_get_memory_requirements2 v1
VK_KHR_get_physical_device_properties2 v1
VK_KHR_image_format_list v1
VK_KHR_maintenance1 v2
VK_KHR_maintenance2 v1
VK_KHR_maintenance3 v1
VK_KHR_push_descriptor v2
VK_KHR_relaxed_block_layout v1
VK_KHR_sampler_mirror_clamp_to_edge v1
VK_KHR_shader_draw_parameters v1
VK_KHR_shader_float16_int8 v1
VK_KHR_storage_buffer_storage_class v1
VK_KHR_surface v25
VK_KHR_swapchain v70
VK_KHR_swapchain_mutable_format v1
VK_EXT_shader_viewport_index_layer v1
VK_EXT_vertex_attribute_divisor v3
VK_MVK_macos_surface v2
VK_MVK_moltenvk v12
VK_AMD_negative_viewport_height v1
Created VkInstance with the following Vulkan extensions enabled:
VK_KHR_surface v25
VK_MVK_macos_surface v2
[mvk-info] GPU device:
model: Intel Iris Graphics
type: Integrated
vendorID: 0x8086
deviceID: 0x0a2e
pipelineCacheUUID: 00000000-0000-0000-0000-27130000272C
supports the following Metal Feature Sets:
macOS GPU Family 1 v3
macOS GPU Family 1 v2
macOS GPU Family 1 v1
[mvk-info] Created VkDevice to run on GPU Intel Iris Graphics with the following Vulkan extensions enabled:
VK_KHR_swapchain v70
[mvk-info] Created 2 swapchain images with initial size (1024, 768).
[Wed Jan 23 20:05:42 2019] INFO: Adapter used Google Inc. ANGLE (Vulkan 1.0.95(Intel Iris Graphics (0x00000A2E)))
[Wed Jan 23 20:05:42 2019] INFO: Set screen mode 1024x768 windowed monitor 0
[Wed Jan 23 20:05:42 2019] INFO: Initialized input
[Wed Jan 23 20:05:42 2019] INFO: Initialized user interface
[Wed Jan 23 20:05:42 2019] INFO: Initialized renderer
[Wed Jan 23 20:05:42 2019] INFO: Initialized engine
[mvk-info] Shader library compilation succeeded with warnings (code 4):

Compilation succeeded with:

program_source:92:12: warning: unused variable ‘_uiBlendIndices’
float4 _uiBlendIndices;
^
program_source:88:12: warning: unused variable ‘_uiNormal’
float3 _uiNormal;
^
program_source:95:11: warning: unused variable ‘_uiObjectIndex’
float _uiObjectIndex;
^
program_source:91:12: warning: unused variable ‘_uiBlendWeights’
float4 _uiBlendWeights;
^
program_source:94:12: warning: unused variable ‘_uiCubeTexCoord1’
float4 _uiCubeTexCoord1;
^
program_source:90:12: warning: unused variable ‘_uiTangent’
float4 _uiTangent;
^
program_source:89:12: warning: unused variable ‘_uiTexCoord1’
float2 _uiTexCoord1;
^
program_source:93:12: warning: unused variable ‘_uiCubeTexCoord’
float3 _uiCubeTexCoord;
^

[mvk-info] Shader library compilation succeeded with warnings (code 4):

Compilation succeeded with:

program_source:200:12: warning: unused variable ‘flippedFragCoord’
float4 flippedFragCoord;
^

[mvk-info] Shader library compilation succeeded with warnings (code 4):

Compilation succeeded with:

program_source:200:12: warning: unused variable ‘flippedFragCoord’
float4 flippedFragCoord;
^

[mvk-info] Shader library compilation succeeded with warnings (code 4):

Compilation succeeded with:

program_source:199:12: warning: unused variable ‘flippedFragCoord’
float4 flippedFragCoord;
^

Used resources:
Textures/Ramp.png
Textures/Spot.png
Textures/FishBoneLogo.png
Textures/UI.png
Textures/UrhoDecal.dds
Techniques/NoTexture.xml
RenderPaths/Forward.xml
UI/DefaultStyle.xml
Textures/UrhoIcon.png
Fonts/Anonymous Pro.ttf
Shaders/GLSL/Basic.glsl

It works.

29

:grinning:


#5

Sweet!
Please issue PR, so we can deal with any cross platform issues!

I have been looking forward to the benefits of Vulkan for such a long time.
This could trigger more work, since we can now render using multiple threads etc.
But the return is worth the effort.


#6

Basically this seems to be ok on Mac Os. Open points:

  • I’ve tested on 10.13.6, still not upgraded to Mojave. To compile, I had to change two files:
    Source/ThirdParty/MoltenVK/MoltenVK/MoltenVK/GPUObjects/MVKDevice.mm, changed MTLFeatureSet_macOS_GPUFamily1_v4 to v3
    Source/ThirdParty/MoltenVK/MoltenVK/MoltenVK/Vulkan/mvk_datatypes.mm, changed MTLTextureType2DMultisampleArray to MTLTextureType2DMultisample
    This gave no error apparently.
  • all example works with the exception of 42_PBR, which shows blank. That was expected
  • within some examples, mouse coordinates are off someway. Probably the retina screen adaptation. This is nasty.
  • the system is in general WAY faster; some examples are incredibly speedy;
  • random crashes happen when exiting. This is the recurrent log:

Thread 1 Crashed:: Dispatch queue: MoltenVKQueue-0-0-0.0-Dispatch
0 com.apple.driver.AppleIntelHD5000GraphicsMTLDriver 0x00007fff48b8ad7a IGAccelRenderCommandEncoder::setVertexBuffer(MTLIGAccelBuffer*, unsigned int, unsigned int) + 120
1 11_Physics 0x000000010bfde0aa MVKGraphicsResourcesCommandEncoderState::encodeImpl() + 346
2 11_Physics 0x000000010bfdab2f MVKCommandEncoder::finalizeDrawState() + 63
3 11_Physics 0x000000010bfccee5 MVKCmdDrawIndexed::encode(MVKCommandEncoder*) + 21
4 11_Physics 0x000000010bfd9fac MVKCommandEncoder::encodeSecondary(MVKCommandBuffer*) + 44
5 11_Physics 0x000000010bfd0aeb MVKCmdExecuteCommands::encode(MVKCommandEncoder*) + 43
6 11_Physics 0x000000010bfd97f2 MVKCommandBuffer::submit(MVKQueueCommandBufferSubmission*) + 370
7 11_Physics 0x000000010c00fe6b MVKQueueCommandBufferSubmission::execute() + 59
8 11_Physics 0x000000010c00f0dc invocation function for block in MVKQueue::submit(MVKQueueSubmission*) + 28
9 libdispatch.dylib 0x00007fff76ae65fa _dispatch_call_block_and_release + 12
10 libdispatch.dylib 0x00007fff76adedb8 _dispatch_client_callout + 8
11 libdispatch.dylib 0x00007fff76af3217 _dispatch_queue_serial_drain + 635
12 libdispatch.dylib 0x00007fff76ae6166 _dispatch_queue_invoke + 373
13 libdispatch.dylib 0x00007fff76af3f0d _dispatch_root_queue_drain_deferred_wlh + 332
14 libdispatch.dylib 0x00007fff76af7d21 _dispatch_workloop_worker_thread + 880
15 libsystem_pthread.dylib 0x00007fff76e2ffd2 _pthread_wqthread + 980
16 libsystem_pthread.dylib 0x00007fff76e2fbe9 start_wqthread + 13

Everything seems quite usable anyway. Setting off with retina issues and exiting crashes should do for a good PR.

Now I’ll proceed testing on Ios. :grinning:


#7

johnnycable
Thanks for your detailed report .
I am currently very bussy on some development stuff I doing at my work so my answers will be delayed.
You are right , I forgot to mention that in order to compile the code , the minimal OS version
Should be 10.14 (Mojave).

MoltenVK is the only component that I didn’t modify ,
I guess your workarounds are ok to validate the functionality , but for an actual fix I think an issue
should be submitted against MoltenVK , MoltenVK dev team should provide a solution for OS < 10.14

I will address the other issues once you will complete your validation on iOS…

Thanks again for your help.


#8

johnnycable
Disregard my previous statement .
You can use version 10.13.6 (High Sierra)
To fix the compilation errors (missing enums) , update only Xcode to latest version 10.1
It works for me.


#9

With respect to the 42_PBR example, I have only managed to run it once on Linux, without it turning up black - I am not sure why, as no errors were reported. Something is very wrong in there.


#10

I was suspecting that.
Regarding Ios, I get:

/usr/local/Urho/anglevk/Urho3D/Source/ThirdParty/MoltenVK/MoltenVK/MoltenVK/API/mvk_datatypes.h:336:1: ‘MTLPrimitiveTopologyClass’ is unavailable: not available on iOS

possibly the same problem?
I have to backup before I can upgrade. It’s going to take a couple of days.


#11

Yep , Xcode 10.1 will fix it.

I fixed the issues you mentioned , I pushed them to angle-vulkan branch.
Also I wrote an short howto in the wiki tab


#12

This is incredible! A fantastic contribution.

Thank you!


#13

Thanks for the kind words Miguel.
Nice to hear from a famous Software Guru.

Anyone please share your impressions , a specially if you ran it on any iOS device
I am very curious to know how it runs on various iOS devices in release build configuration
(I have only my 6 Gen. iPad to verify it)


#14

Upgraded Xcode and re-run the Mac Os tests again.
Everything generally appears to be fine. No more exit crashes, speed is ok. No more coords/mouse problems.

Latest finds:
10_rendertotexture doesn’t render anything inside the wall:

the following two are not probably render-related. Anyway:

15_Navigation shows jagged lines on some objects. As it’s the usual resource, could be a demo bug:

50_Urho2dplatformer some enemies are invisible (but they kill you on contact). Again, could be a demo bug. If someone can confirm…

So, almost everything is working.
Wonderful! Great job!


#15

johnnycable
Thanks for the update.

10_rendertotexture - Known issue on Mac , works fine on my iPad, I might work on it during the weekend if I will have some spare time.

15_Navigation - Are you referring to the yellow lines ?
It’s debug geometry , just press space to remove it.

50_Urho2dplatformer - Demo bug , happens also on the default OpenGL/ES backend.

Did you have the time to run it on iOS device ?


#16

I think he was referring to what seems like z-fighting between the mushrooms and the ground.


#17

Thanks for clarifying :slight_smile:
Yep , I verified it , definitely z-fighting .
Needs some debugging.


#18

Yes, I was referring to z-fighting indeed.
Now I’m going to test on Ios


#19

Tested first 10 examples. 1-6 and 8 are ok. 07 - 09 - 10 crashed for various reasons.
07 shows increasing resources consumption and falling fps. Probably leaks and in the end crashes. The VK_Format error is recurrent.

07_billboards_xcode_log.csv (613.2 KB)


#20

09 and 10 crashes for the same reason, some depth stencil conflict apparently. 10 crashes on the spot.

10_rendertotexture_xcode_log.csv (7.4 KB)

HTH