.net bindings for Urho3D


#1

For few months i was working on a C# bindings generator for Urho3D. I am at the point where while still experimental it already looks like it could be usable. Be aware that project is made with my fork in mind, which is a little bit different from upstream Urho3D. However things generator depends upon are few and minor and i would love to get them upstreamed.

A recap on the features: https://github.com/rokups/Urho3D/wiki/C%23-support
Repository with bindings generator: https://github.com/rokups/Urho3D, build with -DURHO3D_CSHARP=ON.
Repository contains sample 102_CSharpProject written in C#.

Differences from urhosharp:

  • Also works with .net framework
  • Supports inheriting c++ classes and overriding their virtual methods
  • Supports multiple inheritance
  • Supports serializing and deserializing managed objects without magical patches to the source
  • Supports access of protected class members
  • Getters and setters are converted to C# properties

Demo:

using System;
using System.Diagnostics;
using System.IO;
using Urho3D;

namespace DemoApplication
{
    [ObjectFactory]
    class RotateObject : LogicComponent
    {
        public RotateObject(Context context) : base(context)
        {
            UpdateEventMask = UseUpdate;
        }

        public override void Update(float timeStep)
        {
            var d = new Quaternion(10 * timeStep, 20 * timeStep, 30 * timeStep);
            Node.Rotate(d);
        }
    }

    class DemoApplication : Application
    {
        private Scene _scene;
        private Viewport _viewport;
        private Node _camera;
        private Node _cube;
        private Node _light;

        public DemoApplication(Context context) : base(context)
        {
        }

        public override void Setup()
        {
            var currentDir = Directory.GetCurrentDirectory();
            EngineParameters[EngineDefs.EpFullScreen] = false;
            EngineParameters[EngineDefs.EpWindowWidth] = 1920;
            EngineParameters[EngineDefs.EpWindowHeight] = 1080;
            EngineParameters[EngineDefs.EpWindowTitle] = "Hello C#";
            EngineParameters[EngineDefs.EpResourcePrefixPaths] = $"{currentDir};{currentDir}/..";
        }

        public override void Start()
        {
            Input.SetMouseVisible(true);

            // Viewport
            _scene = new Scene(Context);
            _scene.CreateComponent<Octree>();

            _camera = _scene.CreateChild("Camera");
            _viewport = new Viewport(Context, _scene, _camera.CreateComponent<Camera>());
            Renderer.SetViewport(0, _viewport);

            // Background
            Renderer.DefaultZone.FogColor = new Color(0.5f, 0.5f, 0.7f);

            // Scene
            _camera.Position = new Vector3(0, 2, -2);
            _camera.LookAt(Vector3.Zero);

            // Cube
            _cube = _scene.CreateChild("Cube");
            var model = _cube.CreateComponent<StaticModel>();
            model.Model = Cache.GetResource<Model>("Models/Box.mdl");
            model.SetMaterial(0, Cache.GetResource<Material>("Materials/Stone.xml"));
            _cube.CreateComponent<RotateObject>();

            // Light
            _light = _scene.CreateChild("Light");
            _light.CreateComponent<Light>();
            _light.Position = new Vector3(0, 2, -1);
            _light.LookAt(Vector3.Zero);

            SubscribeToEvent(CoreEvents.E_UPDATE, args =>
            {
                var timestep = args[Update.P_TIMESTEP].Float;
                Debug.Assert(this != null);
            });
        }
    }

    internal class Program
    {
        public static void Main(string[] args)
        {
            using (var context = new Context())
            {
                using (var application = new DemoApplication(context))
                {
                    application.Run();
                }
            }
        }
    }
}

ImGui editor w/ scripting
#2

Are the generated bindings ephemeral (ie. Fody magic) or could you toss them into a dump repo? Curious what they look like, but not enough set it up (point @ canonical llvm install dir, vs2015, etc).


#3

Looks nice. Good job . Why you are not helping UrhoSharp team to do them all there?
UrhoSharp already is famous and it’s used in lot of projects. If you apply your changes there then it’s will be use in existing projects too.
And you make me happy too :grin:


#4

They could totally be in their own repo. Actually with a tiny bit effort (sorting of input headers) these bindings would become reproducible as well, so tiny changes would result in tiny diffs in the binding output.

This is current full output: https://gist.github.com/rokups/e1239dd2500a4460d92c055e879e8ef9

Yeah llvm thingy needs fixing, build system does not like spaces. Recommendation is just a workaround as i was pushing fast for windows build.
Actually yesterday we got it working with vs2015. Had to remove single constexpr as it was optional anyway. Also fixed several other bugs that cropped up. @Eugene had luck testing it with vs2015.

I believe their methods are broken and their bindings are broken. It will never be possible to use urhosharp with upstream Urho3D while making my bindings work with upstream project would not be that hard, just need a few tiny changes, where urhosharp sprinkles magic calls around the engine in order to compensate for lack of virtual method overriding support. Also urhosharp bindings generator works only on MacOS. This is simply not acceptable.


Also i would like too point out that these bindings are unsafe. Unsafe in a sense that you have to mind what you are doing. For example anything that does not inherit RefCounted may be freed by engine while you holding a reference to it.

For example you may not keep reference to Application.EngineParameters yourself, because lifetime of EngineParameters is managed by it’s parent class. Wrapper must return a reference, otherwise we will not be able to modify it. So if you kept reference to EngineParameters and freed your Application subclass things would go boom.


#5

@Egorbo that’s what I talked about.
Is there any chance to reuse this work in UrhoSharp?
IMO good binding should be able to bind arbitrary user C++ components, resources and things, and that’s why I pin my hopes on this project.
UrhoSharp has its own feature set and community, but really lacks flexibility.


#6

I have no idea about it. I didn’t deep looks to the UrhoSharp source code and you know what I know Xamarin is part of the Microsoft and I’m sure there is a big support behind that. That why I said it’s good to apply your changes in that projects
There are 2 reasons. First we have centralized C# binding version. Second is all the users same as me be sure if you don’t like to continue your project there is someone else exists to continue.
UrhoSharp have some binding problem I found (https://forums.xamarin.com/discussion/123514/urho-spline-class-is-not-exists-in-c) and exists more. It’s good if you merge your job with that project then everybody have full version and about that I think it’s a good idea to talk with @Egorbo and @migueldeicaza

By the way already UrhoSharp is bind to Android, iOS, Windows, Linux, Mac, Xamarin.Forms, and other platforms. Also you can have it in Workbooks that perfect tools who help to test and play with codes before develop something new.


#7

Nice touch with the commented parse-source above the binding. I’ll have to mess with it and see about getting it to spit out Odin bindings.


#8

Ah yeah i had to add these comments in CApi.cpp in order to aid making rules file (basically adding stuff to ignore).

Not sure how much of use this generator will be to you… Most of passes are very C#-specific, whats left is rather minimal infrastructure. However if you do find it usable then that would be terrific. Me and @boberfly were already having some thoughts to also add support for python by generating code for pybind11. But that is another rather big undertaking, although probably not as big as C#.


Another thingy i would like to pull off is get rid of Urho3D dependency for generator. This would allow us to build generator and generate bindings before Urho3D is built. Then we can build a monolithic Urho3D library that exports both C++ and PInvoke interfaces. Then Urho3DNet.dll would import Urho3D.dll directly instead of Urho3DCSharp.dll.


Edit: this method could totally be used to auto-generate bindings for AngelScript and Lua as well. Just need someone who cares about these langs to do it… :slight_smile:


#9

Well , simply amazing work .
I have never been a C# guy , always using C++ or C (performance freak) , but this is amazing.

I am curious about the overhead imposed by InstanceCache.cs
“GetOrAdd()” for new instances creation or “Remove()” for removal (using ConcurrentDictionary).
What will be the performance hit while creating and removing many objects , specifically on mobile devices.


#10

Performance will suck and not even because of InstanceCache. At the moment i am working on mobile game port to android and game stutters when adding/removing too many objects even if code is written in c++.

InstanceCache sucks though. But we have a nasty problem here. What should we do when native function returns a pointer to native object? Every time create a new C# object that wraps same class? That has a potential to leak a lot of memory. If we cache things then cache may be potentially modified outside of main thread so have to account for that. I would love us to have a better solution so if you have ideas please shoot :slight_smile: Also i am unsure if even with concurrent cache things are safe, because shared pointers in urho3d arent thread-safe and it is likely we will be calling RemoveRef() from non-main thread. But hey it worked for Atomic so maybe this will be alright :slight_smile:


#11

I use a fork for Urho3D instead of directly mainstream for two reasons:

  1. Experiments - e.g. UWP support, HoloLens, etc
  2. Urho3D and SDL don’t support scenarios to work with Urho3D as a subview in existing apps (iOS, Android, macOS) - we just wanted to create a simple and small visualizer for (primary) non-gaming apps.
    When there are no SDL changes in the upstream - it takes 15-30 minutes to update bindings and test. SDL-guys love to refactor it every update so it always break my patches.

Do you really need “virtual” methods when there are tons of events for everything which are automatically bound to C#? I agree - there is a mess in the Binder code, I am trying to rewrite it now to be more clean and easy to extend. And yes, the generator works only on macOS, but it’s possible to add Linux support (and Windows via WSL) - it’s just a clang with a few patches to provide bindable clang AST API to C# in order to be able to write simple generators in C# (unlike CppSharp which is quite complicated).

InstanceCache sucks though. But we have a nasty problem here. What should we do when native function returns a pointer to native object? Every time create a new C# object that wraps same class? That has a potential to leak a lot of memory. If we cache things then cache may be potentially modified outside of main thread so have to account for that. I would love us to have a better solution so if you have ideas please shoot :slight_smile: Also i am unsure if even with concurrent cache things are safe, because shared pointers in urho3d arent thread-safe and it is likely we will be calling RemoveRef() from non-main thread. But hey it worked for Atomic so maybe this will be alright :slight_smile:

InstanceCache based on timers is really bad idea - what if your native object is deleted in Urho3D and a new one of a different type is created with the same address? And yes, RemoveRef from non-UI thread won’t work.
the Atomic guys afair took my approach with a cache + callbacks from RefCounted + C# Finalizer which (if needed) dispatches RemoveRef to the game thread.
Took a quick glance to the code, especially Node - what if I serialize a component written in C# to an XML scene. Then restart app and deserialize the scene - what GetComponent() will return? According to your code it will be just Component, I am just trying to say there are lot of hidden details you are going to hit soon :slight_smile:


#12

That depends. Inheriting LogicComponent requires overriding virtual methods. Or even creating Application. These can be worked around one way or the other, but my aim is to be as close to native API as possible. I do need virtuals :wink:

Introducing WSL to build sounds like over-complicating things. I still would not touch that :stuck_out_tongue: Just like something that would require wine in order to build things on linux. That is crazy :slight_smile:
To be honest i would have loved writing this bindings generator in C#, except that core of generator is cppast which is c++ lib and i just wasnt aware of anything comparable which worked in C#. Also bonus points for cppast - does not require clang patches so prebuilt binaries from llvm website can be used on windows and mac. Packages from repos can be used on linux. I spent quite some time trying to get CppSharp to work and compiling clang was a real burden.

Ah interesting! I knew this will be source of problems. Thanks for pointing it out, ill take a look at how Atomic does things :+1:

Actually GetComponent<T>() will return whatever type you pass to it since it is generic. Or null if node does not have component of that type. Node.GetComponents() would also return array of Component subclasses because when deserializing XML a factory calls back into managed world to create managed instance of serialized class. Then this class ends up in the cache and next time managed wrapper sees particular pointer it will reuse instance from the cache.

This is a good point though. This problem could arise when native API returns for example Component* while instance is actually a native subclass of Component. Then wrong wrapper class instance would be created. At least for Urho3D::Object subclasses wrapper could inspect type StringHash and create instance of correct type.


If anything else comes to your mind please speak up. I definitely do not know all the hidden edgecases and you do have a headstart in this. Both projects definitely have things to learn from each other :+1:


C# support status
#13

Well, WSL is a workaround, the patches can be applied to the windows version of Clang too I guess. WSL is not difficult to install - it’s basically just an app in the app store, just click the install :smile:

Then this class ends up in the cache and next time managed wrapper sees particular pointer it will reuse instance from the cache.

Ah, ok then, I didn’t notice the factory.

If anything else comes to your mind please speak up. I definitely do not know all the hidden edgecases and you do have a headstart in this. Both projects definitely have things to learn from each other :+1:

Ok :slight_smile:


#14

Why is it patched though? cppast provides pretty much complete ast without patches. I am sure everyone would be happier if you could avoid depending on patched version of clang.


#15

it’s patched to provide an API for C# in order to work with AST from C#


#16

Heads-up everyone!

Important milestone was reached just now. Bindings generator was finally merged to master branch on my repo, which means code is more useable than ever. Build it with URHO3D_CSHARP=ON and check out 102_CSharpProject sample.

A word of warning: hard dependency on mono was introduced therefore your mileage will wary on windows. It is pretty easy to use entire thing on linux though. I will look into windows deployment during weekend.

Also updated first post somewhat. Take a peek :wink:


#17

@rku
What should i set for LLVM_CONFIG_BINARY and Qt5Widget_DIR?


#18

Build instructions on https://github.com/rokups/Urho3D/wiki/C%23-support

Not sure about Qt thing now. You can ignore it or disable profiling to get rid of the warning


#19

On Mac only Urho3DNet.dll and libUrho3DCSharp.dylib are generated .
No Urho3D.dll and/or Urho3DCSharp.dll

Let me know if I missed something.

mkdir build
cd build

cmake … -DHAVE_CLOCK_GETTIME=0 -DURHO3D_DATABASE_SQLITE=1 -DURHO3D_CSHARP=ON -DLLVM_CONFIG_BINARY=/usr/local/opt/llvm/bin/llvm-config -DURHO3D_SAMPLES=0 -DURHO3D_PHYSICS=1 -DURHO3D_IK=1 -DURHO3D_NETWORK=1 -DURHO3D_NAVIGATION=1

cmake --build .


#20

It is all good. You probably built engine as static library, this is why libUrho3D.dylib is not there. Engine code is in libUrho3DCSharp.dylib in this case. If you added -DBUILD_SHARED_LIBS=ON to cmake parameters then engine would be built as shared library. libUrho3D.dylib would be created and it would contain required C# glue code inside, there would be no libUrho3DCSharp.dylib.