Atomic Game Engine (MIT Urho3D Fork)


#81

Intel HD Graphics GPUs.
My laptop igf - Intel4000HD supports OpenGL 4.0 (Intel® HD Graphics 4000 with 188 ext.)
So I thinks 3.2 is very very old gapi, from previous century :slight_smile:


#82

But factor in Linux drivers and I don’t know when you can start using recent-ish OpenGL if you don’t want to lock out random parts of your audience :slight_smile:


#83

@everyone THANKS!

@cadaver So far so good on the MIT, happy to share the same license along with the code! Major bump in traffic, being picked up places like Phoronix, with a Urho3D shoutout (though he lost the link, boo!) phoronix.com/scan.php?page=n … ine-Opened

@hd_ Those are some good insights and definitely factors, I would say that Urho3D has a higher code standard. We’ve been in more of a “production” mode on Atomic, though it does have a higher standard than much stuff I have seen :wink:

@weitjong I wrote Atomic’s CEF3 integration, been doing browser integrations a long time. Here’s one I did when I worked at GarageGames on Torque3D in 2009: youtube.com/watch?v=ymI80FKBXfE

@gawag We have nothing to do with that Thunderbeast Entertainment site, and “they” is I, and I am on the forums, so if you have a question might just be easier to ask than make so many assumptions :smiley:

@Shylon Indeed, however I have very precise ideas on the tech and how we’re going to build games with it. Lots of stuff in Atomic wouldn’t make sense to PR against Urho3D (and vice versa). There isn’t any way we would have gotten to this point in around 16 months if we had taken any other approach. THUNDERBEAST GAMES LLC has 1 coder, that’s me, and some awesome contributors

I’d like to make it easier to move code between the projects for sure, one snag is that at one point Urho’s “Engine” namespace was renamed to “Urho3D”, which had a cascade effect on Atomic as all our stuff and script bindings were in the Engine namespace, so the Urho3D namespace became the Atomic namespace.

  • Josh

#84

Don’t get me wrong. I wasn’t implying anything with my earlier comment. I just want to say that I have seen the CEF3 integration (using Urho3D as the game engine, of course) the first time there.


#85

[quote=“cadaver”]
If we speak realistically, Urho / Atomic is a good implementation of game engine state of the art from a few years back. It’s certainly usable for getting games done, and the architecture is solid. However it’s not necessarily suitable for the more modern and high-performance methods (Data-oriented design, more utilization of CPU cores, getting good utilization of new to-the-metal graphics APIs). Though I can’t think of any other permissive open source game engines that are ready for that either (*), and most proprietary engines, at least those with a long legacy probably aren’t any better. What I’m saying is that if your goals are high, Urho may not be the “final” open source base to build on so to speak, but you’d be better off starting a new “god” engine project :slight_smile:

(*) Ogre2+ is heading in a future-prepared direction, but it’s just a rendering engine. It makes choices that are oriented around the “AZDO” approach (such as creating large vertex buffers and managing them manually) that may not be universally good choices, for example on mobiles. From what I read the focus on optimizations makes the API hard to use.[/quote]
So “modern engines” use a faster but harder to use approach? Sounds bad for beginners / non-expert game programmers. Generally and optimally one wants a good mix between usability and performance.
Could the approaches be mixed? Like in doing for example a high performance part like vegetation with a more Data-oriented design and normal stuff where one doesn’t need every bit of performance where usability is more important? Like in putting the hard to use but faster stuff in libraries.

How high is the cost for being more “traditional” and “user oriented”? Like 10% slower? 20%? 30%?


#86

Personally I think it is easier compare to 10 years ago, see C++ now with c++11 or 14 it is like python, see GDC and CPP conference engineers all shares their tech and let people know for new techniques, hard part is inventing a new ways, see for example Unreal implemented PBR/PBS and now engine without PBS is not acceptable for most people. :slight_smile:


#87

With the community we have i believe its possible to keep up with allot of they key engine tech such as PBR, the bigger question is where it fits into Atomic and Urho as they each have there minimal target platforms and some of the techniques wont work on these platforms for example getting Screen Space Reflections on mobile and web platforms is extreamly slow. To keep up with everything we will need developers to devote most of there time to areas of the engines for example im happy to dedicate myself to rendering.


#88

[quote=“gawag”]
So “modern engines” use a faster but harder to use approach? Sounds bad for beginners / non-expert game programmers. Generally and optimally one wants a good mix between usability and performance.
Could the approaches be mixed? Like in doing for example a high performance part like vegetation with a more Data-oriented design and normal stuff where one doesn’t need every bit of performance where usability is more important? Like in putting the hard to use but faster stuff in libraries.

How high is the cost for being more “traditional” and “user oriented”? Like 10% slower? 20%? 30%?[/quote]
Any percentage figures are extremely hard or impossible to give. Note that from what I understand I’d put both Unity and Unreal4 in the same category as Urho, which rather value usability over super performance, and also have “fat” entities/components with logic inside them (Of course UE4 does some advanced stuff rendering-wise, and has more features all around.)

Some examples of a data-oriented & threading friendly, but harder for user operations, would be raycasts and navigation. Traditionally these execute synchronously in the main thread, causing immediate framerate drop or stall if they are too slow, but the API use is very simple: you get the result immediately. In an “advanced” engine you would submit a task to do a raycast or navigation query, these get taken for execution by the job scheduler, and you get the result sometime later, without stalling the main thread.

If you’re interested, read up on console developers’ engines, for example Naughty Dog’s. They typically had to take up multicore, memory and SSE related optimizations in their proprietary engines into much greater use earlier, due to for example the PS3’s architecture.


#89

[quote=“cadaver”]
Some examples of a data-oriented & threading friendly, but harder for user operations, would be raycasts and navigation. Traditionally these execute synchronously in the main thread, causing immediate framerate drop or stall if they are too slow, but the API use is very simple: you get the result immediately. In an “advanced” engine you would submit a task to do a raycast or navigation query, these get taken for execution by the job scheduler, and you get the result sometime later, without stalling the main thread.

If you’re interested, read up on console developers’ engines, for example Naughty Dog’s. They typically had to take up multicore, memory and SSE related optimizations in their proprietary engines into much greater use earlier, due to for example the PS3’s architecture.[/quote]

So a big point is getting stuff out of the rendering thread?
Can Urho be “easily” used to calculate render independent stuff like AI and physics in different threads? Could that be made possible/easier?

When programming an AI I think one doesn’t really care about the rendering loop and more about checking positions of enemies, players and other things and about the navigation mesh. Can separate threads change the scene like move nodes or apply forces to physical objects? Could be tricky to schedule every small operation in an efficient way, but could be possible. What can Urho do currently in that direction?


#90

Urho does the same as Unity which allows to put script / logic hooks to physics update substeps (FixedUpdate), which makes it rather hard to implement physics in a separate thread. If we disallowed that, then it could be threaded, provided there’s added synchronization, but again it’s a performance <-> usability tradeoff.

When you program in C++ you can go crazy with your own background-threaded algorithms, especially if they don’t need to modify the scene, or can for example post their updates to some queue handled by the main thread.

As a generic mechanism for threaded AI or logic execution inside the engine, especially when accounting for scripting, I don’t have an idea how it should or could be done.

EDIT: for those interested, I looked a bit more in detail how UE4 does things:

  • There is a game logic thread and a render thread. Both can submit jobs for additional worker threads,
  • Only render thread actually handles GPU objects. Game logic thread must pass “commands” to modify them.
  • For rendering, “proxy” objects exist of renderable game objects on the render thread side. This and the point above is a significant complication compared to e.g. Urho
  • Physics runs concurrently with logic on another thread. By default physics is variable framerate and thus FPS affects results. You don’t get “fixedupdate” callbacks for logic, but optionally fixed time substepping can be enabled. I didn’t check in detail how the logic and physics results are merged (e.g. at what point in time a command like “add force to game object” is applied to the physics engine.)

#91

That hooking vs. threaded physics and AI option could be switched with a CMake option and defines for 0 runtime cost and proper compiler errors (non existing functions). Or those functions are actually called in separate threads… I still would suggest an optional option as one has to pay more attention when working asynchronously.

[quote=“cadaver”]When you program in C++ you can go crazy with your own background-threaded algorithms
…[/quote]

I’ve also been thinking about what options there are.
Someone on the IRC posted this technique: panda3d.org/blog/triple-your-frame-rate/
I’ve seen that technique before somewhere else, it splits the independent rendering steps into different threads to have three “slices”.
Could that be done with Urho as well?

About other ways to multi thread stuff: I came up with two possible ways.
One solution is the job scheduling that you mentioned the Unreal engine uses. Such a job queue sounds rather complicated and like having a real overhead.
The other possible solution I came up with is to partly synchronize threads with mutexes and/or conditional variables. All the stuff that can only be done at a specific moment (when no one else is modifying the scene for example) is done in serial and not parallel. I would have to look into mutexes and conditional variables again but it’s kinda like this:

void NonMainThread()   // multiple threads do this, like for AI, physics or whatever
{
    // plan all operations like pathfinding and the actions to do
    ...
    MutexGuard mg=MutextMainThread.GetMutexGuard();    // wait until the MainThread is ready and lock a mutex via a RAII mutex guard
    // execute all planned actions in this serial part (AKA "synchronized part")
    ...
}   // the mutex guard is destroyed and automatically unlocks the mutex to get back into parallel mode
...
void Update()   // main thread
{
    ...
    MutexMainThread.unlock();  // let every waiting "worker thread" do his planned actions in serial. Every worker thread should
                               // do only one "set" of actions (not unlock, replan and lock again -> possible deadlock).
    MutexMainThread.lock();    // go back into parallel mode
}

I assume the non-main threads can do stuff like scene manipulations if it is synchronized like that with the main thread? Or is there some weird thread ownership like Qt has?

The second method is pretty manual and avoids a possible giant and slow job queue. Also it is more flexible as it can do everything and doesn’t have to rely on special actions queued (depending on the implementation).

All variants I can think of are not that easy/“idiot safe” as a scene change (or another non-thread-safe action) may still be tried in the parallelized part and not the serial part where it is actually safe.

Hehe :stuck_out_tongue:
This interesting “problem” occupied me as well since the multi threading idea came up. Had a partly written reply lying here since then and kept thinking about it.

Is it safe to read from the scene (like node positions or doing raytraces) when the main thread is rendering? The parallel mode has to be entered for every non-tread-safe operation of course. Usually most machines do also have atomic operations where expensive mutexes can be avoided but I doubt we can really use that as most stuff requires multiple operation without someone getting inbetween. BTW: on x86 every read is atomic even without using the special atomic instructions (manually or via C++11 std::atomic).

Edit: I think the synchronizing has to be done with a mutex for every thread that is unlocked by the main thread when it can do one “set”. Has been quite a while since writing my last thread pool or something similar. But unconditional variables had been also required for something in that direction…
Edit2: Oh! I thing the conditional variables are required when one doesn’t want to lock the main thread and just lock or unlock worker threads. Which is not the case here so it should work with simple mutexes. If I’m not mistaken.


#92

We already have a simple job system in Urho which is used by the rendering and animation, and you can also use it for your own tasks. There is no dependency tracking for them though.

In general it should be safe to read the scene, at least if you’re sure that the main thread logic isn’t simultaneously destroying the scene nodes or components you’re accessing. Just be aware the reading a node’s world position can actually trigger modifications when it has to refresh the position from the node hierarchy, but that should converge to the same result even if being called from multiple threads.


#93

Hey cadaver,

You might know of this one already, but I always found this engine’s concept of making everything a task (and the YaTS taskpool) to be really interesting:
http://bouliiii.blogspot.ca/2011/11/point-frag-distributed-game-engine.html
https://github.com/bsegovia/point-frag/blob/master/src/sys/tasking.hpp

Stealing the thread here, but I thought I’d mention it as we’re comparing other engines and their approaches…


#94

Thanks for posting! I’ve been searching for “next-gen” open source engines having e.g. an advanced threading setup, this may well be worth a look.


#95

Raytracing occlusion culling is also interesting, it might be faster in a small buffer with a fast BVH structure than software raster occlusion culling, like his blog suggests and more threadable and more tolerant to higher poly-count occluders. Maybe throwing something like Embree at it might do the trick… :slight_smile:


#96

Hey all, lots of good stuff going on with Atomic. There have been significant updates to the Atomic Editor under Linux over the last couple months. It is still a work in progress, though is getting closer to having a binary distribution :slight_smile:

We have improved TypeScript project support, including compilation of TS in the Atomic Editor!

There has also been work on better API docs, http://docs.atomicgameengine.com/api/modules/atomic.html as well as a bunch of other updates and bug fixes.

There is now a “Master Builds” section on the download page for recent editor builds with Windows, OSX, Android, iOS, and WebGL deployment, with Linux being in progress. This will be automated, however for now it is still best to strategically select a commit :slight_smile: atomicgameengine.com/download/

I really want to apply some focused work on getting the Atomic and Urho source trees closer together, as to facilitate easier code migration between them. This is a pretty big task at this point and probably needs to be broken up into stages.

  • Josh

#97

Hey all,

I’ve been putting a lot of work into C# scripting, here’s a look at realtime C# inspector fields with the Monaco code editor from VSCode, which is integrated via the Chromium WebView :slight_smile:

The editor also integrates with Visual Studio and VSCode and the tooling is capable of generating solutions, here’s the same project in VS:

Atomic supports JavaScript, TypeScript, C#, and languages that ride on JavaScript and C#, like Haxe/CoffeeScript/Etc. This is a look at the TypeScript project support with full intellisense provided by Monaco:

There has also been work on better Android deployment from the editor with support for release apk’s:

Whew! I am looking forward to merging Urho master soon here, it has been some months and a goal is to get the codebases closer together. The graphics agnostic headers will be a huge win as well :slight_smile: :slight_smile: :slight_smile:

  • Josh

#98

I usually say that integrating with existing IDEs instead of reinventing the wheel is better but wow… this does blow away your mind… Not many engines can boast script editor with full autocompletion, syntax highlighting etc… Even if it is made in javascript :wink: Truly amazing work.

As reiterated multiple times in this thread - urho3d could benefit from c++ editor immensely. Since you atomic engine has such thing and engines are basically brothers maybe it would be in the realm of possibility for engines to share at least that part? I bet cadaver would be very happy if that happened especially since atomic editor has way more features and is written in c++.


#99

If the editor code comes from Atomic it still doesn’t change the fact that it would need a (mostly) steadily contributing developer to handle updates, required Atomic -> Urho changes and editor issues. I assume it will be less work than writing an editor from scratch, though.

However what makes greatly sense to me is to think of Atomic as the IDE and project management built on top of Urho, instead of reimplementing those in Urho. It isn’t exactly a reality right now since Atomic has grown so different and has a different featureset in some respects, but it could be that way if the codebases are brought closer.


#100

The first version of the editor was solely in C++, though as soon as we had mature UI script bindings and especially TypeScript support, the UI was rewritten in TS. Atomic has a “ToolCore” library which is shared by command line tooling and the Atomic Editor. So, asset handling, etc are in C++ and steered by script.

Atomic is geared to be a production tool for shipping apps and games. This means we have deadlines that Urho does not and I don’t foresee Atomic and Urho merging. Though, it should definitely be easier to move code between the two, or other forks of them.

Urho has proven a great base to build on and even after all this crunch, I still love working with the code. I am so going to make a game with all this stuff! :smiley: