Migration from custom container library to augmented EASTL

Okay, I’m gonna extract this question into separate thread to keep discussion on-topic.

Currently there’s very good opportunity to move Urho from custom containers to proper 3rd party container library. Most of the porting is already done.

The issue is that this change is breaking. There are adapters that covers common cases but they don’t provide 100% backward compatibility. And names are not in CamelCase /outcry of shock/.

What this migration offers:

  • Container library maintained by companies instead of custom-made library that nobody maintains;
  • Move-semantics friendly containers;
  • Containers that don’t allocate memory dynamically;
  • Algorithms;

Why one may want this change?

When a person is used to work with std container library in its entirety (more than just vector.push_back), using custom Urho Containers becomes a pain. There’re so many things missing. I have refactored Urho3D::Vector<T> two times when I needed something from it and it’s not even close to “enough”.

Small buffer optimization? Nope. Allocators? Nope. Algorithms? Nope. Containers except vector and hash map? Nope. Move semantics? Barely. Spans and string views? Nope.

Urho is outsourcing physics, navigation, image decoding and tons of other tasks. Why it cannot outsource container library too?

So, what do you think? Is it worth it?

  • [supportive] I want to switch to EASTL or std library.
  • [supportive] I don’t need any features from EASTL but I’m ok with switching to it.
  • [against] I don’t want to switch to EASTL but I have a proposal how to make Urho Containers more convinient to use.
  • [against] I don’t want to switch to EASTL and I have nothing to suggest.

0 voters

Are there any tests comparing Urho and EASTL containers? As far as I understand, Urho3D containers are designed to binding with scripting languages and work fast. Who forbids users to use their own containers in their projects? I am not against progress, however, the loss of binding to scripting languages is a regression. C # cannot replace scripting languages because you cannot quickly call managed code from native code. You can only quickly call native code from managed code.

1 Like

When I checked last time, script bindings are not heavily dependent on exact container types. They will need some revise, but I think it’s doable.

About performance… measuring performance properly is hard task that requires time and effort. I may try it when I have time for it.

1 Like

I’m totally voting up for EASTL, I love how Urho3D containers work right now, but the couple of times I had to extend with functionality that’s already in STL I felt like I was failing a job interview. I don’t mind spending 2 days refactoring my code replacing .Push with .push_back.

1 Like

If this change is decided upon, it would probably be best to make it part of Urho 2.0.

Does this mean you are suggesting to wrap EASTL in Urho classes too? Could there be some backwards compatibility this way or engine modifications averted?

1 Like

Yeah, great idea.
Aprove this here to pave the way for getting rbfx in regardless.

Don’t mix topics.
This topic is about this specific feature that I personally missed in Urho for years.

Here is specific scenario:
People want to use Urho. Not some fork, but the master branch. I’m one of these people, but there are more. This is given.
People want to have decent container library in Urho. I’m one of these people, but there are more. This is given too.

What solution do you offer? I expect constructive answer here.

Could you guys pro stl changes divide the issue a little bit? Pls explain the eastl introduction to Urho3D a little technical detailed?

As a proposal, shall we list merits and demerits? Challenges and opportunities? Future plans?

There will be a lot of code changes to Urho as well as your own code. If we gain more, only cost the unification and aesthetics of the engine, its opportunity.

I like the current containers, since they are just simple and good for my usage. But obviously, many people ask more and have their own needs and pains.

Edit: I have to point out that, rbfx’s Editor is not because of the container changes, it is mainly because of the auther’s hard working. I really respect that. So, I don’t want to mix the rbfx’s features with the introduction of stl.

There are many people who are silent in this community, including the leaders and ex-leaders, they may have their own private fork or workflow, or have quit using Urho, but for the containers change, please stand out and speak out.

2 Likes

Writing huge posts is always taxing. I may do it when I get energy for it.
// TODO:

1 Like

I support this idea unless it turns Urho into a gargantuan giant monster where opening the project in the IDE takes 10 minutes , the building process takes 1 hour and the resulted binary is over 30 mb … Also i expect smooth out of box Cmake scripts without any hassle over fixing broken cumbersome scripts. Right now the cmake scripts (at least on windows) are perfect !! I download Urho3d , build it and start working on my game.

3 Likes

Sorry I missed your reply here.

It would be a waste of performance and manpower to do it.

I understand beauty of having all code in single CamelCase style and whatever.
However, one-line trivial wrappers like we have for Sqrt is weird thing to do.

There are CamelCase adapters in augmented EASTL containers to simplty porting. They are disabled by default and do not cover everything.

I too am quite fond of this consistency, and if it would break over this, I think it would be better to use std (to avoid the road to Boost) and change the Urho code style accordingly or leave things the way they are.

If the community is convinced, as long as EASTL itself alone does not break things too much, we may provide some scripts/way to help people migrate their own projects.

As for AngelScript, can we make things automatic? like SWIG for C#? So we just keep lua, AngelScript and scripts based Editor, as long as we don’t have a better alternative yet so discarding any of the parts are not considerable. If people love C#, use it.

So, everyone is happy and Urho is stronger. Am I asking too much (based on the community size and contributor number)?

So you are considering only there two options: break all API compatiblity or do nothing.
This is quite unusual developement position.

I made quick benchmark, feel free to criticize because I’m totally not an expert in benchmarking.
https://gist.github.com/eugeneko/2cbad3b16ca2f8e250d57a17c2d4d207

Run on (12 X 2208 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x6)
  L1 Instruction 32 KiB (x6)
  L2 Unified 256 KiB (x6)
  L3 Unified 9216 KiB (x1)
----------------------------------------------------------------------
Benchmark                            Time             CPU   Iterations
----------------------------------------------------------------------
Trivial_StringVector_EA        1190020 ns      1199777 ns          560
Trivial_StringVector_Urho      5742792 ns      5719866 ns          112
Trivial_StringHashMap_EA       1692161 ns      1689189 ns          407
Trivial_StringHashMap_Urho     6291083 ns      6277902 ns          112
Real_StringVector_EA           1441662 ns      1443273 ns          498
Real_StringVector_Urho         6115403 ns      5998884 ns          112
Real_SmallStringVector_EA       355737 ns       352926 ns         1948
Real_SmallStringVector_Urho    2116384 ns      2128623 ns          345
Real_HugeHashMap_EA          385838650 ns    390625000 ns            2
Real_HugeHashMap_Urho       1074992600 ns   1078125000 ns            1
5 Likes

You are suggesting to break both compatibility and consistency.

One concern I have with the benchmark is that it may be more of a test of the incremental creation of the strings than of the actual containers. What is you used the same object type for all of the containers (maybe key type as well for the maps, but I understand that might be an issue since the hash function needs to exist).

Also, the suggestion is to break comparability and internal Urho consistency in exchange for a (potentially) better preforming set of containers (and likely greater comparability with external library’s that likely use the stl containers (which I believe share the same function names as EASTL, though not the same classes)).

Proposed compatibility changes differ in scope by the orders of magnitude. Smaller breaking change is always better than bigger breaking change, unless you have strong arguments for doing bigger change.

Consistently is weak argument. Urho way to wrap everything is perfectionism borderline to mental disorder. We have garbage code that is suboptimal and need to be maintained just to have fancy names.

Cmon, why on earth do we need Sqrt? Especially considering that we don’t wrap everything because it is impossible to wrap everything. So Urho is already inconsistent. The moment user wants to use anything from standard library, consistency is gone.

So what if we return vector instead of Vector? We already return bool and not Bool, and we use initializer_list instead of… whatever.

1 Like

Thanks! That’s really good point, I knew I missed something.

I tried it with ints.
I found one specific case when Urho wins.

Trivial_IntHashMap_EA       1538931 ns      1568775 ns          498
Trivial_IntHashMap_Urho      769266 ns       784738 ns          896

Small hash maps (e.g. 1000 ints) are qucker to fill due to allocator in Urho hash map.
I may try to write EASTL allocator to mimic this behavior, but I don’t wanna do it now.

In the rest of cases Urho is either same or slower.

Changing the coding style together with switching to a container library would remedy this while upholding consistency.

@Eugene Could you include std containers in your tests?

You know, I don’t oppose the idea of snake case itself, since it’s native code style of c++. Given choice, I would have probably started new library in this style.

But the idea of global style change in Urho is unviable due to many reasons I don’t even care to mention.