Interest in Lua 5.4?

I have to decide to ride either the Lua horse or the Python horse. Even if I wanted both, it is only sane to implement one before the other. I don’t start things unless I can finish them. Finishing one, the question is, what’s the payoff? If that starts yielding dividends, what is the need and impact upon the other? They’re not orthogonal. Do I try to infect Urho3D with Lua 5.4 culture, or with Python 3.x culture? Leading with one, affects the level of effort that will be put towards the other.

What I wanted, was to do Jai culture, or my own language culture. But neither are shipped, and I must get on with real work now.

I do not currently understand all the implications of the GIL. I do know that there’s a serious issue to overcome, that the entire game industry has spoken about this. I think I will look at some 2D Python 3.x stuff, and see if I can figure out where it “falls over”. See if Blender has benchmarks and profiling too.

1 Like

Implications of GIL: only one thread can execute python bytecode at one time. This means that calls to a long-running native function can release GIL enabling other python code on another thread to run. I still do not see how we could do multithreaded scene updates from scripts.

I was thinking more like 20 complex bots entirely scripted, in either Python or Lua. Or user written AI analysis of a Civ-style map, entirely in Python or Lua. Might be piggish, but easy for someone to write up, especially modders. If you had the cores, why not do it? “Interesting” if one language allows this and the other doesn’t.

There is something important in the VFX industry, called Katana, that is used for production pipelines somehow. It does Python and Lua. A user comments on what he ends up doing:

After 3 years of Python scripting, I finally reached that point where I can confidently say that I roughly know, what I am doing. Then I fall in love with Katana and while interface and node graph stuff can be accessed/modified via Python, OpScripts are – for performance reasons – Lua based.

Well, off to something new then. :stuck_out_tongue:

Someone wrote a tech piece on why you choose Python, Lua, or C++ when working with Katana. Her findings:

PYTHON - PERFORMANCE CONSIDERATIONS

Where faster performance is required, Python isn’t always an ideal choice (partly due to the dreaded GIL).

In the context of parameter expressions, a faster alternative to Python expressions is available for simple expressions that reference nodes or parameters. This is called Reference Expressions, please follow the link to the Katana Developer Guide for further information.

Previous releases of Katana have suffered from stability issues when running Python-based AttributeScripts and asset plug-ins out-of-process through the so-called Katana ProcessManager.

While the stability of ProcessManager was improved in the Katana 2.5 release (see TP 128448 in the Katana 2.5v1 Release Notes), the performance of Python especially in the context of scene evaluation is problematic.

Lua offers better performance, making it a preferred scripting language for scene graph processing operations using OpScript nodes.

LUA

Lua is used within the OpScript node in Katana. Using OpScript/Lua it’s possible to access the Op API, which is both faster and more powerful than Python. In particular, the OpScript node allows you to modify the structure of the scene graph hierarchy, such as deleting locations, creating new child locations as well as setting and editing attributes.

Katana is Lua 5.1 or LuaJIT 2.1 (presumed beta3). LuaJIT is default:

KATANA_OPSCRIPT_INTERPRETER
Switches the Lua interpreter used by the OpScript node to either Lua 5.1 (if set to ‘Lua_5_1’) or LuaJIT 2.1 (if unset, or set to ‘LuaJIT_2_1’). LuaJIT provides better performance.

My takeaway from all of this, is you can do scene graph level magic with LuaJIT that you simply can’t do with Python. But whether you can pull it off in Lua 5.4, is unknown. Lua is a simpler programming model than Python. You don’t worry about the GIL, you don’t have to do any elaborate dance to structure your game around it. You can make a highly threaded scripted game if you want to. Katana would seem to be proof of concept of this, their sales literature is all about performance stuff. But in choosing Lua, you do have to deal with Lua, which is “weird”. Not bad, but different.

Blender devs believe that “Python Modifiers / Compositing / Sequence Effects” are AntiFeatures that should never be included in Blender:

Python is not well suited for fast interactive operations on large data sets such as pixels or vertices.

Even in cases where computation can be optimized, Python has a global interpreter lock (GIL), making any operations which use it single threaded.

Using LLVM may be an option (as we already have for OSL), this would be long term project which needs to be carefully integrated into Blender’s architecture.

So again, this is something that Lua can do, that Python cannot.