There is something important in the VFX industry, called Katana, that is used for production pipelines somehow. It does Python and Lua. A user comments on what he ends up doing:
After 3 years of Python scripting, I finally reached that point where I can confidently say that I roughly know, what I am doing. Then I fall in love with Katana and while interface and node graph stuff can be accessed/modified via Python, OpScripts are – for performance reasons – Lua based.
Well, off to something new then.
Someone wrote a tech piece on why you choose Python, Lua, or C++ when working with Katana. Her findings:
PYTHON - PERFORMANCE CONSIDERATIONS
Where faster performance is required, Python isn’t always an ideal choice (partly due to the dreaded GIL).
In the context of parameter expressions, a faster alternative to Python expressions is available for simple expressions that reference nodes or parameters. This is called Reference Expressions, please follow the link to the Katana Developer Guide for further information.
Previous releases of Katana have suffered from stability issues when running Python-based AttributeScripts and asset plug-ins out-of-process through the so-called Katana ProcessManager.
While the stability of ProcessManager was improved in the Katana 2.5 release (see TP 128448 in the Katana 2.5v1 Release Notes), the performance of Python especially in the context of scene evaluation is problematic.
Lua offers better performance, making it a preferred scripting language for scene graph processing operations using OpScript nodes.
Lua is used within the OpScript node in Katana. Using OpScript/Lua it’s possible to access the Op API, which is both faster and more powerful than Python. In particular, the OpScript node allows you to modify the structure of the scene graph hierarchy, such as deleting locations, creating new child locations as well as setting and editing attributes.
Katana is Lua 5.1 or LuaJIT 2.1 (presumed beta3). LuaJIT is default:
Switches the Lua interpreter used by the OpScript node to either Lua 5.1 (if set to ‘Lua_5_1’) or LuaJIT 2.1 (if unset, or set to ‘LuaJIT_2_1’). LuaJIT provides better performance.
My takeaway from all of this, is you can do scene graph level magic with LuaJIT that you simply can’t do with Python. But whether you can pull it off in Lua 5.4, is unknown. Lua is a simpler programming model than Python. You don’t worry about the GIL, you don’t have to do any elaborate dance to structure your game around it. You can make a highly threaded scripted game if you want to. Katana would seem to be proof of concept of this, their sales literature is all about performance stuff. But in choosing Lua, you do have to deal with Lua, which is “weird”. Not bad, but different.
Blender devs believe that “Python Modifiers / Compositing / Sequence Effects” are AntiFeatures that should never be included in Blender:
Python is not well suited for fast interactive operations on large data sets such as pixels or vertices.
Even in cases where computation can be optimized, Python has a global interpreter lock (GIL), making any operations which use it single threaded.
Using LLVM may be an option (as we already have for OSL), this would be long term project which needs to be carefully integrated into Blender’s architecture.
So again, this is something that Lua can do, that Python cannot.