Experimental web build using upstream LLVM WASM backend

Today I have updated the Web DBE for Urho3D to switch from Fastcomp backend to upstream LLVM backend. This latest LLVM upstream can be installed and activated explicitly using EMSDK. It is not the default yet. I am using EMCC 1.39.0 in my experiment. With a few manual tweaks on the CMake build script, I was able to use the new compiler toolchain to compile the Urho3D library, however, it failed to link the library with the sample targets. This was the linker error I got for STATIC build config. Similar assertion error happened for SHARED build config too.

[100%] Linking CXX executable ../../../bin/Urho3DPlayer.html
wasm-ld: /b/s/w/ir/cache/builder/emscripten-releases/llvm-project/lld/wasm/SyntheticSections.cpp:468: virtual void lld::wasm::LinkingSection::writeBody(): Assertion `isec->getComdatName() == comdat' failed.
Stack dump:
0.	Program arguments: /emsdk-master/upstream/bin/wasm-ld -o ../../../bin/Urho3DPlayer.html --allow-undefined --lto-O0 -L/emsdk-master/upstream/emscripten/system/local/lib CMakeFiles/Urho3DPlayer.dir/Urho3DPlayer.cpp.o -L/emsdk-master/upstream/emscripten/system/lib ../../../lib/libUrho3D.a --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -z stack-size=5242880 --initial-memory=134217728 --no-entry --global-base=1024 --relocatable 
 #0 0x00007fec59b09254 PrintStackTraceSignalHandler(void*) (/emsdk-master/upstream/bin/../lib/libLLVM-10svn.so+0x709254)
 #1 0x00007fec59b06fde llvm::sys::RunSignalHandlers() (/emsdk-master/upstream/bin/../lib/libLLVM-10svn.so+0x706fde)
 #2 0x00007fec59b09508 SignalHandler(int) (/emsdk-master/upstream/bin/../lib/libLLVM-10svn.so+0x709508)
 #3 0x00007fec5caea890 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12890)
 #4 0x00007fec5870ee97 raise (/lib/x86_64-linux-gnu/libc.so.6+0x3ee97)
 #5 0x00007fec58710801 abort (/lib/x86_64-linux-gnu/libc.so.6+0x40801)
 #6 0x00007fec5870039a (/lib/x86_64-linux-gnu/libc.so.6+0x3039a)
 #7 0x00007fec58700412 (/lib/x86_64-linux-gnu/libc.so.6+0x30412)
 #8 0x00000000006e7799 lld::wasm::LinkingSection::writeBody() (/emsdk-master/upstream/bin/wasm-ld+0x6e7799)
 #9 0x00000000006d699a lld::wasm::SyntheticSection::finalizeContents() (/emsdk-master/upstream/bin/wasm-ld+0x6d699a)
#10 0x00000000006d0d88 lld::wasm::(anonymous namespace)::Writer::run() (/emsdk-master/upstream/bin/wasm-ld+0x6d0d88)
#11 0x00000000006c9831 lld::wasm::writeResult() (/emsdk-master/upstream/bin/wasm-ld+0x6c9831)
#12 0x00000000006ab67a lld::wasm::(anonymous namespace)::LinkerDriver::link(llvm::ArrayRef<char const*>) (/emsdk-master/upstream/bin/wasm-ld+0x6ab67a)
#13 0x00000000006a61d8 lld::wasm::link(llvm::ArrayRef<char const*>, bool, llvm::raw_ostream&) (/emsdk-master/upstream/bin/wasm-ld+0x6a61d8)
#14 0x000000000041f61b main (/emsdk-master/upstream/bin/wasm-ld+0x41f61b)
#15 0x00007fec586f1b97 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b97)
#16 0x000000000041f1a9 _start (/emsdk-master/upstream/bin/wasm-ld+0x41f1a9)
shared:ERROR: '/emsdk-master/upstream/bin/wasm-ld -o ../../../bin/Urho3DPlayer.html --allow-undefined --lto-O0 -L/emsdk-master/upstream/emscripten/system/local/lib CMakeFiles/Urho3DPlayer.dir/Urho3DPlayer.cpp.o -L/emsdk-master/upstream/emscripten/system/lib ../../../lib/libUrho3D.a --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -z stack-size=5242880 --initial-memory=134217728 --no-entry --global-base=1024 --relocatable' failed (-6)
Source/Tools/Urho3DPlayer/CMakeFiles/Urho3DPlayer.dir/build.make:86: recipe for target 'bin/Urho3DPlayer.html' failed
make[3]: *** [bin/Urho3DPlayer.html] Error 1
CMakeFiles/Makefile2:1426: recipe for target 'Source/Tools/Urho3DPlayer/CMakeFiles/Urho3DPlayer.dir/all' failed
make[2]: *** [Source/Tools/Urho3DPlayer/CMakeFiles/Urho3DPlayer.dir/all] Error 2
CMakeFiles/Makefile2:1438: recipe for target 'Source/Tools/Urho3DPlayer/CMakeFiles/Urho3DPlayer.dir/rule' failed
make[1]: *** [Source/Tools/Urho3DPlayer/CMakeFiles/Urho3DPlayer.dir/rule] Error 2
Makefile:485: recipe for target 'Urho3DPlayer' failed
make: *** [Urho3DPlayer] Error 2

Is there anyone here experiment with the upstream LLVM backend too? Probably I should file a bug report to Emscripten.

3 Likes

It is simpler than I thought to get it working. Just need to bump the CMake minimum required version to something higher than what we have currently. No manual tweaking the build tree and no more linking error. All sample targets built cleanly. Tested the skeletal animation sample locally and it looked good. I didn’t time the build time just now but I feel the link time is much faster, which is what I hope for. This should help to resolve the Web CI build running out of time issue.

4 Likes

With the new backend, the STATIC/WASM build config is so much efficient than before. Previously we could not use all the logical CPU to perform the web build because they would get choked during linking phase, now that is not true anymore (at least on my test). I have now modified the ‘make’ task in Rakefile to take advantage of all the available logical unit automatically, and also altered the Web CI build to use 4 parallel threads instead of 2. The Web CI can now build all the samples (without any exclusion) and still have plenty of time left. That’s the good news.

The bad news are, 1) Emscripten does not support asm.js anymore and 2) the SHARED lib type still hit by the linker error.

From what I have read, the new backend when configured with WASM=0 will emit plain JS as output instead of asm.js. The Emscripten keeps WASM=0 mode for older browsers that do not support WASM (those that support asm.js also support WASM anyway). In my experiment that took forever to complete for our sample build targets. Thus, if there is no objections from others then as part of the migration to the new backend, I will modify Urho3D build system to drop the support for EMSCRIPTEN_WASM=0. Effectively removing that build option all together because WASM will be enabled by default all the way. I will also drop the other experimental build option to build Urho3D as a “MODULE” lib type. It was introduced by me at the time it looked like a nice idea because Emscripten’s linking phase was so slow. So, that will be gone too.

As for the SHARED lib type linker error, I have logged it as an issue in the upstream Emscripten. See https://github.com/emscripten-core/emscripten/issues/9726.

BTW, all the changes are currently only available in my personal fork. I am thinking of merging them into upstream Urho3D before releasing 1.8, despite it maybe a breaking change to some. Again, only when I don’t hear anyone making a noise about this plan.

2 Likes

It works pretty well for my unfinished little Reactor Idle clone experiment. It used to take about 2 or 3 minutes to link, but it links in about 20 seconds now. The execution seems a bit punchier, but that’s pretty hard to quantify in this case since I have made a lot of code changes since I last tested a build. I’m fine with merging the changes, and thanks for your work on this.

1 Like