Add support for Rust v0 symbol mangling scheme

[cerisier]

Attempt to tackle #474 as per the discussion in the issue thread. Fixes #474.

Until now, demangling was handled by shipping emscripten compiled javascript as a blob and the exact details of how it was compiled had been partially lost, only that it used part of https://github.com/nattofriends/c-filtjs.

In order to support Rust demangling, it has been discussed that we'd take the opportunity to include the source code as part of speedscope and document the process.

This PR follows several guidelines:

• Use the same library as c-filtjs was using (GNU libiberty from gcc which is written in pure C).
• No additional runtime dependencies.
• Keep module loading lazy (since demangling is not always needed)
• Optimize for bundle size.

The approach I took is to write a wrapper demangle function in C which routes demangling to the proper demangler.
It uses cplus_demangle_v3 for theItanium C++ ABI and rust_demangle for Rust.
I followed the same logic as llvm::demangle for routing.

This file is then compiled by emscripten into a single JavaScript file embedding the .wasm file.

Improvements

While working on this, I realized that the implementation within llvm support more cases.
I learned about the existence of symbols with up to 4 _ like ____Z which are not handled by libiberty directly.
One improvement could be to use llvm::demangle rather than GNU libiberty and support more cases.

Swift demangling

Swift also has its own mangling scheme and while profilers like Instruments already demangle Swift symbols on macOS, this support is inexistant on Linux. Adn since Swift is being used more and more outside of macOS, it is likely that we'll start to see perf traces of Swift programs recorded on Linux which wouldn't get demangling directly.

[jlfwong]

Was this necessary as part of this change? I'm not opposed per-se, but I don't like upgrading dependencies as part of PRs that introduce other changes in case they need reverting

[cerisier]

Fully understand. I added the explanation in the commit. Sadly the emscripten generated code is a raw JavaScript file that contains features that are not supported by That version of parcel-bundled hence why I had to upgrade it.

[cerisier]

This is required because emscripten glue code now uses es6 features
like logicalAssignment "??=" which is only supported by newer version
of parcel.

[jlfwong]

Hmmm is there any other way of sanely auto-detecting this? __Z is so rare as a prefix it seems unlikely to yield false positives, but _R and _Z seem far more likely and I'd like to avoid incurring this loading cost unless needed

[cerisier]

I don't see any other way sadly.

Unless mistaken, _Z is the default mangling prefix and __Z happens mostly on macOS where symbols get an extra _.

I tested with a raw undemangled C++ perf trace on Linux and speedscope didn't demsngle it. I added this as a result but should have made a separate commit to explain.

[cerisier]

Referencing rust-lang/rust#60705 for cross visibility :)

[jlfwong]

The import was above intentionally at the top of the file to cause eager, non-blocking loading of this module. As written here, the load of this module will only begin after a profile import, which will be much slower.

In this case, I think we want both the import and the loadDemangling to happen eagerly, and then to await the promise in here.

I think the right way of doing that is to eagerly do the import here, and to eagerly create the wasm module in demangle.ts

[cerisier]

From what I read, demangle is already imported eagerly by application.tsx no ?

As for demangle.ts, is it fine if I do this ?

diff --git a/src/lib/demangle/demangle.ts b/src/lib/demangle/demangle.ts
index b69adeb..70f17a0 100644
--- a/src/lib/demangle/demangle.ts
+++ b/src/lib/demangle/demangle.ts
@@ -1,11 +1,13 @@
 import createWasmDemangleModule from './demangle.wasm'
 
+const wasmDemangleModulePromise = createWasmDemangleModule().then((module) => module)
+
 const cache = new Map<string, string>()
 
 export async function loadDemangling(): Promise<(name: string) => string> {
   // This function converts a mangled C++ name such as "__ZNK7Support6ColorFeqERKS0_"
   // into a human-readable symbol (in this case "Support::ColorF::==(Support::ColorF&)")
-  const wasmDemangleModule = await createWasmDemangleModule()
+  const wasmDemangleModule = await wasmDemangleModulePromise
   return cached(wasmDemangleModule.wasm_demangle)
 }

That way, everything is eager from application.tsx.
import ("./demangle") from profile.tsx is awaiting the eager loaded import from application.tsx
and loadDemangling in profile.tsx just await the eager loaded wasmModule triggered by the eager loding of demangle from `application.tsx.

[jlfwong]

It looks like there's a bunch of stuff downstream of the parcel upgrade that's otherwise unrelated to this change. I'm going to fix it, then ask you to rebase this on top of that

[cerisier]

@jlfwong I have an idea that might save us from upgrade parcel. Basically babelize the generated emscripten code so that it's compatible with whatever javascript version is compatibel with parce-bundler at 1.12.4.

Either from the Makefile directly or as part of the bundling. I'll make a test before suggesting a change here.

[jlfwong]

@cerisier that would be appreciated! I'll need to deal with the bundling problems eventually regardless, but would love to land your changes without dealing with that first.

Rather than babel to do this, I might suggest using esbuild in transform mode. You can set the target property or the supported property to achieve this: https://esbuild.github.io/api/#supported

Something like...

esbuild --target=chrome79 --loader:ts emscripten-output.js

[cerisier]

Wonderful I'll do that

[cerisier]

@jlfwong done. I reverted your PR about upgrading parcel in the mean time. Let me know if you want to do it on main first and I'll just re-revert it here.

[jlfwong]

@cerisier Thanks -- I reverted in main. Can you rebase please to make this PR diff easier to read?

[jlfwong]

In the process of trying to fix this problem a different way by switching to esbuild, I realized ran into another challenging requirement: the ability to load things off of file:/// URLs.

This hasn't been an issue to date because speedscope has used neither wasm nor <script type="module" /> anywhere.

Switching to esbuild necessitates module scripts for code splitting, but I can work around it by not using code splitting for the local file:/// builds. This also means that the wasm module introduced in this PR must be inlined as a base64 string for the non-code-split build, since instantiateStreaming and friends will refuse to load from file:/// local builds.

I think this is all resolvable, but a bit of added pain. See #295 for more detail

EDIT: Skimming through the code in this PR, it looks like the wasm file contents are already base64 encoded in here. So, false alarm!

[cerisier]

@jlfwong done

[cerisier]

Prettier fixed

[coveralls]

coverage: 43.35% (-0.02%) from 43.366%
when pulling eb6926d on cerisier:cerisier/rust-demangling
into c04a148 on jlfwong:main.

[jlfwong]

Sorry for all the flux -- I landed the change to migrate to esbuild and deployed everything. Can you try rebasing one more time and make sure everything looks good? You can also now remove the steps needed to cross-compile with esbuild to get rid of the ?? since now we're using esbuild as the main bundler here, so parcel 1 is gone entirely.

[cerisier]

@jlfwong done !

[cerisier]

✅

[jlfwong]

Thanks! Will deploy later today

[jlfwong]

@cerisier How do I easily create a profile which has symbols mangled in this way? I'd like to test locally before I deploy this to make sure it's working in all the various environments speedscope runs in

[cerisier]

I deleted the example trace I was testing this with :(
I'll have to make a new one unless you get it faster.

[cerisier]

@jlfwong https://gist.github.com/cerisier/c74eb3e5e55a08a5fdf443b21818db40

[jlfwong]

@cerisier Thanks! Was able to test it, everything looks good, and this is now deployed on https://www.speedscope.app/ and on npm as [email protected]