You need a JS trampoline to call your WASM and make browser primitives available to it, and IIRC calls into browser code incur some extra overhead, but those are pretty manageable.
Additionally: for high level languages, source code is _much_ smaller than compiled binaries. If your initial needs are simple, your users are likely downloading more than 10x as much code.