profile
viewpoint

Ask questionsText decoding performance abysmally slow.

This issue was originally focused on file reading performance but it was brought to my attention that the issue is in fact with the text decoding, and that the file read operation does not take very long. This issue does still directly impact reading large files as strings, though.

The following paragraph is from the original issue, and still holds true to an extent. I have updated the rest of the issue to talk about the text decoding, though.

The file reading performance of Deno is, well, abysmal. On my machine it takes 9.25 seconds to read a 25mb file using deno_std's fs readFileStrSync function, and additionally, while it's reading the file it pins CPU usage at 100%+. For comparison, perl5 reads the same file in 0.025 seconds.

I have updated the Repro code below to be a test of the text decoder speed. When you run the code, note the 100%+ CPU usage.

Repro code: Deno:

'use strict';

const decoder = new TextDecoder('utf-8');
const data = new Uint8Array(new Array(25e6).fill(65)); //25 mb of ASCII capital letter 'A";
console.log('Starting decode...');
const start = performance.now();
decoder.decode(data);
console.log('Decode finished in ' + ((performance.now() - start)/1000).toFixed(3) + 's');

Example output: Decode finished in 6.134s

There must be something wrong somewhere with the file reading text decoding for it to take so long (and to pin the CPU usage at 100%+ while it's at it). Until this is fixed I can't really continue working on my project in Deno which involves saving and loading significant amounts of data, because as it is, with ~30mb of data between two files it's taking 10+ seconds to load.

% deno -v
deno: 0.21.0
v8: 7.9.304
typescript: 3.6.3

macOS Catalina 10.15 Beta

denoland/deno

Answer questions kitsonk

Hmmm... it is going to be tricky to do this fast given our current op model. There is the zero copy buffer, but Rust doesn't generate JavaScript strings, which is what we need to end up with. The way this is done in the likes of Chromium is injecting the decode function into the isolate, which then utilises native C++ to perform it.

The other option is to author it in Rust, transpile it to WASM and utilise that in the runtime.

It would be good to add this to the benchmarks, because obviously string encoding and decoding is pretty important.

Also, should do more research into other JavaScript implementations that might be more performance tuned. We maybe doing something silly.

useful!

Related questions

{WSL 2} Permission denied (os error 13) hot 1
gRPC in Deno? hot 1
deno remove/uninstall subcommand hot 1
Insight required: Resource (TCP) errors hot 1
TCP accept loop doesn't use for-await hot 1
Support d.ts files hot 1
Restore runtime lib generation capability hot 1
disable flaky tests _048_media_types_jsx and _019_media_types hot 1
Typescript Custom Transformers Support hot 1
reorg directory structure hot 1
Centos 7 GLIBC_2.18 not found hot 1
"deno ast script.ts" hot 1
"deno ast script.ts" hot 1
Can't build master hot 1
Can't build master hot 1
Github User Rank List