profile
viewpoint
Bartek Iwańczuk bartlomieju @straal Warsaw, PL Contributing to @denoland, developer at @straal, author of deno-postgres, interested in TypeScript, Rust and Python.

pull request commentdenoland/deno

port ops to JSON (fs)

@ry rebased on top of master. I'm gonna skip fine-trim of msg.fbs for now, once op_read and op_write use minimal dispatch for sync ops whole file can be deleted

bartlomieju

comment created time in 13 hours

push eventbartlomieju/deno

Ryan Dahl

commit sha 5b2baa5c990fbeae747e952c5dcd7a5369e950b1

Port rest of os ops to JSON (#2802)

view details

Bartek Iwańczuk

commit sha 79f82cf10ed1dbf91346994250d7311a4d74377a

port ops to JSON: compiler, errors, fetch, files (#2804)

view details

Bartek Iwańczuk

commit sha 137f33733d365026903d40e7cde6e34ac6c36dcf

port more ops to JSON (#2809)

view details

Bartek Iwańczuk

commit sha f3c783494c22f254125879fb485d9ff55e405164

port cli/ops/fs.rs to json ops, part 1

view details

Bartek Iwańczuk

commit sha d32dfd698adce832ead657fc2118a31754ffa0a1

port cli/ops/fs.rs to json ops, part 2

view details

Bartek Iwańczuk

commit sha 6ce38165d7b27a79cad0d54a08c853f982d66c41

port cli/ops/fs.rs to json ops, part 3

view details

push time in 13 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 137f33733d365026903d40e7cde6e34ac6c36dcf

port more ops to JSON (#2809)

view details

push time in 13 hours

delete branch bartlomieju/deno

delete branch : port_ops_4

delete time in 13 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 960953a26236b3d02f58ffe0791ad01c8e76661a

remove TODO

view details

push time in 15 hours

Pull request review commentdenoland/deno

port ops to JSON (list insde)

 // Copyright 2018-2019 the Deno authors. All rights reserved. MIT license.-use super::dispatch_flatbuffers::serialize_response;-use super::utils::blocking;-use super::utils::ok_buf;-use super::utils::CliOpResult;-use crate::msg;+use super::dispatch_json::{blocking_json, Deserialize, JsonOp, Value}; use crate::repl; use crate::resources; use crate::state::ThreadSafeState; use deno::*;-use flatbuffers::FlatBufferBuilder;++#[derive(Deserialize)]+#[serde(rename_all = "camelCase")]+struct ReplStartArgs {+  history_file: String,+}  pub fn op_repl_start(   state: &ThreadSafeState,-  base: &msg::Base<'_>,-  data: Option<PinnedBuf>,-) -> CliOpResult {-  assert!(data.is_none());-  let inner = base.inner_as_repl_start().unwrap();-  let cmd_id = base.cmd_id();-  let history_file = String::from(inner.history_file().unwrap());+  args: Value,+  _zero_copy: Option<PinnedBuf>,+) -> Result<JsonOp, ErrBox> {+  let args: ReplStartArgs = serde_json::from_value(args)?; -  debug!("op_repl_start {}", history_file);-  let history_path = repl::history_path(&state.dir, &history_file);+  debug!("op_repl_start {}", args.history_file);+  let history_path = repl::history_path(&state.dir, &args.history_file);   let repl = repl::Repl::new(history_path);   let resource = resources::add_repl(repl); -  let builder = &mut FlatBufferBuilder::new();-  let inner = msg::ReplStartRes::create(-    builder,-    &msg::ReplStartResArgs { rid: resource.rid },-  );-  ok_buf(serialize_response(-    cmd_id,-    builder,-    msg::BaseArgs {-      inner: Some(inner.as_union_value()),-      inner_type: msg::Any::ReplStartRes,-      ..Default::default()-    },-  ))+  Ok(JsonOp::Sync(json!(resource.rid)))+}++#[derive(Deserialize)]+struct ReplReadlineArgs {+  rid: i32,+  prompt: String, }  pub fn op_repl_readline(   _state: &ThreadSafeState,-  base: &msg::Base<'_>,-  data: Option<PinnedBuf>,-) -> CliOpResult {-  assert!(data.is_none());-  let inner = base.inner_as_repl_readline().unwrap();-  let cmd_id = base.cmd_id();-  let rid = inner.rid();-  let prompt = inner.prompt().unwrap().to_owned();+  args: Value,+  _zero_copy: Option<PinnedBuf>,+) -> Result<JsonOp, ErrBox> {+  let args: ReplReadlineArgs = serde_json::from_value(args)?;+  let rid = args.rid;+  let prompt = args.prompt;   debug!("op_repl_readline {} {}", rid, prompt); -  blocking(base.sync(), move || {-    let repl = resources::get_repl(rid)?;+  // TODO(bartlomieju): this is a sync operation, why is it async in repl.ts?

Done

bartlomieju

comment created time in 15 hours

Pull request review commentdenoland/deno

port ops to JSON (list insde)

 // Copyright 2018-2019 the Deno authors. All rights reserved. MIT license.-use super::dispatch_flatbuffers::serialize_response;-use super::utils::blocking;-use super::utils::ok_buf;-use super::utils::CliOpResult;-use crate::msg;+use super::dispatch_json::{blocking_json, Deserialize, JsonOp, Value}; use crate::repl; use crate::resources; use crate::state::ThreadSafeState; use deno::*;-use flatbuffers::FlatBufferBuilder;++#[derive(Deserialize)]+#[serde(rename_all = "camelCase")]+struct ReplStartArgs {+  history_file: String,+}  pub fn op_repl_start(   state: &ThreadSafeState,-  base: &msg::Base<'_>,-  data: Option<PinnedBuf>,-) -> CliOpResult {-  assert!(data.is_none());-  let inner = base.inner_as_repl_start().unwrap();-  let cmd_id = base.cmd_id();-  let history_file = String::from(inner.history_file().unwrap());+  args: Value,+  _zero_copy: Option<PinnedBuf>,+) -> Result<JsonOp, ErrBox> {+  let args: ReplStartArgs = serde_json::from_value(args)?; -  debug!("op_repl_start {}", history_file);-  let history_path = repl::history_path(&state.dir, &history_file);+  debug!("op_repl_start {}", args.history_file);+  let history_path = repl::history_path(&state.dir, &args.history_file);   let repl = repl::Repl::new(history_path);   let resource = resources::add_repl(repl); -  let builder = &mut FlatBufferBuilder::new();-  let inner = msg::ReplStartRes::create(-    builder,-    &msg::ReplStartResArgs { rid: resource.rid },-  );-  ok_buf(serialize_response(-    cmd_id,-    builder,-    msg::BaseArgs {-      inner: Some(inner.as_union_value()),-      inner_type: msg::Any::ReplStartRes,-      ..Default::default()-    },-  ))+  Ok(JsonOp::Sync(json!(resource.rid)))+}++#[derive(Deserialize)]+struct ReplReadlineArgs {+  rid: i32,+  prompt: String, }  pub fn op_repl_readline(   _state: &ThreadSafeState,-  base: &msg::Base<'_>,-  data: Option<PinnedBuf>,-) -> CliOpResult {-  assert!(data.is_none());-  let inner = base.inner_as_repl_readline().unwrap();-  let cmd_id = base.cmd_id();-  let rid = inner.rid();-  let prompt = inner.prompt().unwrap().to_owned();+  args: Value,+  _zero_copy: Option<PinnedBuf>,+) -> Result<JsonOp, ErrBox> {+  let args: ReplReadlineArgs = serde_json::from_value(args)?;+  let rid = args.rid;+  let prompt = args.prompt;   debug!("op_repl_readline {} {}", rid, prompt); -  blocking(base.sync(), move || {-    let repl = resources::get_repl(rid)?;+  // TODO(bartlomieju): this is a sync operation, why is it async in repl.ts?

Fair enough, should I rephrase the comment?

bartlomieju

comment created time in 15 hours

pull request commentdenoland/deno

port ops to JSON (list insde)

Merged master and ported metrics.rs as well

bartlomieju

comment created time in 15 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 79f82cf10ed1dbf91346994250d7311a4d74377a

port ops to JSON: compiler, errors, fetch, files (#2804)

view details

Bartek Iwańczuk

commit sha b0c06bfb5f8075f6aca8a506d75910362e035a3a

Merge branch 'master' into port_ops_4

view details

Bartek Iwańczuk

commit sha 4ddb5f5cc0eea539f0bad38e5c4449c7b8adbe0d

port cli/ops/metrics.rs to json ops

view details

push time in 15 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 79f82cf10ed1dbf91346994250d7311a4d74377a

port ops to JSON: compiler, errors, fetch, files (#2804)

view details

push time in 15 hours

delete branch bartlomieju/deno

delete branch : port_ops_3

delete time in 15 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 349c2451e368af18682927dc98aba7ff735b1a98

port ops to json ops * cli/ops/files.rs: * op_open * op_close * op_seek * cli/ops/fetch.rs: * op_fetch * cli/ops/compiler.rs: * op_cache * op_fetch_source_file * cli/ops/errors.rs: * op_apply_source_map * op_format_error

view details

Bartek Iwańczuk

commit sha 8cdeef6531f17a929a8900c00b0f73e61c568cb6

address review

view details

Bartek Iwańczuk

commit sha 5a475bde9fa07383fbc45195b5000980e1dafe74

add type annotation

view details

Ryan Dahl

commit sha 5b2baa5c990fbeae747e952c5dcd7a5369e950b1

Port rest of os ops to JSON (#2802)

view details

Bartek Iwańczuk

commit sha 510e7c900bcb87c7e776ed2bcc4fbff92812d90a

Merge branch 'master' into port_ops_3

view details

Bartek Iwańczuk

commit sha 01e2f817978499903d820a4221dd7c179b206ca8

port cli/ops/repl.rs to json ops

view details

Bartek Iwańczuk

commit sha 6ae7a37e66a6e8865f0d3ee842a775d4153de359

port cli/ops/net.rs to json ops

view details

Bartek Iwańczuk

commit sha 3040260a8f7daba352c9b8fe61fa363c1ba805de

port cli/ops/resources.rs to json ops

view details

Bartek Iwańczuk

commit sha 1b08a369de68ba270e026d7db09ff65e86efdd85

port cli/ops/random.rs to json ops, cleanup

view details

Bartek Iwańczuk

commit sha 0a5365a2aac711804aa180803ad178ec2afb4a82

port cli/ops/timers.rs to json ops

view details

Bartek Iwańczuk

commit sha e60a1c72985aa8db98036208999c404bd09e2241

port cli/ops/performance.rs to json ops

view details

Bartek Iwańczuk

commit sha 79c5a2b00c8580db4d00fa60c501305d9ed410d0

port cli/ops/permissions.rs to json ops

view details

Bartek Iwańczuk

commit sha decc266afadf31281cab9710996d604cea30598e

test fixes

view details

Bartek Iwańczuk

commit sha 93f9a64bd87989a7e3cf7a0829cc545e5f2a8ecf

fix tests 2

view details

Bartek Iwańczuk

commit sha bc5d12ef223ac38ae849d840f6366217f137e5e5

port cli/ops/workers.rs to json ops

view details

Bartek Iwańczuk

commit sha 3087c001924769254e10c4d8624004dc642f22c7

port cli/ops/process.rs to json ops

view details

push time in 16 hours

push eventbartlomieju/deno

Ryan Dahl

commit sha 5b2baa5c990fbeae747e952c5dcd7a5369e950b1

Port rest of os ops to JSON (#2802)

view details

Bartek Iwańczuk

commit sha 510e7c900bcb87c7e776ed2bcc4fbff92812d90a

Merge branch 'master' into port_ops_3

view details

push time in 16 hours

pull request commentdenoland/deno

port ops to JSON (list insde)

Waiting for #2804

bartlomieju

comment created time in 16 hours

push eventbartlomieju/deno

Ryan Dahl

commit sha 5b2baa5c990fbeae747e952c5dcd7a5369e950b1

Port rest of os ops to JSON (#2802)

view details

push time in 16 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 07c0d0862816014ffa38502255df3a717a241edf

port cli/ops/fs.rs to json ops, part 3

view details

push time in 16 hours

PR opened denoland/deno

port ops to JSON (fs)

Based on #2809 (so blocked by it)

+913 -1419

0 comment

33 changed files

pr created time in 17 hours

create barnchbartlomieju/deno

branch : port_fs_ops_to_json

created branch time in 17 hours

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 0c399a52cf991b524729fbe47a2f36eb011276e7

port cli/ops/process.rs to json ops

view details

push time in 18 hours

issue commentdenoland/deno

Convert all Flatbuffer ops to JSON

I think I'm gonna go ahead and port remaining ops from fs.rs

ry

comment created time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 7ae8a5c08578ab206c4d13eabc294defe5a70c79

port cli/ops/workers.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 27166c9a7fc0ac8a346e88eb9843e924f3f23c39

port cli/ops/workers.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha f72d3f4833abe924c0ccbe06579d8cc9c92fa5df

fix tests 2

view details

push time in a day

pull request commentdenoland/deno

Json ops metrics

@romainPrignon I suggest to start afresh from master - it should be easier than rebasing

romainPrignon

comment created time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha d87b4d6ec1b01d2503e43cb4747ab7c3aed34391

test fixes

view details

push time in a day

issue commentdenoland/deno

Convert all Flatbuffer ops to JSON

@ry I believe io.rs shouldn't be ported - it already uses minimal dispatch

ry

comment created time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 5172a72355c9235bd3cf715514a1d5f7b3efbc5b

port cli/ops/permissions.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha ea0a571d3b1d9d0c8821f1515cd1edbd6f757f19

port cli/ops/performance.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 1af60a8c8b8c467e4fc007e2340402bb51ce7df0

port cli/ops/performance.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 9125726b0948e54838d4cfb0c63b3a76f06e201b

port cli/ops/timers.rs to json ops

view details

push time in a day

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 5a475bde9fa07383fbc45195b5000980e1dafe74

add type annotation

view details

push time in a day

issue commentdenoland/deno

How can I make a http client to send request now?

Deno supports Fetch API

andares

comment created time in a day

pull request commentdenoland/deno

port ops to JSON (net, random, repl, resources)

@ry ready for review as well 😅 This PR should be landed after #2804 to adjust op ids properly

bartlomieju

comment created time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 8cdeef6531f17a929a8900c00b0f73e61c568cb6

address review

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 19b922a936184436e1b812c217d38ddb99ee451b

port cli/ops/random.rs to json ops, cleanup

view details

push time in 2 days

Pull request review commentdenoland/deno

port ops to JSON (compiler, errors, fetch, files)

 export function asyncMsgFromRust(opId: number, ui8: Uint8Array): void {     case OP_READ:       minimal.asyncMsgFromRust(opId, ui8);       break;+    case OP_EXIT:+    case OP_IS_TTY:+    case OP_ENV:+    case OP_EXEC_PATH:     case OP_UTIME:+    case OP_APPLY_SOURCE_MAP:+    case OP_FORMAT_ERROR:+    case OP_CACHE:+    case OP_FETCH_SOURCE_FILE:+    case OP_OPEN:+    case OP_CLOSE:+    case OP_SEEK:+    case OP_FETCH:

Ah, makes sense! Will update

bartlomieju

comment created time in 2 days

startedcrookse/deno-drash

started time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 8577cd8e2ba29fdd77095f8005e6bd9be31c48dc

port cli/ops/net.rs to json ops

view details

Bartek Iwańczuk

commit sha b2ce5c4f8340f64d32fb8abbd77fcdc45957e4a8

port cli/ops/resources.rs to json ops

view details

push time in 2 days

pull request commentdenoland/deno

port ops to JSON (compiler, errors, fetch, files)

@ry I rebased this PR on latest master, but it should be landed after #2802 because op ids are incremented from last id used in that PR

bartlomieju

comment created time in 2 days

PR opened denoland/deno

port ops to JSON (net, random, repl, resources)
  • [x] net.ts
  • [ ] random.rs
  • [x] repl.rs
  • [ ] resources.rs
+149 -277

0 comment

8 changed files

pr created time in 2 days

create barnchbartlomieju/deno

branch : port_ops_4

created branch time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 349c2451e368af18682927dc98aba7ff735b1a98

port ops to json ops * cli/ops/files.rs: * op_open * op_close * op_seek * cli/ops/fetch.rs: * op_fetch * cli/ops/compiler.rs: * op_cache * op_fetch_source_file * cli/ops/errors.rs: * op_apply_source_map * op_format_error

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 720b1695125a505a5a66767fbbb2514422fdae47

fmt

view details

push time in 2 days

push eventbartlomieju/deno

Ryan Dahl

commit sha 0809b06a3938868f364f1343b0de4d5d9686495d

v0.16.0

view details

Bert Belder

commit sha 47c216317f8eb5bf277663a732a79f6b07ba79ef

build: enable 'derive' feature of 'serde' crate

view details

Ryan Dahl

commit sha bc467b265fbe06ace24f5d9536bd8eb36ae4a601

introduce JSON serialization for ops (#2799) Converts env(), exit(), execPath(), utime() and utimeSync() to use JSON instead of flatbuffers.

view details

Bartek Iwańczuk

commit sha 387a6f8699ccd6ca05f8357260f6eeed62691e41

port cli/ops/errors.rs to json ops

view details

Bartek Iwańczuk

commit sha 7d9264077561b3b92ffc69145456378f8029711a

port cli/ops/compiler.rs to json ops

view details

Bartek Iwańczuk

commit sha a1fe53a4f96caa8792060f1f84daf5b91fe61b13

port cli/ops/fetch.rs to json ops

view details

Bartek Iwańczuk

commit sha b3bc54ce40c53d0320c51ed37a51790d95272ff8

fmt

view details

push time in 2 days

push eventbartlomieju/deno

Kitson Kelly

commit sha 6c7d337960b3745a7b614a18150862279ef1c942

Support .d.ts files (#2746) Fixes #1432

view details

Bert Belder

commit sha 31aa7c1a5d8a27c720b6255dc3eceda3707b1826

build: support rust crates that generate sources in their build script

view details

Bert Belder

commit sha e0c1ed96e22dc9a94cb5457c8b3eb2f2cd1af831

build: remove per-crate 'treat_warnings_as_errors' gn flag

view details

Bert Belder

commit sha 7a902fed04e23c2be6133024f7ad8fdebb641c60

build: add 'cap_lints' flag for rust crates Using a specialized flag rather than the generic 'args' option makes build_extra/rust/BUILD.gn shorter and more readable.

view details

Bert Belder

commit sha 7f9c6decc8982dc9dec762d6f2cc77c6bdd3f817

third_party: upgrade rust crates

view details

Ryan Dahl

commit sha 0809b06a3938868f364f1343b0de4d5d9686495d

v0.16.0

view details

Bert Belder

commit sha 47c216317f8eb5bf277663a732a79f6b07ba79ef

build: enable 'derive' feature of 'serde' crate

view details

Ryan Dahl

commit sha bc467b265fbe06ace24f5d9536bd8eb36ae4a601

introduce JSON serialization for ops (#2799) Converts env(), exit(), execPath(), utime() and utimeSync() to use JSON instead of flatbuffers.

view details

push time in 2 days

push eventbartlomieju/deno

Bert Belder

commit sha 31aa7c1a5d8a27c720b6255dc3eceda3707b1826

build: support rust crates that generate sources in their build script

view details

Bert Belder

commit sha e0c1ed96e22dc9a94cb5457c8b3eb2f2cd1af831

build: remove per-crate 'treat_warnings_as_errors' gn flag

view details

Bert Belder

commit sha 7a902fed04e23c2be6133024f7ad8fdebb641c60

build: add 'cap_lints' flag for rust crates Using a specialized flag rather than the generic 'args' option makes build_extra/rust/BUILD.gn shorter and more readable.

view details

Bert Belder

commit sha 7f9c6decc8982dc9dec762d6f2cc77c6bdd3f817

third_party: upgrade rust crates

view details

Bartek Iwańczuk

commit sha e0b17536975941eb8efdd666cef56505dfea1cfc

Merge branch 'master' into port_ops_3

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha d03479f0cd01a3cb5cdf0944f2e6e80112b12d60

fmt

view details

push time in 2 days

issue commentdenoland/deno

Convert all Flatbuffer ops to JSON

I should convert following files tomorrow:

* net.ts
* performance.rs
* permissions.rs
* process.rs
* random.rs
* repl.rs
* resources.rs
* timers.rs
* utils.rs
* workers.rs
ry

comment created time in 2 days

pull request commentdenoland/deno

port ops to JSON (compiler, errors, fetch, files)

@ry this PR should be finished from my side. Please review, I'll rebase it once you land #2802

bartlomieju

comment created time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 3b4c07010741612ddfe68a8b006d34231a2cbad1

fix tests

view details

Bartek Iwańczuk

commit sha 96d85a6f43a9038377e2d351c90215c11c875ad5

port fetch to JSON

view details

Bartek Iwańczuk

commit sha 9d22b6ce14c2a285f7c072d2704dad3c4c307bfd

lint names

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha e805ccda25e1134342b29cd1906c3ceee63e53e7

lint

view details

push time in 2 days

pull request commentdenoland/deno

WIP: port ops to json #3

@ry how should we handle op_write and op_read that still use flatbuffers? Try to land again #2788 or do them in in JSON as hot fix and revisit later?

bartlomieju

comment created time in 2 days

push eventbartlomieju/deno

Kitson Kelly

commit sha 6c7d337960b3745a7b614a18150862279ef1c942

Support .d.ts files (#2746) Fixes #1432

view details

Bartek Iwańczuk

commit sha a6d777599b91e173eb9a5dec95ff7608f5e7da3f

Merge branch 'master' into port_ops_3

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha f3fc06d93d2b1cff760f03835c412b58f7b9d5f2

port open and openSync

view details

Bartek Iwańczuk

commit sha 5174f3591c21912ac9bdde374c57fd6b5d080464

port close, seek, seekSync

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 391dd49af0b2badf60da3335681af3ad3bdc9527

fix op_fetch_source_file

view details

push time in 2 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 05c04f8a82e0f801a9e34acff57ca65ebad9dc36

Update errors.rs

view details

push time in 3 days

startedxi-editor/xi-editor

started time in 3 days

Pull request review commentdenoland/deno

Start porting flatbuffer ops to JSON

+// Copyright 2018-2019 the Deno authors. All rights reserved. MIT license.+use crate::state::ThreadSafeState;+use crate::tokio_util;+use deno::*;+use futures::Future;+use futures::Poll;+pub use serde_derive::Deserialize;+use serde_json::json;+pub use serde_json::Value;++pub type AsyncJsonOp = Box<dyn Future<Item = Value, Error = ErrBox> + Send>;++pub enum JsonOp {+  Sync(Value),+  Async(AsyncJsonOp),+}++fn json_err(err: ErrBox) -> Value {+  use crate::deno_error::GetErrorKind;+  json!({+    "message": err.to_string(),+    "kind": err.kind() as u32,+  })+}++pub type Dispatcher = fn(+  state: &ThreadSafeState,+  args: Value,+  zero_copy: Option<PinnedBuf>,+) -> Result<JsonOp, ErrBox>;++fn serialize_result(+  promise_id: Option<u64>,+  result: Result<Value, ErrBox>,+) -> Buf {+  let value = match result {+    Ok(v) => json!({ "ok": v, "promiseId": promise_id }),+    Err(err) => json!({ "err": json_err(err), "promiseId": promise_id }),+  };+  let vec = serde_json::to_vec(&value).unwrap();+  vec.into_boxed_slice()+}++#[derive(Deserialize)]+#[serde(rename_all = "camelCase")]+struct AsyncArgs {+  promise_id: Option<u64>,+}++pub fn dispatch(+  d: Dispatcher,+  state: &ThreadSafeState,+  control: &[u8],+  zero_copy: Option<PinnedBuf>,+) -> CoreOp {+  let async_args: AsyncArgs = serde_json::from_slice(control).unwrap();

A few lines down there's:

let result = serde_json::from_slice(control)
    .map_err(ErrBox::from)
    .and_then(move |args| d(state, args, zero_copy));

Does it make sense to do from_slice operation twice?

Also to check if op should be sync or async one would have to manually do inside the op:

let promise_id = args.promise_id;
let is_sync = promise_id.is_none();

Can we some somehow abstract it better?

ry

comment created time in 3 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 6cc652baac970d33f5246ce48110d0af7697c988

port cli/ops/compiler.rs to json ops

view details

push time in 3 days

PR opened denoland/deno

WIP: port ops to json #3
+505 -540

0 comment

20 changed files

pr created time in 3 days

create barnchbartlomieju/deno

branch : port_ops_3

created branch time in 3 days

push eventbartlomieju/deno_std

Bartek Iwańczuk

commit sha d7a837d599ca7d5c489d6c74b7bd42d22a3314eb

use relative paths in test imports (#572)

view details

Eugen Cazacu

commit sha ed1b9e0563e0f22eb6415b248d6218e75e500783

print out the failed tests after the summary (#554)

view details

push time in 3 days

delete branch bartlomieju/deno_std

delete branch : fix-sprintf_imports

delete time in 3 days

issue commentdenoland/deno_std

deno test doesn't work on subdirs

I don't think it's a bug. ./fmt is converted to a regex to match files during walk() and it causes to match all files in fmt/ directory.

Actually the error is the same as in #568 - importing non-JS file. For good measure we can always only match .js and .ts files, but it may be surprising that some files are omitted. Anyway... deno test should definitely be as ergonomic as possible, we should also check how other test runners handle that.

ry

comment created time in 3 days

issue commentdenoland/deno

Convert all Flatbuffer ops to JSON

I'll take these for a good start:

* compiler.rs
* errors.rs
* fetch.rs
* files.rs
ry

comment created time in 3 days

push eventbartlomieju/deno

Kitson Kelly

commit sha f0a235563e1eb748f4030d19af3f9a5ac59d2550

Support custom inspection of objects (#2791)

view details

Bartek Iwańczuk

commit sha 389763c04e3102d5b8261a10bb7514ba046fe373

bump test runner revision (#2800)

view details

Bartek Iwańczuk

commit sha b764d1b8ffc4bf5e2ab89bdbd978d708a6da0f24

fix: dynamic import panic (#2792)

view details

Ryan Dahl

commit sha bdc97b3976786bb744a27e59b0f4f28554a682df

Organize dispatch a bit (#2796) Just some clean up reorganization around flatbuffer/minimal dispatch code. This is prep for adding a JSON dispatcher.

view details

push time in 3 days

create barnchbartlomieju/deno_std

branch : fix-sprintf_imports

created branch time in 3 days

push eventbartlomieju/deno_std

Eugen Cazacu

commit sha f1c4c1a1626f4d794f05cb033eddfa43a3502e5c

test all text functions in colors module (#553)

view details

push time in 3 days

delete branch bartlomieju/deno

delete branch : fix-dynamic_import_panic

delete time in 3 days

delete branch bartlomieju/deno

delete branch : chore-bump_test_runner

delete time in 3 days

issue commentdenoland/deno

"deno test" should run all the functions named test* in current directory

Resolved in #2783!

CC @ry

ry

comment created time in 3 days

PR opened denoland/deno

chore: bump test runner

This PR bumps test runner URL to latest revision (https://github.com/denoland/deno_std/commit/4531fa81597c346fd291c1d1a2063789369fb84d) so the problem described in denoland/deno_std#568 no longer occurs

+1 -1

0 comment

1 changed file

pr created time in 3 days

create barnchbartlomieju/deno

branch : chore-bump_test_runner

created branch time in 3 days

push eventbartlomieju/deno_std

Bartek Iwańczuk

commit sha 4531fa81597c346fd291c1d1a2063789369fb84d

fix: add global test registry (#568) After adding deno test command a new problem appeared. If you try running deno test inside this repo test runner will find 80 test files but won't run any tests! This is caused by fact that deno test uses tagged version of standard library which causes test function available inside to repo to be different function that test available in standard lib used by deno test.

view details

push time in 3 days

Pull request review commentdenoland/deno

fix: dynamic import panic

 pub mod tests {     })   } +  #[test]+  fn dyn_import_err2() {+    use std::convert::TryInto;+    // Import multiple modules to demonstrate that after failed dynamic import+    // another dynamic import can still be run+    run_in_task(|| {+      let count = Arc::new(AtomicUsize::new(0));+      let count_ = count.clone();+      let mut isolate = Isolate::new(StartupData::None, false);+      isolate.set_dyn_import(move |_, specifier, referrer| {+        let c = count_.fetch_add(1, Ordering::Relaxed);+        match c {+          0 => assert_eq!(specifier, "foo1.js"),+          1 => assert_eq!(specifier, "foo2.js"),+          2 => assert_eq!(specifier, "foo3.js"),+          _ => unreachable!(),+        }+        assert_eq!(referrer, "dyn_import_error.js");++        let source_code_info = SourceCodeInfo {+          module_url_specified: specifier.to_owned(),+          module_url_found: specifier.to_owned(),+          code: "# not valid JS".to_owned(),

@ry I updated the test and verified that it fails on master. This bit # not valid JS represents the same situation as when that panic was encountered, ie. importing non-JS file

bartlomieju

comment created time in 3 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha e7eabda37ae3dcafda3a6178985d8feef9083a5c

upgrade dyn_import_err2 test

view details

push time in 3 days

issue commentdenoland/registry

Automatically generate API docs for a module

@j-f1 I did a little research. To use only tsc we might want to take a look at getJSDocTags

Relevant discussion

j-f1

comment created time in 3 days

delete branch bartlomieju/deno_std

delete branch : fix-deno_test_no_tests_found

delete time in 3 days

Pull request review commentdenoland/deno_std

fix: add global test registry

 function print(txt: string, newline: boolean = true): void {   Deno.stdout.writeSync(encoder.encode(`${txt}`)); } +declare global {+  interface Window {

Without it TS complains that __DENO_TEST_REGISTRY doesn't exist on Window:

error TS2339: Property '__DENO_TEST_REGISTRY' does not exist on type 'Window'.

► file:///Users/biwanczuk/dev/deno_std/testing/mod.ts:90:23

90   candidates = window.__DENO_TEST_REGISTRY as TestDefinition[];

I can add @ts-ignore but this solution is more idiomatic

bartlomieju

comment created time in 3 days

pull request commentdenoland/deno_std

fix: add global test registry

@ry do you want me to change anything else in this PR?

bartlomieju

comment created time in 4 days

Pull request review commentdenoland/deno

fix: dynamic import panic

 pub mod tests {     })   } +  #[test]+  fn dyn_import_err2() {

Yes... So the error I encountered was raised during dyn_import_done (SyntaxError), I'll rewrite this test case to raise the same error.

bartlomieju

comment created time in 4 days

Pull request review commentdenoland/deno

Organize dispatch a bit

 // Copyright 2018-2019 the Deno authors. All rights reserved. MIT license.-use crate::deno_error::GetErrorKind;-use crate::dispatch_minimal::dispatch_minimal;-use crate::dispatch_minimal::parse_min_record;-use crate::msg; use crate::state::ThreadSafeState;-use crate::tokio_util; use deno::*;-use flatbuffers::FlatBufferBuilder;-use futures;-use futures::Poll;-use hyper;-use hyper::rt::Future;-use tokio_threadpool;  mod compiler;-use compiler::{op_cache, op_fetch_source_file};+mod dispatch_flatbuffers;+mod dispatch_minimal; mod errors;-use errors::{op_apply_source_map, op_format_error};-mod files;-use files::{op_close, op_open, op_read, op_seek, op_write}; mod fetch;-use fetch::op_fetch;+mod files; mod fs;-use fs::{-  op_chdir, op_chmod, op_chown, op_copy_file, op_cwd, op_link,-  op_make_temp_dir, op_mkdir, op_read_dir, op_read_link, op_remove, op_rename,-  op_stat, op_symlink, op_truncate, op_utime,-};+mod io; mod metrics;-use metrics::op_metrics; mod net;-use net::{op_accept, op_dial, op_listen, op_shutdown}; mod os;-use os::{-  op_env, op_exec_path, op_exit, op_home_dir, op_is_tty, op_set_env, op_start,-}; mod performance;-use performance::op_now; mod permissions;-use permissions::{op_permissions, op_revoke_permission}; mod process;-use process::{op_kill, op_run, op_run_status}; mod random;-use random::op_get_random_values; mod repl;-use repl::{op_repl_readline, op_repl_start}; mod resources;-use resources::op_resources; mod timers;-use timers::{op_global_timer, op_global_timer_stop};+mod utils; mod workers;-use workers::{-  op_create_worker, op_host_get_message, op_host_get_worker_closed,-  op_host_post_message, op_worker_get_message, op_worker_post_message,-}; -type CliOpResult = OpResult<ErrBox>;+pub const OP_FLATBUFFER: OpId = 44;+pub const OP_READ: OpId = 1;+pub const OP_WRITE: OpId = 2; -type CliDispatchFn = fn(-  state: &ThreadSafeState,-  base: &msg::Base<'_>,-  data: Option<PinnedBuf>,-) -> CliOpResult;--pub type OpSelector = fn(inner_type: msg::Any) -> Option<CliDispatchFn>;--#[inline]-fn empty_buf() -> Buf {-  Box::new([])-}--const FLATBUFFER_OP_ID: OpId = 44;--pub fn dispatch_all(+pub fn dispatch(   state: &ThreadSafeState,   op_id: OpId,   control: &[u8],   zero_copy: Option<PinnedBuf>,-  op_selector: OpSelector, ) -> CoreOp {   let bytes_sent_control = control.len();   let bytes_sent_zero_copy = zero_copy.as_ref().map(|b| b.len()).unwrap_or(0);-  let op = if op_id != FLATBUFFER_OP_ID {-    let min_record = parse_min_record(control).unwrap();-    dispatch_minimal(state, op_id, min_record, zero_copy)-  } else {-    dispatch_all_legacy(state, control, zero_copy, op_selector)-  };-  state.metrics_op_dispatched(bytes_sent_control, bytes_sent_zero_copy);-  op-}--/// Processes raw messages from JavaScript.-/// This functions invoked every time Deno.core.dispatch() is called.-/// control corresponds to the first argument of Deno.core.dispatch().-/// data corresponds to the second argument of Deno.core.dispatch().-pub fn dispatch_all_legacy(-  state: &ThreadSafeState,-  control: &[u8],-  zero_copy: Option<PinnedBuf>,-  op_selector: OpSelector,-) -> CoreOp {-  let base = msg::get_root_as_base(&control);-  let inner_type = base.inner_type();-  let is_sync = base.sync();-  let cmd_id = base.cmd_id();--  debug!(-    "msg_from_js {} sync {}",-    msg::enum_name_any(inner_type),-    is_sync-  ); -  let op_func: CliDispatchFn = match op_selector(inner_type) {-    Some(v) => v,-    None => panic!("Unhandled message {}", msg::enum_name_any(inner_type)),-  };--  let op_result = op_func(state, &base, zero_copy);--  let state = state.clone();--  match op_result {-    Ok(Op::Sync(buf)) => {-      state.metrics_op_completed(buf.len());-      Op::Sync(buf)+  let op = match op_id {+    OP_READ => {+      dispatch_minimal::dispatch(io::op_read, state, control, zero_copy)     }-    Ok(Op::Async(fut)) => {-      let result_fut = Box::new(-        fut-          .or_else(move |err: ErrBox| -> Result<Buf, ()> {-            debug!("op err {}", err);-            // No matter whether we got an Err or Ok, we want a serialized message to-            // send back. So transform the DenoError into a Buf.-            let builder = &mut FlatBufferBuilder::new();-            let errmsg_offset = builder.create_string(&format!("{}", err));-            Ok(serialize_response(-              cmd_id,-              builder,-              msg::BaseArgs {-                error: Some(errmsg_offset),-                error_kind: err.kind(),-                ..Default::default()-              },-            ))-          })-          .and_then(move |buf: Buf| -> Result<Buf, ()> {-            // Handle empty responses. For sync responses we just want-            // to send null. For async we want to send a small message-            // with the cmd_id.-            let buf = if buf.len() > 0 {-              buf-            } else {-              let builder = &mut FlatBufferBuilder::new();-              serialize_response(-                cmd_id,-                builder,-                msg::BaseArgs {-                  ..Default::default()-                },-              )-            };-            state.metrics_op_completed(buf.len());-            Ok(buf)-          })-          .map_err(|err| panic!("unexpected error {:?}", err)),-      );-      Op::Async(result_fut)+    OP_WRITE => {+      dispatch_minimal::dispatch(io::op_write, state, control, zero_copy)     }-    Err(err) => {-      debug!("op err {}", err);-      // No matter whether we got an Err or Ok, we want a serialized message to-      // send back. So transform the DenoError into a Buf.-      let builder = &mut FlatBufferBuilder::new();-      let errmsg_offset = builder.create_string(&format!("{}", err));-      let response_buf = serialize_response(-        cmd_id,-        builder,-        msg::BaseArgs {-          error: Some(errmsg_offset),-          error_kind: err.kind(),-          ..Default::default()-        },-      );-      state.metrics_op_completed(response_buf.len());-      Op::Sync(response_buf)+    OP_FLATBUFFER => {+      dispatch_flatbuffers::dispatch(state, op_id, control, zero_copy)

op_id can be removed?

ry

comment created time in 4 days

Pull request review commentdenoland/deno

perf: sanitize build artifacts using ASAN/LSAN

 Deno* deno_new(deno_config config) {       // If no snapshot is provided, we initialize the context with empty       // main source code and source maps.       deno::InitializeContext(isolate, context);+    } else {+      delete[] d->snapshot_.data;

Okay so I figured what causes this leak, however I'm not yet sure how to fix it.

The problem is in will_snapshot test case in //cli/isolate.rs; this test does a snapshot of a short script and then tries to instantiate a new isolate with that snapshot. Isolate.snapshot() returns Snapshot1 which explicitly says in comment:

/// The type returned from deno_snapshot_new. Needs to be dropped.

During creation of new isolate (Isolate::new) this snapshot is passed in deno_config as load_snapshot (which has a type of Snapshot2 not Snapshot1). However there is no way to tell that this is the type of snapshot on C++ side and Rust's borrow checker prevents an explicitly drop after calling libdeno::deno_new.

bartlomieju

comment created time in 4 days

Pull request review commentdenoland/deno

perf: sanitize build artifacts using ASAN/LSAN

 Deno* deno_new(deno_config config) {       // If no snapshot is provided, we initialize the context with empty       // main source code and source maps.       deno::InitializeContext(isolate, context);+    } else {+      delete[] d->snapshot_.data;

But makes tests fail 😓 still digging...

bartlomieju

comment created time in 4 days

Pull request review commentdenoland/deno

perf: sanitize build artifacts using ASAN/LSAN

 Deno* deno_new(deno_config config) {       // If no snapshot is provided, we initialize the context with empty       // main source code and source maps.       deno::InitializeContext(isolate, context);+    } else {+      delete[] d->snapshot_.data;

@ry adding this bit makes LSAN happy. I followed this test and it indeed seems to fix the issue.

bartlomieju

comment created time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha dd739889a6a237e7172652ce2db9765deefc4739

sanitize leak from snapshot creator

view details

push time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha fad5ce69d8e6f136ee224a7a42f1dce63d084d15

fmt

view details

push time in 5 days

pull request commentdenoland/deno

feat: use v8::MicrotaskPolicy::kExplicit

With check_last_exception removed, the results are more promising:

third_party/wrk/mac/wrk -d 10s --latency http://127.0.0.1:4555
Running 10s test @ http://127.0.0.1:4555
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   192.76us  444.83us  12.51ms   95.73%
    Req/Sec    39.87k     2.52k   43.02k    87.00%
  Latency Distribution
     50%  109.00us
     75%  114.00us
     90%  149.00us
     99%    2.55ms
  793274 requests in 10.00s, 38.58MB read
Requests/sec:  79319.90
Transfer/sec:      3.86MB
bartlomieju

comment created time in 5 days

Pull request review commentdenoland/deno

feat: use v8::MicrotaskPolicy::kExplicit

 impl Isolate {     self.check_last_exception()   } +  fn run_microtasks(&self) -> Result<(), ErrBox> {+    unsafe { libdeno::deno_run_microtasks(self.libdeno_isolate) };+    self.check_last_exception()

Removed.

bartlomieju

comment created time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 88e6e6579d1fd75c00cdbc207c4221e6297c461d

remove unneeded check

view details

push time in 5 days

pull request commentdenoland/deno

feat: use v8::MicrotaskPolicy::kExplicit

Results

This PR

13:41 $ third_party/wrk/mac/wrk -d 10s --latency http://127.0.0.1:4555
Running 10s test @ http://127.0.0.1:4555
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   204.46us  465.60us  10.14ms   95.44%
    Req/Sec    38.68k     3.56k   42.98k    89.60%
  Latency Distribution
     50%  111.00us
     75%  117.00us
     90%  168.00us
     99%    2.67ms
  777553 requests in 10.10s, 37.82MB read
Requests/sec:  76974.46
Transfer/sec:      3.74MB

Latest release:

13:47 $ third_party/wrk/mac/wrk -d 10s --latency http://127.0.0.1:4555
Running 10s test @ http://127.0.0.1:4555
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   183.57us  372.93us   5.53ms   95.52%
    Req/Sec    40.72k     2.25k   44.34k    75.25%
  Latency Distribution
     50%  106.00us
     75%  111.00us
     90%  157.00us
     99%    2.46ms
  818606 requests in 10.10s, 39.81MB read
Requests/sec:  81050.03
Transfer/sec:      3.94MB

I'll profile V8 as well in the evening

bartlomieju

comment created time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 29afa00c8966b1b05497de40132a6d7828fb21e2

pass user_data to deno_run_microtasks

view details

push time in 5 days

Pull request review commentdenoland/deno

feat: use v8::MicrotaskPolicy::kExplicit

 void deno_terminate_execution(Deno* d_) {   deno::DenoIsolate* d = reinterpret_cast<deno::DenoIsolate*>(d_);   d->isolate_->TerminateExecution(); }++void deno_run_microtasks(Deno* d_) {

Thanks for the patch @afinch7. If I may could you guys explain why UserDataScope is needed here? I suspect it's because deno_run_microtasks goes through all microtasks resolving promises which in turn can have associated data passed between Rust and V8. (sorry for lame question, C++ is not my strongest feat)

bartlomieju

comment created time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha b34c55335d89e57145d43817c500f847e1e88371

bring back old unit test

view details

push time in 5 days

PR opened denoland/deno

perf: sanitize build artifacts using ASAN/LSAN

This PR is intended to sanitize more build targets.

Currently only libdeno_test is built and run with ASAN/LSAN

+2 -1

0 comment

1 changed file

pr created time in 5 days

create barnchbartlomieju/deno

branch : perf-sanitize_artifacts

created branch time in 5 days

issue commentdenoland/registry

Further improvements to HTML code viewer

Just FYI I intend to pull out JSDoc for all export from a file using TypeScript AST, but I'll start working on it in the second part of the week. @j-f1 in the meantime feel free to hack around on your own

j-f1

comment created time in 5 days

push eventbartlomieju/deno

Bartek Iwańczuk

commit sha 838e811aa7cfe62e157dd6e98ed4be58aa91a7cf

use v8::MicrotaskPolicy::kExplicit

view details

push time in 5 days

PR opened denoland/deno

feat: use v8::MicrotaskPolicy::kExplicit
+43 -17

0 comment

5 changed files

pr created time in 5 days

more