profile
viewpoint
Robert Gabriel Jakabosky Neopallium Guangzhou, China http://ca.linkedin.com/in/neopallium Senior Rust Engineer

Neopallium/llvm-lua 109

Automatically exported from code.google.com/p/llvm-lua

Neopallium/lua-clang-cindex 12

FFI bindings to libclang's CIndex interface.

Neopallium/lua-buf 9

Mutable buffer object for Lua.

Neopallium/lludp_dissector 6

Wireshark dissector for the LLUDP protocol

lectio/go-json-hal 2

JSON+HAL Go Library for use in Lectio and other projects

Neopallium/faxpp 2

Fast XML Pull Parser

Neopallium/libgit2-backends 2

Standalone ODB backends for the libgit2 library

lectio/imap-facade-openproject 1

IMAP server façade for OpenProject work packages

Neopallium/cr-rs 1

Rust safe wrapper for cr.h: A Simple C Hot Reload Header-only Library

Neopallium/cr-sys 1

Rust raw bindings for cr.h: A Simple C Hot Reload Header-only Library

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> Module<T> {         let complexity = asset_compliance             .iter()             .flat_map(|req| req.conditions())-            .fold(0usize, |complexity, condition| {-                let (claims, issuers) = condition.complexity();-                complexity.saturating_add(claims.saturating_mul(match issuers {-                    0 => default_issuer_count,-                    _ => issuers,-                }))-            });-        if let Ok(complexity_u32) = u32::try_from(complexity) {-            if complexity_u32 <= T::MaxConditionComplexity::get() {-                return Ok(());-            }+            .fold(0u32, |total, condition| {+                let complexity = condition.complexity(default_issuer_count);+                total.saturating_add(complexity)+            })+            // NB: If the compliance requirements are empty (0 complexity),+            // then use the count of requirements.+            .max(asset_compliance.len() as u32);

Originally I wanted to throw an error when the user tries to add empty compliance requirements, but that broke a lot of tests. So to keep this PR simple, I added the above max logic.

I created a Jira ticket for using an error instead of this fix. MESH-1781

Neopallium

comment created time in 20 hours

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> Module<T> {         let complexity = asset_compliance             .iter()             .flat_map(|req| req.conditions())-            .fold(0usize, |complexity, condition| {-                let (claims, issuers) = condition.complexity();-                complexity.saturating_add(claims.saturating_mul(match issuers {-                    0 => default_issuer_count,-                    _ => issuers,-                }))-            });-        if let Ok(complexity_u32) = u32::try_from(complexity) {-            if complexity_u32 <= T::MaxConditionComplexity::get() {-                return Ok(());-            }+            .fold(0u32, |total, condition| {+                let complexity = condition.complexity(default_issuer_count);+                total.saturating_add(complexity)+            })+            // NB: If the compliance requirements are empty (0 complexity),+            // then use the count of requirements.+            .max(asset_compliance.len() as u32);

Also note that each compliance requirement has both sender & receiver side conditions, but it is ok for one side to be empty. So it is easier to just make sure that the total complexity is at least as large as the number of requirements (asset_compliance.len()).

Neopallium

comment created time in 20 hours

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> Module<T> {         let complexity = asset_compliance             .iter()             .flat_map(|req| req.conditions())-            .fold(0usize, |complexity, condition| {-                let (claims, issuers) = condition.complexity();-                complexity.saturating_add(claims.saturating_mul(match issuers {-                    0 => default_issuer_count,-                    _ => issuers,-                }))-            });-        if let Ok(complexity_u32) = u32::try_from(complexity) {-            if complexity_u32 <= T::MaxConditionComplexity::get() {-                return Ok(());-            }+            .fold(0u32, |total, condition| {+                let complexity = condition.complexity(default_issuer_count);+                total.saturating_add(complexity)+            })+            // NB: If the compliance requirements are empty (0 complexity),+            // then use the count of requirements.+            .max(asset_compliance.len() as u32);

If the total number of requirements is larger then the complexity of the non-empty requirements, then it will still use the "larger" value. (i.e. 100 requirements and only one with complex 30, then the final complex would be 100).

Also note that the maximum complexity for asset compliance requirements is very low (T::MaxConditionComplexity == 50 right now).

Neopallium

comment created time in 20 hours

push eventPolymathNetwork/Polymesh

Robert G. Jakabosky

commit sha cd2147ac52d2cda702bea8c2166986a99986af28

Code cleanup.

view details

push time in a day

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl Condition {     }      /// Returns worst case complexity of a condition.-    pub fn complexity(&self) -> (usize, usize) {-        (self.condition_type.complexity(), self.issuers.len())+    pub fn complexity(&self, default_issuer_count: usize) -> u32 {+        let issuers = match self.issuers.len() {+            0 => default_issuer_count,+            count => count,+        };+        self.condition_type+            .count()+            // NB: `max(1)` Make sure issuer count is not zero.

It is fine as is.

Neopallium

comment created time in a day

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> Module<T> {         let complexity = asset_compliance             .iter()             .flat_map(|req| req.conditions())-            .fold(0usize, |complexity, condition| {-                let (claims, issuers) = condition.complexity();-                complexity.saturating_add(claims.saturating_mul(match issuers {-                    0 => default_issuer_count,-                    _ => issuers,-                }))-            });-        if let Ok(complexity_u32) = u32::try_from(complexity) {-            if complexity_u32 <= T::MaxConditionComplexity::get() {-                return Ok(());-            }+            .fold(0u32, |total, condition| {+                let complexity = condition.complexity(default_issuer_count);+                total.saturating_add(complexity)+            })+            // NB: If the compliance requirements are empty (0 complexity),+            // then use the count of requirements.+            .max(asset_compliance.len() as u32);

This is to stop someone from adding a large number of empty requirements. Users should not be adding empty compliance requirements.

Neopallium

comment created time in a day

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl Condition {     }      /// Returns worst case complexity of a condition.-    pub fn complexity(&self) -> (usize, usize) {-        (self.condition_type.complexity(), self.issuers.len())+    pub fn complexity(&self, default_issuer_count: usize) -> u32 {+        let issuers = match self.issuers.len() {+            0 => default_issuer_count,+            count => count,+        };+        self.condition_type+            .count()+            // NB: `max(1)` Make sure issuer count is not zero.+            .saturating_mul(issuers.max(1))+            .try_into()+            .unwrap_or(u32::MAX)+    }++    /// Return number of claims, issuers, and claim_types.+    ///+    /// This is used for weight calculation.+    pub fn counts(&self) -> (u32, u32, u32) {

No. It is fine as a tuple.

Neopallium

comment created time in a day

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 pub trait WeightInfo {     fn change_compliance_requirement(s: u32, r: u32) -> Weight;     fn replace_asset_compliance(c: u32) -> Weight;     fn reset_asset_compliance() -> Weight;++    fn condition_costs(conditions: u32, claims: u32, issuers: u32, claim_types: u32) -> Weight;++    fn add_compliance_requirement_full(sender: &[Condition], receiver: &[Condition]) -> Weight {+        let (_, claims, issuers, claim_types) =+            conditions_total_counts(sender.iter().chain(receiver.iter()));+        Self::add_compliance_requirement(sender.len() as u32, receiver.len() as u32)+            .saturating_add(Self::condition_costs(0, claims, issuers, claim_types))+    }++    fn change_compliance_requirement_full(req: &ComplianceRequirement) -> Weight {+        let (_, claims, issuers, claim_types) = req.counts();+        Self::change_compliance_requirement(+            req.sender_conditions.len() as u32,+            req.receiver_conditions.len() as u32,+        )+        .saturating_add(Self::condition_costs(0, claims, issuers, claim_types))+    }++    fn replace_asset_compliance_full(reqs: &[ComplianceRequirement]) -> Weight {+        let (conditions, claims, issuers, claim_types) =+            conditions_total_counts(reqs.iter().flat_map(|req| req.conditions()));+        Self::replace_asset_compliance(reqs.len() as u32).saturating_add(Self::condition_costs(+            conditions,+            claims,+            issuers,+            claim_types,+        ))+    }

Oops. Forgot to change the weight functions on the extrinsics.

Neopallium

comment created time in a day

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 use polymesh_common_utilities::{     benchs::{AccountIdOf, User, UserBuilder},     TestUtilsFn, };-use polymesh_primitives::{asset::AssetType, TrustedFor, TrustedIssuer};+use polymesh_primitives::{asset::AssetType, ClaimType, Scope, TrustedFor, TrustedIssuer};+use sp_std::convert::TryFrom;  const MAX_DEFAULT_TRUSTED_CLAIM_ISSUERS: u32 = 3; const MAX_TRUSTED_ISSUER_PER_CONDITION: u32 = 3; const MAX_SENDER_CONDITIONS_PER_COMPLIANCE: u32 = 3; const MAX_RECEIVER_CONDITIONS_PER_COMPLIANCE: u32 = 3; const MAX_COMPLIANCE_REQUIREMENTS: u32 = 2; +const MAX_CONDITIONS: u32 = 10;+const MAX_CONDITION_TYPE_CLAIMS: u32 = 10;+const MAX_CONDITION_ISSUERS: u32 = 10;+const MAX_CONDITION_ISSUER_CLAIM_TYPES: u32 = 10;+/*+const MIN_CONDITIONS: u32 = 1;+const MIN_CONDITION_TYPE_CLAIMS: u32 = 1;+const MIN_CONDITION_ISSUERS: u32 = 1;+const MIN_CONDITION_ISSUER_CLAIM_TYPES: u32 = 1;+*/++const CLAIM_TYPES: &'static [ClaimType] = &[+    ClaimType::Accredited,+    ClaimType::Affiliate,+    ClaimType::BuyLockup,+    ClaimType::SellLockup,+    ClaimType::CustomerDueDiligence,+    ClaimType::KnowYourCustomer,+    ClaimType::Jurisdiction,+    ClaimType::Exempted,+    ClaimType::Blocked,+    ClaimType::InvestorUniqueness,+    ClaimType::NoType,+    ClaimType::InvestorUniquenessV2,+];

It only has use in the benchmarks. It is fine where it is.

Neopallium

comment created time in a day

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> ComplianceRequirementBuilder<T> {     } } +fn setup_conditions_bench<T: Config + TestUtilsFn<AccountIdOf<T>>>(+    conditions: u32,+    claims: u32,+    issuers: u32,+    claim_types: u32,+) -> Vec<Condition> {+    let issuers = make_issuers::<T>(issuers, Some(claim_types as usize));+    let conditions = make_conditions(conditions, Some(claims as usize), &issuers);+    conditions+}++fn conditions_bench(conditions: Vec<Condition>) {+    let encoded = conditions.encode();+    let decoded = Vec::<Condition>::decode(&mut encoded.as_slice())+        .expect("This shouldn't fail since we just encoded a `Vec<Condition>` value.");+    if !conditions.eq(&decoded) {+        panic!("This shouldn't fail.");+    }+}+ benchmarks! {     where_clause { where T: TestUtilsFn<AccountIdOf<T>> } +    /*+    conditions_cost {+        let c in 1..MAX_CONDITIONS;++        let conditions = setup_conditions_bench::<T>(c, MIN_CONDITION_TYPE_CLAIMS, MIN_CONDITION_ISSUERS, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_claims_cost {+        let c in 1..MAX_CONDITION_TYPE_CLAIMS;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, c, MIN_CONDITION_ISSUERS, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_issuers_cost {+        let i in 1..MAX_CONDITION_ISSUERS;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, MIN_CONDITION_TYPE_CLAIMS, i, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_issuer_claim_types_cost {+        let t in 1..MAX_CONDITION_ISSUER_CLAIM_TYPES;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, MIN_CONDITION_TYPE_CLAIMS, MIN_CONDITION_ISSUERS, t);+    }: {+        conditions_bench(conditions);+    }+    */

Originally I wanted to use these cost functions, but the generated weights were wrong. Each of the generated cost functions were added a large fixed weight, but we only want to increase the cost for large inputs.

Neopallium

comment created time in a day

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 impl<T: Config> ComplianceRequirementBuilder<T> {     } } +fn setup_conditions_bench<T: Config + TestUtilsFn<AccountIdOf<T>>>(+    conditions: u32,+    claims: u32,+    issuers: u32,+    claim_types: u32,+) -> Vec<Condition> {+    let issuers = make_issuers::<T>(issuers, Some(claim_types as usize));+    let conditions = make_conditions(conditions, Some(claims as usize), &issuers);+    conditions+}++fn conditions_bench(conditions: Vec<Condition>) {+    let encoded = conditions.encode();+    let decoded = Vec::<Condition>::decode(&mut encoded.as_slice())+        .expect("This shouldn't fail since we just encoded a `Vec<Condition>` value.");+    if !conditions.eq(&decoded) {+        panic!("This shouldn't fail.");+    }+}+ benchmarks! {     where_clause { where T: TestUtilsFn<AccountIdOf<T>> } +    /*+    conditions_cost {+        let c in 1..MAX_CONDITIONS;++        let conditions = setup_conditions_bench::<T>(c, MIN_CONDITION_TYPE_CLAIMS, MIN_CONDITION_ISSUERS, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_claims_cost {+        let c in 1..MAX_CONDITION_TYPE_CLAIMS;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, c, MIN_CONDITION_ISSUERS, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_issuers_cost {+        let i in 1..MAX_CONDITION_ISSUERS;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, MIN_CONDITION_TYPE_CLAIMS, i, MIN_CONDITION_ISSUER_CLAIM_TYPES);+    }: {+        conditions_bench(conditions);+    }++    condition_issuer_claim_types_cost {+        let t in 1..MAX_CONDITION_ISSUER_CLAIM_TYPES;++        let conditions = setup_conditions_bench::<T>(MIN_CONDITIONS, MIN_CONDITION_TYPE_CLAIMS, MIN_CONDITION_ISSUERS, t);+    }: {+        conditions_bench(conditions);+    }+    */

I wanted to show the results of trying to split the complexity parameters into multiple cost functions.

Neopallium

comment created time in a day

PullRequestReviewEvent

Pull request review commentPolymathNetwork/Polymesh

MESH-1773 & MESH-1777 condition cost function

 pub fn make_issuer<T: IdentityConfig + TestUtilsFn<AccountIdOf<T>>>(id: u32) -> ///   - It could have more complexity if `TrustedIssuer::trusted_for` is a vector but not on ///   benchmarking of add/remove. That could be useful for benchmarking executions/evaluation of ///   complience requiriments.

That is talking about the cost of executing the compliance requirements during an asset transfer. Which is out of scope for this PR.

Neopallium

comment created time in a day

PullRequestReviewEvent
PullRequestReviewEvent

PR opened PolymathNetwork/Polymesh

Mesh 1773 condition cost function

changelog

modified logic

  • Use a cost function for Condition type for more accurate call weights.
+284 -35

0 comment

6 changed files

pr created time in 2 days

create barnchPolymathNetwork/Polymesh

branch : MESH-1773-condition-cost-function

created branch time in 2 days

PR opened PolymathNetwork/Polymesh

Fix order of calls in identity pallet.

Fix the order of extrinsics in the Identity pallet.

+19 -19

0 comment

1 changed file

pr created time in 2 days

PullRequestReviewEvent

push eventPolymathNetwork/Polymesh

Robert Gabriel Jakabosky

commit sha f6afeeea4f3028788d9441fb3c0714a8f3665a11

Fix schema for InvestorZKProofData type. (#1194)

view details

Robert Gabriel Jakabosky

commit sha a172d85410305bb535a071f449593ceb284e33f8

Merge branch 'develop' into fix_multi_node_script

view details

push time in 7 days

more