profile
viewpoint
Jonathan Turner jonathandturner @Microsoft New Zealand jonathanturner.org Current: @azure and @nushell. Previously: @rust-lang, PM for TypeScript @microsoft.

AzureAD/microsoft-authentication-library-for-js 1212

Microsoft Authentication Library (MSAL) for JS

jonathandturner/AudioScope-ng2 32

AudioScope demo written in Angular 2 + TypeScript

jonathandturner/actress 4

Experimenting with a simple actor system in Rust

jonathandturner/api-extractor.com-website 0

GitHub Pages website for api-extractor.com

jonathandturner/azure-functions-rs 0

Create Azure Functions with Rust!

jonathandturner/azure-rest-api-specs 0

The source for REST API specifications for Microsoft Azure.

jonathandturner/azure-sdk-for-js 0

An isomorphic javascript sdk for Azure services

jonathandturner/blog.rust-lang.org 0

The Rust Programming Language Blog

issue commentnushell/nushell

Calling external command inside each block returns "Received unexpected type from pipeline (row)" error

This may be specific to $it.name and each. A variation of the above appears to work correctly:

> ls *.txt | get name | each { ^echo $it }
CBenoit

comment created time in 41 minutes

pull request commentAzure/azure-sdk-for-js

[Identity] Success logging (Both in Interactive and in DeviceCode)

@sadasant - I don't think this needs its own changelog entry, though if you think so please feel free to add.

sadasant

comment created time in 3 hours

PullRequestReviewEvent

pull request commentAzure/azure-sdk-for-js

[Identity] Interactive Browser Credential samples idea

@sadasant - have you had a chance to look at the .NET sample docs for identity? I like how they mix them into the markdown (I believe we've done similar with KV if I'm remembering right

https://github.com/Azure/azure-sdk-for-net/blob/feature/identity/140/sdk/identity/Azure.Identity/samples/ClientSideUserAuthentication.md

sadasant

comment created time in 3 hours

pull request commentnushell/nushell

Remove it expansion

@fdncred - I believe we used echo $it at the very beginning and it stuck around, even though it doesn't do anything helpful (it doesn't print to the screen only sends the same info it receives back out the pipeline).

And yeah the two examples you showed I wrote down for later, too.

For the first one, if there's a single value coming through the pipeline having to write each feels a bit awkward.

For the second, I have no idea how that works to be honest. I wanted to go ahead and wrap up the PR so we could start testing but that one in particular should have failed but didn't. I made a note to come back to it and see why.

jonathandturner

comment created time in 6 hours

delete branch jonathandturner/nushell

delete branch : bump_0_21_1

delete time in 16 hours

push eventnushell/nushell

Jonathan Turner

commit sha a6fdee4a5179f4126f9887bb86674d1f4863bb73

bump to 0.21.1 (#2702) * bump to 0.21.1 * bump trash version

view details

push time in 16 hours

PR merged nushell/nushell

bump to 0.21.1
+206 -206

0 comment

31 changed files

jonathandturner

pr closed time in 16 hours

push eventjonathandturner/nushell

Jonathan Turner

commit sha 9356d6ea9a39f56a4d8c8238c78456c3a625762b

bump trash version

view details

push time in 16 hours

PR opened nushell/nushell

bump to 0.21.1
+203 -203

0 comment

31 changed files

pr created time in 17 hours

create barnchjonathandturner/nushell

branch : bump_0_21_1

created branch time in 17 hours

delete branch jonathandturner/nushell

delete branch : remove_it_expansion2

delete time in 17 hours

push eventnushell/nushell

Jonathan Turner

commit sha 6951fb440ca35861e4c8cd16ed498f1acd32686b

Remove it expansion (#2701) * Remove it-expansion, take 2 * Cleanup * silly update to test CI

view details

push time in 17 hours

PR merged nushell/nushell

Remove it expansion

This PR removes automatic it-expansion. With it, we no longer assume each is called around uses of $it.

This means that uses of $it are now expected to be inside of an explicit or an implicit block. In effect, $it is now the default name of values given to blocks.

No longer correct: echo 1 + 3 | echo $it Now: echo 1 + 3 | each { echo $it } or echo $(1 + 3)

We realize that this may have some pretty deep impacts in user code, depending on how much it-expansion was leveraged. We'd like to attempt to make the transition, and if it's too painful to continue to refine. Without knowing the full extent of the impact, we won't know exactly where the design should land.

Previous discussions in discord requested that we make this simplification. In looking into it, it should also help us refine how variables are scoped so that it's easier to change the default name to something else if you want to refer to a value given to an outer block (not part of this PR but may be part of future improvements)

+175 -642

0 comment

95 changed files

jonathandturner

pr closed time in 17 hours

push eventjonathandturner/nushell

Jonathan Turner

commit sha 0eded1834cc15d5d7776018a6847f8a4da0fbc06

silly update to test CI

view details

push time in 19 hours

Pull request review commentnushell/nushell

Change Reject To Error Out When Invalid Column is Provided

 pub fn select_fields(obj: &Value, fields: &[String], tag: impl Into<Tag>) -> Val     out.into_value() } -pub fn reject_fields(obj: &Value, fields: &[String], tag: impl Into<Tag>) -> Value {+pub fn reject_fields(+    obj: &Value,+    fields: &[Tagged<String>],+    tag: impl Into<Tag>,+) -> Result<Value, ShellError> {     let mut out = TaggedDictBuilder::new(tag);      let descs = obj.data_descriptors(); +    for field in fields {+        if descs.iter().all(|desc| desc != &field.item) {+            return Err(ShellError::labeled_error(+                "Invalid column",+                "invalid column",+                field.tag.clone(),+            ));+        }+    }+

Yeah exactly. When you have a mix of rows that have different shapes. For these I think we should detect if we successfully removed the column at least once. If we get to the end and never did, we should give the error then.

JosephTLyons

comment created time in 20 hours

PullRequestReviewEvent

pull request commentnushell/nushell

Improve argument escaping when running external commands for non-literals

Looks like you're hitting some clippy lints:

error: explicit lifetimes given in parameter types where they could be elided (or replaced with `'_` if needed by type declaration)
   --> crates\nu-cli\src\commands\classified\external.rs:518:1
    |
518 | fn escape_double_quotes<'a>(argument: &'a str) -> Cow<'a, str> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `-D clippy::needless-lifetimes` implied by `-D warnings`

You can check these locally with cargo clippy --all --features=extra

CBenoit

comment created time in 20 hours

push eventnushell/nushell

Chris Gillespie

commit sha 502c9ea70679252b52a3c7666f028b37a11c587e

Radix added to str decimal conversion (#2696)

view details

push time in 20 hours

PR merged nushell/nushell

Radix option added to str decimal conversion

Fixes #2536, at least at a basic level, not built into main parser

  1. Add a Radix option for str to decimal conversions
  2. Auto-parse if there is a binary or hex prefix in the str
+96 -27

1 comment

1 changed file

gillespiecd

pr closed time in 20 hours

issue closednushell/nushell

Support alternative integer shapes, such as binary and hexadecimal

Is your feature request related to a problem? Please describe. I've gotten in the habit of using the Chrome console for binary and hexadecimal math using bitwise operators. It's nice, but having this possible in the terminal would be a nice QOL feature. Atm 0b101011 and 0xffa3b both are interpreted as strings.

Describe the solution you'd like It would be fantastic to be able to express numbers using other "literal" formats, like what is possible in most modern programming languages. The = command should be able to parse 0b101011 and 0xffa3b as numbers. Having the to command also support "binary", "hexadecimal", and "octal" would be nice as well.

Describe alternatives you've considered Since the only context I would personally use this is with the = command, this could also be made instead into another package. I'd want bitwise operators eventually, but those require the | symbol, which is already in use. For that, a separate plugin that supports bitwise operators might be preferable.

Additional context N/A

closed time in 20 hours

EmNudge

pull request commentnushell/nushell

Radix option added to str decimal conversion

Looks good! Makes me wonder if we should add support for these to the Nu language itself.

gillespiecd

comment created time in 20 hours

PR opened nushell/nushell

Remove it expansion

This PR removes automatic it-expansion. With it, we no longer assume each is called around uses of $it.

This means that uses of $it are now expected to be inside of an explicit or an implicit block. In effect, $it is now the default name of values given to blocks.

No longer correct: echo 1 + 3 | echo $it Now: echo 1 + 3 | each { echo $it } or echo $(1 + 3)

We realize that this may have some pretty deep impacts in user code, depending on how much it-expansion was leveraged. We'd like to attempt to make the transition, and if it's too painful to continue to refine. Without knowing the full extent of the impact, we won't know exactly where the design should land.

Previous discussions in discord requested that we make this simplification. In looking into it, it should also help us refine how variables are scoped so that it's easier to change the default name to something else if you want to refer to a value given to an outer block (not part of this PR but may be part of future improvements)

+173 -640

0 comment

95 changed files

pr created time in 20 hours

create barnchjonathandturner/nushell

branch : remove_it_expansion2

created branch time in 21 hours

push eventjonathandturner/jonathandturner.github.io

GitHub Actions

commit sha b1f66cd58682aa2b918f6de827076ef68c61e862

Automatic deploy

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha e88668083f84515d3d8b68e6d8b83bfdc70cbc5e

Add feed

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

GitHub Actions

commit sha be9342dd6e047f47c3fa04247fc5b8647e805a70

Automatic deploy

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 32404d11ead94c540d4a9279f3b53322cde3a207

change CNAME

view details

Jonathan Turner

commit sha c442d0c382f1310e7b8245b3d2b071d21a4cff78

FIX

view details

Jonathan Turner

commit sha ef2d2ea102d8fafbc6a869c965df8995654db965

Add back CNAME

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha c8e2cd290e8bce80184527e69e2e2531aa5a245e

Create CNAME

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

GitHub Actions

commit sha 19d7a5d2b8193a3eb71dbffa6a4cec1cefb4e7b0

Automatic deploy

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 69688af7038b76fd8788f51f4635f84a499983ab

Delete CNAME

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 78a6bec40d5884a47a4e2bc2bc135317dd29212d

Delete CNAME

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

GitHub Actions

commit sha 705a8726749251357fa4c26e9b4026203d65b57d

Automatic deploy

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha e9a85bda2628d350dd92c7985e1289d1696e7a53

Add username

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 4b9f90291bc5ea8d92ce3f827f874729d54809c3

Rename token

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 5f184ae3e9ecb4279023854a3df142493c459a1d

Try someone else's action

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha ce6191a2b789da1cd3765783ed78f0119171e14f

Move to source

view details

push time in a day

create barnchjonathandturner/jonathandturner.github.io

branch : source

created branch time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 30e2450b088626efb8b95c61e4f181e52530b1ee

Try a CNAME

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 21ac37dd5857e5c961717bd7de987f717da3cf09

Update main.yml

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha e21ba98b002d47d950f5421d9e549f66fc3706f0

Try a different base url

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha c6400cc3b4885562051d6e4fda04aab0434b66da

Add back theme

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 45454e04c11d1d80f13ad8096a986a5ad6ca7628

Remove theme temporarily

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 0b75b20cd67c833acf0bee1818368b2d7e8d86f1

Create main.yml

view details

push time in a day

push eventjonathandturner/jonathandturner.github.io

Jonathan Turner

commit sha 9a241eff82cc640efd701c7d47392a50cdaa6042

Move to zola

view details

push time in a day

issue commentbennetthardwick/simple-dev-blog-zola-starter

Instructions to install theme no longer work?

Yup, those fixed it for me. Thanks!

jonathandturner

comment created time in a day

issue openedbennetthardwick/simple-dev-blog-zola-starter

Instructions to install theme no longer work?

I tried following the steps from here: https://www.getzola.org/themes/simple-dev-blog/

But when I try to run zola serve at the end, I get:

Error: Failed to render section '/home/jonathan/Source/sites/my_amazing_site/content/_index.md'
Reason: Failed to render 'index.html' (error happened in 'base.html').
Reason: Function call 'resize_image' failed
Reason: `resize_image`: Cannot find path: icon.png

Any ideas?

created time in 2 days

Pull request review commentnushell/nushell

Change Reject To Error Out When Invalid Column is Provided

 pub fn select_fields(obj: &Value, fields: &[String], tag: impl Into<Tag>) -> Val     out.into_value() } -pub fn reject_fields(obj: &Value, fields: &[String], tag: impl Into<Tag>) -> Value {+pub fn reject_fields(+    obj: &Value,+    fields: &[Tagged<String>],+    tag: impl Into<Tag>,+) -> Result<Value, ShellError> {     let mut out = TaggedDictBuilder::new(tag);      let descs = obj.data_descriptors(); +    for field in fields {+        if descs.iter().all(|desc| desc != &field.item) {+            return Err(ShellError::labeled_error(+                "Invalid column",+                "invalid column",+                field.tag.clone(),+            ));+        }+    }+

I was wondering about that. we want to check that at least one of the rows has a column with that name, right? If some don't but others do, seems like that'd still be okay.

JosephTLyons

comment created time in 2 days

PullRequestReviewEvent

push eventnushell/this_week_in_nu

Jonathan Turner

commit sha 9ef4bd4cac751b7d47f7a6e80c3754881ab99299

Create This week in Nu 61.md

view details

push time in 3 days

push eventnushell/contributor-book

Haraman Johal

commit sha 94972d40210210923681db2de1742bee27e41f33

Fix typos and style for readability

view details

Haraman Johal

commit sha f5a680b4d9a90e91477e9f3d98f5239994ee3166

Wording

view details

Jonathan Turner

commit sha 7a83a3063c34754ccdf6207721278b91b44f7729

Merge pull request #31 from HaramanJohal/readability Readability

view details

push time in 3 days

PR merged nushell/contributor-book

Readability

I read the contributor book this week and though I found it very helpful, I came across a few typos and confusing wordings. I have made some changes here that are a mixture of corrections and slight wording adjustments which I think improve the readability.

Happy to discuss any thoughts and adjust the PR if necessary :)

+48 -53

1 comment

9 changed files

HaramanJohal

pr closed time in 3 days

pull request commentnushell/contributor-book

Readability

Thanks for fixing these! Much appreciated

HaramanJohal

comment created time in 3 days

push eventnushell/book

Arnau Siches

commit sha 26c9d119eab6c158adec1198f33d5389f45cdef8

Fix pick to select rename In version 0.13.0 `pick` was renamed to `select` but some examples were left behind. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Jonathan Turner

commit sha f224abff044685bfb9cbef95e027fb59f57e7203

Merge pull request #140 from arnau/fix_pick_select Fix pick to select rename

view details

push time in 3 days

PR merged nushell/book

Fix pick to select rename

In version 0.13.0 pick was renamed to select but some examples were left behind.

+13 -13

1 comment

3 changed files

arnau

pr closed time in 3 days

pull request commentnushell/book

Fix pick to select rename

Thanks!

arnau

comment created time in 3 days

push eventnushell/book

Warren Seine

commit sha 58f2f4911677225bc7b33e9665e2ad1a820da6cb

📝 Add `date` equivalent.

view details

Jonathan Turner

commit sha 32c4e67ae04a44340efc97d28ebc5159daeb4e04

Merge pull request #139 from warrenseine/patch-2 📝 Add `date` equivalent.

view details

push time in 3 days

PR merged nushell/book

📝 Add `date` equivalent.
+1 -0

1 comment

1 changed file

warrenseine

pr closed time in 3 days

pull request commentnushell/book

📝 Add `date` equivalent.

Thanks!

warrenseine

comment created time in 3 days

push eventnushell/cookbook

Arnau Siches

commit sha 488eeea7fed7174f0c5b97823676457065a90989

Fix split command example Change from `split-column` style to `split column`. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Arnau Siches

commit sha f24601fedb54e0cf9bba858fcffd24f59d98847c

Fix parse example The `parse` command works on a single text input or on a list of inputs. This fix adds `lines` to fix the expected result. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Arnau Siches

commit sha a08e094c69d479b039ec007468c76b3785ab6d40

Fix str command Change the `str` command from flag style to subcommand. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Jonathan Turner

commit sha f7e535405be5a447cd1f48618518bd0133503f78

Merge pull request #19 from arnau/fix_split_examples Fix split command example

view details

push time in 3 days

PR merged nushell/cookbook

Fix split command example

Change from split-column style to split column.

Also, add lines to make the equivalent example using parse output the same.

Tested with nu 0.21.0

+3 -3

1 comment

1 changed file

arnau

pr closed time in 3 days

pull request commentnushell/cookbook

Fix split command example

:+1:

arnau

comment created time in 3 days

push eventnushell/cookbook

Arnau Siches

commit sha bc8ad2fd16dd2dc73360672e6d9652cf8a0fb40a

Fix pick to select rename In version 0.13.0 `pick` was renamed to `select`. This change fixes an example that wasn't renamed. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Jonathan Turner

commit sha fd19c778ebbed1d94647d914abf54fb44980d9a1

Merge pull request #18 from arnau/fix_pick_select Fix pick to select rename

view details

push time in 3 days

PR merged nushell/cookbook

Fix pick to select rename

In version 0.13.0 pick was renamed to select. This change fixes an example that wasn't renamed.

+1 -1

1 comment

1 changed file

arnau

pr closed time in 3 days

pull request commentnushell/cookbook

Fix pick to select rename

Thanks!

arnau

comment created time in 3 days

pull request commentAzure/azure-sdk-for-js

[Identity] Azure Arc and ManagedIdentityCredential refactoring

@sadasant - this definitely feels complete enough to move to being a full PR and to pull in folks who have worked on Arc on other languages for their feedback.

Thanks for taking the time to work on the refactor. The code felt readable and I was able to track where this new functionality was being added and could see where it was related to existing functionality.

sadasant

comment created time in 4 days

PullRequestReviewEvent

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it+                //therefore pipeline idx, pipeline and registry has to be passed here+                None+            }+            Expression::Literal(literal) => {+                match literal {+                    nu_protocol::hir::Literal::Number(number) => match number {+                        nu_protocol::hir::Number::Int(_) => Some(SyntaxShape::Int),+                        nu_protocol::hir::Number::Decimal(_) => Some(SyntaxShape::Number),+                    },+                    nu_protocol::hir::Literal::Size(_, _) => Some(SyntaxShape::Unit),+                    nu_protocol::hir::Literal::String(_) => Some(SyntaxShape::String),+                    //Rest should have failed at parsing stage?+                    nu_protocol::hir::Literal::GlobPattern(_) => Some(SyntaxShape::String),+                    nu_protocol::hir::Literal::Operator(_) => Some(SyntaxShape::Operator),+                    nu_protocol::hir::Literal::ColumnPath(_) => Some(SyntaxShape::ColumnPath),+                    nu_protocol::hir::Literal::Bare(_) => Some(SyntaxShape::String),+                }+            }+            //Synthetic are expressions that are generated by the parser and not inputed by the user+            //ExternalWord is anything sent to external commands (?)+            Expression::ExternalWord => Some(SyntaxShape::String),+            Expression::Synthetic(_) => Some(SyntaxShape::String),++            Expression::Binary(_) => Some(SyntaxShape::Math),+            Expression::Range(_) => Some(SyntaxShape::Range),+            Expression::List(_) => Some(SyntaxShape::Table),+            Expression::Boolean(_) => Some(SyntaxShape::String),++            Expression::Path(_) => Some(SyntaxShape::ColumnPath),+            Expression::FilePath(_) => Some(SyntaxShape::Path),+            Expression::Block(_) => Some(SyntaxShape::Block),+            Expression::ExternalCommand(_) => Some(SyntaxShape::String),+            Expression::Table(_, _) => Some(SyntaxShape::Table),+            Expression::Command => Some(SyntaxShape::String),+            Expression::Invocation(_) => Some(SyntaxShape::Block),+            Expression::Garbage => unreachable!("Should have failed at parsing stage"),+        }+    }++    fn get_shape_of_binary_arg_or_insert_dependency(+        &mut self,+        //var depending on shape of expr (arg)+        (var, expr): (&VarUsage, &SpannedExpression),+        //source_bin is binary having var on one and expr on other side+        source_bin: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<Option<SyntaxShape>, ShellError> {+        trace!("Getting shape of binary arg {:?} for var {:?}", expr, var);+        if let Some(shape) = self.get_shape_of_expr_or_insert_dependency(+            var,+            (op_of(source_bin), source_bin.span),+            expr,+        ) {+            trace!("> Found shape: {:?}", shape);+            Ok(match shape {+                SyntaxShape::Math => {+                    //If execution happens here, the situation is as follows:+                    //There is an Binary expression (source_bin) with a variable on one side+                    //and a binary (lets call it "deep binary") on the other:+                    //e.G. $var + 1 * 1+                    //Now we try to infer the shapes inside the deep binary, compute the resulting+                    //shape based on the operator (see get_result_shape_of) and return that.+                    //That won't work if one of the deeper binary left/right expr is a variable.+                    //Then we insert an element into+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //+                    //If the deeper binary contains a binary on one side, we check if that binary+                    //has a computable result type (e.G. has no variable in it) by recursively+                    //calling this function and if so return it.+                    //If the result type is not computable (the deep deep binary had a variable), we substitute+                    //the deep deep binary side of the deep binary with a variable (fake_var) and+                    //insert a dependency from the fake_var to the deep deep binary in+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //The $var on the source_bin will then (as described above) depend on the deep+                    //binary (as it has a variable (fake_var)) on one side.+                    //+                    //The dependencies gets resolved at the end, when most information about all+                    //variables is accessable.+                    //+                    //+                    //+                    //+                    //Expression is of type binary+                    //We have to descend deeper into tree+                    //And compute result shape based on operator+                    let bin = match &expr.expr {+                        Expression::Binary(bin) => bin,+                        _ => unreachable!("SyntaxShape::Math means expression binary"),+                    };+                    match (&bin.left.expr, &bin.right.expr) {+                        //$it should give shape in get_shape_of_expr_or_insert_dependency+                        //Therefore the following code is not correct!+                        ////Substitute+                        //(+                        //    Expression::Variable(Variable::It(_it_span)),+                        //    Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //)+                        //    | (+                        //        Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //        Expression::Variable(Variable::It(_it_span)),+                        //    ) => {+                        //        //TODO deduce type of $it and insert into+                        //        //dependencies_on_result_type+                        //        None+                        //    }+                        //(+                        //    Expression::Variable(Variable::It(_l_it)),+                        //    Expression::Variable(Variable::It(_r_it)),+                        //) => {+                        //    //TODO deduce type of $it and return it (based on operator)+                        //    None+                        //}+                        (+                            Expression::Variable(Variable::Other(_, _)),+                            Expression::Variable(Variable::Other(_, _)),+                        )+                        | (Expression::Variable(Variable::Other(_, _)), _)+                        | (_, Expression::Variable(Variable::Other(_, _))) => {+                            //Example of this case is: $foo + $bar * $baz+                            //foo = var (depending of shape of arg (bar * baz))+                            self.dependencies_on_result_type.push((+                                var.clone(),+                                op_of(source_bin),+                                expr.clone(),+                            ));+                            None+                        }+                        (Expression::Binary(_), Expression::Binary(_)) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, r_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&r_fake_var, &bin.right),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    //Both sides could be evaluated+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, Some(_)) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    self.substitute_right_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &r_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (None, None) => {+                                    //Substitute both with fake var and insert dependencies+                                    let mut fake_bin = bin.clone();+                                    fake_bin.right.expr = r_fake_var_expr;+                                    fake_bin.left.expr = l_fake_var_expr;+                                    let op = op_of(source_bin);+                                    self.dependencies_on_result_type.push((+                                        var.clone(),+                                        op,+                                        SpannedExpression::new(+                                            Expression::Binary(fake_bin),+                                            source_bin.span,+                                        ),+                                    ));+                                    None+                                }+                            }+                        }+                        //After here every invocation on get_shape_of_expr_or_insert_dependency(expr) should+                        //give a result shape+                        (Expression::Binary(_), _) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_expr_or_insert_dependency(+                                    &r_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.right,+                                ),+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, _) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    unreachable!("At this point shape should be deducable!")+                                }+                            }+                        }+                        (_, Expression::Binary(_)) => {+                            let (l_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let (r_fake_var, r_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_expr_or_insert_dependency(+                                    &l_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.left,+                                ),+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&r_fake_var, &bin.right),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (_, None) => {+                                    self.substitute_right_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &r_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (None, Some(_)) => {+                                    unreachable!("At this point shape should be deducable!")+                                }+                            }+                        }+                        (_, _) => {+                            let (l_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let (r_fake_var, _) = self.fake_var_generator.next_as_expr();+                            match (+                                self.get_shape_of_expr_or_insert_dependency(&l_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.left),+                                self.get_shape_of_expr_or_insert_dependency(&r_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.right)+                            ) {+                                ( Some(l_shape), Some(r_shape) ) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                _ => unreachable!("This should be unreachable as neither expr is real var or binary")++                            }+                        }+                    }+                }+                _ => Some(shape),+            })+        } else {+            trace!("> Could not deduce shape in expr");+            Ok(None)+        }+    }++    fn substitute_right_with_fake_var_and_insert_dependencies(+        &mut self,+        //Bin in which to substitute+        bin: &Binary,+        //The var with which to substitute (as usage and expr)+        r_fake_var_expr: &Expression,+        //The source bin having var on one side and above bin on other+        (source_bin, var): (&SpannedExpression, &VarUsage),+    ) {+        let mut fake_bin = Box::new(bin.clone());+        fake_bin.right.expr = r_fake_var_expr.clone();+        let op = op_of(source_bin);+        self.dependencies_on_result_type.push((+            var.clone(),+            op,+            SpannedExpression::new(Expression::Binary(fake_bin), source_bin.span),+        ));+    }+    fn substitute_left_with_fake_var_and_insert_dependencies(+        &mut self,+        //Bin in which to substitute+        bin: &Binary,+        //The var with which to substitute (as usage and expr)+        l_fake_var_expr: &Expression,+        //The source bin having var on one side and above bin on other+        (source_bin, var): (&SpannedExpression, &VarUsage),+    ) {+        let mut fake_bin = Box::new(bin.clone());+        fake_bin.left.expr = l_fake_var_expr.clone();+        let op = op_of(source_bin);+        self.dependencies_on_result_type.push((+            var.clone(),+            op,+            SpannedExpression::new(Expression::Binary(fake_bin), source_bin.span),+        ));+    }++    fn get_shapes_in_list_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        bin_spanned: &SpannedExpression,+        list: &[SpannedExpression],+        (_pipeline_idx, _pipeline): (usize, &Commands),+        _registry: &CommandRegistry,+    ) -> Option<Vec<SyntaxShape>> {+        let shapes_in_list = list+            .iter()+            .filter_map(|expr| {+                self.get_shape_of_expr_or_insert_dependency(+                    var,+                    (Operator::Equal, bin_spanned.span),+                    expr,+                )+            })+            .collect_vec();+        if shapes_in_list.is_empty() {+            None+        } else {+            Some(shapes_in_list)+        }+    }++    fn infer_shapes_between_var_and_expr(+        &mut self,+        (var, expr): (&VarUsage, &SpannedExpression),+        var_side: BinarySide,+        //Binary having expr on one side and var on other+        bin_spanned: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering shapes between var {:?} and expr {:?}", var, expr);+        let bin = match &bin_spanned.expr {+            Expression::Binary(bin) => bin,+            _ => unreachable!(),+        };+        if let Expression::Literal(Literal::Operator(op)) = bin.op.expr {+            match &op {+                //For || and && we insert shapes decay able to bool+                Operator::And | Operator::Or => {+                    let shapes = get_shapes_decay_able_to_bool();+                    // shapes.push(SyntaxShape::Math);+                    self.checked_insert(+                        &var,+                        VarShapeDeduction::from_usage_with_alternatives(&var.span, &shapes),+                    )?;+                }+                Operator::Contains | Operator::NotContains => {+                    self.checked_insert(+                        var,+                        vec![VarShapeDeduction::from_usage(+                            &var.span,+                            &SyntaxShape::String,+                        )],+                    )?;+                }+                Operator::In | Operator::NotIn => {+                    match var_side {+                        BinarySide::Left => match &expr.expr {+                            Expression::List(list) => {+                                if !list.is_empty() {+                                    let shapes_in_list = self+                                        .get_shapes_in_list_or_insert_dependency(+                                            var,+                                            bin_spanned,+                                            &list,+                                            (pipeline_idx, pipeline),+                                            registry,+                                        );+                                    match shapes_in_list {+                                        None => {}+                                        Some(shapes_in_list) => {+                                            self.checked_insert(+                                                var,+                                                VarShapeDeduction::from_usage_with_alternatives(+                                                    &var.span,+                                                    &shapes_in_list,+                                                ),+                                            )?;+                                        }+                                    }+                                }+                            }+                            //REVIEW is var in table legal? Shouldn't the rhs in "$p in [a b c]" be +                            //parsed as a list?+                            Expression::Table(_, _) => {}++                            Expression::Literal(_)+                            | Expression::ExternalWord+                            | Expression::Synthetic(_)+                            | Expression::Variable(_)+                            | Expression::Binary(_)+                            | Expression::Range(_)+                            | Expression::Block(_)+                            | Expression::Path(_)+                            | Expression::FilePath(_)+                            | Expression::ExternalCommand(_)+                            | Expression::Command+                            | Expression::Invocation(_)+                            | Expression::Boolean(_)+                            | Expression::Garbage => {unreachable!("Parser should have rejected code. In only applicable with rhs of type List")}+                        },+                        BinarySide::Right => {+                            self.checked_insert(+                                var,+                                VarShapeDeduction::from_usage_with_alternatives(+                                    &var.span,+                                    &[SyntaxShape::Table],+                                ),+                            )?;+                        }+                    }+                }+                Operator::Modulo => {+                    self.checked_insert(+                        var,+                        VarShapeDeduction::from_usage_with_alternatives(+                            &var.span,+                            &[SyntaxShape::Int, SyntaxShape::Number],+                        ),+                    )?;+                }+                Operator::Equal+                | Operator::NotEqual+                | Operator::LessThan+                | Operator::GreaterThan+                | Operator::LessThanOrEqual+                | Operator::GreaterThanOrEqual+                | Operator::Plus+                | Operator::Minus => {+                    if let Some(shape) = self.get_shape_of_binary_arg_or_insert_dependency(+                        (var, expr),+                        bin_spanned,+                        (pipeline_idx, pipeline),+                        registry,+                    )? {+                        match shape {+                            SyntaxShape::Int | SyntaxShape::Number => {+                                self.checked_insert(+                                    var,+                                    VarShapeDeduction::from_usage_with_alternatives(+                                        &var.span,+                                        &[SyntaxShape::Number, SyntaxShape::Int],+                                    ),+                                )?;+                            }+                            SyntaxShape::Unit => {+                                self.checked_insert(+                                    var,+                                    VarShapeDeduction::from_usage_with_alternatives(+                                        &var.span,+                                        &[SyntaxShape::Unit],+                                    ),+                                )?;+                            }+                            s => unreachable!(format!(+                                "Shape of {:?} should have failed at parsing stage",+                                s+                            )),+                        }+                    }+                }+                Operator::Multiply => {+                    if let Some(shape) = self.get_shape_of_binary_arg_or_insert_dependency(+                        (var, expr),+                        bin_spanned,+                        (pipeline_idx, pipeline),+                        registry,+                    )? {+                        if shape == SyntaxShape::Unit {+                            //TODO at the moment unit * unit is not possible+                            //As soon as more complex units land this changes!+                            self.checked_insert(+                                var,+                                vec![VarShapeDeduction::from_usage(+                                    &var.span,+                                    &SyntaxShape::Number,+                                )],+                            )?;+                        } else if shape == SyntaxShape::Number || shape == SyntaxShape::Int {+                            self.checked_insert(+                                var,+                                VarShapeDeduction::from_usage_with_alternatives(+                                    &var.span,+                                    &[SyntaxShape::Number, SyntaxShape::Unit],+                                ),+                            )?;+                        } else {+                            unreachable!("Only int or number or unit in binary with op = * allowed")+                        }+                    }+                }+                Operator::Divide => {+                    if let Some(shape) = self.get_shape_of_binary_arg_or_insert_dependency(+                        (var, expr),+                        bin_spanned,+                        (pipeline_idx, pipeline),+                        registry,+                    )? {+                        match shape {+                            SyntaxShape::Int | SyntaxShape::Number => {+                                match var_side {+                                    BinarySide::Left => {+                                        //Binary is on left, number / int on right+                                        self.checked_insert(+                                            var,+                                            VarShapeDeduction::from_usage_with_alternatives(+                                                &var.span,+                                                &[+                                                    SyntaxShape::Number,+                                                    SyntaxShape::Int,+                                                    SyntaxShape::Unit,+                                                ],+                                            ),+                                        )?;+                                    }+                                    BinarySide::Right => {+                                        //TODO currently no unit type is supports 1/unit. This+                                        //changes if there would be ever more complex unit types+                                        //e.G. Frequency+                                        self.checked_insert(+                                            var,+                                            VarShapeDeduction::from_usage_with_alternatives(+                                                &var.span,+                                                &[SyntaxShape::Number, SyntaxShape::Int],+                                            ),+                                        )?;+                                    }+                                }+                            }+                            SyntaxShape::Unit => {+                                match var_side {+                                    BinarySide::Left => {+                                        //Must be unit / unit

Do you mean the shape of division should be unit divided by unit? This might be too restrictive. Both of these are valid (or at least should be valid in the future):

12ms / 12ms
12ms / 12
LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it+                //therefore pipeline idx, pipeline and registry has to be passed here+                None+            }+            Expression::Literal(literal) => {+                match literal {+                    nu_protocol::hir::Literal::Number(number) => match number {+                        nu_protocol::hir::Number::Int(_) => Some(SyntaxShape::Int),+                        nu_protocol::hir::Number::Decimal(_) => Some(SyntaxShape::Number),+                    },+                    nu_protocol::hir::Literal::Size(_, _) => Some(SyntaxShape::Unit),+                    nu_protocol::hir::Literal::String(_) => Some(SyntaxShape::String),+                    //Rest should have failed at parsing stage?+                    nu_protocol::hir::Literal::GlobPattern(_) => Some(SyntaxShape::String),+                    nu_protocol::hir::Literal::Operator(_) => Some(SyntaxShape::Operator),+                    nu_protocol::hir::Literal::ColumnPath(_) => Some(SyntaxShape::ColumnPath),+                    nu_protocol::hir::Literal::Bare(_) => Some(SyntaxShape::String),+                }+            }+            //Synthetic are expressions that are generated by the parser and not inputed by the user+            //ExternalWord is anything sent to external commands (?)+            Expression::ExternalWord => Some(SyntaxShape::String),+            Expression::Synthetic(_) => Some(SyntaxShape::String),++            Expression::Binary(_) => Some(SyntaxShape::Math),+            Expression::Range(_) => Some(SyntaxShape::Range),+            Expression::List(_) => Some(SyntaxShape::Table),+            Expression::Boolean(_) => Some(SyntaxShape::String),++            Expression::Path(_) => Some(SyntaxShape::ColumnPath),+            Expression::FilePath(_) => Some(SyntaxShape::Path),+            Expression::Block(_) => Some(SyntaxShape::Block),+            Expression::ExternalCommand(_) => Some(SyntaxShape::String),+            Expression::Table(_, _) => Some(SyntaxShape::Table),+            Expression::Command => Some(SyntaxShape::String),+            Expression::Invocation(_) => Some(SyntaxShape::Block),+            Expression::Garbage => unreachable!("Should have failed at parsing stage"),+        }+    }++    fn get_shape_of_binary_arg_or_insert_dependency(+        &mut self,+        //var depending on shape of expr (arg)+        (var, expr): (&VarUsage, &SpannedExpression),+        //source_bin is binary having var on one and expr on other side+        source_bin: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<Option<SyntaxShape>, ShellError> {+        trace!("Getting shape of binary arg {:?} for var {:?}", expr, var);+        if let Some(shape) = self.get_shape_of_expr_or_insert_dependency(+            var,+            (op_of(source_bin), source_bin.span),+            expr,+        ) {+            trace!("> Found shape: {:?}", shape);+            Ok(match shape {+                SyntaxShape::Math => {+                    //If execution happens here, the situation is as follows:+                    //There is an Binary expression (source_bin) with a variable on one side+                    //and a binary (lets call it "deep binary") on the other:+                    //e.G. $var + 1 * 1+                    //Now we try to infer the shapes inside the deep binary, compute the resulting+                    //shape based on the operator (see get_result_shape_of) and return that.+                    //That won't work if one of the deeper binary left/right expr is a variable.+                    //Then we insert an element into+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //+                    //If the deeper binary contains a binary on one side, we check if that binary+                    //has a computable result type (e.G. has no variable in it) by recursively+                    //calling this function and if so return it.+                    //If the result type is not computable (the deep deep binary had a variable), we substitute+                    //the deep deep binary side of the deep binary with a variable (fake_var) and+                    //insert a dependency from the fake_var to the deep deep binary in+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //The $var on the source_bin will then (as described above) depend on the deep+                    //binary (as it has a variable (fake_var)) on one side.+                    //+                    //The dependencies gets resolved at the end, when most information about all+                    //variables is accessable.+                    //+                    //+                    //+                    //+                    //Expression is of type binary+                    //We have to descend deeper into tree+                    //And compute result shape based on operator+                    let bin = match &expr.expr {+                        Expression::Binary(bin) => bin,+                        _ => unreachable!("SyntaxShape::Math means expression binary"),+                    };+                    match (&bin.left.expr, &bin.right.expr) {+                        //$it should give shape in get_shape_of_expr_or_insert_dependency+                        //Therefore the following code is not correct!+                        ////Substitute+                        //(+                        //    Expression::Variable(Variable::It(_it_span)),+                        //    Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //)+                        //    | (+                        //        Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //        Expression::Variable(Variable::It(_it_span)),+                        //    ) => {+                        //        //TODO deduce type of $it and insert into+                        //        //dependencies_on_result_type+                        //        None+                        //    }+                        //(+                        //    Expression::Variable(Variable::It(_l_it)),+                        //    Expression::Variable(Variable::It(_r_it)),+                        //) => {+                        //    //TODO deduce type of $it and return it (based on operator)+                        //    None+                        //}+                        (+                            Expression::Variable(Variable::Other(_, _)),+                            Expression::Variable(Variable::Other(_, _)),+                        )+                        | (Expression::Variable(Variable::Other(_, _)), _)+                        | (_, Expression::Variable(Variable::Other(_, _))) => {+                            //Example of this case is: $foo + $bar * $baz+                            //foo = var (depending of shape of arg (bar * baz))+                            self.dependencies_on_result_type.push((+                                var.clone(),+                                op_of(source_bin),+                                expr.clone(),+                            ));+                            None+                        }+                        (Expression::Binary(_), Expression::Binary(_)) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, r_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&r_fake_var, &bin.right),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    //Both sides could be evaluated+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, Some(_)) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    self.substitute_right_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &r_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (None, None) => {+                                    //Substitute both with fake var and insert dependencies+                                    let mut fake_bin = bin.clone();+                                    fake_bin.right.expr = r_fake_var_expr;+                                    fake_bin.left.expr = l_fake_var_expr;+                                    let op = op_of(source_bin);+                                    self.dependencies_on_result_type.push((+                                        var.clone(),+                                        op,+                                        SpannedExpression::new(+                                            Expression::Binary(fake_bin),+                                            source_bin.span,+                                        ),+                                    ));+                                    None+                                }+                            }+                        }+                        //After here every invocation on get_shape_of_expr_or_insert_dependency(expr) should+                        //give a result shape+                        (Expression::Binary(_), _) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_expr_or_insert_dependency(+                                    &r_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.right,+                                ),+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, _) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    unreachable!("At this point shape should be deducable!")+                                }+                            }+                        }+                        (_, Expression::Binary(_)) => {+                            let (l_fake_var, _) = self.fake_var_generator.next_as_expr();

With this fake var pieces, I'm wondering if what we need is instead a flattening phase. That way, we don't need to embed it into the inference code, but instead can run a pass over the AST that flattens all the nested expressions and assigns the simple expressions to unique variables.

Like:

$var * 1 + 1

from above would be replaced with:

$unique_var_12345 = $var * 1
$unique_var_12346 = $unique_var_12345 + 1

This type of expansion is pretty common with compilers, and I believe if we do it correctly should help simplify the code here and also help us iron out any corner cases which might arise.

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it+                //therefore pipeline idx, pipeline and registry has to be passed here+                None+            }+            Expression::Literal(literal) => {+                match literal {+                    nu_protocol::hir::Literal::Number(number) => match number {+                        nu_protocol::hir::Number::Int(_) => Some(SyntaxShape::Int),+                        nu_protocol::hir::Number::Decimal(_) => Some(SyntaxShape::Number),+                    },+                    nu_protocol::hir::Literal::Size(_, _) => Some(SyntaxShape::Unit),+                    nu_protocol::hir::Literal::String(_) => Some(SyntaxShape::String),+                    //Rest should have failed at parsing stage?+                    nu_protocol::hir::Literal::GlobPattern(_) => Some(SyntaxShape::String),+                    nu_protocol::hir::Literal::Operator(_) => Some(SyntaxShape::Operator),+                    nu_protocol::hir::Literal::ColumnPath(_) => Some(SyntaxShape::ColumnPath),+                    nu_protocol::hir::Literal::Bare(_) => Some(SyntaxShape::String),+                }+            }+            //Synthetic are expressions that are generated by the parser and not inputed by the user+            //ExternalWord is anything sent to external commands (?)+            Expression::ExternalWord => Some(SyntaxShape::String),+            Expression::Synthetic(_) => Some(SyntaxShape::String),++            Expression::Binary(_) => Some(SyntaxShape::Math),+            Expression::Range(_) => Some(SyntaxShape::Range),+            Expression::List(_) => Some(SyntaxShape::Table),+            Expression::Boolean(_) => Some(SyntaxShape::String),++            Expression::Path(_) => Some(SyntaxShape::ColumnPath),+            Expression::FilePath(_) => Some(SyntaxShape::Path),+            Expression::Block(_) => Some(SyntaxShape::Block),+            Expression::ExternalCommand(_) => Some(SyntaxShape::String),+            Expression::Table(_, _) => Some(SyntaxShape::Table),+            Expression::Command => Some(SyntaxShape::String),+            Expression::Invocation(_) => Some(SyntaxShape::Block),+            Expression::Garbage => unreachable!("Should have failed at parsing stage"),+        }+    }++    fn get_shape_of_binary_arg_or_insert_dependency(+        &mut self,+        //var depending on shape of expr (arg)+        (var, expr): (&VarUsage, &SpannedExpression),+        //source_bin is binary having var on one and expr on other side+        source_bin: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<Option<SyntaxShape>, ShellError> {+        trace!("Getting shape of binary arg {:?} for var {:?}", expr, var);+        if let Some(shape) = self.get_shape_of_expr_or_insert_dependency(+            var,+            (op_of(source_bin), source_bin.span),+            expr,+        ) {+            trace!("> Found shape: {:?}", shape);+            Ok(match shape {+                SyntaxShape::Math => {+                    //If execution happens here, the situation is as follows:+                    //There is an Binary expression (source_bin) with a variable on one side+                    //and a binary (lets call it "deep binary") on the other:+                    //e.G. $var + 1 * 1+                    //Now we try to infer the shapes inside the deep binary, compute the resulting+                    //shape based on the operator (see get_result_shape_of) and return that.+                    //That won't work if one of the deeper binary left/right expr is a variable.+                    //Then we insert an element into+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //+                    //If the deeper binary contains a binary on one side, we check if that binary+                    //has a computable result type (e.G. has no variable in it) by recursively+                    //calling this function and if so return it.+                    //If the result type is not computable (the deep deep binary had a variable), we substitute+                    //the deep deep binary side of the deep binary with a variable (fake_var) and+                    //insert a dependency from the fake_var to the deep deep binary in+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //The $var on the source_bin will then (as described above) depend on the deep+                    //binary (as it has a variable (fake_var)) on one side.+                    //+                    //The dependencies gets resolved at the end, when most information about all+                    //variables is accessable.+                    //+                    //+                    //+                    //+                    //Expression is of type binary+                    //We have to descend deeper into tree+                    //And compute result shape based on operator+                    let bin = match &expr.expr {+                        Expression::Binary(bin) => bin,+                        _ => unreachable!("SyntaxShape::Math means expression binary"),+                    };+                    match (&bin.left.expr, &bin.right.expr) {+                        //$it should give shape in get_shape_of_expr_or_insert_dependency+                        //Therefore the following code is not correct!+                        ////Substitute+                        //(+                        //    Expression::Variable(Variable::It(_it_span)),+                        //    Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //)+                        //    | (+                        //        Expression::Variable(Variable::Other(_var_name, _var_span)),+                        //        Expression::Variable(Variable::It(_it_span)),+                        //    ) => {+                        //        //TODO deduce type of $it and insert into+                        //        //dependencies_on_result_type+                        //        None+                        //    }+                        //(+                        //    Expression::Variable(Variable::It(_l_it)),+                        //    Expression::Variable(Variable::It(_r_it)),+                        //) => {+                        //    //TODO deduce type of $it and return it (based on operator)+                        //    None+                        //}+                        (+                            Expression::Variable(Variable::Other(_, _)),+                            Expression::Variable(Variable::Other(_, _)),+                        )+                        | (Expression::Variable(Variable::Other(_, _)), _)+                        | (_, Expression::Variable(Variable::Other(_, _))) => {+                            //Example of this case is: $foo + $bar * $baz+                            //foo = var (depending of shape of arg (bar * baz))+                            self.dependencies_on_result_type.push((+                                var.clone(),+                                op_of(source_bin),+                                expr.clone(),+                            ));+                            None+                        }+                        (Expression::Binary(_), Expression::Binary(_)) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, r_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&r_fake_var, &bin.right),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    //Both sides could be evaluated+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, Some(_)) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    self.substitute_right_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &r_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (None, None) => {+                                    //Substitute both with fake var and insert dependencies+                                    let mut fake_bin = bin.clone();+                                    fake_bin.right.expr = r_fake_var_expr;+                                    fake_bin.left.expr = l_fake_var_expr;+                                    let op = op_of(source_bin);+                                    self.dependencies_on_result_type.push((+                                        var.clone(),+                                        op,+                                        SpannedExpression::new(+                                            Expression::Binary(fake_bin),+                                            source_bin.span,+                                        ),+                                    ));+                                    None+                                }+                            }+                        }+                        //After here every invocation on get_shape_of_expr_or_insert_dependency(expr) should+                        //give a result shape+                        (Expression::Binary(_), _) => {+                            let (l_fake_var, l_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let (r_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&l_fake_var, &bin.left),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                                self.get_shape_of_expr_or_insert_dependency(+                                    &r_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.right,+                                ),+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (None, _) => {+                                    self.substitute_left_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &l_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (Some(_), None) => {+                                    unreachable!("At this point shape should be deducable!")+                                }+                            }+                        }+                        (_, Expression::Binary(_)) => {+                            let (l_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let (r_fake_var, r_fake_var_expr) =+                                self.fake_var_generator.next_as_expr();+                            let fake_bin = change_op_to_assignment(expr.clone());+                            match (+                                self.get_shape_of_expr_or_insert_dependency(+                                    &l_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.left,+                                ),+                                self.get_shape_of_binary_arg_or_insert_dependency(+                                    (&r_fake_var, &bin.right),+                                    &fake_bin,+                                    (pipeline_idx, pipeline),+                                    registry,+                                )?,+                            ) {+                                (Some(l_shape), Some(r_shape)) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                (_, None) => {+                                    self.substitute_right_with_fake_var_and_insert_dependencies(+                                        bin,+                                        &r_fake_var_expr,+                                        (source_bin, var),+                                    );+                                    None+                                }+                                (None, Some(_)) => {+                                    unreachable!("At this point shape should be deducable!")+                                }+                            }+                        }+                        (_, _) => {+                            let (l_fake_var, _) = self.fake_var_generator.next_as_expr();+                            let (r_fake_var, _) = self.fake_var_generator.next_as_expr();+                            match (+                                self.get_shape_of_expr_or_insert_dependency(&l_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.left),+                                self.get_shape_of_expr_or_insert_dependency(&r_fake_var,+                                    (Operator::Equal, source_bin.span),+                                    &bin.right)+                            ) {+                                ( Some(l_shape), Some(r_shape) ) => {+                                    Some(get_result_shape_of(l_shape, &bin.op, r_shape)?)+                                }+                                _ => unreachable!("This should be unreachable as neither expr is real var or binary")++                            }+                        }+                    }+                }+                _ => Some(shape),+            })+        } else {+            trace!("> Could not deduce shape in expr");+            Ok(None)+        }+    }++    fn substitute_right_with_fake_var_and_insert_dependencies(+        &mut self,+        //Bin in which to substitute+        bin: &Binary,+        //The var with which to substitute (as usage and expr)+        r_fake_var_expr: &Expression,+        //The source bin having var on one side and above bin on other+        (source_bin, var): (&SpannedExpression, &VarUsage),+    ) {+        let mut fake_bin = Box::new(bin.clone());+        fake_bin.right.expr = r_fake_var_expr.clone();+        let op = op_of(source_bin);+        self.dependencies_on_result_type.push((+            var.clone(),+            op,+            SpannedExpression::new(Expression::Binary(fake_bin), source_bin.span),+        ));+    }+    fn substitute_left_with_fake_var_and_insert_dependencies(+        &mut self,+        //Bin in which to substitute+        bin: &Binary,+        //The var with which to substitute (as usage and expr)+        l_fake_var_expr: &Expression,+        //The source bin having var on one side and above bin on other+        (source_bin, var): (&SpannedExpression, &VarUsage),+    ) {+        let mut fake_bin = Box::new(bin.clone());+        fake_bin.left.expr = l_fake_var_expr.clone();+        let op = op_of(source_bin);+        self.dependencies_on_result_type.push((+            var.clone(),+            op,+            SpannedExpression::new(Expression::Binary(fake_bin), source_bin.span),+        ));+    }++    fn get_shapes_in_list_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        bin_spanned: &SpannedExpression,+        list: &[SpannedExpression],+        (_pipeline_idx, _pipeline): (usize, &Commands),+        _registry: &CommandRegistry,+    ) -> Option<Vec<SyntaxShape>> {+        let shapes_in_list = list+            .iter()+            .filter_map(|expr| {+                self.get_shape_of_expr_or_insert_dependency(+                    var,+                    (Operator::Equal, bin_spanned.span),+                    expr,+                )+            })+            .collect_vec();+        if shapes_in_list.is_empty() {+            None+        } else {+            Some(shapes_in_list)+        }+    }++    fn infer_shapes_between_var_and_expr(+        &mut self,+        (var, expr): (&VarUsage, &SpannedExpression),+        var_side: BinarySide,+        //Binary having expr on one side and var on other+        bin_spanned: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering shapes between var {:?} and expr {:?}", var, expr);+        let bin = match &bin_spanned.expr {+            Expression::Binary(bin) => bin,+            _ => unreachable!(),+        };+        if let Expression::Literal(Literal::Operator(op)) = bin.op.expr {+            match &op {+                //For || and && we insert shapes decay able to bool+                Operator::And | Operator::Or => {+                    let shapes = get_shapes_decay_able_to_bool();+                    // shapes.push(SyntaxShape::Math);+                    self.checked_insert(+                        &var,+                        VarShapeDeduction::from_usage_with_alternatives(&var.span, &shapes),+                    )?;+                }+                Operator::Contains | Operator::NotContains => {+                    self.checked_insert(+                        var,+                        vec![VarShapeDeduction::from_usage(+                            &var.span,+                            &SyntaxShape::String,+                        )],+                    )?;+                }+                Operator::In | Operator::NotIn => {+                    match var_side {+                        BinarySide::Left => match &expr.expr {+                            Expression::List(list) => {+                                if !list.is_empty() {+                                    let shapes_in_list = self+                                        .get_shapes_in_list_or_insert_dependency(+                                            var,+                                            bin_spanned,+                                            &list,+                                            (pipeline_idx, pipeline),+                                            registry,+                                        );+                                    match shapes_in_list {+                                        None => {}+                                        Some(shapes_in_list) => {+                                            self.checked_insert(+                                                var,+                                                VarShapeDeduction::from_usage_with_alternatives(+                                                    &var.span,+                                                    &shapes_in_list,+                                                ),+                                            )?;+                                        }+                                    }+                                }+                            }+                            //REVIEW is var in table legal? Shouldn't the rhs in "$p in [a b c]" be +                            //parsed as a list?

We've got a few things to clean up (not as part of this PR) in the names of things. The Table below should actually be an inner table. You're right, that in your example that should be a list.

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it+                //therefore pipeline idx, pipeline and registry has to be passed here+                None+            }+            Expression::Literal(literal) => {+                match literal {+                    nu_protocol::hir::Literal::Number(number) => match number {+                        nu_protocol::hir::Number::Int(_) => Some(SyntaxShape::Int),+                        nu_protocol::hir::Number::Decimal(_) => Some(SyntaxShape::Number),+                    },+                    nu_protocol::hir::Literal::Size(_, _) => Some(SyntaxShape::Unit),+                    nu_protocol::hir::Literal::String(_) => Some(SyntaxShape::String),+                    //Rest should have failed at parsing stage?+                    nu_protocol::hir::Literal::GlobPattern(_) => Some(SyntaxShape::String),+                    nu_protocol::hir::Literal::Operator(_) => Some(SyntaxShape::Operator),+                    nu_protocol::hir::Literal::ColumnPath(_) => Some(SyntaxShape::ColumnPath),+                    nu_protocol::hir::Literal::Bare(_) => Some(SyntaxShape::String),+                }+            }+            //Synthetic are expressions that are generated by the parser and not inputed by the user+            //ExternalWord is anything sent to external commands (?)+            Expression::ExternalWord => Some(SyntaxShape::String),+            Expression::Synthetic(_) => Some(SyntaxShape::String),++            Expression::Binary(_) => Some(SyntaxShape::Math),+            Expression::Range(_) => Some(SyntaxShape::Range),+            Expression::List(_) => Some(SyntaxShape::Table),+            Expression::Boolean(_) => Some(SyntaxShape::String),++            Expression::Path(_) => Some(SyntaxShape::ColumnPath),+            Expression::FilePath(_) => Some(SyntaxShape::Path),+            Expression::Block(_) => Some(SyntaxShape::Block),+            Expression::ExternalCommand(_) => Some(SyntaxShape::String),+            Expression::Table(_, _) => Some(SyntaxShape::Table),+            Expression::Command => Some(SyntaxShape::String),+            Expression::Invocation(_) => Some(SyntaxShape::Block),+            Expression::Garbage => unreachable!("Should have failed at parsing stage"),+        }+    }++    fn get_shape_of_binary_arg_or_insert_dependency(+        &mut self,+        //var depending on shape of expr (arg)+        (var, expr): (&VarUsage, &SpannedExpression),+        //source_bin is binary having var on one and expr on other side+        source_bin: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<Option<SyntaxShape>, ShellError> {+        trace!("Getting shape of binary arg {:?} for var {:?}", expr, var);+        if let Some(shape) = self.get_shape_of_expr_or_insert_dependency(+            var,+            (op_of(source_bin), source_bin.span),+            expr,+        ) {+            trace!("> Found shape: {:?}", shape);+            Ok(match shape {+                SyntaxShape::Math => {+                    //If execution happens here, the situation is as follows:+                    //There is an Binary expression (source_bin) with a variable on one side+                    //and a binary (lets call it "deep binary") on the other:+                    //e.G. $var + 1 * 1+                    //Now we try to infer the shapes inside the deep binary, compute the resulting+                    //shape based on the operator (see get_result_shape_of) and return that.+                    //That won't work if one of the deeper binary left/right expr is a variable.+                    //Then we insert an element into+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //+                    //If the deeper binary contains a binary on one side, we check if that binary+                    //has a computable result type (e.G. has no variable in it) by recursively+                    //calling this function and if so return it.+                    //If the result type is not computable (the deep deep binary had a variable), we substitute+                    //the deep deep binary side of the deep binary with a variable (fake_var) and+                    //insert a dependency from the fake_var to the deep deep binary in+                    //VarSyntaxShapeDeductor.dependencies_on_result_type+                    //The $var on the source_bin will then (as described above) depend on the deep+                    //binary (as it has a variable (fake_var)) on one side.+                    //+                    //The dependencies gets resolved at the end, when most information about all+                    //variables is accessable.+                    //+                    //+                    //+                    //+                    //Expression is of type binary+                    //We have to descend deeper into tree+                    //And compute result shape based on operator+                    let bin = match &expr.expr {+                        Expression::Binary(bin) => bin,+                        _ => unreachable!("SyntaxShape::Math means expression binary"),+                    };+                    match (&bin.left.expr, &bin.right.expr) {+                        //$it should give shape in get_shape_of_expr_or_insert_dependency+                        //Therefore the following code is not correct!

Where does this code come from?

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument

Can you say a bit more about "we assume every variable in an optional positional is used as this optional argument"? I'm not sure what this means.

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it+                //therefore pipeline idx, pipeline and registry has to be passed here+                None+            }+            Expression::Literal(literal) => {+                match literal {+                    nu_protocol::hir::Literal::Number(number) => match number {+                        nu_protocol::hir::Number::Int(_) => Some(SyntaxShape::Int),+                        nu_protocol::hir::Number::Decimal(_) => Some(SyntaxShape::Number),+                    },+                    nu_protocol::hir::Literal::Size(_, _) => Some(SyntaxShape::Unit),+                    nu_protocol::hir::Literal::String(_) => Some(SyntaxShape::String),+                    //Rest should have failed at parsing stage?+                    nu_protocol::hir::Literal::GlobPattern(_) => Some(SyntaxShape::String),+                    nu_protocol::hir::Literal::Operator(_) => Some(SyntaxShape::Operator),+                    nu_protocol::hir::Literal::ColumnPath(_) => Some(SyntaxShape::ColumnPath),+                    nu_protocol::hir::Literal::Bare(_) => Some(SyntaxShape::String),+                }+            }+            //Synthetic are expressions that are generated by the parser and not inputed by the user+            //ExternalWord is anything sent to external commands (?)+            Expression::ExternalWord => Some(SyntaxShape::String),+            Expression::Synthetic(_) => Some(SyntaxShape::String),++            Expression::Binary(_) => Some(SyntaxShape::Math),+            Expression::Range(_) => Some(SyntaxShape::Range),+            Expression::List(_) => Some(SyntaxShape::Table),+            Expression::Boolean(_) => Some(SyntaxShape::String),++            Expression::Path(_) => Some(SyntaxShape::ColumnPath),+            Expression::FilePath(_) => Some(SyntaxShape::Path),+            Expression::Block(_) => Some(SyntaxShape::Block),+            Expression::ExternalCommand(_) => Some(SyntaxShape::String),+            Expression::Table(_, _) => Some(SyntaxShape::Table),+            Expression::Command => Some(SyntaxShape::String),+            Expression::Invocation(_) => Some(SyntaxShape::Block),+            Expression::Garbage => unreachable!("Should have failed at parsing stage"),+        }+    }++    fn get_shape_of_binary_arg_or_insert_dependency(+        &mut self,+        //var depending on shape of expr (arg)+        (var, expr): (&VarUsage, &SpannedExpression),+        //source_bin is binary having var on one and expr on other side+        source_bin: &SpannedExpression,+        (pipeline_idx, pipeline): (usize, &Commands),+        registry: &CommandRegistry,+    ) -> Result<Option<SyntaxShape>, ShellError> {+        trace!("Getting shape of binary arg {:?} for var {:?}", expr, var);+        if let Some(shape) = self.get_shape_of_expr_or_insert_dependency(+            var,+            (op_of(source_bin), source_bin.span),+            expr,+        ) {+            trace!("> Found shape: {:?}", shape);+            Ok(match shape {+                SyntaxShape::Math => {+                    //If execution happens here, the situation is as follows:+                    //There is an Binary expression (source_bin) with a variable on one side+                    //and a binary (lets call it "deep binary") on the other:+                    //e.G. $var + 1 * 1+                    //Now we try to infer the shapes inside the deep binary, compute the resulting+                    //shape based on the operator (see get_result_shape_of) and return that.+                    //That won't work if one of the deeper binary left/right expr is a variable.+                    //Then we insert an element into

Can you say more about what it means that this inference won't work if there is a binary expression that has a variable deeper inside of it?

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as+    // https://github.com/nushell/rfcs/pull/3+    // didn't land+    // self.infer_shapes_in_rows(_rows)?;+    //fn infer_shape_in_column(+    //    &mut self,+    //    var: &VarUsage,+    //    col_idx: usize,+    //    rows: &[Vec<SpannedExpression>],+    //) -> Result<(), ShellError> {+    //    //Within a col there is equal type forcing+    //    let (op, span) = (Operator::Equal, rows[col_idx][0].span);+    //    rows.iter()+    //        .filter_map(|r| r.get(col_idx))+    //        .filter_map(|cell| self.get_shape_of_expr_or_insert_dependency(var, (op, span), cell))+    //        .next()+    //        .map_or(Ok(()), |shape| {+    //            self.checked_insert(var, vec![VarShapeDeduction::from_usage(&var.span, &shape)])?;+    //            Ok(())+    //        })+    //}+    //fn infer_shapes_in_rows(&mut self, rows: &[Vec<SpannedExpression>]) -> Result<(), ShellError> {+    //    //Iterate over all cells+    //    for (_row_idx, _row) in rows.iter().enumerate() {+    //        for (col_idx, cell) in _row.iter().enumerate() {+    //            //if cell is var+    //            if let Expression::Variable(Variable::Other(name, span)) = &cell.expr {+    //                let var = VarUsage::new(name, span);+    //                self.infer_shape_in_column(&var, col_idx, rows)?;+    //            }+    //        }+    //    }+    //    Ok(())+    //}++    fn get_shape_of_expr_or_insert_dependency(+        &mut self,+        var: &VarUsage,+        (op, span): (Operator, Span),+        expr: &SpannedExpression,+    ) -> Option<SyntaxShape> {+        match &expr.expr {+            Expression::Variable(Variable::Other(name, _)) => {+                self.dependencies+                    .push((var.clone(), (op, span), VarUsage::new(name, &expr.span)));+                None+            }+            Expression::Variable(Variable::It(_)) => {+                //TODO infer tpye of $it

typo

LhKipp

comment created time in 5 days

Pull request review commentnushell/nushell

Change alias shape inference to proposal of RFC#4

+use crate::CommandRegistry;+use nu_errors::ShellError;+use nu_parser::SignatureRegistry;+use nu_protocol::{+    hir::{+        Binary, Block, ClassifiedCommand, Commands, Expression, Literal, NamedArguments,+        NamedValue, Operator, SpannedExpression, Variable,+    },+    NamedType, PositionalType, Signature, SyntaxShape,+};+use nu_source::Span;+use serde::{Deserialize, Serialize};+use std::{collections::HashMap, hash::Hash};++use itertools::{merge_join_by, EitherOrBoth, Itertools};+use log::trace;++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarDeclaration {+    pub name: String,+    // type_decl: Option<UntaggedValue>,+    // scope: ?+    pub span: Span,+}++#[derive(Debug, Clone)]+pub enum Deduction {+    VarShapeDeduction(Vec<VarShapeDeduction>),+    //A deduction for VarArgs will have a different layout than for a normal var+    //Therefore Deduction is implemented as a enum+    // VarArgShapeDeduction(VarArgShapeDeduction),+}++// That would be one possible layout for a var arg shape deduction+// #[derive(Debug, Clone, Serialize, Deserialize)]+// pub struct VarArgShapeDeduction {+//     /// Spans pointing to the source of the deduction.+//     /// The spans locate positions within the tag of var_decl+//     pub deduced_from: Vec<Span>,+//     pub pos_shapes: Vec<(PositionalType, String)>,+//     pub rest_shape: Option<(SyntaxShape, String)>,+// }++#[derive(Debug, Clone, Serialize, Deserialize)]+pub struct VarShapeDeduction {+    pub deduction: SyntaxShape,+    /// Spans pointing to the source of the deduction.+    /// The spans locate positions within the tag of var_decl+    pub deducted_from: Vec<Span>,+}++impl VarShapeDeduction {+    pub fn from_usage(usage: &Span, deduced_shape: &SyntaxShape) -> VarShapeDeduction {+        VarShapeDeduction {+            deduction: *deduced_shape,+            deducted_from: vec![*usage],+        }+    }++    pub fn from_usage_with_alternatives(+        usage: &Span,+        alternatives: &[SyntaxShape],+    ) -> Vec<VarShapeDeduction> {+        alternatives+            .iter()+            .map(|shape| VarShapeDeduction::from_usage(usage, shape))+            .collect()+    }+}++struct FakeVarGen {+    counter: usize,+    fake_var_prefix: String,+}+impl FakeVarGen {+    pub fn new() -> Self {+        FakeVarGen {+            counter: 0,+            fake_var_prefix: "$DFSasfjqiDFJSnSbwbqWF".to_string(),+        }+    }+    pub fn next(&mut self) -> String {+        let mut fake_var = self.fake_var_prefix.clone();+        fake_var.push_str(&self.counter.to_string());+        self.counter += 1;+        fake_var+    }++    pub fn next_as_expr(&mut self) -> (VarUsage, Expression) {+        let var = self.next();+        (+            VarUsage::new(&var, &Span::unknown()),+            Expression::Variable(Variable::Other(var, Span::unknown())),+        )+    }+}++pub struct VarSyntaxShapeDeductor {+    //Initial set of caller provided var declarations+    var_declarations: Vec<VarDeclaration>,+    //Inferences for variables+    inferences: HashMap<VarUsage, Deduction>,+    //Var depending on another var via a operator+    //First is a variable+    //Second is a operator+    //Third is a variable+    dependencies: Vec<(VarUsage, (Operator, Span), VarUsage)>,+    //A var depending on the result type of a spanned_expr+    //First argument is var,+    //Second is binary containing var op and result_bin_expr+    //Third is binary expr, which result shape var depends on+    //This list is populated for binaries like: $var + $baz * $bar+    dependencies_on_result_type: Vec<(VarUsage, Operator, SpannedExpression)>,++    fake_var_generator: FakeVarGen,+}++#[derive(Clone, Debug, Eq)]+pub struct VarUsage {+    pub name: String,+    /// Span describing where this var is used+    pub span: Span,+    //See below+    //pub scope: ?+}+impl VarUsage {+    pub fn new(name: &str, span: &Span) -> VarUsage {+        VarUsage {+            name: name.to_string(),+            span: *span,+        }+    }+}++impl PartialEq<VarUsage> for VarUsage {+    // When searching through the expressions, only the name of the+    // Variable is available. (TODO And their scope). Their full definition is not available.+    // Therefore the equals relationship is relaxed+    fn eq(&self, other: &VarUsage) -> bool {+        // TODO when scripting is available scope has to be respected+        self.name == other.name+        // && self.scope == other.scope+    }+}++impl Hash for VarUsage {+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {+        self.name.hash(state)+    }+}++impl From<VarDeclaration> for VarUsage {+    fn from(decl: VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}+impl From<&VarDeclaration> for VarUsage {+    fn from(decl: &VarDeclaration) -> Self {+        //Span unknown as var can be used in multiple places including none+        VarUsage::new(&decl.name, &Span::unknown())+    }+}++//REVIEW these 4 functions if correct types are returned+fn get_shapes_allowed_in_table_header() -> Vec<SyntaxShape> {+    vec![SyntaxShape::String]+}++fn get_shapes_allowed_in_path() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int, SyntaxShape::String]+}++fn get_shapes_decay_able_to_bool() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn get_shapes_allowed_in_range() -> Vec<SyntaxShape> {+    vec![SyntaxShape::Int]+}++fn op_of(bin: &SpannedExpression) -> Operator {+    match &bin.expr {+        Expression::Binary(bin) => match bin.op.expr {+            Expression::Literal(Literal::Operator(oper)) => oper,+            _ => unreachable!(),+        },+        _ => unreachable!(),+    }+}+fn change_op_to_assignment(mut bin: SpannedExpression) -> SpannedExpression {+    match &mut bin.expr {+        Expression::Binary(bin) => {+            match &mut bin.op.expr {+                Expression::Literal(Literal::Operator(op)) => {+                    //Currently there is no assignment operator.+                    //Plus does the same thing+                    *op = Operator::Equal;+                }+                _ => unreachable!(),+            }+        }+        _ => unreachable!(),+    }+    bin+}++//TODO in the future there should be a unit interface+//which offers this functionality; SyntaxShape::Unit would then be+//SyntaxShape::Unit(UnitType)+/// Get the resulting type if op is applied to l_shape and r_shape+/// Throws error if types are not coerceable+///+fn get_result_shape_of(+    l_shape: SyntaxShape,+    op_expr: &SpannedExpression,+    r_shape: SyntaxShape,+) -> Result<SyntaxShape, ShellError> {+    let op = match op_expr.expr {+        Expression::Literal(Literal::Operator(op)) => op,+        _ => unreachable!("Passing anything but the op expr is invalid"),+    };+    //TODO one should check that the types are coerceable.+    //There is some code for that in the evaluator already.+    //One might reuse it.+    //For now we ignore this issue+    Ok(match op {+        Operator::Equal+        | Operator::NotEqual+        | Operator::LessThan+        | Operator::GreaterThan+        | Operator::In+        | Operator::NotIn+        | Operator::And+        | Operator::Or+        | Operator::LessThanOrEqual+        | Operator::GreaterThanOrEqual+        | Operator::Contains+        | Operator::NotContains => {+            //TODO introduce syntaxshape boolean+            SyntaxShape::Int+        }+        Operator::Plus | Operator::Minus => {+            //l_type +/- r_type gives l_type again (if no weird coercion)+            l_shape+        }+        Operator::Multiply => {+            if l_shape == SyntaxShape::Unit || r_shape == SyntaxShape::Unit {+                SyntaxShape::Unit+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Divide => {+            if l_shape == r_shape {+                SyntaxShape::Number+            } else if l_shape == SyntaxShape::Unit {+                l_shape+            } else {+                SyntaxShape::Number+            }+        }+        Operator::Modulo => SyntaxShape::Number,+    })+}++enum BinarySide {+    Left,+    Right,+}++impl VarSyntaxShapeDeductor {+    /// Deduce vars_to_find in block.+    /// Returns: Mapping from var_to_find -> Vec<shape_deduction>+    /// in which each shape_deduction is one possible deduction for the variable.+    /// If a variable is used in at least 2 places with different+    /// required shapes, that do not coerce into each other,+    /// an error is returned.+    /// If Option<Deduction> is None, no deduction can be made (for example if+    /// the variable is not present in the block).+    pub fn infer_vars(+        vars_to_find: &[VarDeclaration],+        block: &Block,+        registry: &CommandRegistry,+    ) -> Result<Vec<(VarDeclaration, Option<Deduction>)>, ShellError> {+        trace!("Deducing shapes for vars: {:?}", vars_to_find);++        let mut deducer = VarSyntaxShapeDeductor {+            var_declarations: vars_to_find.to_owned(),+            inferences: HashMap::new(),+            // block,+            dependencies: Vec::new(),+            dependencies_on_result_type: Vec::new(),+            fake_var_generator: FakeVarGen::new(),+        };+        deducer.infer_shape(block, registry)?;++        deducer.solve_dependencies();+        trace!("Found shapes for vars: {:?}", deducer.inferences);++        Ok(deducer+            .var_declarations+            .iter()+            .map(|decl| {+                let usage: VarUsage = decl.into();+                let deduction = match deducer.inferences.get(&usage) {+                    Some(vec) => Some(vec.clone()),+                    None => None,+                };+                (decl.clone(), deduction)+            })+            .collect())+    }++    fn infer_shape(&mut self, block: &Block, registry: &CommandRegistry) -> Result<(), ShellError> {+        trace!("Infering vars in shape");+        for pipeline in &block.block {+            self.infer_pipeline(pipeline, registry)?;+        }+        Ok(())+    }++    pub fn infer_pipeline(+        &mut self,+        pipeline: &Commands,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in pipeline");+        for (cmd_pipeline_idx, classified) in pipeline.list.iter().enumerate() {+            match &classified {+                ClassifiedCommand::Internal(internal) => {+                    if let Some(signature) = registry.get(&internal.name) {+                        //When the signature is given vars directly used as named or positional+                        //arguments can be deduced+                        //e.G. cp $var1 $var2+                        if let Some(positional) = &internal.args.positional {+                            //Infer shapes in positional+                            self.infer_shapes_based_on_signature_positional(+                                positional, &signature,+                            )?;+                        }+                        if let Some(named) = &internal.args.named {+                            //Infer shapes in named+                            self.infer_shapes_based_on_signature_named(named, &signature)?;+                        }+                    }+                    //vars in expressions can be deduced by their usage+                    //e.G. 1..$var ($var is of type Int)+                    if let Some(positional) = &internal.args.positional {+                        //Infer shapes in positional+                        for (_pos_idx, pos_expr) in positional.iter().enumerate() {+                            self.infer_shapes_in_expr(+                                (cmd_pipeline_idx, pipeline),+                                pos_expr,+                                registry,+                            )?;+                        }+                    }+                    if let Some(named) = &internal.args.named {+                        trace!("Infering vars in named exprs");+                        for (_name, val) in named.iter() {+                            if let NamedValue::Value(_, named_expr) = val {+                                self.infer_shapes_in_expr(+                                    (cmd_pipeline_idx, pipeline),+                                    named_expr,+                                    registry,+                                )?;+                            }+                        }+                    }+                }+                ClassifiedCommand::Expr(spanned_expr) => {+                    trace!(+                        "Infering shapes in ClassifiedCommand::Expr: {:?}",+                        spanned_expr+                    );+                    self.infer_shapes_in_expr(+                        (cmd_pipeline_idx, pipeline),+                        spanned_expr,+                        registry,+                    )?;+                }+                ClassifiedCommand::Dynamic(_) | ClassifiedCommand::Error(_) => unimplemented!(),+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_positional(+        &mut self,+        positionals: &[SpannedExpression],+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in positionals");+        //TODO currently correct inference for optional positionals is not implemented.+        // See https://github.com/nushell/nushell/pull/2486 for a discussion about this+        // For now we assume every variable in an optional positional is used as this optional+        // argument+        trace!("Positionals len: {:?}", positionals.len());+        for (pos_idx, positional) in positionals.iter().enumerate().rev() {+            trace!("Handling pos_idx: {:?} of type: {:?}", pos_idx, positional);+            if let Expression::Variable(Variable::Other(var_name, _)) = &positional.expr {+                let deduced_shape = {+                    if pos_idx >= signature.positional.len() {+                        if let Some((shape, _)) = &signature.rest_positional {+                            Some(shape)+                        } else {+                            None+                        }+                    } else {+                        match &signature.positional[pos_idx].0 {+                            PositionalType::Mandatory(_, shape)+                            | PositionalType::Optional(_, shape) => Some(shape),+                        }+                    }+                };+                trace!(+                    "Found var: {:?} in positional_idx: {:?} of shape: {:?}",+                    var_name,+                    pos_idx,+                    deduced_shape+                );+                if let Some(shape) = deduced_shape {+                    self.checked_insert(+                        &VarUsage::new(var_name, &positional.span),+                        vec![VarShapeDeduction::from_usage(&positional.span, shape)],+                    )?;+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_based_on_signature_named(+        &mut self,+        named: &NamedArguments,+        signature: &Signature,+    ) -> Result<(), ShellError> {+        trace!("Infering vars in named");+        for (name, val) in named.iter() {+            if let NamedValue::Value(span, spanned_expr) = &val {+                if let Expression::Variable(Variable::Other(var_name, _)) = &spanned_expr.expr {+                    if let Some((named_type, _)) = signature.named.get(name) {+                        if let NamedType::Mandatory(_, shape) | NamedType::Optional(_, shape) =+                            named_type+                        {+                            trace!(+                                "Found var: {:?} in named: {:?} of shape: {:?}",+                                var_name,+                                name,+                                shape+                            );+                            self.checked_insert(+                                &VarUsage::new(var_name, span),+                                vec![VarShapeDeduction::from_usage(span, shape)],+                            )?;+                        }+                    }+                }+            }+        }+        Ok(())+    }++    fn infer_shapes_in_expr(+        &mut self,+        (pipeline_idx, pipeline): (usize, &Commands),+        spanned_expr: &SpannedExpression,+        registry: &CommandRegistry,+    ) -> Result<(), ShellError> {+        match &spanned_expr.expr {+            Expression::Binary(_) => {+                trace!("Infering vars in bin expr");+                self.infer_shapes_in_binary_expr((pipeline_idx, pipeline), spanned_expr, registry)?;+            }+            Expression::Block(b) => {+                trace!("Infering vars in block");+                self.infer_shape(&b, registry)?;+            }+            Expression::Path(path) => {+                trace!("Infering vars in path");+                match &path.head.expr {+                    //PathMember can't be var yet (?)+                    //TODO Iterate over path parts and find var when implemented+                    Expression::Invocation(b) => self.infer_shape(&b, registry)?,+                    Expression::Variable(Variable::Other(var_name, span)) => {+                        self.checked_insert(+                            &VarUsage::new(var_name, span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_path(),+                            ),+                        )?;+                    }+                    _ => (),+                }+            }+            Expression::Range(range) => {+                trace!("Infering vars in range");+                if let Some(range_left) = &range.left {+                    if let Expression::Variable(Variable::Other(var_name, _)) = &range_left.expr {+                        self.checked_insert(+                            &VarUsage::new(var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &spanned_expr.span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+                if let Some(range_right) = &range.right {+                    if let Expression::Variable(Variable::Other(var_name, span)) = &range_right.expr+                    {+                        self.checked_insert(+                            &VarUsage::new(&var_name, &spanned_expr.span),+                            VarShapeDeduction::from_usage_with_alternatives(+                                &span,+                                &get_shapes_allowed_in_range(),+                            ),+                        )?;+                    }+                }+            }+            Expression::List(inner_exprs) => {+                trace!("Infering vars in list");+                for expr in inner_exprs {+                    self.infer_shapes_in_expr((pipeline_idx, pipeline), expr, registry)?;+                }+            }+            Expression::Invocation(invoc) => {+                trace!("Infering vars in invocation: {:?}", invoc);+                self.infer_shape(invoc, registry)?;+            }+            Expression::Table(header, _rows) => {+                self.infer_shapes_in_table_header(header)?;+                // Shapes within columns can be heterogenous as long as+                // https://github.com/nushell/rfcs/pull/3+                // didn't land+                // self.infer_shapes_in_rows(_rows)?;+            }+            Expression::Variable(_) => {}+            Expression::Literal(_) => {}+            Expression::ExternalWord => {}+            Expression::Synthetic(_) => {}+            Expression::FilePath(_) => {}+            Expression::ExternalCommand(_) => {}+            Expression::Command => {}+            Expression::Boolean(_) => {}+            Expression::Garbage => {}+        };++        Ok(())+    }++    fn infer_shapes_in_table_header(+        &mut self,+        header: &[SpannedExpression],+    ) -> Result<(), ShellError> {+        for expr in header {+            if let Expression::Variable(Variable::Other(name, _)) = &expr.expr {+                let var = VarUsage::new(name, &expr.span);+                self.checked_insert(+                    &var,+                    VarShapeDeduction::from_usage_with_alternatives(+                        &var.span,+                        &get_shapes_allowed_in_table_header(),+                    ),+                )?;+            }+        }+        Ok(())+    }++    // Shapes within columns can be heterogenous as long as

Was this section of commented code intended?

LhKipp

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventnushell/nushell

Chris Gillespie

commit sha 1d833ef9726dff383d5e01929d884359045807e0

Set weather chars as emoji only (#2691)

view details

push time in 5 days

PR merged nushell/nushell

Set weather chars as emoji only

Closes #2657

  1. Set weather chars as emoji only to prevent issues with column rendering due to ansi bytes
echo [ [name, icon]; ['Clear', $(char sun)] ['Clouds', $(char clouds)] ['Rain', $(char rain)] ['Fog', $(char fog)] ['Snow', $(char snow)] ['Thunderstorm', $(char thunderstorm)] ]

image

Clouds and snow look kind of bland though... maybe there is a better choice of unicode.

+9 -9

1 comment

1 changed file

gillespiecd

pr closed time in 5 days

issue closednushell/nushell

Table wrap of a utf-8 value gives incorrect width

Describe the bug

> echo [[bar]; [$(char rain)] ]
+---+-------------+
| # |     bar     |
+---+-------------+
| 0 | ☔ |
+---+-------------+

To Reproduce Steps to reproduce the behavior:

  1. See above

Expected behavior The above should be able to have the table properly aligned

Screenshots

Screenshot from 2020-10-10 15-51-07

closed time in 5 days

jonathandturner

pull request commentnushell/nushell

Set weather chars as emoji only

Thanks!

gillespiecd

comment created time in 5 days

pull request commentnushell/nushell

improve error message for CoerceError

@peterthejohnston - if you're interested in continuing on this PR, I think it's a good direction that will help improve the errors

peterthejohnston

comment created time in 5 days

delete branch jonathandturner/nushell

delete branch : draft_of_new_vars

delete time in 5 days

PR closed nushell/nushell

finish draft of new vars

An alternate draft to #2604

In this one, we introduce a few different variables. This PR is meant more for experimentation, so that we can play with this approach and see what we like and don't like about it.

In this, we now have three special variables:

  • $row - the current row. When used, it will do each-expansion and iterate over each row
  • $table - the whole stream as a table. When used, the whole table is collected and available as a value
  • $it - the first value in the stream. When used, only one value is read from the stream and used

cc @andrasio

+411 -146

0 comment

23 changed files

jonathandturner

pr closed time in 5 days

delete branch jonathandturner/nushell

delete branch : remove_it_expansion

delete time in 5 days

PR closed nushell/nushell

Remove it expansion

DRAFT

This PR removes it-expansion. We've had a few points of feedback that it can feel confusing to use it-expansion, as you don't ever get to see exactly what is being expanded or how. Instead, people have asked that each is required and that in Nu this is used to specifically point to when we're processing things a row at a time.

You can see the kinds of changes this causes by looking in the tests that were updated in this PR. Ignoring dropping the unnecessary | echo $it updates, you'll see the main set of updates. String interpolation and uses of $it including $it variable paths now will need each wrapping around them.

Unrelated, but part of PR: I fixed an issue with stdin into external commands. We can take this separately, but I wasn't able to pass the test suite reliably without it.

+241 -626

0 comment

78 changed files

jonathandturner

pr closed time in 5 days

push eventnushell/nushell

morbatex

commit sha 0d8064ed2d9b2ebd609256dfea1a95b79616163b

Add rounding functionalties (#2672) * added math round * added math floor * added math ceil * added math.md examples * moved the detection of nonnumerical values in ceil/floor/round * math round now works on streams * math floor now works on streams * math ceil now works on streams

view details

push time in 5 days

PR merged nushell/nushell

Add rounding functionalties

This PR adds math ceil, math floor and math round.

eg:

> echo [1.5 2.3 -3.1] | math ceil
───┬────
 0 │  2
 1 │  3
 2 │ -3
───┴────
> echo [1.5 2.3 -3.1] | math floor
───┬────
 0 │  1
 1 │  2
 2 │ -4
───┴────
> echo [1.5 2.3 -3.1] | math round
───┬────
 0 │  2
 1 │  2
 2 │ -3
───┴────

This PR will close #2671

Checklist

  • [x] Code compiles
  • [x] Fix any lint/clippy issues
  • [x] Fix any merge conflicts
  • [x] Add unit tests / e2e tests
  • [x] Add/update documentation
+329 -3

5 comments

8 changed files

morbatex

pr closed time in 5 days

issue closednushell/nushell

Add round subcommand: math round

Is your feature request related to a problem? Please describe. It would be nice if one could round mathematical values.

Describe the solution you'd like A math round command could be added which rounds the input: echo [1.5 2.3 -3.1] | math round => [2 2 -3]

If it's ok I would implement this.

closed time in 5 days

morbatex

pull request commentnushell/nushell

Add rounding functionalties

These look good. Thanks!

morbatex

comment created time in 5 days

push eventnushell/nushell

rjboas

commit sha cc06ea4d873c0eef038ec3723c390e1e4fe4a84a

Add Tau constant (#2673) Adds Tau constant using meval::Context. Also adds a test to match pi's. Note: Tau ends up not being more precise than 2*pi. Resolves: #2258

view details

push time in 5 days

PR merged nushell/nushell

Add Tau constant to math eval

Adds Tau constant to math eval using meval::Context.

Also adds a test to match pi's.

Note: Tau ends up not being more precise than 2*pi.

Resolves: #2258

+15 -1

2 comments

2 changed files

rjboas

pr closed time in 5 days

issue closednushell/nushell

Add `Tau` Constant to `Math Eval`

According to the documentation:

https://doc.rust-lang.org/nightly/std/f64/consts/constant.TAU.html

The Tau constant will be stabilized in Rust 1.47. We could add this to our list of constants (currently pi and e) in Math Eval when Rust 1.47 is officially released.

closed time in 5 days

JosephTLyons

pull request commentnushell/nushell

Add Tau constant to math eval

@rjboas thanks!

rjboas

comment created time in 5 days

pull request commentnushell/nushell

Add Tau constant to math eval

Not sure why circleci is failing, but this should pass. I'm going to go ahead and land and cross my fingers I don't break something

rjboas

comment created time in 5 days

push eventnushell/nushell

Joseph T. Lyons

commit sha db590369a8cd064457556e2a96a0b5bd825d1b68

Fix filesize "B" regression (#2688)

view details

push time in 6 days

PR merged nushell/nushell

Fix filesize "B" regression

A small regression was introduced where the "B" on the file size was dropped. Here is the 0.21 release:

Screen Shot 2020-10-20 at 10 47 53 PM

Here is Nushell compiled from master today:

Screen Shot 2020-10-20 at 10 47 37 PM

This PR puts back the "B"

+5 -2

1 comment

1 changed file

JosephTLyons

pr closed time in 6 days

pull request commentnushell/nushell

Fix filesize "B" regression

Thanks!

JosephTLyons

comment created time in 6 days

push eventnushell/book

Arnau Siches

commit sha 4d78ace9eeeba91eed9ead21382d6f8e7c2fd9fb

Fix `from` command examples. The `from` command changed from a `from-toml` approach to `from toml` at some point. This change aligns the examples with it. Signed-off-by: Arnau Siches <asiches@gmail.com>

view details

Jonathan Turner

commit sha f80673b3a8c7eb4a08da524dc571a563368fd8ad

Merge pull request #138 from arnau/fix_from_examples Fix `from` command examples.

view details

push time in 6 days

PR merged nushell/book

Fix `from` command examples.

The from command changed from a from-toml approach to from toml at some point. This change aligns the examples with it.

+28 -28

1 comment

9 changed files

arnau

pr closed time in 6 days

more