profile
viewpoint

lib/pq 6230

Pure Go Postgres driver for database/sql

mjibson/goread 2417

RSS reader in go on app engine; formerly goread.io

mjibson/go-dsp 703

Digital Signal Processing for Go

mjibson/esc 585

A simple file embedder for Go

mjibson/moggio 354

A multi-source music player in Go

mjibson/sqlfmt 286

SQL formatter with width-aware output

mjibson/goon 218

Autocaching interface to the app engine datastore for Go

mjibson/acre 15

langserver client for acme in rust

mjibson/aw 12

Go integration for acme

mjibson/hots.dog 11

Heroes of the Storm aggregator; formerly https://hots.dog/

push eventMaterializeInc/materialize

Jessica Laughlin

commit sha ab69a179150fd3facb333e71e2e514e6eec6a511

sql: support empty maps (#4874)

view details

push time in 11 hours

push eventMaterializeInc/materialize

Nikhil Benesch

commit sha 266ecd20d8059f73c9d403688fd82e72561c4508

sql: fix grammar in comment in func selection code

view details

Nikhil Benesch

commit sha 32162db5985ec343c7b904dbd779717d040a78e9

Merge pull request #4879 from MaterializeInc/benesch-patch-1 sql: fix grammar in comment in func selection code

view details

push time in 13 hours

delete branch MaterializeInc/materialize

delete branch : benesch-patch-1

delete time in 13 hours

PR opened MaterializeInc/materialize

Reviewers
sql: jam recursive casts into the global cast map

@sploiselle this is one possible approach to making can_cast aware of map/list types. I'm decently happy with it, but the type of CastTemplate is pretty horrible, so marking this as a draft for now to give us the space to try to think of something better.


Casting lists and map types is tricky, because the casts must proceed recursively on the element types. So casts for these types were previously handled separately from the other types of casts. A small bug resulted from this design, as the "can_cast" function was unaware of these special list and map casts.

Restructure the cast map so that it can describe the recursive casts necessary for maps and lists. The gist is that the cast map now stores "cast templates" which can describe a suite of casts, like "string to any list", rather than just individual casts between two concrete types.

+170 -173

0 comment

2 changed files

pr created time in 13 hours

create barnchMaterializeInc/materialize

branch : benesch-patch-1

created branch time in 14 hours

push eventMaterializeInc/materialize

Nikhil Benesch

commit sha 962ce81c27d16f3a6debe3277ff4f246d523928f

sql: merge coerce_arg_to_types and coerce_args_to_type No one calls the non-bulk version directly, so just merge the two functions together. This is pure code movement.

view details

Nikhil Benesch

commit sha 67f81ae00161417af53579354a2d86e97c531b02

sql: mark ParamType::MapAny as polymorphic Also use an exhaustive match statement, rather than the matches! macro, so that future additions to ParamType cause compiler errors in this method.

view details

Nikhil Benesch

commit sha e4f541e89270e107f503af52814a91da5a53a4d3

sql: replumb polymorphic type resolution Rather than resolving polymorphic types by constructing a ParamList containing the resolved concrete types, teach coerce_args_to_types to perform polymorphic type resolution based on the function's original ParamList. This ensures that ParamList only ever describes the *abstract* parameters to a function, and doesn't need to do double duty representing both the abstract and concrete parameters.

view details

Nikhil Benesch

commit sha 51cb13bcdaac93d21b95cd2682ea06070414fb06

sql: avoid desaturation in function selection

view details

Nikhil Benesch

commit sha f177de4a8cd1412ca4e6218eb03bf26245388fb7

repr: auto-derive a ScalarBaseType enum ScalarBaseType mirrors ScalarType, except that data associated with each variant has been stripped. For example, ScalarType::Decimal(usize, usize) becomes ScalarBaseType::Decimal. This supplants the "desaturate" function, which manually "zeroed-out" the data fields. Using ScalarBaseType is clearer than matching on desaturated ScalarTypes, in my opinion, and the enum-kinds crate makes maintaining the extra enum essentially free.

view details

Nikhil Benesch

commit sha 36e208b582f5ed9fa9f915121b7fca0bf518da94

Merge pull request #4875 from benesch/sql-polymorphism repr,sql: get rid of desaturation

view details

push time in 14 hours

PR merged MaterializeInc/materialize

repr,sql: get rid of desaturation

Twiddle some interfaces around so that we no longer need to "desaturate" scalar types when planning casts and functions. Details in commit messages.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+176 -204

0 comment

7 changed files

benesch

pr closed time in 14 hours

Pull request review commentMaterializeInc/materialize

repr,sql: get rid of desaturation

 impl ParamList {     /// - All arguments passed to `MapAny` must be `ScalarType::Map`s with the     ///   same type of value in each key, value pair.     ///-    /// # Errors-    /// - If `typs` is inconsistent with these constraints.-    fn resolve_polymorphic_types(-        &self,-        typs: &[Option<ScalarType>],-    ) -> Result<ParamList, anyhow::Error> {-        // Early return if polymorphic constraints are unnecessary.-        let p = match self {-            ParamList::Exact(p) | ParamList::Repeat(p) => p,-        };--        if !p.iter().any(|p| p.is_polymorphic()) {-            return Ok(self.clone());-        }-+    /// Returns `Some` if the constraints was successfully resolved, or `None`

D'oh, thanks! Going to fix in post so that I can stack another PR on top of this one.

benesch

comment created time in 14 hours

Pull request review commentMaterializeInc/materialize

repr,sql: get rid of desaturation

 impl ParamList {     /// - All arguments passed to `MapAny` must be `ScalarType::Map`s with the     ///   same type of value in each key, value pair.     ///-    /// # Errors-    /// - If `typs` is inconsistent with these constraints.-    fn resolve_polymorphic_types(-        &self,-        typs: &[Option<ScalarType>],-    ) -> Result<ParamList, anyhow::Error> {-        // Early return if polymorphic constraints are unnecessary.-        let p = match self {-            ParamList::Exact(p) | ParamList::Repeat(p) => p,-        };--        if !p.iter().any(|p| p.is_polymorphic()) {-            return Ok(self.clone());-        }-+    /// Returns `Some` if the constraints was successfully resolved, or `None`

nit: s/constraints was/constraints were/

benesch

comment created time in 17 hours

push eventMaterializeInc/materialize

Brandon W Maister

commit sha 64d25b58e281f48c617b42490b19ec488f4634fa

mzconduct: Use relative paths instead of changing directory (#4877) This fixes an occasional source of bugs, And one explicit bug that I ran into. More importantly brings mzconduct more inline with the conventions of mzcompose, and is thus a step on the way to #4769

view details

push time in 15 hours

PR merged MaterializeInc/materialize

mzconduct: Use relative paths instead of changing directory

This fixes an occasional source of bugs, And one explicit bug that I ran into.

More importantly brings mzconduct more inline with the conventions of mzcompose, and is thus a step on the way to #4769

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+104 -82

0 comment

1 changed file

quodlibetor

pr closed time in 15 hours

push eventMaterializeInc/materialize

Brandon W Maister

commit sha b531aa1daac749dc4db7a8312f562fd70b67a108

Reduce the number of metrics we send for tracking "in maintenance" We added the `mz_arrangement_maintenance_active_info` metric to track whether any specific arrangement is spending all of its time in maintenance. Unfortunately that means that it grows with the number of arrangements, possibly to hundreds or thousands of metrics. With this new metric we will only see one metric per worker. It is still possible to determine which arrangement is spending all its time in mainenance by looking at the `mz_arrangement_maintenance_seconds_total` metric, which will show the proportion of time taken up by each arrangement's maintenance.

view details

Brandon W Maister

commit sha 5894d256ba8898bc89c905b9ee9d8b9d217379ac

Merge pull request #4863 from quodlibetor/reduce-arrangement-maintenance-count Reduce the number of metrics we send for tracking "in maintenance"

view details

push time in 15 hours

PR merged MaterializeInc/materialize

Reviewers
Reduce the number of metrics we send for tracking "in maintenance"

We added the mz_arrangement_maintenance_active_info metric to track whether any specific arrangement is spending all of its time in maintenance. Unfortunately that means that it grows with the number of arrangements, possibly to hundreds or thousands of metrics.

With this new metric we will only see one metric per worker. It is still possible to determine which arrangement is spending all its time in mainenance by looking at the mz_arrangement_maintenance_seconds_total metric, which will have more detail about which arrangements are actually taking up time for any given interval.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+27 -22

1 comment

1 changed file

quodlibetor

pr closed time in 15 hours

pull request commentMaterializeInc/materialize

Reduce the number of metrics we send for tracking "in maintenance"

I've verified that the per-workflow metrics are showing up correctly in the billing demo.

quodlibetor

comment created time in 15 hours

PR opened MaterializeInc/materialize

mzconduct: Use relative paths instead of changing directory

This fixes an occasional source of bugs, And one explicit bug that I ran into.

More importantly brings mzconduct more inline with the conventions of mzcompose, and is thus a step on the way to #4769

+104 -82

0 comment

1 changed file

pr created time in 15 hours

push eventMaterializeInc/materialize

Sean Loiselle

commit sha d920e617cf0daeb900702a75e68aeeb86cbdbaa9

sql/plan: disallow ListAny + ArrayAny's acceptance of String vals

view details

Sean Loiselle

commit sha 202ae6785fbe8869cedc9d08f15f539854d9e80d

Merge pull request #4873 from sploiselle/misleading-list-type-comment sql/plan: remove misleading comment about ListAny/ArrayAny

view details

push time in 15 hours

PR merged MaterializeInc/materialize

sql/plan: remove misleading comment about ListAny/ArrayAny

I left a misleading comment that sounded like a TODO but was not. The original comment was meant only to explain that ListAny and ArrayAny looked like they accepted strings when they didn't; now they do, so the comment is moot.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+12 -5

3 comments

2 changed files

sploiselle

pr closed time in 15 hours

Pull request review commentMaterializeInc/materialize

sql/plan: remove misleading comment about ListAny/ArrayAny

 SELECT LIST[[NULL]] || LIST[1]  query error no overload for i32 list list list || i32 list: Cannot concatenate i32 list list list and i32 list SELECT LIST[[[1]]] || LIST[2]++# Literal text cannot be implicitly cast to list+query error no overload for i32 list || string: arguments cannot be implicitly cast to any implementation's parameters; try providing explicit casts+SELECT LIST[1] || {2}::text

Not my finest day's work.

sploiselle

comment created time in 15 hours

push eventMaterializeInc/materialize

Frank McSherry

commit sha d48a69690f9251f8465e33f3e3af670531c86a94

deduplicate coalesce arguments (#4876)

view details

push time in 15 hours

PR merged MaterializeInc/materialize

Deduplicate coalesce arguments

This PR deduplicates arguments to coalesce, which allows us to remove the coalesce if all arguments are the same (which happens in using(_) joins). This fix allows inner joins with using syntax to plan as delta queries, once the coalesce is elided.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+4 -0

0 comment

1 changed file

frankmcsherry

pr closed time in 15 hours

PR opened MaterializeInc/materialize

Reviewers
Deduplicate coalesce arguments

This PR deduplicates arguments to coalesce, which allows us to remove the coalesce if all arguments are the same (which happens in using(_) joins). This fix allows inner joins with using syntax to plan as delta queries, once the coalesce is elided.

+4 -0

0 comment

1 changed file

pr created time in 16 hours

PR opened MaterializeInc/materialize

Reviewers
repr,sql: get rid of desaturation

Twiddle some interfaces around so that we no longer need to "desaturate" scalar types when planning casts and functions. Details in commit messages.

+176 -204

0 comment

7 changed files

pr created time in 17 hours

PR closed MaterializeInc/materialize

Reviewers
sql: resolve a TODO in function planning

Arrays and lists now support casting from string, so we no longer need to special case concatenation in the function selection logic.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+43 -57

3 comments

1 changed file

benesch

pr closed time in 17 hours

pull request commentMaterializeInc/materialize

sql: resolve a TODO in function planning

Superseded by #4873. I'll open a new PR for the rest.

benesch

comment created time in 17 hours

Pull request review commentMaterializeInc/materialize

sql/plan: remove misleading comment about ListAny/ArrayAny

 SELECT LIST[[NULL]] || LIST[1]  query error no overload for i32 list list list || i32 list: Cannot concatenate i32 list list list and i32 list SELECT LIST[[[1]]] || LIST[2]++# Literal text cannot be implicitly cast to list+query error no overload for i32 list || string: arguments cannot be implicitly cast to any implementation's parameters; try providing explicit casts+SELECT LIST[1] || {2}::text

This looks like a syntax error! Missing quotes?

sploiselle

comment created time in 18 hours

push eventMaterializeInc/materialize

sploiselle

commit sha d39aea063da183522efb4b2c307d35833c99e8f6

sql/plan: coalesce plan_list_cast + plan_map_cast

view details

sploiselle

commit sha 0fd1537158fa35bad5538bc45ebaa525d6b34c32

sql/plan: typeconv comment edits/func renames

view details

Brandon W Maister

commit sha f1bf62f6f12f61fea01ae1d8597f9bfbafa1d7ba

Update rust toolchain to 1.48

view details

Brandon W Maister

commit sha b0409204d8e2ccd6ee8d67dce2e54e9310d6ab8d

Update bin/doc now that stable rustc does intra-doc links

view details

Sean Loiselle

commit sha 555e20bdbc60e6a70e89a37796d9e2fdad3e540a

Merge pull request #4851 from sploiselle/iterative-casts sql/plan: unify iterative casts

view details

Brandon W Maister

commit sha 3064858267a14a60ac6e0f020a2394950d47300d

Update linked-hash-map There is a long-fixed issue[1] that we hit on this upgrade. Honestly I'm surprised that dependabot didn't upgrade us on this. [1]: https://github.com/contain-rs/linked-hash-map/pull/100

view details

Brandon W Maister

commit sha 415b1e8c859930dbe15a37bc267f7c9a96f8e9f2

Handle new clippy lint unnecessary_lazy_evaluations

view details

Brandon W Maister

commit sha eb5e3bc7324e56faa4b5e1442d05f23d8017b451

Handle new clippy lint manual_strip

view details

Brandon W Maister

commit sha 137cb90f91f26d7fe5201cdb77b10703ce1276db

Handle new clippy lint needless_return

view details

Brandon W Maister

commit sha d059b97cf4d11d83fa4a78f0455aedebfb9c4c9c

Handle new clippy lint single_char_push_str

view details

Ruchir Khaitan

commit sha acd17bd09f41e33532c0e4891715903e5a61caf2

*: let users hint the expected number of values for a group key (#4807)

view details

Brandon W Maister

commit sha b5fb4487fab3e37c5fe566e3d56de86149864e7b

Merge pull request #4866 from quodlibetor/update-rust-1.48 Update Rust to 1.48

view details

Brandon W Maister

commit sha 066945dd61c883e5d5d70ea576ed70fd5202d170

Update regex and friends It no longer needs utf8-ranges.

view details

Brandon W Maister

commit sha 45c6f347543c1196a3942a037d22be6686b9e6f5

Update bstr autocfg fnv itoa siphasher

view details

Brandon W Maister

commit sha 0efe10d4eaa1d1a42d1f2968428fbeb9d2c6d797

Update proc-macro related packages

view details

Brandon W Maister

commit sha e3ca1561b97083d553323a8883547297fb38c574

Merge pull request #4870 from quodlibetor/update-deps Update several dependencies that have been left behind by dependabot

view details

Jessica Laughlin

commit sha 7d9bc41c0600218d5cea6832cf08195138101dbb

sql: support nested map types (#4868) Adds support for nested map types like: '{a=>{b=>1}}'::map[text=>map[text=>int]] Map keys must be of type text and non-null. Map values can be any non-custom type. Support for nested maps with custom value types is forthcoming.

view details

Chris Golden

commit sha 72e0d5a91108fb62324ecf26687de3aab41d5035

Merge branch 'main' into chris/dump-topic-schema

view details

push time in 18 hours

pull request commentMaterializeInc/materialize

sql/plan: remove misleading comment about ListAny/ArrayAny

Ah, I had forgotten that text literals are of unknown type at this point in function selection.

sploiselle

comment created time in 18 hours

more