profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/alexey-milovidov/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

VadimPlh/Arrival 12

How to use TLA+ / TLA+ specification of the ClickHouse replication protocol

amosbird/ClickHouse 5

ClickHouse is a free analytic DBMS for big data.

alexey-milovidov/awesome-cpp 1

A curated list of awesome C++ (or C) frameworks, libraries, resources, and shiny things. Inspired by awesome-... stuff.

alexey-milovidov/firstcontributions.github.io 1

Web app for First Contributions

alexey-milovidov/font-selector 1

Monospace Font Selector

alexey-milovidov/poco 1

POCO C++ Libraries - Cross-platform C++ libraries with a network/internet focus.

alexey-milovidov/russian-conferences-articles 1

Коллективная расшифровка видеодокладов c Российских конференций, митапов в статьи

alexey-milovidov/up-for-grabs.net 1

This is a list of projects which have curated tasks specifically for new contributors. These issues are a great way to get started with a project, or to help share the load of working on open source projects. Jump in!

myrrc/ba-thesis 1

HSE Qualification course work (bachelor thesis)

virbenka/pulls_dashboard 1

dashboard for systematic view of pull requests for a selected repository

issue openedVertamedia/clickhouse-grafana

Difficult to use if only Date column is present (and there is no DateTime column)

Workaround: add WITH toDateTime($dateCol) AS undefined to the query. But that's exceptionally ugly.

created time in 12 hours

pull request commentClickHouse/ClickHouse

Add aggregate function for exponential smoothing

Looks like "moving average" that I've implemented does not make any physical (natural) sense, and "moving sum" should have the last value multiplied by 1 / k, so the sum over sequence of 1, 1, 1, 1... will yield 1.

alexey-milovidov

comment created time in 13 hours

pull request commentClickHouse/ClickHouse

Feature/create simple lambda function

@l1t1 Not in near time.

User defined SQL functions are substituted (as expression) firstly and only then optionally compiled and evaluated. We cannot do infinite substitution, that's why recursion is not supported.

Realist007

comment created time in 19 hours

issue commentClickHouse/ClickHouse

How to use post submission of URL function?

@amosbird Is it about using url table function similarly to the new-born executable table function to perform arbitrary transformations on data? Sounds nice :)

linqingping

comment created time in 20 hours

push eventClickHouse/ClickHouse

Azat Khuzhin

commit sha c77c4e22ae1e5242247413b5874d29d5a188354e

Add MySQL query into after transformation in case of BadQuery Since it may differs from the user input

view details

Azat Khuzhin

commit sha ef161f10f73b93c5dcac1f511c9273dd8b0facbb

gtest_transform_query_for_external_database: print input query

view details

Azat Khuzhin

commit sha 0c2f612a860fb64c0ef39951fc4a18ab20529050

gtest_transform_query_for_external_database: add test with multiple columns in IN

view details

Azat Khuzhin

commit sha e4e584a3de7ace0dd67cc55b84a25de92eecc204

Do not transform (k,v) IN ((K, V)) -> (k, v) IN (K, V) during parsing Before this patch the query `(k,v) IN ((1, 2))` was transformed `(k,v) IN (1,2)` by ClickHouse, and this produce the following error while sending such query to MySQL: DB::Exception: mysqlxx::BadQuery: Operand should contain 2 column(s) (There is a workaround though, add another value for IN section, i.e. `(k,v) IN ((1,2),(0,0))`, but this is icky) v0: (k,v) IN ((1,2)) -> (k,v) IN tuple((1,2)) But these is not supported by transformQueryForExternalDatabase() v1: (k,v) IN ((1,2)) -> (k,v) IN ((1,2)) But now you will get an error for the following query: SELECT ('foo', 'bar') IN (('foo', 'bar')) Now you will get: Logical error: 'Trying to get name of not a column: ExpressionList'. (from two places, first is DB::ActionsMatcher::visit, easy to fix by inherit ASTExpressionList from ASTWithAlias, but the second is evaluteConstantExpression(), which requires additional support for these case while creating a set in DB::createBlockForSet()) v2: (k,v) IN ((1,2)) -> (k,v) IN tuple((1,2)) So instead I prefer these, and add support for this case in transformQueryForExternalDatabase() in the next patch.

view details

Azat Khuzhin

commit sha f40ab891ebd6d077ab58230658470dad40c6dc5d

Support tuple with zero/one arguments in transformQueryForExternalDatabase()

view details

Azat Khuzhin

commit sha 5ad9cb9b797abe498ae558f9c233e16cd89e41b4

Add test_storage_mysql::test_mysql_in with multiple columns in IN

view details

Azat Khuzhin

commit sha b51c1809220ae159a0de5abae9058784e07270af

tests/00132_sets: remove one more tuple nesting CI report [1]: 2021-09-11 14:25:31 [5656ccb14ed9] 2021.09.11 14:25:31.498685 [ 18561 ] {6096a73f-2066-4c78-9f99-73f013c96deb} <Error> executeQuery: Code: 43. DB::Exception: Invalid types for IN function: Tuple(UInt8, Tuple(UInt8, UInt8)) and Tuple(Tuple(Tuple(UInt8, Tuple(UInt8, UInt8)), Tuple(UInt8, Tuple(UInt8, UInt8)))).: While processing (1 AS number, (2, 3) AS tuple) IN tuple(((1, (2, 3)), (4, (5, 6)))). (ILLEGAL_TYPE_OF_ARGUMENT) (version 21.11.1.1) (from [::1]:32780) (comment: 00132_sets.sql) (in query: SELECT number, tuple FROM (SELECT 1 AS number, (2, 3) AS tuple) WHERE (number, tuple) IN (((1, (2, 3)), (4, (5, 6))));), Stack trace (when copying this message, always include the lines below): [1]: https://clickhouse-test-reports.s3.yandex.net/28888/5ad9cb9b797abe498ae558f9c233e16cd89e41b4/fast_test.html#fail1

view details

Azat Khuzhin

commit sha 976874d57b1f2087db9ae4760bee48d3b149a1ae

tests/01016_input_null_as_default: remove one more tuple nesting CI report [1]: 01016_input_null_as_default: [ FAIL ] - having stderror: Code: 53. DB::Exception: Bad size of tuple. Expected size: 1, actual size: 2.: While executing ValuesBlockInputFormat: data for INSERT was parsed from stdin. (TYPE_MISMATCH) [1]: https://clickhouse-test-reports.s3.yandex.net/28888/5ad9cb9b797abe498ae558f9c233e16cd89e41b4/fast_test.html#fail1

view details

Azat Khuzhin

commit sha c99bd06195d3d1b17db11bafb78ec32a59da724a

copier: fix partition_key alias using WITH After those changes it is not possible to add an alias for the tuple element anymore, i.e.: $ clickhouse-local -q 'select ((1,2) = (1,2) as foo), foo' 1 1 $ clickhouse-upstream local -q 'select ((1,2) = ((1,2) as foo)), foo' 1 (1,2) But these can be done using WITH statement.

view details

alexey-milovidov

commit sha c4b3add85904e6dba5b044317e3118bd6e465622

Update Query.cpp

view details

alexey-milovidov

commit sha e16e8fa286a6a3ca119b464cfef2115f00efa6a7

Update ExpressionElementParsers.cpp

view details

alexey-milovidov

commit sha 20d8523a2ebe22c93266e56fc097e273e4c8c68b

Merge pull request #28888 from azat/mysql-in-fix Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

view details

push time in 20 hours

PR merged ClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` ) pr-bugfix

Changelog category (leave one):

  • Bug Fix (user-visible misbehaviour in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md): Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. (k,v) IN ((1, 2)) )

Detailed description: Note, there are few backward incompatible changes after this PR:

copier partition_key

After those changes it is not possible to add an alias for the tuple element anymore, i.e.:

$ clickhouse-local -q 'select ((1,2) = (1,2) as foo), foo'
1       1
$ clickhouse-upstream local -q 'select ((1,2) = ((1,2) as foo)), foo'
1       (1,2)

These can be done using WITH statement, but it means that you cannot run old clickhouse-copier with the newer clickhouse-server, since it relies on that syntax.

Superfluous brackets in the IN section:

SELECT number, tuple FROM (SELECT 1 AS number, (2, 3) AS tuple) WHERE (number, tuple) IN (((1, (2, 3)), (4, (5, 6)))); -- fail
SELECT number, tuple FROM (SELECT 1 AS number, (2, 3) AS tuple) WHERE (number, tuple) IN ( (/*number*/1, /*tuple*/(2, 3)), (/*number*/4, /*tuple*/(5, 6)) ); -- ok

See also comments in the code and patch descriptions

+86 -25

1 comment

8 changed files

azat

pr closed time in 20 hours

PullRequestReviewEvent

Pull request review commentClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

 bool isCompatible(const IAST & node)          /// A tuple with zero or one elements is represented by a function tuple(x) and is not compatible,         /// but a normal tuple with more than one element is represented as a parenthesized expression (x, y) and is perfectly compatible.-        if (name == "tuple" && function->arguments->children.size() <= 1)-            return false;+        /// So to support tuple with zero or one elements we can clear function name to get (x) instead of tuple(x)+        if (name == "tuple")+        {+            if (function->arguments->children.size() <= 1)+            {+                function->name.clear();

Do I understand correctly that we reliying on a fact that MySQL can successfully parse and disambiguare lists with one element: (1), ((1, 2))?

azat

comment created time in 20 hours

PullRequestReviewEvent

Pull request review commentClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

 bool isCompatible(const IAST & node)          /// A tuple with zero or one elements is represented by a function tuple(x) and is not compatible,         /// but a normal tuple with more than one element is represented as a parenthesized expression (x, y) and is perfectly compatible.-        if (name == "tuple" && function->arguments->children.size() <= 1)-            return false;+        /// So to support tuple with zero or one elements we can clear function name to get (x) instead of tuple(x)+        if (name == "tuple")+        {+            if (function->arguments->children.size() <= 1)+            {+                function->name.clear();

This is a dirty trick. I did not aware that we can represent a tuple as function with empty name.

Ok as it is used only locally and the resulting AST is not passed around.

But also we need to think should we exploit or not exploit this trick somewhere else.

azat

comment created time in 20 hours

PullRequestReviewEvent

Pull request review commentClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

 static void check(     std::string transformed_query = transformQueryForExternalDatabase(         query_info, state.getColumns(), IdentifierQuotingStyle::DoubleQuotes, "test", "table", state.context); -    EXPECT_EQ(transformed_query, expected);+    EXPECT_EQ(transformed_query, expected) << query;

Did not know about that :+1:

azat

comment created time in 20 hours

PullRequestReviewEvent

push eventazat/ClickHouse

alexey-milovidov

commit sha e16e8fa286a6a3ca119b464cfef2115f00efa6a7

Update ExpressionElementParsers.cpp

view details

push time in 20 hours

Pull request review commentClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

 bool ParserParenthesisExpression::parseImpl(Pos & pos, ASTPtr & node, Expected &         return false;     } +    /// special case for one-element tuple     if (expr_list.children.size() == 1 && is_elem)     {-        node = expr_list.children.front();-    }-    else-    {-        auto function_node = std::make_shared<ASTFunction>();-        function_node->name = "tuple";-        function_node->arguments = contents_node;-        function_node->children.push_back(contents_node);-        node = function_node;+        auto * ast_literal = expr_list.children.front()->as<ASTLiteral>();+        /// but only if it's argument is not tuple,

it's -> its

azat

comment created time in 20 hours

PullRequestReviewEvent

push eventazat/ClickHouse

alexey-milovidov

commit sha c4b3add85904e6dba5b044317e3118bd6e465622

Update Query.cpp

view details

push time in 20 hours

Pull request review commentClickHouse/ClickHouse

Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )

 void Query::executeImpl()         case CR_SERVER_LOST:             throw ConnectionLost(errorMessage(mysql_driver), err_no);         default:-            throw BadQuery(errorMessage(mysql_driver), err_no);+            /// Add query to the exception message, since it may differs from the user input query.+            /// (also you can use this and create query with an error to see what query ClickHouse created)+            throw BadQuery(errorMessage(mysql_driver) + "(query: " + query_string + ")", err_no);

Looks like a missing whitespace.

azat

comment created time in 20 hours

PullRequestReviewEvent

push eventClickHouse/ClickHouse

Azat Khuzhin

commit sha 7f5f0e364dc3cc65ef0c7d0702991cd1538e24a5

Increase timeout in 2010_lc_native 10 seconds it not enough [1]: 2010_lc_native: [ FAIL ] 15.53 sec. - having stderror: Traceback (most recent call last): File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 302, in <module> main() File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 298, in main insertLowCardinalityRowWithIndexOverflow() File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 259, in insertLowCardinalityRowWithIndexOverflow readException(s) File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 185, in readException assert(readVarUInt(s) == 2) File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 65, in readVarUInt byte = readStrict(s)[0] File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 34, in readStrict cur = s.recv(size) socket.timeout: timed out [1]: https://clickhouse-test-reports.s3.yandex.net/29060/7f1160c466a968c680d1bc73e0ad5e9419d0b6d0/functional_stateless_tests_(debug).html#fail1 Server log: 2021.09.17 21:02:53.734388 [ 682 ] {123456} <Trace> ContextAccess (default): Access granted: INSERT(x) ON test_0tmdh4.tab 2021.09.17 21:03:04.835044 [ 682 ] {123456} <Error> executeQuery: Code: 117. DB::Exception: Index for LowCardinality is out of range. Dictionary size is 1, but found index with value 72057594037927936. (INCORRECT_DATA) (version 21.11.1.8109) (from [::ffff:127.0.0.1]:59512) (in query: insert into test_0tmdh4.tab format TSV settings input_format_defaults_for_omitted_fields=0), Stack trace (when copying this message, always include the lines below): And according to trace_log the problem is slow unwind: 2021-09-17 21:02:55.103893 libunwind::LocalAddressSpace::get32 2021-09-17 21:02:56.151960 libunwind::LocalAddressSpace::get8 2021-09-17 21:03:03.168447 libunwind::LocalAddressSpace::getULEB128

view details

alexey-milovidov

commit sha 2a9c6fb9fc173a2eb24e4a28293654a870862df7

Merge pull request #29153 from azat/test-2010_lc_native-fix Increase timeout in 2010_lc_native

view details

push time in 20 hours

PR merged ClickHouse/ClickHouse

Increase timeout in 2010_lc_native pr-not-for-changelog

10 seconds it not enough 1:

2010_lc_native:                                                         [ FAIL ] 15.53 sec. - having stderror:
Traceback (most recent call last):
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 302, in <module>
    main()
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 298, in main
    insertLowCardinalityRowWithIndexOverflow()
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 259, in insertLowCardinalityRowWithIndexOverflow
    readException(s)
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 185, in readException
    assert(readVarUInt(s) == 2)
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 65, in readVarUInt
    byte = readStrict(s)[0]
  File "/usr/share/clickhouse-test/queries/0_stateless/2010_lc_native.python", line 34, in readStrict
    cur = s.recv(size)
socket.timeout: timed out

Server log:

2021.09.17 21:02:53.734388 [ 682 ] {123456} <Trace> ContextAccess (default): Access granted: INSERT(x) ON test_0tmdh4.tab
2021.09.17 21:03:04.835044 [ 682 ] {123456} <Error> executeQuery: Code: 117. DB::Exception: Index for LowCardinality is out of range. Dictionary size is 1, but found index with value 72057594037927936. (INCORRECT_DATA) (version 21.11.1.8109) (from [::ffff:127.0.0.1]:59512) (in query: insert into test_0tmdh4.tab format TSV settings input_format_defaults_for_omitted_fields=0), Stack trace (when copying this message, always include the lines below):

And according to trace_log the problem is slow unwind:

2021-09-17 21:02:55.103893 libunwind::LocalAddressSpace::get32
2021-09-17 21:02:56.151960 libunwind::LocalAddressSpace::get8
2021-09-17 21:03:03.168447 libunwind::LocalAddressSpace::getULEB128

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Cc: @KochetovNicolai

+3 -3

0 comment

1 changed file

azat

pr closed time in 20 hours

issue commentClickHouse/ClickHouse

Incorrect work of join_use_nulls

In new versions, subquery result was changed to Nullable. See the changelog for 21.9: https://github.com/ClickHouse/ClickHouse/blob/master/CHANGELOG.md#backward-incompatible-change

Fallenyasha

comment created time in 20 hours

issue commentClickHouse/ClickHouse

A list of benchmarks

QuestDB vs ClickHouse on time series ingestion performance: https://questdb.io/blog/2021/06/16/high-cardinality-time-series-data-performance/

alexey-milovidov

comment created time in 20 hours

push eventClickHouse/ClickHouse

fuzzERot

commit sha dba428c443c1630e8a99c25b4ef6211aa6656d4c

Implementation of h3ToGeoBoundary

view details

alexey-milovidov

commit sha 4f073a436bcdbe531a61e0669669ff1ae949e85b

Update h3ToGeoBoundary.cpp

view details

alexey-milovidov

commit sha b467df98d36ffeb2b719b25feb2d7f25abc9a170

Update 02006_h3_to_geo_boundary.sql

view details

alexey-milovidov

commit sha 10a314dd46d1921b2dace86fb1cf2a20d87b4d70

Update h3ToGeoBoundary.cpp

view details

mergify[bot]

commit sha d4ac1fdba06ec24191dab93b81bf78cc316fabe1

Merge branch 'master' into h3-functions

view details

alexey-milovidov

commit sha 3eed1f06ea8e41a74884d12f962a52837de447c9

Merge pull request #28952 from fuzzERot/h3-functions Implementation of h3ToGeoBoundary

view details

push time in 21 hours

PR merged ClickHouse/ClickHouse

Implementation of h3ToGeoBoundary can be tested pr-feature doc-alert

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Implements the h3ToGeoBoundary function.

Detailed description / Documentation draft:

This PR implements h3ToGeoBoundary function, which takes a H3 index and returns array consisting of pairs (longitude, latitude). This pairs corresponds to the boundary of the provided H3 index.

Relates to: #17708

+184 -0

2 comments

6 changed files

fuzzERot

pr closed time in 21 hours

push eventClickHouse/ClickHouse

robot-clickhouse

commit sha dc260f912eafd85e20e1292ecd6b263b9ef10cb1

Backport #29063 to 21.7: Merging #27963

view details

alexey-milovidov

commit sha 4bd89d9f6f5385d18d91b2eab0d6544749d8ea2a

Merge pull request #29149 from ClickHouse/backport/21.7/29063 Backport #29063 to 21.7: Merging #27963

view details

push time in 21 hours

delete branch ClickHouse/ClickHouse

delete branch : backport/21.7/29063

delete time in 21 hours

PR merged ClickHouse/ClickHouse

Backport #29063 to 21.7: Merging #27963 pr-backport

Original pull-request #29063 Cherry-pick pull-request #29137

This pull-request is a last step of an automated backporting. Treat it as a standard pull-request: look at the checks and resolve conflicts. Merge it only if you intend to backport changes to the target branch, otherwise just close it.

+36 -7

0 comment

4 changed files

robot-clickhouse

pr closed time in 21 hours

pull request commentClickHouse/ClickHouse

Add Cors

A test needed.

FArthur-cmd

comment created time in 21 hours

Pull request review commentClickHouse/ClickHouse

Add Cors

 namespace ErrorCodes     extern const int HTTP_LENGTH_REQUIRED; } +namespace+{+    /// Process options request. Usefull for CORS.+    void processOptionsRequest(HTTPServerResponse & response, const Poco::Util::LayeredConfiguration & config)+    {+        /// If answer for options request was not defined, return 501 to client.+        if (!config.has("http_options_response"))+        {+            response.setStatusAndReason(HTTPResponse::HTTP_NOT_IMPLEMENTED);+            response.send();+        }+        else+        {+            /// otherwise fill response.+            Strings config_keys;+            config.keys("http_options_response", config_keys);+            for (const std::string & config_key : config_keys)+            {+                if (config_key == "header" || config_key.starts_with("header["))+                {+                    response.add(config.getString("http_options_response." + config_key + ".name", "Empty header"),+                                 config.getString("http_options_response." + config_key + ".value", ""));+                    response.setKeepAlive(false);

Why'd you set keep alive to false in a loop?

FArthur-cmd

comment created time in 21 hours