profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/inikep/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

percona/percona-server 808

Percona Server

inikep/lzbench 533

lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors

inikep/lizard 414

Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.

inikep/XWRT 31

XWRT (XML-WRT) is an efficient XML/HTML/text compressor

percona/rocksdb 5

A library that provides an embeddable, persistent key-value store for fast storage.

inikep/zstd 2

Zstandard - Fast real-time compression algorithm

inikep/mysql-server 1

MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.

inikep/rocksdb 1

A library that provides an embeddable, persistent key-value store for fast storage.

DataCompression/FiniteStateEntropy 0

New generation entropy coder

inikep/lz4 0

Extremely Fast Compression algorithm

push eventinikep/mysql-5.6

Zsolt Parragi

commit sha 4e665e98b8949fb72960f5baadd5dc83998cd555

FB8-87, FB8-119: Supporting dynamic scheduling in MTS (#999) (#999) Summary: Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: https://github.com/facebook/mysql-5.6/commit/5846968 Reference commit: https://github.com/facebook/mysql-5.6/commit/d6217c9 Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/999 Differential Revision: D14883857 (https://github.com/facebook/mysql-5.6/commit/ae0e0512be44668a8f28843277216cf601f0d227) fbshipit-source-id: cdee7a7444f

view details

Manuel Ung

commit sha 5e6cd912d20d8eaf0e2cd450e41eae7dec7c6d92

FB8-118: Port slow query log rotation (#1015) (#1015) Summary: Jira issue: https://jira.percona.com/browse/FB8-118 Reference Patch: https://github.com/facebook/mysql-5.6/commit/3445f81 Porting notes: 1. fixed PS-1484: slowlog rotation and expiration doesn't work with the default slow query log name 2. `max_slowlog_files` was replaced with `slowlog_space_limit` ---------- https://github.com/facebook/mysql-5.6/commit/3445f81 ---------- This diff is a combination of the following two commits from Percona: https://github.com/percona/percona-server/commit/725a238c https://github.com/percona/percona-server/commit/85c461c8 Adds two new variables `max_slowlog_size` and `max_slowlog_files` as documented here: https://www.percona.com/doc/percona-server/LATEST/flexibility/slowlog_rotation.html Originally Reviewed By: hermanlee Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1015 Reference Patch: https://github.com/facebook/mysql-5.6/commit/b857c4840db Differential Revision: D15158133 (https://github.com/facebook/mysql-5.6/commit/06ea967d8b24c7a5c929acc1e26d0ae56392ccb1) fbshipit-source-id: b14ee4a7a9e

view details

Yash Botadra

commit sha 5d4e0e63d26c70b524b1bb2670645796a11bca8c

FB8-117: Add thread ID option for SQL SET command (#1012) (#1012) Summary: Jira issue: https://jira.percona.com/browse/FB8-117 Reference Patch: https://github.com/facebook/mysql-5.6/commit/19f5aad Added thread ID option to the SQL SET command. The syntax of the command is as follows: `SET SESSION [THREAD_ID] var_name = var_value [, [SESSION [THREAD_ID]] var_name = var_value]` Notes: - The thread ID can be specified only with the SESSION keyword - If no thread ID is provided, it is assumed to be the current thread - Similarly, if own thread ID is provided, it is equivalent to `SET var_name = var_value` - If the thread with given ID does not exist, it will fail - For commands like `SET SESSION THREAD_ID var1 = var_value1, var2 = var_value2, var3 = var_value3` the SESSION THREAD_ID will apply to all three variables as explained in https://dev.mysql.com/doc/refman/5.6/en/set-variable.html Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1012 Reviewed By: lloyd Differential Revision: D15773495 (https://github.com/facebook/mysql-5.6/commit/7638d74b3f56863c9c1d929037c07d757ae70c9c) Pulled By: lth fbshipit-source-id: c64e1ee47ac

view details

Zsolt Parragi

commit sha b2ccc77fc2f29ab69c2be10aa8f27a30e1894a4a

FB8-221: Change SET GLOBAL [super_]read_only to not acquire the global read lock (#1008) (#1008) Summary: Jira ticket: https://jira.percona.com/browse/FB8-221 Reference commit: https://github.com/facebook/mysql-5.6/commit/d8657c3d Reference commit: https://github.com/facebook/mysql-5.6/commit/a3f808d Reference commit: https://github.com/facebook/mysql-5.6/commit/f2dfbd0 Change SET GLOBAL [super_]read_only to not acquire the global read lock Calling SET GLOBAL [super_]read_only acquires both the global read lock and the global commit lock before setting the flag. Acquiring the global read lock will cause the SET statement to wait on long running DML transactions. We think it is safe in our environment to not acquire this lock, so that SET GLOBAL [super_]read_only does not need to wait on long running DML transactions. The side effect of not acquiring the global read lock is some DDL statements may complete after super_read_only is set, but that behavior is acceptable. This behavior can be changed back to the original by SET global.legacy_global_read_lock_mode=1 Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1008 Differential Revision: D15360477 (https://github.com/facebook/mysql-5.6/commit/ee8f2789318183e7aa80fe7310da97373315d39d) Pulled By: yizhang82 fbshipit-source-id: e86e93ece4b

view details

Bertan Ari

commit sha c2c34de41a9803df6c0a62f3fddfa4c78b26dd11

Adding Rows_examined and Rows_sent implementation from 5.6 Summary: These two fields were added to 5.6 but missing in 8.0. I noticed previous implementation was buggy so added a new implementation which is capturing more rows_examine operations. There were also no MTR tests so I added new tests to cover these fields. Differential Revision: D15999862 (https://github.com/facebook/mysql-5.6/commit/60041178418cf3f8869d4f5a47f70bd332d27b9a) fbshipit-source-id: 70e49aea419

view details

Yi Zhang

commit sha 906a410d082f87364b4121af51e0f87925db5844

Properly support SSL options in mysqlslap Summary: Mysqlslap doesn't apply options the same way for the initial connection and the workers - as a result the workers missing a lot of options passed from command line, including SSL. This fixes the SSL part only. I'll address the bigger issue with probably a bit refactoring with a separate fix (which is going to require quite a bit more testing). Differential Revision: D16015149 (https://github.com/facebook/mysql-5.6/commit/ff6cc9f5dd9e7e913e804fcd9ab33895a49729c2) fbshipit-source-id: 3c25da7b5fc

view details

Manuel Ung

commit sha c6ab0016c85b5ec4bd58a0670f66bbc9ca0eac76

Response attributes Summary: Partial port of response attributes (http://github.com/facebook/mysql-5.6/commit/bbd97ab). The functionality provided here (current db and sysvar changes) are actually provided already by upstream (https://dev.mysql.com/doc/refman/5.7/en/session-state-tracking.html). Once our clients switch over to using the upstream implementation, we can drop portions of this patch (response attributes themselves may still be useful). Reference patch: http://github.com/facebook/mysql-5.6/commit/bbd97ab http://github.com/facebook/mysql-5.6/commit/0c95773 http://github.com/facebook/mysql-5.6/commit/2e79af2 http://github.com/facebook/mysql-5.6/commit/c0ed8c3 http://github.com/facebook/mysql-5.6/commit/d031181 http://github.com/facebook/mysql-5.6/commit/5ff932a http://github.com/facebook/mysql-5.6/commit/f9b38c97b01 Differential Revision: D18702006 (https://github.com/facebook/mysql-5.6/commit/98c56cf647235c7c81538114cbdee311db2e205a) fbshipit-source-id: 8a90b93179d

view details

Herman Lee

commit sha 5a7bdea172bc9e58a7420c48112db80e3c5d4ebb

Fix mysqldump handling of varbinary columns Summary: https://bugs.mysql.com/bug.php?id=96053 8.0.13 added _binary tags to blobs in the mysqldump output. However, memory allocated for the insert statement did not account of the 8 bytes of the "_binary " string properly. This resulted in mysqldump writing to memory beyond the buffer allocated for the string buffer before it is reallocated with a new larger size. The fix is to account for the 8 bytes of the "_binary " string during the realloc calculation. Reviewed By: lloyd Differential Revision: D16088056 (https://github.com/facebook/mysql-5.6/commit/5e15108bcf25e811af70a688031ff5597e30e22e) fbshipit-source-id: 7d8cd4398f1

view details

Yi Zhang

commit sha 2cf7a698e1cfa80cd68e677516e0b0d03bb79174

Fix x.connection_unixsocket socket path too long Summary: In some cases these tests are failing with path too long with unix socket name: ``` -in main, line 0:ERROR: No such file or directory, while connecting to SOCKET (code 2002) +in main, line 0:ERROR: UNIX Socket file name too long, size should be less or equal 107 (code 2005) not ok ``` In our platform, the limit is 108 bytes: ``` /* Structure describing the address of an AF_LOCAL (aka AF_UNIX) socket. */ struct sockaddr_un { __SOCKADDR_COMMON (sun_); char sun_path[108]; /* Path name. */ }; ``` If you count the sandcastle path - the 2 out of the 3 socket names are 108 which is exactly 1 character more than 107: ``` /mnt/btrfs/trunk-git-mysql80-int-58142895-1561679946/_build-8.0-ASan/mysql-test/var/tmp/12/mysqld.dir.socket -> 108 /mnt/btrfs/trunk-git-mysql80-int-58142895-1561679946/_build-8.0-ASan/mysql-test/var/tmp/12/mysqld.x.custom -> 106 /mnt/btrfs/trunk-git-mysql80-int-58142895-1561679946/_build-8.0-ASan/mysql-test/var/tmp/12/mysqld_valid.sock -> 108 ``` Interestingly we did have a check in mysqltestrun.pl: ``` # Set the "tmp" directory if (!$opt_tmpdir) { $opt_tmpdir = "$opt_vardir/tmp" unless $opt_tmpdir; my $res = check_socket_path_length("$opt_tmpdir/mysqld.NN.sock", $opt_parallel); ``` however that length is only 105 (after taking the parllel worker ID into account): ``` /mnt/btrfs/trunk-git-mysql80-int-58142895-1561679946/_build-8.0-ASan/mysql-test/var/tmp/xx/mysqld.NN.sock -> 105 ``` So at the end of the day, any socket name should be less than "mysqld.NN.sock" which is 14 characters. Reviewed By: lloyd Differential Revision: D16079675 (https://github.com/facebook/mysql-5.6/commit/f60efa85054e340f030b1d0e4ff5e51b5ed7f293) fbshipit-source-id: 784389167e3

view details

Yi Zhang

commit sha 77f67f40094496834b72daae6303bfcaf98512ea

Disable mysqlx plugin in tests by default Summary: This is a fun one. We've been observing random unexplained connections from the test suite, something like the following: ``` Audit_null_connection_change_user 0 -Audit_null_connection_connect 10 -Audit_null_connection_disconnect 10 -Audit_null_connection_pre_authenticate 10 +Audit_null_connection_connect 11 +Audit_null_connection_disconnect 11 +Audit_null_connection_pre_authenticate 11 ``` There is a rogue connection somewhere into the process. However we weren't able to find out exactly where it came from. Then we lucked out: in one test case it is failing and print out parts of the query: ``` @@ -25,6 +25,7 @@ WHERE line LIKE ( '%SELECT%' ); incoming_query 'The_original_query' +@tls_version DROP TABLE test_log; SET GLOBAL general_log = DEFAULT; SET GLOBAL general_log_file= old_general_log_file; mysqltest: Result length mismatch ``` The extra tls_version is the last section (separated by ' ') from the following SELECT in the test: ``` SELECT substring_index( line, ' ', -1 ) AS incoming_query FROM test_log WHERE line LIKE ( '%SELECT%' ); ``` This means we are looking for a SELECT with tls_version query, and it is coming from mysqlx: ``` sql_result.query( "SELECT skip_networking, skip_name_resolve, have_ssl='YES', " "ssl_key, " "ssl_ca, ssl_capath, ssl_cert, ssl_cipher, ssl_crl, " "ssl_crlpath, tls_version;"); ``` The query itself is coming from mysqlx startup and it is done from another thread asynchronously, which explains the race condition where we would only see it randomly popping up in test cases. This also explains why we only see it in 8.0 but never in 5.6. For now, the easy thing to do is to disable mysqlx globally in the test suite, and fortunately the mysqlx plugin test suite has a x/t/suite.opt that enables mysqlx by default and that overrides the --skip-mysqlx. I've verified that mysqlx is disabled in other test cases (by inserting a SHOW PLUGINS) ``` +mysqlx DISABLED DAEMON NULL GPL +mysqlx_cache_cleaner ACTIVE AUDIT NULL GPL ``` and enabled properly in mysqlx test cases (by inserting a SHOW PLUGINS) ``` +mysqlx ACTIVE DAEMON NULL GPL +mysqlx_cache_cleaner ACTIVE AUDIT NULL GPL ``` Note that I also had to do a few things besides globally disable mysqlx: 1. enable mysqlx for sys_var suites because they test mysqlx variables and those tests are safe as they don't do any show processlist stuff 2. Rebaseline tests that has mysqlx artifacts. For reference, the original commit that enabled mysqlx across the board is this commit: aed56f7708413ffe5ee600a20bea838875173440, WL#11538. 3. make xplugin_wait_for_interface.inc more resilient by doing nothing if xplugin is not enabled Keep in mind this will lose some X plugin coverage in non-X-plugin scenarios (xplugin still has its own test suite) but I doubt it matters that much for us. Differential Revision: D16106437 (https://github.com/facebook/mysql-5.6/commit/57c33122ba5c461e488609fd25835e32044cdd4a) fbshipit-source-id: fbc3a9ae4d4

view details

Zhicheng Zhu

commit sha f2683ef083e1f05450d8fd1d5b065d5512b92a5b

Add table for queries_used and queries_empty Summary: Adding queries_used and empty_queries instruments for performance schema. Since this diff is more than 500 lines. Let me talk about the code structure. This diff contains three parts 1. Adding instrument data(query_stat) 2. Create performance_schema.table_statistics_per_table for querying query_stat 3. Adding tests and update existing tests There are three things I thought about when writing this code. * Where should I add the query_stat and how to do the aggregation for query_stats? After reading other pfs code. Since query_stat is a part of table_stat, I added it to pfs_stat.h * Since we create a new stat which is related to table. There are two places -- PFS_table and PFS_table_share we need to considered. For details, look at pfs_instr.cc/pfs_instr.h and pfs_instr_class.cc/pfs_instr_class.h * PFS_table. -- This is per table instance, which means there can be multiple PFS_table point to the same sql table * PFS_table_share -- All the table instance will have a pointer points to this object. If a sql table is flushed or closed, we will dump all the PFS_table query_stat data to the PFS_table_share query_stat. * Where can we collect the data? Based on our 5.6 implementation. If there is an io event, then that mean we have a query and we should bump the queries_used counter. If the sql command is select and sent_rows is 0, then we should bump the empty_queries counter. In this case we need to modify pfs_end_table_io_wait_v1 function to know if there is a query, then bump counters at pfs_unbind_table_v1. Originally I was thinking put all those logic into pfs_end_table_io_wait_v1, but I realized that a single query can have multiple io operations. This we should only bump query_stat counter in pfs_unbind_table_v1. For details, look at pfs.cc I basically use the storage/perfschema/table_tlws_by_table.cc as the template. Differential Revision: D15974071 (https://github.com/facebook/mysql-5.6/commit/c24ec1b76a9aa6e43d469aec7ece946c4962f759) fbshipit-source-id: 6531dda8ca7

view details

Abhinav Sharma

commit sha fd7996dab83004116a17ad88caf4aaa993548514

Fixing memory leak when order by is used to dump selected tables Summary: `order_by` needs to be freed after every invocation of `dump_tables()`. In `dump_selected_tables()` we were not doing this. Upstream bug report: https://bugs.mysql.com/bug.php?id=96178 Reviewed By: lloyd Differential Revision: D16189662 (https://github.com/facebook/mysql-5.6/commit/072b4ff1a0104998ffe9c943dd597777a4911d33) fbshipit-source-id: 3ecb554e70a

view details

Abhinav Sharma

commit sha 6ce8d58c9d007ecf403b021866b5e189459ef53f

FB8-181: Maintain a whitelist of semi-sync slaves which can ACK transactions (#1017) (#1017) Summary: Jira issue: https://jira.percona.com/browse/FB8-181 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ee761fc Reference Patch: https://github.com/facebook/mysql-5.6/commit/68f34521e82 Reference Patch: https://github.com/facebook/mysql-5.6/commit/dfb038d48d0 Reference Patch: https://github.com/facebook/mysql-5.6/commit/de1e8c7bfe7 Reference Patch: https://github.com/facebook/mysql-5.6/commit/26835a39459 A whitelist of semi-sync slaves can be maintained using a new sys-var `rpl_semi_sync_master_whitelist`. The value of this variable is a comma separated list of UUIDs. ACK from any slave which is not in the whitelist will be ignored. The whitelist is persisted in file `semi_sync_master_whitelist.info` to survive server restarts. Porting notice: You have to use `set persist.rpl_semi_sync_master_whitelist` to keep value across server restarts. Originally Reviewed By: Tema Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1017 Differential Revision: D15285223 (https://github.com/facebook/mysql-5.6/commit/b5d9d0426353162a1529dcf22406c23334203b16) Pulled By: yizhang82 fbshipit-source-id: c638d32de0f

view details

Bertan Ari

commit sha 773e21c2524c2a4c749f6386fff6204deb012895

Adding SUM_ROWS_DELETED,SUM_ROWS_INSERTED and SUM_ROWS_UPDATED columns to events_statements_summary_by_user_by_event_name table. Summary: Extended events_statements_summary_by_user_by_event_name table to track deleted rows under the column of SUMS_ROWS_DELETED. Also added columns to track inserted and deleted rows. Reviewed By: lloyd Differential Revision: D16444830 (https://github.com/facebook/mysql-5.6/commit/8dc6387a0f16109b71d7f2d99874cc4e470c8008) fbshipit-source-id: 7a5de1c8980

view details

Zhicheng Zhu

commit sha c67a558a9a03e5162074e60906e42e6e6810d32c

Add queries_used and queries_empty to sys table view Summary: as title said add queries_used and queries_empty to schema_table_statistics Reviewed By: lloyd Differential Revision: D16569535 (https://github.com/facebook/mysql-5.6/commit/e0746739e0f7e55f4c79468eb2877e92a84217a9) fbshipit-source-id: 5b7ca66d4c2

view details

Zsolt Parragi

commit sha f2d120fafed0a6adfc4d9afc8b50f15c63162963

FB8-98, FB8-126, FB8-127: Introducing high priority DDLs (#1011) (#1011) Summary: Jira ticket: https://jira.percona.com/browse/FB8-98 Reference commit: 2b59a06 Reference commit: f816fee Reference commit: 397fa65 Reference commit: 00323f7 Reference commit: bffb8d7 Reference commit: e9cb575 Reference commit: b145f26 Reference commit: 7e41852 Reference commit: 434d496 Reference commit: https://github.com/facebook/mysql-5.6/commit/ee63260f74b -------- 2b59a06 -------- This diff introduces high priority DDLs. If enabled, DDLs will kill the blocking connections that hold shared metadata locks. Below are the details: - Added high_priority_ddl system variable that will allow DDL to kill blocking connections which hold shared locks. High priority DDLs are only available to admin users. By default the functionality is turned off. - Only the shared locks (read/write) will be affected, which are DML queries such as select, insert, update, etc. Any lock higher than upgradable MDL (i.e. from conflicting DDL) will not be affected. - When killing the blocking connections, their transactions will automatically be rolled back. This is the same behavior as `kill processlist_id` command. - The DDL will kill blocking connections at the end of the timeout instead of instantly. Admin can control the waiting period by setting the session lock_wait_timeout before issuing the DDL. -------- 397fa65 -------- This feature has been dropped. Adding high_priority syntax in DDL commands This diff introduces `HIGH_PRIORITY` syntax in some DDL commands and optimize command. The purpose is the same as the system variable `high_priority_ddl` which is described in D4784076 (https://github.com/facebook/mysql-5.6/commit/2b59a066be4eaaa94c7b5950454d6decb405e114). The list of commands that support `HIGH_PRIORITY` syntax is: - create/drop/alter/truncate table - create/drop index - create/drop trigger - optimize table With this diff, there are two ways of issuing high priority DDLs: e.g. `CREATE HIGH_PRIORITY INDEX ...` Recap of the high_priority ddl feature: - Only the shared locks (read/write) will be affected, which are DML queries such as select, insert, update, etc. Any lock higher than upgradable MDL (i.e. from conflicting DDL) will not be affected. - When killing the blocking connections, their transactions will automatically be rolled back. This is the same behavior as `kill processlist_id` command. - The DDL will kill blocking connections at the end of the timeout instead of instantly. Admin can control the waiting period by setting the session lock_wait_timeout before issuing the DDL. -------- b145f26 -------- Add LOCK TABLES as a supported command for high priority ddl Although LOCK TABLES is not a ddl command, it is useful to have it support taking mdl locks with high priority, via the high_priority_ddl session variable. -------- 7e41852 -------- Add high_priority_lock_wait_timeout system variable The original lock_wait_timeout is used for all queries and also the long running slave threads. It'll be better that high_priority commands (DDLs) use a separate system variable for timeout, which can be used across the master/slaves without messing with lock_wait_timeout. This diff introduces a new system variable high_priority_lock_wait_timeout that can be set for high_priority command's lock wait timeout, after which the command's thread will kill shared locks that are blocking the command. The new variable doesn't apply to regular queries. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1011 Differential Revision: D14884073 (https://github.com/facebook/mysql-5.6/commit/bddf9738de5d920144915ff1e809fb789254f7be) Pulled By: hermanlee fbshipit-source-id: 4af9020c7f7

view details

Luqun Lou

commit sha 2dd13d7be4fc0cf01cf8000fac821d2e93e87093

slave high priority DDL Summary: Adds support for enabling high priority ddl on slave threads. Depends on D14884073 (https://github.com/facebook/mysql-5.6/commit/bddf9738de5d920144915ff1e809fb789254f7be) Differential Revision: D16421520 (https://github.com/facebook/mysql-5.6/commit/6ab2e5f08a1e4bcee76c357403f7b5f5d1dc599c) fbshipit-source-id: bb53d705f66

view details

Artem Danilov

commit sha 8545e44738c791fa2ef7495f819cf07010f77c89

Add kill_conflicting_connections option for admin user Summary: Add kill_conflicting_connections session variable to allow 'set read_only=1' command to go through by killing all connections conflicting for metadata locks. The kill_conflicting_connections affects any command running within the session, not only 'set read_only=1'. Any user can set this variable but it will take effect only for super users and will be silently ignored for everyone else. This should help with acquiring global read only locks when it is hold by DDL commands. The kill_conflicting_connections will try to kill session conflicting for COMMIT lock as well, but it will hardly help as most of connections holding COMMIT lock are not really killable. Note hi-pri ddl kills only sessions conflicting for object level locks but kill_conflicting_connections affects both object level and scoped lock. Only the later is involved in 'set read_only = 1' so hi-pri ddl won't kill it. Reference Patch: https://github.com/facebook/mysql-5.6/commit/f2ed7c3e4ce Reviewed By: lloyd Differential Revision: D9563242 (https://github.com/facebook/mysql-5.6/commit/f2ed7c3e4cedb4c57705ff9111a3058cb2b1e2b3) fbshipit-source-id: e1b63e49d29

view details

push time in 7 days

push eventinikep/mysql-5.6

Abhinav Sharma

commit sha b1a615b1e13be9015c7b80c5ea71be52fe0232f8

FB8-199: Add an ability to insert sleeps in the dump thread before waiting for new events (#984) (#984) Summary: Jira issue: https://jira.percona.com/browse/FB8-199 Reference Patch: https://github.com/facebook/mysql-5.6/commit/af3dc25310e In some scenarios it might make sense to sleep before locking and waiting for an update in the latest binlog file to reduce contention. Originally Reviewed By: midom Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/984 Reviewed By: lloyd Differential Revision: D14421323 (https://github.com/facebook/mysql-5.6/commit/b7765e7d64ca2069609f7733e3c5aeeab8cd20e1) Pulled By: lth fbshipit-source-id: 06ec72570e1

view details

Herman Lee

commit sha 8589118e50feca584a74f1403ed2374646213495

FB8-143: Print out stack trace of symbol names in one line to the error log (#1000) (#1000) Summary: Jira issue: https://jira.percona.com/browse/FB8-143 Reference Patch: https://github.com/facebook/mysql-5.6/commit/b1538c5 Consolidate the stack trace of the symbol names into a single error log line output that can be processed easily by log aggregators. Originally Reviewed By: asandryh Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1000 Reviewed By: lloyd Differential Revision: D14652441 (https://github.com/facebook/mysql-5.6/commit/4d658567cacc6db0caa961883ad25d3246967aba) Pulled By: lth fbshipit-source-id: 77782a92bb1

view details

Jay Edgar

commit sha aff1ca250fc52b40e96b551cdcdef99511b57288

FB8-106: Add more information to duplicate key error (#998) (#998) Summary: Jira issue: https://jira.percona.com/browse/FB8-106 Reference Patch: https://github.com/facebook/mysql-5.6/commit/2c7f717 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f3a0ed0 The current error information for duplicate key lists the key value and key name. In some scenarios this may not be sufficient. Add a variable that, when enabled, includes the tableame for the key and the query text (or up to 512 bytes of it) into the error message. The new variable, improved_dup_key_error, will change the output error message to contain this new information. Originally Reviewed By: hermanlee Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/998 Reviewed By: lloyd Differential Revision: D14652434 (https://github.com/facebook/mysql-5.6/commit/bf0a8afd6e5555fb5bdc001ad6424c7fae280a26) Pulled By: lth fbshipit-source-id: 96c3ab5bc5a

view details

Yura Sorokin

commit sha 7f6a6147e1ed79adbe216499b3c1b004297ffe2a

Fix FB8-232 (Unable to build fb-mysql-8.0.13 with -DWITH_VALGRIND=ON) (#1033) (#1033) Summary: https://jira.percona.com/browse/FB8-232 Fixed problem with 'trashing' non-POD object of type 'buf_buddy_free_t' in 'buf_buddy_alloc_zip()'. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1033 Reviewed By: lloyd Differential Revision: D16262721 (https://github.com/facebook/mysql-5.6/commit/0bbcbcc10d57cc413d79b4895e4b565be37dc33a) Pulled By: lth fbshipit-source-id: 5b809f1b0d5

view details

Yi Zhang

commit sha fa5c3b7ddf301a289ca9a9bf0a9c1a07709d0da2

Add rocksdb_bulk_load_allow_sk option in mysqldump - this enables bulk loading secondary keys in mysqldump output Summary: See title Reference Patch: https://github.com/facebook/mysql-5.6/commit/309f324ea3238006e9f2c0cadfc62b93de461dae Differential Revision: D14864271 (https://github.com/facebook/mysql-5.6/commit/4984fdd88d71151330e62173c4dfaf48a994698f) fbshipit-source-id: ac50d0b7e06

view details

przemyslaw.skibinski@percona.com

commit sha 22d6884e817fdc09a30c17f322f8e5c394bda2ab

FB8-46: Port v5.1 Extra Stats: Command Timers (#989) (#989) Summary: Jira issue: https://jira.percona.com/browse/FB8-46 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f2d6fdc With additional improvements: 1. Variables moved from global to session from https://github.com/facebook/mysql-5.6/commit/459f669 2. Added time tracking for "sp_instr_stmt::exec_core", "mysql_open_cursor", "Execute_sql_statement::execute_server_code" and "Prepared_statement::execute" from https://github.com/facebook/mysql-5.6/commit/4554e16 Adds command, slave_command, pre-exec, parse, and execution timers. This ports these timers from v5.1 to 5.6, using the new timers. I also added an mtr test to confirm they are at least partly working. Originally Reviewed By: chip Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/989 Differential Revision: D14883715 (https://github.com/facebook/mysql-5.6/commit/8f0ffc0d89cf29d8930b4fe413fdfa09b3ee954e) Pulled By: yizhang82 fbshipit-source-id: e39699e203a

view details

Abhinav Sharma

commit sha 25fc8cd077ecec55a3fb8e5c8e4537723b2a8681

FB8-92: Checking for NULL while preparing write set according to binlog_row_image (#990) (#990) Summary: Jira issue: https://jira.percona.com/browse/FB8-92 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f3dc487 Field::cmp_binary_offset returns 0 when comparing a NULL text field and an empty string leading to the field not being logged when the value is changed from NULL to empty string. This fix first checks if both AI and BI fields are not NULL before comparing them. Originally Reviewed By: santoshbanda Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/990 Differential Revision: D14652424 (https://github.com/facebook/mysql-5.6/commit/b6a8e1d03664f7ba45f2bc10ac51003db2fa3544) fbshipit-source-id: 61a8844cc3a

view details

Zsolt Parragi

commit sha 2714cc7ec79eccd16ed551844db00d80ae2d8522

FB8-75: Periodic fsync in select into outfile controlled by sysvar (#992) (#992) Summary: Jira ticket: https://jira.percona.com/browse/FB8-75 Reference commit: https://github.com/facebook/mysql-5.6/commit/14b1086 Two variables are added, GLOBAL variables with SESSION override. select_into_file_fsync_size, which controls the size of the buffer after which a fsync will be done. Needs to be a multiple of 1024 After each fsync you can optionally sleep for select_into_file_fsync_timeout milliseconds. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/992 Differential Revision: D14883775 (https://github.com/facebook/mysql-5.6/commit/3f41610405981035af525795b825793a0e7f6da4) Pulled By: yizhang82 fbshipit-source-id: 4a9fc141a4e

view details

Zsolt Parragi

commit sha 9d3ba28c68d6509974769d59bed5927bc7f403bd

FB8-79: Add option to mysqlbinlog to print encountered gtids on exit (#994) (#994) Summary: Jira ticket: https://jira.percona.com/browse/FB8-79 Reference commit: https://github.com/facebook/mysql-5.6/commit/49f77bd Reference commit: https://github.com/facebook/mysql-5.6/commit/2306b6e Reference commit: https://github.com/facebook/mysql-5.6/commit/eaa782f Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/994 Differential Revision: D14883835 (https://github.com/facebook/mysql-5.6/commit/f219b9acdda631596f794ef6f8f22c1fc0beaec4) fbshipit-source-id: 3c15f90d081

view details

Manuel Ung

commit sha 7887a88ec5f76f18967789f3b675daeb867d4933

FB8-78: Add filesort_max_file_size option (#993) (#993) Summary: Jira issue: https://jira.percona.com/browse/FB8-78 Reference Patch: https://github.com/facebook/mysql-5.6/commit/11ef068 Reference Patch: https://github.com/facebook/mysql-5.6/commit/4a91b98aa46 Add the filesort_max_file_size variable setting which specifies the maximum number of bytes a filesort file can take. If it is exceeded, then the query fails. This is very similar to the tmp_table_max_file_size variable. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/993 Differential Revision: D14652353 (https://github.com/facebook/mysql-5.6/commit/cc088df2854093614bb26461b0a443c5eab995f2) fbshipit-source-id: 10d2ee4fd62

view details

przemyslaw.skibinski@percona.com

commit sha a0c6026d1003075b76fa39ca2ff9568eb0129183

FB8-206: Avoid executing status var functions during `SHOW STATUS` when not required (#1009) (#1009) Summary: Jira issue: https://jira.percona.com/browse/FB8-206 Reference Patch: https://github.com/facebook/mysql-5.6/commit/bf0baf3f7af Status var functions were always executed irrespective of matching string in the `LIKE` clause of `SHOW STATUS`. This caused execution of some heavy status var functions like `show_jemalloc_*` every time `SHOW STATUS` was invoked. In this change we execute the status var function only for vars whose names are strict prefixes of the wildcard specified in the `LIKE` clause. Since we're checking for strict prefix match, we might get false positives in some cases. We cannot use normal wildcard matching (like other function-less status vars) because some var names are constructed recursively in `show_status_array()` (e.g. `Rpl_semi_sync_master_trx_wait_histogram%` is composed out of `Rpl_semi_sync_master` and `trx_wait_histogram`). Originally Reviewed By: midom, anirbanr-fb Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/1009 Differential Revision: D14884044 (https://github.com/facebook/mysql-5.6/commit/9697af8c377782dc0f1b13329ce3619fd4795dfd) fbshipit-source-id: bcb1ecdd971

view details

Volodymyr Verovkin

commit sha e8506ca6e2ec690b75c0a60b9c5bd4bd3c2b1d78

FB8-180: Implemented TTL base cache eviction policy for "open table" and "table description" caches Summary: Jira issue: https://jira.percona.com/browse/FB8-180 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ec9e122 MySQL has variable "flush_time". If it is set to a nonzero value, all tables are closed every "flush_time" seconds to free up resources and synchronize unflushed data to disk. This option is best used only on systems with minimal resources. I added variable "flush_only_old_cache_entries" which were open more than "flush_time" seconds. Originally Reviewed By: anirbanr-fb Differential Revision: D14884057 (https://github.com/facebook/mysql-5.6/commit/3444872fea86cac5f5864ae468f4adaa18892637) fbshipit-source-id: cd43c3a333e

view details

Zsolt Parragi

commit sha 639bab5d78233cdac1421a92e7c0161f711fe206

FB8-67: Publish git hash in db log and via global status (#991) (#991) Summary: Jira ticket: https://jira.percona.com/browse/FB8-67 Reference commit: https://github.com/facebook/mysql-5.6/commit/e4f22e0 Reference commit: https://github.com/facebook/mysql-5.6/commit/f00855b Reference commit: https://github.com/facebook/mysql-5.6/commit/92bec02 Reference commit: https://github.com/facebook/mysql-5.6/commit/408ea32ed2 Rocksdb hash and date is set to "none" until rocksdb is added. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/991 Reviewed By: lloyd Differential Revision: D14603932 (https://github.com/facebook/mysql-5.6/commit/b59a89f15f4a1691ddaa4d736402b2eb2399d247) Pulled By: lth fbshipit-source-id: f6fe510abef

view details

Zsolt Parragi

commit sha 50672da599fba33b34ebd289066c392fee0234f1

FB8-132: NUMA support (#987) (#987) Summary: Jira ticket: https://jira.percona.com/browse/FB8-132 Reference commit: https://github.com/facebook/mysql-5.6/commit/e50c754879828c1d0b4f40cc13475ec7adea9afc Reference commit: https://github.com/facebook/mysql-5.6/commit/286e975 WebScaleSQL Feature: NUMA Support Credits for research and implementation: Jeremy Cole and Davi Arnaut This patch provides startup options: * flush-caches: Flush and purge buffers/caches * numa-nodebind: Run mysqld with CPU affinity Note, the original patch provided numa-interleave, but that is supported natively by 8.0 now. It also provides a config option: * innodb_buffer_pool_populate: pre-allocation of buffer pool memory at start up: -- Use MAP_POPULATE if supported (Linux 2.6.23 and higher) -- Forced pre-allocation using memset Automation will pass in the right value to this option if it needs to bind mysqld to a particular socket. That will happen via systemd scripts. This is only adding the capability on the mysqld_safe side to be able to pass in numactl to the server. The turning on of this feature will need more testing and slow roll, and that will happen in different diffs on the ops repo. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/987 Reviewed By: lloyd Differential Revision: D14652411 (https://github.com/facebook/mysql-5.6/commit/b5d4a679b5065b747c5734279617f5d8efaee11b) Pulled By: lth fbshipit-source-id: 7b1dd922612

view details

Yi Zhang

commit sha c556804c149bd3c07bcf673a9491482e9dd9472e

mysqlslap honors protocol Summary: I find out in my testing that mysqlslap doesn't really use the --protocol parameter in its workers. The bigger problem seems to be that it doesn't apply some of its parameters in the worker thread, --protocol is just one of them. I'm not sure I have time to test all the parameters here so I'm just adding the fix for --protocol. The testing for mysqlslap seems be rather minimal in MySQL in general. Reference Patch: https://github.com/facebook/mysql-5.6/commit/284346c05771015accb212885e41f51d6fc49cb0 Differential Revision: D15286198 (https://github.com/facebook/mysql-5.6/commit/a6e1773cccf8c9e6e97a6c1cda0909e2d14bd309) fbshipit-source-id: e154672c37a

view details

Satya Bodapati

commit sha d31f8c31143be012b0f914cc3566c68a307a9057

FB8-59: Database admission control (#969) (#969) Summary: JIRA: https://jira.percona.com/browse/FB8-59 Reference patch: https://github.com/facebook/mysql-5.6/commit/5cbfab0 Reference patch: https://github.com/facebook/mysql-5.6/commit/0ebaf75 Reference patch: https://github.com/facebook/mysql-5.6/commit/cf7ce48 Reference patch: https://github.com/facebook/mysql-5.6/commit/3842a8f Reference patch: https://github.com/facebook/mysql-5.6/commit/bcbd77a Reference patch: https://github.com/facebook/mysql-5.6/commit/e6c2b87 Reference patch: https://github.com/facebook/mysql-5.6/commit/928c2a5 Reference patch: https://github.com/facebook/mysql-5.6/commit/86587af Reference patch: https://github.com/facebook/mysql-5.6/commit/7870828 Reference patch: https://github.com/facebook/mysql-5.6/commit/0151b45 Reference patch: https://github.com/facebook/mysql-5.6/commit/75de821 Reference patch: https://github.com/facebook/mysql-5.6/commit/08d4174 Reference patch: https://github.com/facebook/mysql-5.6/commit/c1e5b4f Reference patch: https://github.com/facebook/mysql-5.6/commit/08368d4 Reference patch: https://github.com/facebook/mysql-5.6/commit/a2198e0 Reference patch: https://github.com/facebook/mysql-5.6/commit/0730342 Reference patch: https://github.com/facebook/mysql-5.6/commit/d868c1b Reference patch: https://github.com/facebook/mysql-5.6/commit/645fc5e Reference patch: https://github.com/facebook/mysql-5.6/commit/0b130f1ebc9 Reference patch: https://github.com/facebook/mysql-5.6/commit/f49a4a74945 <Comment by Percona developers> This patch ports Database admission control. The multi-tenancy plugin changes are not ported. This code will be eventually moved to new plugin of audit type. There are possible improvements with code. Below are the deferred items (makes it easier for review. Can compare with 5.6 code) 1. Remove MT_RESOURCE_ATTRS. we only need database name (also removes the typedefs) 2. Remove enums enum_multi_tenancy_resource_type & enum_multi_tenancy_return_type 3. Remove the usage of 'multi tenancy' word thd->is_real_trans is not ported from 5.6. Instead an equivalent is used. This is verified in 5.6 and it works well. (5.6 equivalent is: thd->transaction.all.ha_list == NULL;) "!thd->get_transaction()->is_active(Transaction_ctx::SESSION)" is used instead of thd->is_real_trans. The above check seems sufficient but after porting thd->is_real_trans, it should be possible to replace the above check with thd->is_real_trans </end> ------- https://github.com/facebook/mysql-5.6/commit/5cbfab0 -------- Add two global sys vars max_running_queries and max_waiting_queries. They control maximum number of running queries on a database and maximum waiting queries when max_running_queries limit is crossed on that database. A value of 0 implies no limits are applied for the queries. If max_waiting_queries limit is crossed, then new queries will simply fail. For waiting threads SHOW PROCESSLIST will show the state as 'waiting for admission'. Only dagtabase set at the session level is considered in these checks. Admission checks are by-passed in the following cases 1. Query is run by super user. 2. Query is run by replication threads. 3. No database is set for the session. 4. max_running_queries is 0. Performance testing 1. start MySQL using mtr > mtr --start --mysqld=--default_storage_engine=InnoDB 2. Connect to mysql using > mysql --socket=mysql-test/var/tmp/mysqld.1.sock --user=root 3. Set innodb_flush_log_at_trx_commit=2. mysql> set global.innodb_flush_log_at_trx_commit=2 4. Check rows inserted using > mysqladmin extended-status --user=root -i 1 -r --socket=mysql-test/var/tmp/mysqld.1.sock status | grep "Innodb_rows_inserted" 5. Run mysqlslap using > mysqlslap --auto-generate-sql --number-of-queries=5000000 --concurrency=$concurrency --auto-generate-sql-load-type=write --auto-generate-sql-add-autoincrement --csv=a --user=root --socket=mysql-test/var/tmp/mysqld.1.sock == Innodb_rows_inserted with concurrency=100 == 38106 (max_running_queries=0, max_waiting_queries=0) 38733 (max_running_queries=1000, max_waiting_queries=0) 38365 (max_running_queries=100, max_waiting_queries=0) 40308 (max_running_queries=70, max_waiting_queries=0) 43790 (max_running_queries=50, max_waiting_queries=0) 47321 (max_running_queries=20, max_waiting_queries=0) 49473 (max_running_queries=10, max_waiting_queries=0) 54851 (max_running_queries=5, max_waiting_queries=0) == Innodb_rows_inserted with concurrency=100 == 26516 (max_running_queries=0, max_waiting_queries=0) 44494 (max_running_queries=10, max_waiting_queries=0) 51193 (max_running_queries=5, max_waiting_queries=0) The above numbers show the improvement in quality of service. == Innodb_rows_inserted with concurrency=20 == 47228 (max_running_queries=0, max_waiting_queries=0) 47107 (max_running_queries=100, max_waiting_queries=0) 47082 (max_running_queries=20, max_waiting_queries=0) 47515 (max_running_queries=10, max_waiting_queries=0) * There isn't much difference under low concurrency workloads. * There isn't much difference with non zero max_waiting_queries values greater than max_running_queries. With values lower than max_running_queries, lots of queries simply fail, so the qps in this scenario is of no value. Original Reviewers: tianx, kradhakrishnan, jtolmer, jkedgar ----- https://github.com/facebook/mysql-5.6/commit/0ebaf75 ----- Added the following status variables: 1. Database_admission_control_aborted_queries gives the total number of queries aborted because of queue overfill. 2. Database_admission_control_running_queries gives the total number of running queries across all the databases. 3. Database_admission_control_waiting_queries gives the total number of waiting queries across all the databases. ----- https://github.com/facebook/mysql-5.6/commit/cf7ce48 ------ In order to avoid possible mysqld hangs, we need an option to blacklist specific commands in admission control. Example usage is set global.admission_control_filter='BEGIN,COMMIT,SET'; An empty value for admission_control_filter implies no commands are blacklisted in the admission control. ---- https://github.com/facebook/mysql-5.6/commit/bcbd77a ------ Initial version of admission control applied limits on each query statement in multi query packet. The behavior is modified to the following: A thread enters admission control when executing the first non-filtered command in the multi query packet and exits after executing all the query statements in the multi query packet. This diff also fixes a bug where admission control limits are not initialized during mysqld startup. ----- https://github.com/facebook/mysql-5.6/commit/928c2a5 -------- Get accurate status variables Hold global admission control locks to get accurate value of running/waiting queries. Initial stats were not accurate causing test failures ------- https://github.com/facebook/mysql-5.6/commit/86587af ----------- Added SHOW command filter in admission control. ------- https://github.com/facebook/mysql-5.6/commit/7870828 ------ The original ac_map was a map of unique_ptr<Ac_info>. When accessing the value we had to use a reference to the value. This diff changes that so that we use a shared_ptr<Ac_info> instead so we can just pass it around have have reference counting automatically. ---- https://github.com/facebook/mysql-5.6/commit/75de821 ------ Fixing the following two issues in admission control when changing database is in a multi-statement query: - If the session hasn't set a database, admission control will allow arbitrary number of multi-query packet to run, even the packet starts with `use [db_name]`. - If the session already has a default database, the admission control will use the limit of the default database, even if we change database in the multi-query packet. This diff fixes the above issues in the following way: - The admission control flag (is_in_ac) is reset when a `use` command is executed. So in the multi-statement query, the subsequent sub-queries are not bypassed. - Admission control uses the `thd->db` directly instead of attribute map, so the attribute map doesn't need to store the session database. - `USE [db_name]` statement is added to `admission_control_filter`. In addition, I also changed the multitenancy plugin interface to use a wrapper object to pass down the connection_attrs_map, query_attrs_map, and session database info. Attribute maps will not be modified. --------- https://github.com/facebook/mysql-5.6/commit/c1e5b4f -------- Fix resource leaks by admission_control_by_trx - Rollback will need to mark is_real_trans - Query resource needs to be released when THD is terminated. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/969 Reviewed By: lloyd Differential Revision: D14567747 (https://github.com/facebook/mysql-5.6/commit/03e9575388139aa676104cfc7f655cff7980e019) Pulled By: lth fbshipit-source-id: 5c1fdbde73c

view details

push time in 8 days

push eventinikep/mysql-5.6

Przemyslaw Skibinski

commit sha f830e336a828413b64d91a72e68ede2a963e4fd7

Fix Oracle's MTR tests when using --slave-parallel-workers=X Some tests are failing with MTS turned on with e.g. `--mysqld=--slave-parallel-workers=4`. Disable MTS for these tests with `--slave-parallel-workers=0`.

view details

Yura Sorokin

commit sha 70a9a30bc8d6fa50119af5c1ca669382210e1de8

Refactored 'main.all_persisted_variables' MTR test case (#1103) Reviewed By: lloyd Pulled By: lth fbshipit-source-id: b4405a9ed5d

view details

Yi Zhang

commit sha e72ee66783f87010340614dbbe6755b4554a6dd8

Add innodb experimental vars into sys_vars Summary: When enabling InnoDB 8.0 experimental sys vars in tools repo, we need to update the corresponding tests here as well. I'm not going to add test for them (since they are experimental) so I'm just rebaselining those tests for now. 4 of them are read-only so we need to bump the counter in all_presisted_variables as well. Reviewed By: luqun Differential Revision: D21759066 fbshipit-source-id: a9449e8d319

view details

Przemyslaw Skibinski

commit sha edd854eef95f6f049b3b69211de1546a76aac19f

Fix build and test errors for 8.0 Differential Revision: D7827421 (https://github.com/facebook/mysql-5.6/commit/43b6d7e25674f2219295892629ce9cc7e7e1bc05) Reference Patch: https://github.com/facebook/mysql-5.6/commit/c9bf1570398 Reference Patch: https://github.com/facebook/mysql-5.6/commit/5fc1ecad8fb Reference Patch: https://github.com/facebook/mysql-5.6/commit/bbd078c669b Reference Patch: https://github.com/facebook/mysql-5.6/commit/379db8c75de fbshipit-source-id: 5601f2fc864

view details

Herman Lee

commit sha 68bef9c574c07b86537308b7f6a8abfec6b6f0d7

Allow server to bootstrap with unknown variables Summary: When mysqld initializes for the first time, the --initialize parameter is passed in, which puts mysqld into a bootstrapping mode. In this mode, mysqld initializes the data directories and then exists. For 8.0.20, plugins are no longer loaded. However, the config file used for the normal mysqld process is also used for the bootstrapping process. Any plugin variables specified in the config file may result in the bootstrap process failing because those variables now considered unknown. To fix this, extend ignoring unknown options to include variables during the bootstrap process. Reviewed By: george-reynya Differential Revision: D26284550 fbshipit-source-id: dbc6773e06d

view details

Aliaksei Sandryhaila

commit sha 8d8606b15516770ec2261f1cd8c77dc8860cd06a

Port Control Cross-Table Access to 8.0 Summary: Port our v5.6 sysvar allow_noncurrent_db_rw, which detects when a query accesses data not in the current database. Depending on its setting, when a query tries to read or write to another database, it will: ON: Allow it. LOG: Allow it, but log that it occurred. LOG_WARN: Allow it, log it occurred, and send an error to client. OFF: Block it, and send an error to the client. Reference patch: https://github.com/facebook/mysql-5.6/commit/b5f54b4 Differential Revision: D6071054 (https://github.com/facebook/mysql-5.6/commit/d04eb6a30f4148e5aebbf850988554fa2a727185) fbshipit-source-id: 690ab034efe

view details

Steaphan Greene

commit sha 159d62f47821b74c071145926a27703be92465cf

Port BinLog BaseDirs Summary: Add global variables to list directory for binlog files and binlog index file. The variables are binlog_file_basedir and binlog_index_basedir. Reference patch: https://github.com/facebook/mysql-5.6/commit/3ccbf59 Differential Revision: D6155155 (https://github.com/facebook/mysql-5.6/commit/31a2eb2f651c0971cb626c080997060343bdb9d9) fbshipit-source-id: da30ee3447f

view details

Aliaksei Sandryhaila

commit sha 446a9ced0c111af96e672756bed555f960e0a103

Port fast timer support from 5.6 to 8.0 Summary: This replaces the fast_timers we added in 5.1, using the timers which are now included in upstream v5.6 for perfschema. Initializes the timers, selects which one to use, points the global "my_timer_now" function pointer to the correct timer, puts the stats for that timer in the global struct "my_timer", and sets the new SHOW STATUS variable "Timer_in_use" to indicate which was chosen. Added my_timer_ok mtr test to confirm the timer chosen is not "None". Added my_timer_good mtr test to confirm the timer chosen is millisecond precision or better. All timers are type ulonglong, and are stored in native timer units. Provides my_timer_now() function pointer to get current time. Provides my_timer_since(t) function to get time passed since time t, automatically adjusted for the estimated timer overhead. Provides my_timer_since_and_update(&t) function to get time passed since time t, adjusted for the estimated timer overhead. Also updates t to the current time, skipping that overhead. To count the time not counted here, add my_timer.overhead for each call to this function. Provides my_timer_to_seconds(t) function to convert native timer units to seconds (including fractions, in a double) for the current timer. Also adds _milliseconds and _microseconds versions of this. Provides microseconds_to_my_timer(t) function to convert microseconds to native timer units. Adds a new show variable type "SHOW_TIMER" which stores its value as a 64-bit unsigned integer, in native timer units, but displays as a double, in seconds, automatically converting it based on the current timer's frequency. Since this conversion is only done for display, no precision is ever lost in the actual variable, but the variable is still shown as seconds with a fractional component, just like doubles. This is the base for all our various stats which require timers. Reference patch: https://github.com/facebook/mysql-5.6/commit/614f9b0 Differential Revision: D6167189 (https://github.com/facebook/mysql-5.6/commit/4dad5e610f8aab7a20aab68465d733b648efd901) fbshipit-source-id: a183d5551b2

view details

Artem Danilov

commit sha 88f1fac5203e6c4193461f3f2635e578b584e89b

Port Extra Stats: callers of os_file_flush Summary: Report on the callers of os_file_flush in SHOW INNODB STATUS. This is displayed in the TABLESPACE section as 'fsync callers:' Tihe diff is amended from 5.6 branch to remove DIRTY_BUFFER category as it was not used and ARCHIVE category as I couldn't find a place to put it in Reference patch: https://github.com/facebook/mysql-5.6/commit/3e6bf37 Differential Revision: D6176814 (https://github.com/facebook/mysql-5.6/commit/e086c773efd1d284599ec83fa6f64a78211ad0aa) fbshipit-source-id: 9c60880903f

view details

Artem Danilov

commit sha 90cc32c64c6e10500d46f48ae7b373a7eeb4c68d

Port Extra Stats: Insert Buffer and Adaptive Hash Summary: Add insert buffer and adaptive hash stats to SHOW STATUS Add these to SHOW STATUS: Innodb_adaptive_hash_hits Adaptive hash search hits Innodb_adaptive_hash_misses Adaptive hash search misses Innodb_ibuf_merges Number of secondary index pages updated with merged records Innodb_ibuf_inserts Number of records added to the insert buffer Innodb_ibuf_delete_marks Number of records added to the delete mark buffer Innodb_ibuf_deletes Number of records added to the delete buffer Innodb_ibuf_size Size of the insert buffer in pages Reference patch: https://github.com/facebook/mysql-5.6/commit/1678522 Differential Revision: D6187284 (https://github.com/facebook/mysql-5.6/commit/e518b865c7c98b01e31daefbdb33bfbd0e8b6016) fbshipit-source-id: 6ab66d292b7

view details

Steaphan Greene

commit sha a617f6595370b98c9b7610a2defed98b0d0fe78a

Port v5.1 Extra Stats: InnoDB Lock Status Summary: Add InnoDB lock status details to SHOW [INNODB] STATUS Add these to SHOW STATUS: Innodb_row_lock_deadlocks Number of transaction aborts from deadlock Innodb_row_lock_wait_timeouts Number of transaction aborts from lock wait Innodb_purge_pending Number of transactions for which purge must be done. This is already named 'History list length' in SHOW INNODB STATUS Added to TRANSACTIONS section in SHOW INNODB STATUS: Lock stats: 0 deadlocks, 0 lock wait timeouts Added the innodb_lock_status mtr test. Reference patch: https://github.com/facebook/mysql-5.6/commit/fd4d7ae Differential Revision: D6211850 (https://github.com/facebook/mysql-5.6/commit/a76918055c3aab9bc21b34fce5a027c00112d80a) fbshipit-source-id: f5a2e500d5a

view details

Santosh Praneeth Banda

commit sha a95a12c57a0cdcdb6a1a977778df7f35432c983d

Check how max_binlog_cache_size affects LOAD DATA INFILE Summary: Need to check whether load data infile fails if it tries to create a binlog cache fails with size greater than max_binlog_cache_size. Its failing with ER_TRANS_CACHE_FULL error. Added a test case so that any future modifications in the code notifies the error. Reference patch: https://github.com/facebook/mysql-5.6/commit/cd75236 Differential Revision: D6214445 (https://github.com/facebook/mysql-5.6/commit/325cbd58169e41fec060b6d926c20ec6854995e9) fbshipit-source-id: 8860520edd9

view details

Rongrong Zhong

commit sha 28e188a52816bf93a2bf4869c78f81c49ec79966

Port Add --timeout option for mysqldump to 5.6 Summary: Allow per-session control of server user variables for mysqldump, thus avoiding session timeouts on write blockign (mostly in NFS environments) This adds --timeout option, which will set: * net_write_timeout * wait_timeout for mysqldump's connection to MySQL server Reference patch: https://github.com/facebook/mysql-5.6/commit/0b35262 Differential Revision: D6253820 (https://github.com/facebook/mysql-5.6/commit/be2dbf9427ad8a988c6f9d4756a0321fbd106d31) fbshipit-source-id: dc4a267dc18

view details

Rongrong Zhong

commit sha 4d52961c56a2462ce1b517441e44b065dbaff4c7

Port Stats: binlog slave offset to 5.6 Summary: This uses a variation of code from the Google patch to display the binlog name and offset currently downloaded by a replication slave SHOW PROCESSLIST; Id User Host db Command Time State Info 3 root localhost:47190 test Query 1 Writing to net INSERT INTO t1 SELECT * FROM t1 5 root localhost:47194 NULL Binlog Dump 1 NULL slave offset: master-bin.000001 4 (Original Reviewed By: steaphan) Reference patch: https://github.com/facebook/mysql-5.6/commit/3b05df3 Differential Revision: D6272842 (https://github.com/facebook/mysql-5.6/commit/6fd6a9351b200ed28460b8739930957cad21b53b) fbshipit-source-id: b8e18086d03

view details

Manuel Ung

commit sha 4070107d42093c26f9dc8ed43ad2db6f908da98a

Add client error codes to distinguish between network timeout and loss Summary: Previously, mysql clients returned CR_SERVER_LOST on any network exception, including timeout. This adds two new client error codes `CR_NET_READ_INTERRUPTED` and `CR_NET_WRITE_INTERRUPTED` which is returned on network timeout. `CR_SERVER_LOST` is still returned for everything else. Reference patches: https://github.com/facebook/mysql-5.6/commit/e3052e1 https://github.com/facebook/mysql-5.6/commit/e5b99cd Differential Revision: D6325225 (https://github.com/facebook/mysql-5.6/commit/323786b849944b9e7a2d9a59d47eb0ae0450106e) fbshipit-source-id: b7f356ffc9c

view details

Aliaksei Sandryhaila

commit sha 945723eb65d0577f686a239cee61c1627df2d3ea

Do not use timestamp adjustment Summary: mi->clock_diff_with_master stores the time difference of master and slave clocks. seconds_behind_master is adjusted by this difference every time it is queried. However the current method of detecting clock_diff_with_master is broken because it does not compensate for network delays etc. And since we know that all of our clocks are already synced, we can always set clock_diff_with_master to zero. Reference patch: https://github.com/facebook/mysql-5.6/commit/6340f0a Differential Revision: D6336275 (https://github.com/facebook/mysql-5.6/commit/7251630c5b19995571738b18234f69eaef502ebe) fbshipit-source-id: d4aab4701ea

view details

Aliaksei Sandryhaila

commit sha 79b3d90976f0eb9ee10c2dfedd7683b619d71397

Add mysqldump option to print ordering key Summary: The --print-ordering-key option to mysqldump causes mysqldump to get the key which is used to order the rows for each table, and print it to the dump file. This will help to efficiently run a diff on two mysqldumps of the same database at different points of time. Reference patch: https://github.com/facebook/mysql-5.6/commit/0f2578a Differential Revision: D6364804 (https://github.com/facebook/mysql-5.6/commit/42333e7696d2f6cb4710fab49dfa6416b5b288fd) fbshipit-source-id: f69a97d906e

view details

Aliaksei Sandryhaila

commit sha 820d0b450db10fea0d543a7fd9db0010fc95c575

Port v5.1: Output row counts in mysqldump Summary: Output the number of rows dumped by mysqldump in comment after each table. Reference patch: https://github.com/facebook/mysql-5.6/commit/9939293 Differential Revision: D6366330 (https://github.com/facebook/mysql-5.6/commit/13f919e34a8e0d2a3839d261e216cd9ab1995b9a) fbshipit-source-id: 7f0b1214ed6

view details

Aliaksei Sandryhaila

commit sha 2f1fe954822d2cab69865bf113e0c02469956f78

New conf option for SO_SNDBUF in dump thread. Summary: A new configuration option rpl_send_buffer_size is added to control the TCP send buffer size of the socket used in dump thread. Reference patch: https://github.com/facebook/mysql-5.6/commit/ac90867 Differential Revision: D6377432 (https://github.com/facebook/mysql-5.6/commit/a107c9cf9d05794f80585c164ea3190ff5afcaa8) fbshipit-source-id: d9b0099277c

view details

Steaphan Greene

commit sha 08e15e7c23a184939d211756b29d757aefb7bd49

Port v5.1 Extra Stats: Remove Transactions Summary: Disable including transactions in SHOW INNODB STATUS. Transactions are already visible in information_schema.innodb_trx. This change removes them from the SHOW INNODB STATUS output in order to keep the size managable. It also removes the size limit on the output to avoid truncation. (Originally Reviewed By: chipi) Reference patch: https://github.com/facebook/mysql-5.6/commit/151bcc5 Differential Revision: D6437907 (https://github.com/facebook/mysql-5.6/commit/0833a230cf05a54a1ec4c9d27510ee6a73c8eca8) fbshipit-source-id: 967a9157694

view details

push time in 8 days

push eventinikep/mysql-5.6

Tian Xia

commit sha 751b62e236171c9ef7a55fe0ae373200eb5225bd

FB8-84: Add global variable `max_nonsuper_connections` (#950) (#950) Summary: JIRA: https://jira.percona.com/browse/FB8-84 Reference Patch: https://github.com/facebook/mysql-5.6/commit/478fa2a Reference Patch: https://github.com/facebook/mysql-5.6/commit/8a269ee4f97 Reference Patch: https://github.com/facebook/mysql-5.6/commit/c53bf248a27 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ae0d38ae0c8 The new global variable `max_nonsuper_connections` will enforce the limit for the sum of all non-admin connections. This will be useful to limit regular user connections while still allow super users to connect to server. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/950 Reviewed By: lloyd Differential Revision: D13987618 (https://github.com/facebook/mysql-5.6/commit/72abd2ed75fb07eacf527995b76da9ec6caeefee) Pulled By: lth fbshipit-source-id: 6455e5c2861

view details

Abhinav Sharma

commit sha 0ec9d3b42f920edac2a426d7440621d165dcb1dd

FB8-105: Stop slave immediately in MTS if partial trx in the relay log can be rollbacked (#977) (#977) Summary: Jira issue: https://jira.percona.com/browse/FB8-105 Reference Patch: https://github.com/facebook/mysql-5.6/commit/cd39ae1 Reference Patch: https://github.com/facebook/mysql-5.6/commit/0623a07 Stop slave immediately in MTS if partial trx in the relay log can be rollbacked In MTS stop slave can take a minute to complete if the last transaction is partially downloaded from the master. The slave waits for a minute for the master to send the rest of the trx and then finally gives up. Strictly, this wait is only required if the partial trx cannot be rollbacked safely (e.g. trx on non-transactional engine, DDLs etc.). This change checks if there are no jobs queued in the worker threads and if the partial trx can be rollbacked, if yes, it immediately stops the slave. Originally Reviewed By: tianx Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/977 Reviewed By: lloyd Differential Revision: D14387911 (https://github.com/facebook/mysql-5.6/commit/64886ac703597efb65c0d188916125d9f906edf1) Pulled By: lth fbshipit-source-id: 79decc490ad

view details

Francois Richard

commit sha 4fea07d080a587a049782b46c88963898475ac44

FB8-128: track Semisync_ack Engine_commit time in slow query log (#982) (#982) Summary: Jira issue: https://jira.percona.com/browse/FB8-128 Reference Patch: https://github.com/facebook/mysql-5.6/commit/2dd16b3 Reference Pull: https://github.com/facebook/mysql-5.6/pull/681 Originally Reviewed By: lth Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/982 Reviewed By: lloyd Differential Revision: D14421316 (https://github.com/facebook/mysql-5.6/commit/29729c9656685bd78063213d3ff9deb1b9a0059c) Pulled By: lth fbshipit-source-id: 8ec16de3049

view details

Abhinav Sharma

commit sha f7bde93c813bf3e2b87f8c8148449da6c2c0d8c7

FB8-187: New variable binlog_rows_event_max_rows to control the number of rows in a single rows event (#974) (#974) Summary: Jira issue: https://jira.percona.com/browse/FB8-187 Reference Patch: https://github.com/facebook/mysql-5.6/commit/dfe6746cdfd Number of rows in a single event can be indirectly controlled with binlog_rows_event_max_size. This new variable will give more direct control to the user. Originally Reviewed By: anirbanr-fb Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/974 Reviewed By: lloyd Differential Revision: D14421324 (https://github.com/facebook/mysql-5.6/commit/e0c5da26ef044e71db0fad10769135560f419d36) Pulled By: lth fbshipit-source-id: 4b6898b2f14

view details

Manuel Ung

commit sha 7cdd25a05ff3586cf56cfe6dfa47fa4785d41379

FB8-172: Add variable to disable full table/index scans (#970) (#970) Summary: Jira issue: https://jira.percona.com/browse/FB8-172 Reference Patch: https://github.com/facebook/mysql-5.6/commit/b90e801 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ecef4b2 There are cases where it is always unacceptable for a client to be doing full table scans. To prevent this from happening, add a new variable optimizer_full_scan. When it is turned off, it will return a new error code ER_FULL_SCAN_DISABLED. For best results, use with optimizer_force_index_for_range. Currently, it does not fallback to alternative plans because we check at the end of query planning. EXPLAIN queries with derived tables do not populate select_options in the JOIN structure properly. This means that we could error if we tried to run an explain on a query with derived tables. Instead of reading from the JOIN struct, read from the LEX struct on the THD, as this is where the flag is original set on during query parsing. This bug also means that in upstream, we increment some status variables tracking full table scans despite the fact that only an explain statement was done. This seems to have been fixed in 8.0 though. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/970 Reviewed By: lloyd Differential Revision: D14567846 (https://github.com/facebook/mysql-5.6/commit/707435385a820e1df294e36daf3aaa7973cdd7d8) Pulled By: lth fbshipit-source-id: f7a7ed38322

view details

Anirban Rahut

commit sha b79e52021d562a194818b5fc58bd13e1eda8b8d6

FB8-139: TraceID support in MySQLD (#981) (#981) Summary: Jira issue: https://jira.percona.com/browse/FB8-139 Reference Patch: https://github.com/facebook/mysql-5.6/commit/18c1e44 Enables traces to be passed in as query attributes by clients and traces (instruction costs) to be collected on server side. Originally Reviewed By: abhinav04sharma Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/981 Reviewed By: lloyd Differential Revision: D14387874 (https://github.com/facebook/mysql-5.6/commit/482ae55a3c4cb91cd2cbde00688d31614e8b3760) Pulled By: lth fbshipit-source-id: a18b3abf052

view details

Zsolt Parragi

commit sha dc06bf15bec8ff5074637c8b0bf58ae8d43e9ddf

FB8-104: enabling subsecond lock_wait_timeout (#980) (#980) Summary: For a first review, I only added the code changes to this pull request for two reasons: * a full test is still running, and I'll have to check the results (I do see a few non output only failures because timeouts, so looks like I missed some nsec-sec conversion in this diff - I'll fix that and update). And checking output changes / rerecording will take time. * compared to the 5.6 patch, the code change for 8.0 is huge, and affects the service api - reviewing only the code changes is probably a good idea Jira ticket: https://jira.percona.com/browse/FB8-104 Reference commit: https://github.com/facebook/mysql-5.6/commit/c21adb7 Reference commit: https://github.com/facebook/mysql-5.6/commit/25f0f276d13 Currently lock_wait_timeout is an integer type variable which only have seconds resolution. We would want to extend the variable to double/decimal type, so it can have milliseconds or microseconds resolution, such as lock_wait_timeout = 1.23. The integral part should still mean seconds to keep backward compatibility. The range of valid values expanded to [ 0 .. 604800 ] to allow timeouts below 1 second; such as lock_wait_timeout = 0.005. Values that are too small will effectively mean not waiting for locks. This update also changes the output of lock_wait_timeout to be decimal; note the extra .000000 appended to the seconds. SELECT global.lock_wait_timeout; global.lock_wait_timeout 604800.000000 Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/980 Reviewed By: lloyd Differential Revision: D14349342 (https://github.com/facebook/mysql-5.6/commit/a2ddd7a7261f94f7d7aca1018608972033c8ecc0) Pulled By: lth fbshipit-source-id: 5b6e53b91f1

view details

push time in 9 days

push eventinikep/mysql-5.6

Manuel Ung

commit sha 14c7c61a2e3dce613dc8337539ba9916d3ba2522

Don't disable SSL on socket Summary: 8.0 disables SSL for socket connections. We still do auth on socket connections and turning off SSL means all SSL-only users cannot connect through socket. Differential Revision: D7961099 (https://github.com/facebook/mysql-5.6/commit/fa74396cfc10bff71be9b1bc44c54ac76a4b59cb) fbshipit-source-id: df9d2af972a

view details

Manuel Ung

commit sha 2e46a2137acbac9c6606a60ea42fbfb4158b16d9

Return early from mysql_next_result if connection is closed Summary: Some clients incorrectly call mysql_next_result after mysql_close. This results in a memory allocation due to MYSQL_TRACE_STAGE allocating mysql->extension on demand. Because there isn't another mysql_close call, this memory gets leaked. Differential Revision: D7961414 (https://github.com/facebook/mysql-5.6/commit/7665232b8d1d11a7cea9ada4ef1928edb3cc51e6) fbshipit-source-id: 76043e92d12

view details

Santosh Praneeth Banda

commit sha 99a3ace27c945af3d66beac07ba070e4042bd32e

option to log only comments of a query when using RBR Summary: option to log only comments of a query when using RBR. Note that comments at the beginning of the query are logged. Other comments in the middle or end of the query are ignored. Reference Patch: https://github.com/facebook/mysql-5.6/commit/fcd892a Differential Revision: D7717532 (https://github.com/facebook/mysql-5.6/commit/fa9c631315dcbf5f0e75b055508fe03574fa5919) fbshipit-source-id: 0ffd1fe73b5

view details

Manuel Ung

commit sha 54ae70ea22735e794b78992cc3b3f626188c605e

Optionally log all @@global variable changes to error log Summary: A new global variable, i.e. a sysvar, log_global_var_changes was introduced. Its default value is false. If it is set to true, all global variable changes will be logged into error log. The corresponding command line option is also supported. Instead of modifying every Sys_var_*::global_update function, just use get_one_variable instead to extract text form of the variable. The feature of logging global variables whose value was set either from command line options or my.cnf with different value than its default value is dropped. This is because: - This feature never worked because the error log has not been initialized when variables are being set. The output is printed to stderr, never makes it to a log. - Equivalent functionality can be found in performance_schema.variable_info table (https://dev.mysql.com/doc/refman/8.0/en/variables-info-table.html) Reference patch: https://github.com/facebook/mysql-5.6/commit/edb78db Differential Revision: D8730324 (https://github.com/facebook/mysql-5.6/commit/5e48ee8b58871ef0f9843b630e571b3a0a339313) fbshipit-source-id: 3d8bba53fef

view details

Abhinav Sharma

commit sha 531f62ffe243dfc8a60c9dd21be610a5546a4068

Implemented COMPLETE binlog row image format Summary: When binlog_row_image is complete, mysql will log all columns in the before row images, but only changed columns in after row image Reference patches: https://github.com/facebook/mysql-5.6/commit/e2428b6423723cab47c4f2b8434e2413bcb0e7c4 https://github.com/facebook/mysql-5.6/commit/3ccefedf0986a64e7e63ec33ba534a4c3c0cbbe9 https://github.com/facebook/mysql-5.6/commit/d29492d28cf5b434496e06a2505a2c99f6677f43 Differential Revision: D8520660 (https://github.com/facebook/mysql-5.6/commit/871ab9d3c73ecff373667a042a6bf3a9bf176daf) fbshipit-source-id: 045fc8d20b4

view details

Rongrong Zhong

commit sha a9a74d60719a0351ff3fcb652bb99edbc0cc0cfa

force stats recalculation when "show table status" is called Summary: Make innodb_stats_on_metadata a session variable and enforced for both transient and persistent stats. Reference patch: https://github.com/facebook/mysql-5.6/commit/3e82c24 Differential Revision: D8794479 (https://github.com/facebook/mysql-5.6/commit/475d60e689561c08a9ee91bdb1aa72dba737288a) fbshipit-source-id: 7314ea1251b

view details

Aliaksei Sandryhaila

commit sha d082f1e07a32487f5ce91899f3bcfbd37d62336a

Changing long_query_time to base on total query time including lock wait time Summary: Slow log and long_query_time only log slow queries based on the time after locks are acquired. It doesn't reflect the time a query spends on waiting for locks. This diff fixes long_query_time to reflect the total clock time a query spends after sending to the server. Reference patch: https://github.com/facebook/mysql-5.6/commit/95427d7 Reviewed By: lloyd Differential Revision: D8795023 (https://github.com/facebook/mysql-5.6/commit/00c05e895b2df45f0c234648a4f219f9321f859b) fbshipit-source-id: 70e5c2239e2

view details

Manuel Ung

commit sha 3c9230928950727cb6b4602cb467e6d835ee39f6

Add long_query_time option to mysqldump Summary: The slow log currently contains noise from mysqldump because their queries are usually longer running. This adds an option to mysqldump to specify long_query_time as a session variable to avoid this problem. Specifying --long_query_time=0 for mysqldump means using the server's value instead of explicitly setting it. It does not set long_query_time to 0. Reference patch: https://github.com/facebook/mysql-5.6/commit/76d7986 Differential Revision: D8905202 (https://github.com/facebook/mysql-5.6/commit/ab1521f2e39a3c2adc23915a61a1833deb959a07) fbshipit-source-id: 041d29ecabd

view details

Aliaksei Sandryhaila

commit sha 8eb4b00283a95badf625b94a8eb94da1627ffc35

added IF EXISTS to the ALTER TABLE command Summary: Added the ability to alter a table only if it exists. This was requested for replication purposes. Reference patch: https://github.com/facebook/mysql-5.6/commit/216c036 Reviewed By: lloyd Differential Revision: D8822284 (https://github.com/facebook/mysql-5.6/commit/71a8559120fc4b7ddb4b0303f1d02970e0cf78a3) fbshipit-source-id: c3614c307df

view details

Manuel Ung

commit sha bba3e6952758f3896a0c354a272ac7545a878592

Log semi-sync tailer's host/ip information Summary: Log semi-sync tailer's host/ip information Reference patch: https://github.com/facebook/mysql-5.6/commit/dfeef11 Differential Revision: D8867279 (https://github.com/facebook/mysql-5.6/commit/d1464beb7de988674c23e1d04b7765fd27513c8c) fbshipit-source-id: 93566cb1c3e

view details

Aliaksei Sandryhaila

commit sha 1386c1834c5722b3e7ed017d7f5a28086721c844

deleted copyright text and helper text on start of CLI Summary: When opening the CLI you no longer have space wasted on copyright and text about opening help. Closes #694 Reference patch: https://github.com/facebook/mysql-5.6/commit/a19a389 Reviewed By: lloyd Differential Revision: D8881138 (https://github.com/facebook/mysql-5.6/commit/634a84588a87f8da00e15f663311a113264e4a3d) fbshipit-source-id: 5211b6c32ef

view details

Aliaksei Sandryhaila

commit sha 8fd4134a97b74923f80a7fb6b5e309c65d544fb6

Remove mysql client warning about passwords Summary: This warning annoys people, and it's pretty stupid since it just makes you do it in even less secure ways (like env vars). Reference patch: https://github.com/facebook/mysql-5.6/commit/ec66b6e Reviewed By: lloyd Differential Revision: D8881245 (https://github.com/facebook/mysql-5.6/commit/780927dc2aa8c13f5f16206670a43c88a986f593) fbshipit-source-id: 27a8ee187df

view details

Aliaksei Sandryhaila

commit sha 2a0f7e95abc2f401d12b777e2ce53657f064fbf5

change handshake to include rpm version Summary: Changes the handshake sent over by mysqld to "version_comment" rather than "version", which contains more information. Reference patch: https://github.com/facebook/mysql-5.6/commit/b98a293 https://github.com/facebook/mysql-5.6/commit/292d5461 Reviewed By: lloyd Differential Revision: D8881186 (https://github.com/facebook/mysql-5.6/commit/af106e5209c0b7596c49525a72733bc5aa39caa9) fbshipit-source-id: 905f3beb2df

view details

Jiacheng Feng

commit sha 8b957eda65d649ba0835c3ea033c6c14e8dcbd23

add statistics for recreated rows in InnoDB Summary: Add two global status variables 'innodb_row_recreations' and 'innodb_row_recreation_steps' at innodb layer to get statistics info for recreated rows in innodb. This diff implemented the status variables at innodb layer instead of server layer is because server layer doesn't really care about row recreation that is specific to innodb. 'innodb_row_recreations' will be increased by one each time innodb goes to old versions to find a row. 'innodb_row_recreation_step' will be increased by the steps of the number of versions which innodb goes through to find the row. The corresponding innodb status variables are defined in srv_stats. Reference patch: https://github.com/facebook/mysql-5.6/commit/9cecd2c Differential Revision: D8882644 (https://github.com/facebook/mysql-5.6/commit/89fe9bb7cda9cc88bee7444c3ea8f45a8f56a9d1) fbshipit-source-id: 6be95217dc3

view details

Tian Xia

commit sha f17b917c0438b98eaf5c889cb7def9bd0bcc270b

Added optional commenting of the @@GLOBAL.GTID_PURGED line in mysqldump Summary: Added a new allowed value for --set-gtid-purged command linbe argument for mysqldump. --set-gtid-purged=COMMENTED will output the SET GLOBAL.GTID_PURGED information in a comment. (This code is already accepted in 8.0.17) Also, changed the logic of extracting the GTIDS. They are now extracted from the results of START TRANSACTION. (Remaining part of the patch). Reference Patch: https://github.com/facebook/mysql-5.6/commit/344a333 Reference Patch: D1515198 Reference Patch: https://github.com/facebook/mysql-5.6/commit/9adb44b120b Reference Patch: https://github.com/facebook/mysql-5.6/commit/015559b4efa Differential Revision: D8266014 (https://github.com/facebook/mysql-5.6/commit/384f980b85f3a0af8b71a03adea86bfa2e5781cc) fbshipit-source-id: 64ccab7f4f6

view details

Santosh Praneeth Banda

commit sha dffba82eab4e135199a7afa928a3208ac39e9491

Option to ignore views in backups Summary: See title Reference patch: https://github.com/facebook/mysql-5.6/commit/d07fedf Differential Revision: D9092424 (https://github.com/facebook/mysql-5.6/commit/c9852e5efdd50814828c611d263ed4cebe4460d4) fbshipit-source-id: 4d6c511a55e

view details

Santosh Praneeth Banda

commit sha d8ebd7177ff206fe1bc7f2c7b9ce8be2769d4eb5

Change binlog index file format to include previous gtid set. Summary: Format of binlog index file is changed to include previous gtid set. This is used during server start-up where gtid_executed and gtid_purged are initialized without scanning and opening binary logs. The functions find_log_pos, find_next_log, are changed so that they skip previous gtid set(stored as binary string) in the index file while reading. The new change will handle both old and new formats. The initialization of gtid_sets is called (init_gtid_sets()) before opening binlog (open_binlog()) during server start-up. This is done to avoid the extra work of opening the active file again to write the previous gtid log event. This also avoids writing the the previous gtid set again in the index file which needs to be done if init_gtid_sets() is called after open_binlog(). Added a mapping from binlog name to previous gtid set in that binlog in memory(previous_gtid_set_map) so that opening binlog files is not necessary by the dump thread when a slave reconnects. Modified the logic in find_first_log_not_in_gtid_set so that it reverse iterates over the in memory data structure previous_gtid_set_map. Added a test case to check gtid_executed and gtid_purged are initialized properly on both master and slave after FLUSH LOGS, PURGE BINARY LOGS command and also after restarting server several times. No extra tests are added to test the scenario where gtid_mode=OFF and reading binlog index files since binlog.binlog_index tests all those scenarios. Reference Patch: https://github.com/facebook/mysql-5.6/commit/ecf5786 Reference Patch: https://github.com/facebook/mysql-5.6/commit/05e5f04 Differential Revision: D8923357 (https://github.com/facebook/mysql-5.6/commit/667dcc92636f70d151eb293c1ff5f10310b32382) fbshipit-source-id: 343859ba084

view details

Santosh Praneeth Banda

commit sha 693b7e7df1101621cd6b29712d7b699d22a9c87e

find binlog by gtid functionality and mysqlbinlog should be more user friendly with gtids Summary: A new query FIND BINLOG GTID='gtid' functionality is given. It prints the binlog file name and the starting position where the requested gtid event is physically present. The query reverse iterates over the previous_gtid_set_map to find the binlog file where the gtid is present and scans the resulted binlog file comapring the Gtid_log_events with the requested gtid to find the starting position of the corresponding Gtid_log_event. A cache of gtid and physical coordinates is helpful to speed up this procedure, i would like to evaluate this option in another task. New options --start-gtid --find-gtid-position are added to the mysqlbinlog utility. Both require --index-file option which should specify path to the binlog index file. Index file is required to build previous_gtid_set_map which avoids opening each binlog file to read previous_gtid_log_event. --start-gtid dumps every event starting from the Gtid_log_event corresponding to the given GTID. --find-gtid-position prints the binlog file name and starting binlog position of Gtid_log_event corresponding to the given GTID. Reference Patch: https://github.com/facebook/mysql-5.6/commit/374f475 Reference Patch: https://github.com/facebook/mysql-5.6/commit/bfe57d1 Reference Patch: https://github.com/facebook/mysql-5.6/commit/11a5671 Reference Patch: https://github.com/facebook/mysql-5.6/commit/bfe57d1 Differential Revision: D9005165 (https://github.com/facebook/mysql-5.6/commit/c3fd2fa93cc011d667f997d7e8bb67b9c2e7999a) fbshipit-source-id: dae53a56ad7

view details

Yoshinori Matsunobu

commit sha 568de8684489827830440698cb3b693b6b112e32

mysqldump supporting RocksDB Summary: This diff makes mysqldump support RocksDB, by the following two features. 1. Added an option --rocksdb. This tells mysqldump to use START TRANSACTION WITH CONSISTENT ROCKSDB SNAPSHOT instead of INNODB SNAPSHOT, when using together with --single-transaction. 2. Added an option --order-by-primary-desc. This tells mysqldump to take backup order by PRIMARY KEY DESC. This is not engine dependent, but helps RocksDB when using reverse order column family for primary key. This diff adds a new session variable rocksdb_skip_fill_cache (default false, and enabled at mysqldump), and set ReadOptions.fill_cache = false if turned on. This avoids mysqldump filling block cache. Reference Patch: https://github.com/facebook/mysql-5.6/commit/d59aea0 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ca1f197 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ca0f3507354 Differential Revision: D9108738 (https://github.com/facebook/mysql-5.6/commit/c8503fd21727123e90ee5a7dd6d726d4d3ced415) fbshipit-source-id: b39c64ff8bf

view details

Aliaksei Sandryhaila

commit sha 8b9547ab321b382a0077865e36a9b6fe9b556c60

Added new option --stop-gtid Summary: Added new option --stop-gtid that stops reading binlogs once it gets the first binlog event with a gtid not included in the given gtid set Allow --stop-gtid to be used with --index-file option. Reference patch: https://github.com/facebook/mysql-5.6/commit/632864c Reviewed By: lloyd Differential Revision: D9138382 (https://github.com/facebook/mysql-5.6/commit/fdd85d38ac3569584adb4639afc2520b22eac68b) fbshipit-source-id: a7598cf021e

view details

push time in 9 days

push eventinikep/mysql-5.6

Santosh Praneeth Banda

commit sha 410b822d2d3c964862071eec05eae16a5aa74a3d

FB8-80: Display row information in processlist for rbr threads (#953) (#953) Summary: Jira ticket: https://jira.percona.com/browse/FB8-80 Reference Patch: https://github.com/facebook/mysql-5.6/commit/564824a Set row_query using the fields unpacked from the row event. Use row_query in show processlist. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/953 Reviewed By: lloyd Differential Revision: D14137212 (https://github.com/facebook/mysql-5.6/commit/c3bd1c6fdad506527f5e26ad5e9866fa90c00a6c) Pulled By: lth fbshipit-source-id: 85e53d756e5

view details

Manuel Ung

commit sha 8b16930f846bdad17fb92287f098653ee09e80c4

FB8-167: Add variable to force range plans if force index is used (#967) (#967) Summary: Jira ticket: https://jira.percona.com/browse/FB8-167 Reference Patch: https://github.com/facebook/mysql-5.6/commit/b01ff6a Currently, when force index is used, full table scans are highly penalized, but it is still possible to do a full index scan which is expensive. In many cases, we only use force index when we know there is better way to traverse the index, and not necessarily because we want a full index scan. To address these cases, add a session variable called optimizer_force_index_for_range, which when turned on, will try to search for a 'range' plan before falling back to an index plan. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/967 Reviewed By: lloyd Differential Revision: D14387089 (https://github.com/facebook/mysql-5.6/commit/f19d0c28b985595b8b0ab101339e62b683888e0d) Pulled By: lth fbshipit-source-id: 29a3701d2eb

view details

push time in 9 days

PullRequestReviewEvent

push eventinikep/mysql-5.6

Manuel Ung

commit sha 191228fa35c76b0f40d36cdea2d9356ebe2da9dc

FB8-61: Add lock information to lock timeout error message (#961) (#961) Summary: Jira issue: https://jira.percona.com/browse/FB8-61 Reference Patch: https://github.com/facebook/mysql-5.6/commit/c2dbe68 Notice: Changes in `ha_rocksdb.cc` and files in `mysql-test/suite/rocksdb` were not ported. Currently, on lock timeout, we only give ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction which has no information on what type of lock timed out. This change augments the message with the lock type to say something like ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction: Timeout on index: test/t1.GEN_CLUST_INDEX which shows which type of lock timed out as well as the associated object. There two cases to handle here. For error messages coming from the storage engine, the error is stored onto the transaction, which will get forwarded to the SQL layer for display through `handler::get_error_message`.. For MDL locks in the SQL layer, we are directly using the information in the MDL lock to display the error message. I didn't touch NDB because I didn't bother getting that to build. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/961 Differential Revision: D14137402 (https://github.com/facebook/mysql-5.6/commit/8b2b92ff428efff2a0126729443c947a28448d18) Pulled By: hermanlee fbshipit-source-id: 4aa5c777148

view details

Zsolt Parragi

commit sha 13e0f2a62a864f2f205e16fdf3ef0a27fb6ecbe9

FB8-134: Fix crash during shutdown in audit plugin code path (#957) (#957) Summary: Jira ticket: https://jira.percona.com/browse/FB8-134 Reference patch: https://github.com/facebook/mysql-5.6/commit/0fc059a ------ 0fc059a ------ On server shutdown, there exists a race condition that can trigger a segfault if an audit plugin is installed. The root cause because the `THD::audit_class_plugins` array is accessed concurrently without locks. This is the sequence of actions that can trigger a crash: 1. The shutdown thread calls close_connections on all connected clients. 2. The connection thread will call into the audit plugin because it will receive a network error from `close_connections`, and will log the error into the audit plugin. The thread then adds the plugin to `THD::audit_class_plugins`, but it has not updated `audit_class_mask` yet. 3. The shutdown thread sees that `audit_class_mask` is not set, and also adds the plugin into `THD::audit_class_plugins`, but it has only extended the array, without setting the value of the 2nd element of the array. 4. The connection thread then goes ahead and updates `audit_class_mask` and finishes `acquire_plugins`. It then calls `mysql_audit_notify` and sees the array of size 2, and when it loops over the 2nd element, it segfaults because the value is uninitalized. The fix is to add a lock per THD that protects these data structures. They should be relatively uncontended since the shutdown thread is the only thread that calls `acquire_plugins` on another THD. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/957 Differential Revision: D14236916 (https://github.com/facebook/mysql-5.6/commit/767a603b353e19ff5a7a9234b1188305141b8219) Pulled By: hermanlee fbshipit-source-id: e31d48eb629

view details

Manuel Ung

commit sha 6eadef6287f41a7ec9882bd70e23cf0c2f9cd31a

Revert "WL#4321 skip-grant-tables imply skip-networking" Summary: We still have cases where we want to connect to mysql via a port without grants. Differential Revision: D14333193 (https://github.com/facebook/mysql-5.6/commit/23b537af5a0e827f6ff39e95b232484caa6eec10) fbshipit-source-id: 03aa6d8f58c

view details

Santosh Praneeth Banda

commit sha 37084152ef4b6ee84be80ad2b846d786a3433b27

FB8-48: Option to run triggers on slave for row-based events (#946) (#946) Summary: JIRA: https://jira.percona.com/browse/FB8-48 Reference Patch: https://github.com/facebook/mysql-5.6/commit/8225c64 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f5466d6 Reference Patch: https://github.com/facebook/mysql-5.6/commit/87e3650 Port the slave_run_triggers_for_rbr feature from mariadb 10.1.1. When using statement based replication slave executes the sql statments which runs the slave side triggers. Since in row based replication, slave applies the row events directly to the storage engine, triggers on the slave table are not executed. Add functionality to run triggers on slave side when executing row based events. The following triggers are invoked: * Update_row_event runs an UPDATE trigger * Delete_row_event runs a DELETE trigger * Write_row_event action depends on whether the operation will require foreign key checks: 1) when FK checks are not necessary, the operation will invoke a DELETE trigger if the record to be modified existed in the table. After that, an INSERT trigger will be invoked. 2) when FK checks are necessary, either an UPDATE or or a combination of DELETE and INSERT triggers will be invoked. slave_run_triggers_for_rbr option controls the feature. Default value is NO which don't invoke trigger for row-based events; Setting the option to YES will cause the SQL slave thread to invoke triggers for row based events; setting it to LOGGING will also cause the changes made by the triggers to be written into the binary log. There is a basic protection against triggers being invoked both on the master and slave. If the master modifies a table that has triggers, it will produce row-based binlog events with the "triggers were invoked for this event" flag. The slave will not invoke any triggers for flagged events. optionally disable binlogging while executing triggers Online schema change (OSC) creates triggers only on master side. Since RBR logs row changes made by triggers, sql_thread will hit errors due to missing tables on slaves. Add a session variable SQL_LOG_BIN_TRIGGERS to optionally disable binlogging for trigger statements so that RBR and OSC are fine with each other. disable_sql_log_bin_triggers flag is added in the TABLE_LIST struct to track tables that are opened during trigger execution. This flag is used to skip writing Table_map_log_events for such tables. With sql_log_bin_triggers enabled, the trigger changes which may be necessary for slave are not propagated. To avoid this, slave_run_triggers_for_rbr option must be enabled on slave. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/946 Reviewed By: lloyd Differential Revision: D13972562 (https://github.com/facebook/mysql-5.6/commit/a5e5b466a05baebaed3e14ee6f7e165acf6dd5a8) Pulled By: lth fbshipit-source-id: 3aa1abda384

view details

Abhinav Sharma

commit sha 3095b5ad74238cdabddef89e9b229ecf4b897fe7

FB8-82: Removing unchanged columns from AI when when row image is not FULL (#942) (#942) Summary: JIRA: https://jira.percona.com/browse/FB8-82 Reference Patch: https://github.com/facebook/mysql-5.6/commit/b1fec60 Some update and upsert (insert ... on duplicate key update) queries used to log columns which have not changed in the after image (when using RBR) even when the image format is not FULL. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/942 Reviewed By: lloyd Differential Revision: D13930763 (https://github.com/facebook/mysql-5.6/commit/171a6f517c71b54840d9b9a063326b0b45cace4b) Pulled By: lth fbshipit-source-id: 271fc27a431

view details

Abhinav Sharma

commit sha 31ed9b4098fe96e555d5dee4a2a59f19af04e2f7

FB8-131: Option to shutdown master if semi-sync is being switched off while there are active un-acked trxns (#966) (#966) Summary: Jira issue: https://jira.percona.com/browse/FB8-131 Reference Patch: https://github.com/facebook/mysql-5.6/commit/2d47615 Reference Patch: https://github.com/facebook/mysql-5.6/commit/d1ea741 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f5085f4 Reference Patch: https://github.com/facebook/mysql-5.6/commit/9204c65 Option to shutdown master if semi-sync is being switched off while there are active un-acked trxns Semi-sync master can be switched off manually or because of a timeout while waiting for an ack. This change introduces a new variable rpl_semi_sync_master_crash_if_active_trxs which when set to true will cause the master to shutdown (after printing an error msg) if semi-sync is being switched off while there are active un-acked transactions. This prevents any un-acked trx from committing on the master. Also delete pid file when crashing after timeout with unacked trx. If rpl_semi_sync_master_crash_if_active_trxs = true, the server crashes after a timeout if there are any active unacked transactions. mysqld_safe expects the pid file to be deleted if the expectation is to not bring the intance back up again. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/966 Differential Revision: D14308888 (https://github.com/facebook/mysql-5.6/commit/55756acf8a6d240398b2a78355f0b74c2d3f518e) Pulled By: hermanlee fbshipit-source-id: e1ee8daf01a

view details

Manuel Ung

commit sha ff10045010d1173dd49ce11efc76e8917b65ba24

FB8-186: Extend mysqld_safe to LD_PRELOAD shared libraries (#979) (#979) Summary: Jira issue: https://jira.percona.com/browse/FB8-186 Reference Patch: https://github.com/facebook/mysql-5.6/commit/16b02494411 This will allow us to override some glibc functions like localtime_r. Originally Reviewed By: hermanlee Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/979 Reviewed By: lloyd Differential Revision: D14335103 (https://github.com/facebook/mysql-5.6/commit/3e72f695686c72158ccd3a364bcd3eb8e5c3b19a) Pulled By: lth fbshipit-source-id: 7ca49a3cb58

view details

Yura Sorokin

commit sha 2e86f6050206661c7d4dd793b8d43046a3564459

FB8-53: Expose user certificate details to command line (#959) (#959) Summary: Jira ticket: https://jira.percona.com/browse/FB8-53 Reference Patch: https://github.com/facebook/mysql-5.6/commit/f29eb03 Reference Patch: https://github.com/facebook/mysql-5.6/commit/c631017 'THD' class extended with the following convenience methods: - 'has_net_vio()' - 'get_net_vio()' - 'has_net_vio_ssl_arg()' - 'get_net_vio_ssl_arg()' - 'get_peer_certificate_info()' Implemented new 'Fill_authinfo_list' class derived from 'Do_THD_Impl' which populates 'INFORMATION_SCHEMA.AUTHINFO' table. This is a change to help the implementation of SSL based authentication efforts. Currently the certificate details are unavailable to the sys admin. The v3 extensions usually carry important information for identifying the user. This change adds a new table to the information schema to expose auth details. It is organized by process ID like processlist table. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/959 Reviewed By: lloyd Differential Revision: D14137362 (https://github.com/facebook/mysql-5.6/commit/3889d13317a2d4953d3a5d088955652b8ae88f35) Pulled By: lth fbshipit-source-id: 7fbd7ced083

view details

push time in 9 days

PullRequestReviewEvent

push eventinikep/mysql-5.6

Przemyslaw Skibinski

commit sha 02ca0a2f17c35bc31845c80083523893c5a6fb07

Fix clang-5+ and gcc-5+ compilation issues Reviewed By: lloyd Differential Revision: D19301431 (https://github.com/facebook/mysql-5.6/commit/42a3a731679c7f7f98d6e8f64526fef9c652c473) Pulled By: lth fbshipit-source-id: d52b885d630

view details

Yura Sorokin

commit sha d8698306dbc5dd31317267b9117e340eff2a7c64

Refactored 'main.all_persisted_variables' MTR test case (#1103) Reviewed By: lloyd Pulled By: lth fbshipit-source-id: b4405a9ed5d

view details

Yi Zhang

commit sha b9a695a75e43320a0e2ce844492b66de9f27c9f0

Add innodb experimental vars into sys_vars Summary: When enabling InnoDB 8.0 experimental sys vars in tools repo, we need to update the corresponding tests here as well. I'm not going to add test for them (since they are experimental) so I'm just rebaselining those tests for now. 4 of them are read-only so we need to bump the counter in all_presisted_variables as well. Reviewed By: luqun Differential Revision: D21759066 fbshipit-source-id: a9449e8d319

view details

Przemyslaw Skibinski

commit sha 7df2ca44caa26e23041500883a7379587227b130

Fix build and test errors for 8.0 Differential Revision: D7827421 (https://github.com/facebook/mysql-5.6/commit/43b6d7e25674f2219295892629ce9cc7e7e1bc05) Reference Patch: https://github.com/facebook/mysql-5.6/commit/c9bf1570398 Reference Patch: https://github.com/facebook/mysql-5.6/commit/5fc1ecad8fb Reference Patch: https://github.com/facebook/mysql-5.6/commit/bbd078c669b Reference Patch: https://github.com/facebook/mysql-5.6/commit/379db8c75de fbshipit-source-id: 5601f2fc864

view details

Herman Lee

commit sha 9aa251a769fb883381b699e5e0bb788536b7b658

Allow server to bootstrap with unknown variables Summary: When mysqld initializes for the first time, the --initialize parameter is passed in, which puts mysqld into a bootstrapping mode. In this mode, mysqld initializes the data directories and then exists. For 8.0.20, plugins are no longer loaded. However, the config file used for the normal mysqld process is also used for the bootstrapping process. Any plugin variables specified in the config file may result in the bootstrap process failing because those variables now considered unknown. To fix this, extend ignoring unknown options to include variables during the bootstrap process. Reviewed By: george-reynya Differential Revision: D26284550 fbshipit-source-id: dbc6773e06d

view details

Aliaksei Sandryhaila

commit sha a149a6d0d2dadf2697bc00f0af677fe25aeb7993

Port Control Cross-Table Access to 8.0 Summary: Port our v5.6 sysvar allow_noncurrent_db_rw, which detects when a query accesses data not in the current database. Depending on its setting, when a query tries to read or write to another database, it will: ON: Allow it. LOG: Allow it, but log that it occurred. LOG_WARN: Allow it, log it occurred, and send an error to client. OFF: Block it, and send an error to the client. Reference patch: https://github.com/facebook/mysql-5.6/commit/b5f54b4 Differential Revision: D6071054 (https://github.com/facebook/mysql-5.6/commit/d04eb6a30f4148e5aebbf850988554fa2a727185) fbshipit-source-id: 690ab034efe

view details

Steaphan Greene

commit sha 36244b887e755a6a249d298410469a50ad2b9318

Port BinLog BaseDirs Summary: Add global variables to list directory for binlog files and binlog index file. The variables are binlog_file_basedir and binlog_index_basedir. Reference patch: https://github.com/facebook/mysql-5.6/commit/3ccbf59 Differential Revision: D6155155 (https://github.com/facebook/mysql-5.6/commit/31a2eb2f651c0971cb626c080997060343bdb9d9) fbshipit-source-id: da30ee3447f

view details

Aliaksei Sandryhaila

commit sha 556702a147ac5b2cfa619a5a81595ac9fa90f586

Port fast timer support from 5.6 to 8.0 Summary: This replaces the fast_timers we added in 5.1, using the timers which are now included in upstream v5.6 for perfschema. Initializes the timers, selects which one to use, points the global "my_timer_now" function pointer to the correct timer, puts the stats for that timer in the global struct "my_timer", and sets the new SHOW STATUS variable "Timer_in_use" to indicate which was chosen. Added my_timer_ok mtr test to confirm the timer chosen is not "None". Added my_timer_good mtr test to confirm the timer chosen is millisecond precision or better. All timers are type ulonglong, and are stored in native timer units. Provides my_timer_now() function pointer to get current time. Provides my_timer_since(t) function to get time passed since time t, automatically adjusted for the estimated timer overhead. Provides my_timer_since_and_update(&t) function to get time passed since time t, adjusted for the estimated timer overhead. Also updates t to the current time, skipping that overhead. To count the time not counted here, add my_timer.overhead for each call to this function. Provides my_timer_to_seconds(t) function to convert native timer units to seconds (including fractions, in a double) for the current timer. Also adds _milliseconds and _microseconds versions of this. Provides microseconds_to_my_timer(t) function to convert microseconds to native timer units. Adds a new show variable type "SHOW_TIMER" which stores its value as a 64-bit unsigned integer, in native timer units, but displays as a double, in seconds, automatically converting it based on the current timer's frequency. Since this conversion is only done for display, no precision is ever lost in the actual variable, but the variable is still shown as seconds with a fractional component, just like doubles. This is the base for all our various stats which require timers. Reference patch: https://github.com/facebook/mysql-5.6/commit/614f9b0 Differential Revision: D6167189 (https://github.com/facebook/mysql-5.6/commit/4dad5e610f8aab7a20aab68465d733b648efd901) fbshipit-source-id: a183d5551b2

view details

Artem Danilov

commit sha 8a47d05ed25304cd72f45edc77d6bb0326a0a75b

Port Extra Stats: callers of os_file_flush Summary: Report on the callers of os_file_flush in SHOW INNODB STATUS. This is displayed in the TABLESPACE section as 'fsync callers:' Tihe diff is amended from 5.6 branch to remove DIRTY_BUFFER category as it was not used and ARCHIVE category as I couldn't find a place to put it in Reference patch: https://github.com/facebook/mysql-5.6/commit/3e6bf37 Differential Revision: D6176814 (https://github.com/facebook/mysql-5.6/commit/e086c773efd1d284599ec83fa6f64a78211ad0aa) fbshipit-source-id: 9c60880903f

view details

Artem Danilov

commit sha 4a9cc150bf81edab378c159d1095c50528ea902f

Port Extra Stats: Insert Buffer and Adaptive Hash Summary: Add insert buffer and adaptive hash stats to SHOW STATUS Add these to SHOW STATUS: Innodb_adaptive_hash_hits Adaptive hash search hits Innodb_adaptive_hash_misses Adaptive hash search misses Innodb_ibuf_merges Number of secondary index pages updated with merged records Innodb_ibuf_inserts Number of records added to the insert buffer Innodb_ibuf_delete_marks Number of records added to the delete mark buffer Innodb_ibuf_deletes Number of records added to the delete buffer Innodb_ibuf_size Size of the insert buffer in pages Reference patch: https://github.com/facebook/mysql-5.6/commit/1678522 Differential Revision: D6187284 (https://github.com/facebook/mysql-5.6/commit/e518b865c7c98b01e31daefbdb33bfbd0e8b6016) fbshipit-source-id: 6ab66d292b7

view details

Steaphan Greene

commit sha 1b79cc4e99131c0c729c53d7f8dc02c243653f74

Port v5.1 Extra Stats: InnoDB Lock Status Summary: Add InnoDB lock status details to SHOW [INNODB] STATUS Add these to SHOW STATUS: Innodb_row_lock_deadlocks Number of transaction aborts from deadlock Innodb_row_lock_wait_timeouts Number of transaction aborts from lock wait Innodb_purge_pending Number of transactions for which purge must be done. This is already named 'History list length' in SHOW INNODB STATUS Added to TRANSACTIONS section in SHOW INNODB STATUS: Lock stats: 0 deadlocks, 0 lock wait timeouts Added the innodb_lock_status mtr test. Reference patch: https://github.com/facebook/mysql-5.6/commit/fd4d7ae Differential Revision: D6211850 (https://github.com/facebook/mysql-5.6/commit/a76918055c3aab9bc21b34fce5a027c00112d80a) fbshipit-source-id: f5a2e500d5a

view details

Santosh Praneeth Banda

commit sha 4c616d766814b00c3aae591f74831f231c25f580

Check how max_binlog_cache_size affects LOAD DATA INFILE Summary: Need to check whether load data infile fails if it tries to create a binlog cache fails with size greater than max_binlog_cache_size. Its failing with ER_TRANS_CACHE_FULL error. Added a test case so that any future modifications in the code notifies the error. Reference patch: https://github.com/facebook/mysql-5.6/commit/cd75236 Differential Revision: D6214445 (https://github.com/facebook/mysql-5.6/commit/325cbd58169e41fec060b6d926c20ec6854995e9) fbshipit-source-id: 8860520edd9

view details

Rongrong Zhong

commit sha 229bb62ae1ae3936b774d29a5f3e74cbc3fa8d4d

Port Add --timeout option for mysqldump to 5.6 Summary: Allow per-session control of server user variables for mysqldump, thus avoiding session timeouts on write blockign (mostly in NFS environments) This adds --timeout option, which will set: * net_write_timeout * wait_timeout for mysqldump's connection to MySQL server Reference patch: https://github.com/facebook/mysql-5.6/commit/0b35262 Differential Revision: D6253820 (https://github.com/facebook/mysql-5.6/commit/be2dbf9427ad8a988c6f9d4756a0321fbd106d31) fbshipit-source-id: dc4a267dc18

view details

Rongrong Zhong

commit sha 2c9c56428b120e42d9778de9bc6eed5f9e22908e

Port Stats: binlog slave offset to 5.6 Summary: This uses a variation of code from the Google patch to display the binlog name and offset currently downloaded by a replication slave SHOW PROCESSLIST; Id User Host db Command Time State Info 3 root localhost:47190 test Query 1 Writing to net INSERT INTO t1 SELECT * FROM t1 5 root localhost:47194 NULL Binlog Dump 1 NULL slave offset: master-bin.000001 4 (Original Reviewed By: steaphan) Reference patch: https://github.com/facebook/mysql-5.6/commit/3b05df3 Differential Revision: D6272842 (https://github.com/facebook/mysql-5.6/commit/6fd6a9351b200ed28460b8739930957cad21b53b) fbshipit-source-id: b8e18086d03

view details

Manuel Ung

commit sha 863b27d29014ad26d1b0167fa94aa2e3b5f6e3e3

Add client error codes to distinguish between network timeout and loss Summary: Previously, mysql clients returned CR_SERVER_LOST on any network exception, including timeout. This adds two new client error codes `CR_NET_READ_INTERRUPTED` and `CR_NET_WRITE_INTERRUPTED` which is returned on network timeout. `CR_SERVER_LOST` is still returned for everything else. Reference patches: https://github.com/facebook/mysql-5.6/commit/e3052e1 https://github.com/facebook/mysql-5.6/commit/e5b99cd Differential Revision: D6325225 (https://github.com/facebook/mysql-5.6/commit/323786b849944b9e7a2d9a59d47eb0ae0450106e) fbshipit-source-id: b7f356ffc9c

view details

Aliaksei Sandryhaila

commit sha c9c3e2ac7778c66dae5b4ac03c3ad567e706d337

Do not use timestamp adjustment Summary: mi->clock_diff_with_master stores the time difference of master and slave clocks. seconds_behind_master is adjusted by this difference every time it is queried. However the current method of detecting clock_diff_with_master is broken because it does not compensate for network delays etc. And since we know that all of our clocks are already synced, we can always set clock_diff_with_master to zero. Reference patch: https://github.com/facebook/mysql-5.6/commit/6340f0a Differential Revision: D6336275 (https://github.com/facebook/mysql-5.6/commit/7251630c5b19995571738b18234f69eaef502ebe) fbshipit-source-id: d4aab4701ea

view details

Aliaksei Sandryhaila

commit sha b7b0a5fc85798d449ae926b8dbf7d61825f0ffae

Add mysqldump option to print ordering key Summary: The --print-ordering-key option to mysqldump causes mysqldump to get the key which is used to order the rows for each table, and print it to the dump file. This will help to efficiently run a diff on two mysqldumps of the same database at different points of time. Reference patch: https://github.com/facebook/mysql-5.6/commit/0f2578a Differential Revision: D6364804 (https://github.com/facebook/mysql-5.6/commit/42333e7696d2f6cb4710fab49dfa6416b5b288fd) fbshipit-source-id: f69a97d906e

view details

Aliaksei Sandryhaila

commit sha 415c4265550d662744893849dd5e1b2ab9cf5c14

Port v5.1: Output row counts in mysqldump Summary: Output the number of rows dumped by mysqldump in comment after each table. Reference patch: https://github.com/facebook/mysql-5.6/commit/9939293 Differential Revision: D6366330 (https://github.com/facebook/mysql-5.6/commit/13f919e34a8e0d2a3839d261e216cd9ab1995b9a) fbshipit-source-id: 7f0b1214ed6

view details

Aliaksei Sandryhaila

commit sha 6d2768341622ee9f67c5644315878dac348a62e9

New conf option for SO_SNDBUF in dump thread. Summary: A new configuration option rpl_send_buffer_size is added to control the TCP send buffer size of the socket used in dump thread. Reference patch: https://github.com/facebook/mysql-5.6/commit/ac90867 Differential Revision: D6377432 (https://github.com/facebook/mysql-5.6/commit/a107c9cf9d05794f80585c164ea3190ff5afcaa8) fbshipit-source-id: d9b0099277c

view details

Steaphan Greene

commit sha a375a20b9e0e0851eacd6ea2c18f2bce62ef7a4a

Port v5.1 Extra Stats: Remove Transactions Summary: Disable including transactions in SHOW INNODB STATUS. Transactions are already visible in information_schema.innodb_trx. This change removes them from the SHOW INNODB STATUS output in order to keep the size managable. It also removes the size limit on the output to avoid truncation. (Originally Reviewed By: chipi) Reference patch: https://github.com/facebook/mysql-5.6/commit/151bcc5 Differential Revision: D6437907 (https://github.com/facebook/mysql-5.6/commit/0833a230cf05a54a1ec4c9d27510ee6a73c8eca8) fbshipit-source-id: 967a9157694

view details

push time in 10 days

push eventinikep/mysql-5.6

Satya Bodapati

commit sha cfca90c73016aefd059579f0357fe2b1a752a115

FB8-47 : Increase max user name length to 80 (#937) (#937) Summary: https://jira.percona.com/browse/FB8-47 The max user name length is increased to 80 chars (utf8mb3) Since the system tables are InnoDB tables and because Primary key is on the full username (80 *3 = 240 bytes) and other columns, InnoDb initialization with 4k page size fails. This is new limitation. Reference Patch: https://github.com/facebook/mysql-5.6/commit/3dbbcfc This mitigates upstream bug: http://bugs.mysql.com/3083 User name length limit of 16 was too short, so made it 80. 80 is effectively the limit without making additional changes because of MyISAM's 1000 byte limit on key length and the primary key of mysql.columns_priv. To compute the size of the key for that table, the lengths of the char fields are summed and then multipled by 3, resulting in a limit on the max size of user name lengths being 81 characters. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/937 Reviewed By: lloyd Differential Revision: D13890369 (https://github.com/facebook/mysql-5.6/commit/1075e5717f04d69102592bdd5c3a352c25f5e49c) Pulled By: lth fbshipit-source-id: 2ec67345fb8

view details

Tian Xia

commit sha 21542d7bf5fd39e903318f238b885b284d5decd8

Per Database Read-Only Summary: Implements per-database scope read-only setting. The read-only can be turned on in `alter database` statement. Privilege to set read-only follows alter-database ACL. ALTER DATABASE db_name [SUPER_READ_ONLY | READ_ONLY = FALSE | TRUE]] `SUPER_READ_ONLY = TRUE` will prevent all write transactions from committing for any user (including super users). `READ_ONLY = TRUE` will prevent any write transaction from committing for regular users, while super users can still write to the database. `SUPER_READ_ONLY = FALSE` will turn off super_read_only on the database, and the database remains on read_only (for regular users), `READ_ONLY = FALSE` will turn off read_only on the database, The READ_ONLY database status can be shown by `show create database`. mysql> alter database test read_only = true; Query OK, 1 row affected (0.00 sec) mysql> show create database test; +----------+---------------------------------------------------------------------------+ | Database | Create Database | +----------+---------------------------------------------------------------------------+ | test | CREATE DATABASE `test` /*!40100 DEFAULT CHARACTER SET latin1 READ_ONLY */ | +----------+---------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> alter database test super_read_only = true; Query OK, 1 row affected (0.00 sec) mysql> show create database test; +----------+---------------------------------------------------------------------------------+ | Database | Create Database | +----------+---------------------------------------------------------------------------------+ | test | CREATE DATABASE `test` /*!40100 DEFAULT CHARACTER SET latin1 SUPER_READ_ONLY */ | +----------+---------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> alter database test super_read_only = false; Query OK, 1 row affected (0.00 sec) mysql> show create database test; +----------+---------------------------------------------------------------------------+ | Database | Create Database | +----------+---------------------------------------------------------------------------+ | test | CREATE DATABASE `test` /*!40100 DEFAULT CHARACTER SET latin1 READ_ONLY */ | +----------+---------------------------------------------------------------------------+ 1 row in set (0.00 sec) Details: * READ_ONLY flag is persisted in db.ops and survives restarts. * Alter database will only return when no write transaction is still committing. Uncommitted write transactions will fail after alter database succeeds. * DB options are stored in each thread (using hash map), so checking the read_only options doesn't grab a global lock. Only db_read_only that is turned on is stored in the local hash map. * The first time thead local hash map is initialized (once), shared db option map will be locked and accessed. Since the db options are not automatically loaded into the shared hash map (cache) at the beginning, we have to explicitly load the db options during the first time any thread initializes its local hash map. * create/alter/delete database DDLs will iterate through thread array to update local db opt (if needed), similar to how show processlist works. * write transactions are checked against local hashmap at commit time (either explicit commit or auto-commit). This will make the explicit commit and auto-commits consistent, and avoid problems such that explicit commit may become a "read-only" transaction if we block write-statements at the parsing time. * The list of databases accessed in a write transaction is obtained through the metadata locks (MDL) that the transaction holds. As I noted in the code, this may rollback some false positives, such as the read-only DB in the list may not actually be modified in the transaction (for cross-db transactions). We do not differentiate such cases. Reference patch: https://github.com/facebook/mysql-5.6/commit/8194409 https://github.com/facebook/mysql-5.6/commit/c971129 https://github.com/facebook/mysql-5.6/commit/e26e9b2 https://github.com/facebook/mysql-5.6/commit/a9f6f3d https://github.com/facebook/mysql-5.6/commit/b1ff9a6 https://github.com/facebook/mysql-5.6/commit/548c7ca Differential Revision: D9379904 (https://github.com/facebook/mysql-5.6/commit/329fb74dd261320d81c726cc0ae14104ba96f0d2) fbshipit-source-id: e1bc15f9155

view details

Zsolt Parragi

commit sha 74649506441d96bed5805e11866c9aee92507867

FB8-120: Added db_metadata options to create database command (#972) (#972) Summary: JIRA ticket: https://jira.percona.com/browse/FB8-120 Reference commit: https://github.com/facebook/mysql-5.6/commit/2f08e3a Reference commit: https://github.com/facebook/mysql-5.6/commit/a21e84f Reference commit: https://github.com/facebook/mysql-5.6/commit/5983ba3181a Reference commit: https://github.com/facebook/mysql-5.6/commit/414870abeee Added new option db_metadata to the create database `CREATE DATABASE [IF NOT EXISTS] db_name DB_METADATA [=] metadata` The new option is stored in the db.opt file along with other database options like charset and read_only. The options accepts any string value. If not specified, it writes empty string as the default value in the db.opt file. To retrieve the option value for a particular database, the show command can be used as follows: `SHOW CREATE DATABASE db_name` Example Usage: create database test2 charset utf8 read_only = true db_metadata '{\'shard_name\':\'myshard_for_test2\'}'; show create database test2; Output: +----------+--------------------------------------------------------------------------------------------------------------------------+ | Database | Create Database | +----------+--------------------------------------------------------------------------------------------------------------------------+ | test2 | CREATE DATABASE `test2` /*!40100 DEFAULT CHARACTER SET utf8 READ_ONLY DB_METADATA {'shard_name':'myshard_for_test2'} */ | +----------+--------------------------------------------------------------------------------------------------------------------------+ Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/972 Differential Revision: D14883632 (https://github.com/facebook/mysql-5.6/commit/1cae2d47620e8ecad3ae6f6496c88070efe2fc03) Pulled By: yizhang82 fbshipit-source-id: ced7e589419

view details

Yi Zhang

commit sha 33934a610e69b8e7e742c2ff9c746c652b8f57cf

Add DB_METADATA to INFORMATION_SCHEMA.SCHEMATA Summary: Add DB_METADATA to INFORMATION_SCHEMA.SCHEMATA for each database for easy query by OLM --- 8.0 porting comments 'SCHEMATA_EXT' Information Schema table is implemented as Data Dictionary system view. Reference Patch: https://github.com/facebook/mysql-5.6/commit/64bf72c276 Differential Revision: D13510326 (https://github.com/facebook/mysql-5.6/commit/09ba1e41992dcc439455da0d1b41794324620cc1) fbshipit-source-id: 31b974dee32

view details

Przemyslaw Skibinski

commit sha e965af308940f086be67b9c7cfef8940c1a227a2

FB8-116, FB8-152, FB8-83, FB8-125: Implement trx metadata in binlog (#965) (#965) Summary: Jira ticket: https://jira.percona.com/browse/FB8-116 Jira ticket: https://jira.percona.com/browse/FB8-152 Jira ticket: https://jira.percona.com/browse/FB8-83 Jira ticket: https://jira.percona.com/browse/FB8-125 Reference commit: https://github.com/facebook/mysql-5.6/commit/316e0e5 Reference commit: https://github.com/facebook/mysql-5.6/commit/a85b8d3 Reference commit: https://github.com/facebook/mysql-5.6/commit/21edb26 Reference commit: https://github.com/facebook/mysql-5.6/commit/96de6c4 Reference commit: https://github.com/facebook/mysql-5.6/commit/e1cb53ba0f9 Reference commit: https://github.com/facebook/mysql-5.6/commit/bd25e29 Reference commit: https://github.com/facebook/mysql-5.6/commit/e392a18d40e Reference commit: https://github.com/facebook/mysql-5.6/commit/fa4f588a248 Reference commit: https://github.com/facebook/mysql-5.6/commit/68f8c9a58fa Reference commit: https://github.com/facebook/mysql-5.6/commit/4519ab421f8 Reference commit: https://github.com/facebook/mysql-5.6/commit/42bdfc8 Reference commit: https://github.com/facebook/mysql-5.6/commit/3950491 Reference commit: https://github.com/facebook/mysql-5.6/commit/bc17d30 Reference commit: https://github.com/facebook/mysql-5.6/commit/eb7b1df Reference commit: https://github.com/facebook/mysql-5.6/commit/6a27bd42dfe Reference commit: https://github.com/facebook/mysql-5.6/commit/9503743bf59 Reference commit: https://github.com/facebook/mysql-5.6/commit/7b270a0ad3b Reference commit: https://github.com/facebook/mysql-5.6/commit/e6f031af712 Reference commit: https://github.com/facebook/mysql-5.6/commit/7a0da2c9e94 Reference commit: https://github.com/facebook/mysql-5.6/commit/bb9f4aae4fd Porting notes: The deadlock/ASAN bugfixes were skipped - I wasn't able to reproduce the issue mentioned in the commit messages even without the patches, and the pathes themselves caused early guaranteed crashes in the server. -------- 316e0e5 -------- If enabled, For every trx before the commit event is written we write a rows query event which contains meta data in JSON format wrapped inside a comment. Storing timestamps in the meta data to calculate slave lag in millisecond granularity. -------- a85b8d3 -------- Supporting binlog trx meta data for SBR. The meta data event is written right before every query event in a trx representing a statement. -------- 21edb26 -------- Milli-seconds behind master was not calculated correctly for MTS. With this change we calculate it just like seconds behind master. One difference between these is that sec timestamp is available for every event but msec timestamp is only available once per query/rows event in form of trx metadata inside a Rows_query_event. -------- bd25e29 -------- A separate event for trx metadata was causing an unreasonable increase in binlog size. Also, now we're not writing rows query event if there are no comments in the query and `log_only_query_comments` is enabled. -------- eb7b1df -------- Log meta data event before logging the rows query event for a statement. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/965 -------- b8a600a -------- When reset_seconds_behind_master is disabled, in the case where last_master_timestamp is still 0, check to determine if the I/O thread is running before returning 0 or NULL. In the case where replication is broken, the sbm value should be ever increasing, or set to NULL, and not set to 0. Differential Revision: D14652380 (https://github.com/facebook/mysql-5.6/commit/3847e40e264198139286ed5be59879df5d1d7539) Pulled By: yizhang82 fbshipit-source-id: 4875c64d8f5

view details

Abhinav Sharma

commit sha 8160da5a8f38dc97b2694ad36e57dc036eedf0db

Handle rows_query events correctly when filters are specified Summary: When database or table filter was specified along with verbose mode, mysqlbinlog would print rows_query events of filtered transactions. Fixed that behavior in this change by buffering the rows_query event until the filtering decision is taken in the following rows or query event. Note that the "# at <pos>" comment in the output of mysqlbinlog was always broken for buffered events (like int_var and user_var events). This diff does not fix that behavior for rows_query. The result is that the "at" comment for buffered events are printed consecutively before the actual events. For 8.0: the binlog_gtid.binlog_rows_query_with_filter requires the --binlog-trx-meta-data patch to be ported, so it will currently fail. Reference Patch: https://github.com/facebook/mysql-5.6/commit/b45040ad5a Differential Revision: D10219146 (https://github.com/facebook/mysql-5.6/commit/3431ef0a46b2844b8480a5da59ee35d2c93877c1) fbshipit-source-id: 64e7130f58d

view details

Satya Bodapati

commit sha ae76e44a89df15d142b057f7295cc50746c9defa

FB8-102 : Add posix_fadvise(FADV_DONTNEED) to flush buffered pages (#955) (#955) Summary: JIRA : https://jira.percona.com/browse/FB8-102 Referenced patch : https://github.com/facebook/mysql-5.6/commit/ab9d60b When InnoDB opens a .ibd or log file in O_DIRECT mode, request the os to flush the buffer cache. Otherwise, direct IO is serialized when buffered pages for the file existing within the filesystem cache. Originally Reviewed By: alxyang Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/955 Reviewed By: lloyd Differential Revision: D14034476 (https://github.com/facebook/mysql-5.6/commit/5c17d94db61181dfb5f6a1dd285edf99fe8fceba) Pulled By: lth fbshipit-source-id: e5725e8c39d

view details

Zsolt Parragi

commit sha 91ac6997180cbc81672b4ca6bfc6e736ee5b9b4a

FB8-49: Don't remove padding chars for binary column in RBR (#947) (#947) Summary: Jira issue: https://jira.percona.com/browse/FB8-49 Reference patch: https://github.com/facebook/mysql-5.6/commit/915b131 MySQL client doesn't remove padding chars for binary columns. Do the same for RBR. Tools parsing the replication stream vs. the client output can then use memcmp to compare keys. Otherwise, the tools need to handle padding characters correctly. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/947 Differential Revision: D13972586 (https://github.com/facebook/mysql-5.6/commit/2b73efa353366d28411d56b2b474cedcb6698247) Pulled By: hermanlee fbshipit-source-id: a1bf6e34e76

view details

push time in 10 days

PullRequestReviewEvent

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+connection master;+create table t1 (id int primary key, value int) engine = rocksdb;++insert into t1 values (1,1),(2,2),(3,3);++# auto commit failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+--error 2013+insert into t1 values (4,4);++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;+select * from t1 order by id;+++connection master;

--source include/rpl_connection_master.inc

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+--disable_warnings+--source include/have_rocksdb.inc+--source include/have_binlog_format_row.inc+--source include/master-slave.inc

master-slave should be the last include

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+--rocksdb_max_subcompactions=1+--rocksdb_default_cf_options=target_file_size_base=100k;max_bytes_for_level_multiplier=1;max_bytes_for_level_base=1m;target_file_size_multiplier=1

--rocksdb_ should be always --loose_rocksdb_ in case RocksDB is not compiled

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+--source include/have_rocksdb.inc++DELIMITER //;+CREATE PROCEDURE compact_start()+BEGIN+ select variable_value into @c from performance_schema.global_status where variable_name='rocksdb_compact_write_bytes';+END//+CREATE PROCEDURE compact_end()+BEGIN+  select case when variable_value-@c > 0 then 'true' else 'false' end as checked from performance_schema.global_status where variable_name='rocksdb_compact_write_bytes';+END//+DELIMITER ;//++--disable_warnings+DROP TABLE IF EXISTS t1;+--enable_warnings

Should be removed as it it no longer required.

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+connection master;

--source include/rpl_connection_master.inc

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+connection master;+create table t1 (id int primary key, value int) engine = rocksdb;++insert into t1 values (1,1),(2,2),(3,3);++# auto commit failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+--error 2013+insert into t1 values (4,4);++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;+select * from t1 order by id;+++connection master;+# tx failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+begin;+insert into t1 values (5,5);+insert into t1 values (6,6);+--error 2013+commit;++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;+select * from t1 order by id;+++connection master;

--source include/rpl_connection_master.inc

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+connection master;+create table t1 (id int primary key, value int) engine = rocksdb;++insert into t1 values (1,1),(2,2),(3,3);++# auto commit failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+--error 2013+insert into t1 values (4,4);++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;

--source include/rpl_connection_slave.inc

ldonoso

comment created time in 11 days

Pull request review commentpercona/percona-server

PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104

+connection master;+create table t1 (id int primary key, value int) engine = rocksdb;++insert into t1 values (1,1),(2,2),(3,3);++# auto commit failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+--error 2013+insert into t1 values (4,4);++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;+select * from t1 order by id;+++connection master;+# tx failure at engine commit, crash and restart (roll forward with binlog)+--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect+eval set session debug= $myrocks_debug_set_code;+begin;+insert into t1 values (5,5);+insert into t1 values (6,6);+--error 2013+commit;++--enable_reconnect+--source include/wait_until_connected_again.inc++# Primary instance restart needs this, otherwise rpl_end fails.+--let $rpl_server_number = 1+--source include/rpl_reconnect.inc++select * from t1 order by id;+source include/sync_slave_sql_with_master.inc;++connection slave;

--source include/rpl_connection_slave.inc

ldonoso

comment created time in 11 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventinikep/mysql-5.6

Abhinav Sharma

commit sha 074f5f34f61eb8853a24338993e9c2f06905cbfd

Remove unflushed events warning when trx metadata is enabled on SBR Summary: mysqlbinlog expects rows_query event only in RBR. It prints an unflushed event warning when it parses a transaction which has rows_query but not a rows events with STMT_END_F flag set. Since there is no direct way to tell the binlog format in mysqlbinlog, we'll just forgo the warning for rows_event which contain metadata. Reference Patch: https://github.com/facebook/mysql-5.6/commit/89e1c03ac Reviewed By: lloyd Differential Revision: D10054923 (https://github.com/facebook/mysql-5.6/commit/a658c4d4a73533842adf0832b6c9a2ae0f5039d6) fbshipit-source-id: 2446a74fa4f

view details

Manuel Ung

commit sha 2d3c9ccedbf77d3e25378c86ff4f0053868f3303

Fix mysqldump querying column stats in 5.6 Summary: mysqldump in 8.0 queries from information_schema.column_statistics without a server version check. This causes backups from 5.6 servers to fail. Fix by adding a version check. This fixes upstream bug https://bugs.mysql.com/bug.php?id=89825 Differential Revision: D10207120 (https://github.com/facebook/mysql-5.6/commit/783ce94828b18220044e52d249b095669f7ce539) fbshipit-source-id: bf81eda5ea6

view details

Zsolt Parragi

commit sha 7e3af1a9b6c1d5b860b0db4a4412b1d1bedbbe23

FB8-103: Heartbeat event should carry the latest master timestamp (#951) (#951) Summary: Jira issue: https://jira.percona.com/browse/FB8-103 Reference patch: https://github.com/facebook/mysql-5.6/commit/eb9a3db Reference patch: https://github.com/facebook/mysql-5.6/commit/5803e4f Reference patch: https://github.com/facebook/mysql-5.6/commit/746c217 Compared to the 5.6 patch: * fixed the rpl_heartbeat_timestamp testcase, which resulted in the test executor killing all servers because of a connection drop * split the rpl_heartbeat_zero_timestamp test into two Heartbeat events should carry the now() timestamp from the master and the last_master_timstamp from the slave. In other words, HB should always carry the lastet master timestamp. This will help services connected to slaves know if they are lagging even when no real events are flowing in the system. Also, for this to work last_master_timestamp should be updated whenever a HB event is received. Let master only send HB timestamp while waiting for new trx, this will fix the SBM being inaccurate when the slave is stopped and trying to catchup from the master. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/951 Reviewed By: lloyd Differential Revision: D14034449 (https://github.com/facebook/mysql-5.6/commit/e14e52798d589ea74d0c85d7b8e35c278ca1c464) Pulled By: lth fbshipit-source-id: 265303cd6f5

view details

Manuel Ung

commit sha fa9b66c87d2f4106fd55892beba71e0a91ceeaf9

Allow mysqladmin to call COM_SHUTDOWN for 5.6 servers Summary: Because we skipped 5.7, there was no longer any compatibility code that allows clients to call COM_SHUTDOWN for 5.6 servers. Add it back in until we have finished upgrading all servers to 8.0. Differential Revision: D12984296 (https://github.com/facebook/mysql-5.6/commit/548efd55b4e2032ba385c683a208e19b3f3b0264) fbshipit-source-id: e539ad1fc09

view details

Anirban Rahut

commit sha 76e8b2b60c2898367046a833d3b96a9091deb4f3

Fix base64 dump of BINLOG event by preventing event split Summary: mysqlbinlog verbose mode in case of multi-row BINLOG statement caches events for verbose output and prints it after BINLOG statement is finished. BINLOG statement must not be splitted into multiple statements because row events require Table_map_event Borrowed from MDEV-16372 Differential Revision: D13039422 (https://github.com/facebook/mysql-5.6/commit/7e7fbbebca93c91e983593370166eca3dc78e853) fbshipit-source-id: 78ffb2aea24

view details

przemyslaw.skibinski@percona.com

commit sha 30c10f481a00e38322388c61d0e1b2007f4290f1

FB8-69: Don't force flush relay log info (#935) (#935) Summary: With gtid enabled, flushing relay log info is not necessary. The force flushing is required to ensure recovery of MTS in non-gtid mode. The coordinator thread relies on positions in the relay log info repository during recovery. With GTID, this is not required and sql_thread doesn't even need to do any recovery. The slave (io_thread) uses auto-positioning directly. JIRA: https://jira.percona.com/browse/FB8-69 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ed39d49 Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/935 Reviewed By: lloyd Differential Revision: D13874140 (https://github.com/facebook/mysql-5.6/commit/a4d4b0bd84c4e7b5d86b1049f39380b8b3b5c3a8) Pulled By: lth fbshipit-source-id: 033c2f149bb

view details

Jay Edgar

commit sha 20b5a717306659ca77d2c90ad2cba57765bd1be7

FB8-100: Handle out-of-space on binlog cache file (#938) (#938) Summary: JIRA: https://jira.percona.com/browse/FB8-100 Reference Patch: https://github.com/facebook/mysql-5.6/commit/e29ff14 The binlog cache file code is designed to handle an error that would cause binlog corruption. It does this by checking the error for a set of values that it expects and when it finds one it marks the cache to indicate an incident has occurred and this mark keeps it from writing corrupt entries to the binlog. Unfortunately, during an out-of-space event, there is a lower level error that gets set which is not being overwritten when we set the expected error. This means that when the code checks for the expected error it is not found - the original lower level error is found. This diff modifies the checking code to scan through all associated conditions (which is a list of all encountered errors) to see if the expected error is present anywhere in the list. This diff can be reverted if/when https://bugs.mysql.com/bug.php?id=72457 is resolved upstream. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/938 Reviewed By: lloyd Differential Revision: D13924297 (https://github.com/facebook/mysql-5.6/commit/3603beb2acc2e2fb1d6a4426fb6a415be34a1095) Pulled By: lth fbshipit-source-id: 200d9123c1e

view details

Abhinav Sharma

commit sha e3b0956328f316df2a4fca6a96837646469158e5

FB8-77: Changed the behavior of purging logs. Now deleting logs first and then updating the index (#943) (#943) Summary: JIRA: https://jira.percona.com/browse/FB8-77 Reference Patch: https://github.com/facebook/mysql-5.6/commit/832be30 Reference Patch: https://github.com/facebook/mysql-5.6/commit/0d85250 Reference Patch: https://github.com/facebook/mysql-5.6/commit/e8be5be Changed the behavior of purging logs. Now deleting logs first and then updating the index. Previously when purging logs, the index was updated before deleting the logs. This creates problems the disk is full. We should first delete logs to free up disk and then update index. When index file is updated, it is first copied to a crash safe index file, updates are made to the crash safe file and finally the crash safe file is renamed. During purge, all this should happen only when at least one log is being removed. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/943 Reviewed By: lloyd Differential Revision: D13930811 (https://github.com/facebook/mysql-5.6/commit/97ae20bd3aa24aee6d3ad87ec7833931c451d887) Pulled By: lth fbshipit-source-id: 077653f287d

view details

Przemyslaw Skibinski

commit sha b300090b9953a7e4a98d20fc28bfed8d72c468f8

FB8-86: Delete crash safe index file if index file exists (#939) (#939) Summary: JIRA: https://jira.percona.com/browse/FB8-86 Reference Patch: https://github.com/facebook/mysql-5.6/commit/78fa270 Reference Patch: https://github.com/facebook/mysql-5.6/commit/72c25d615d8 Reference Patch: https://github.com/facebook/mysql-5.6/commit/78b6b8bc826 Reference Patch: https://github.com/facebook/mysql-5.6/commit/00be4805f47 Crash safe index file is created when any change is made to the index file. The index file is first copied over to the crash safe one and then stuff is added/removed from the crash safe index file. Finally, the old index file is deleted and crash safe index file is renamed to the index file. There might be a scenario where mysql crashes after stuff is changed in the crash safe file but before the old index file is deleted. In this case we should delete the crash safe index file at restart. If we don't delete it subsequent changes to the index file can corrupt it because it can be copied to the old crash safe file which already has some data in it. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/939 Reviewed By: lloyd Differential Revision: D13924478 (https://github.com/facebook/mysql-5.6/commit/98a250fbd89f49f678c3ce9ad8a98f8fbd7b8148) Pulled By: lth fbshipit-source-id: 157947a2d38

view details

Santosh Praneeth Banda

commit sha 8c0c042b130038b7a39fed74036e48b8cbaacae2

FB8-56: Create a new mode SEMI_STRICT for slave_exec_mode (#941) (#941) Summary: JIRA: https://jira.percona.com/browse/FB8-56 Reference Patch: https://github.com/facebook/mysql-5.6/commit/9d97282 Create a new mode SEMI_STRICT which ignores HA_ERR_RECORD_CHANGED and HA_ERR_KEY_NOT_FOUND errors. A new status counter will track the usage of this new mode. Logging a warning/error message may cause error log to get flooded with same error message, so I decided against it. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/941 Reviewed By: lloyd Differential Revision: D13924607 (https://github.com/facebook/mysql-5.6/commit/286494fd1ae2189e6c01060ada36bdcdcdccc686) Pulled By: lth fbshipit-source-id: 8ab53b11f8e

view details

Zsolt Parragi

commit sha e70e1088376394da0384a3f105a88bcf02c34581

FB8-96: Remove special formatting for user name when writing to audit plugin log (#948) (#948) Summary: Jira issue: https://jira.percona.com/browse/FB8-96 Reference patch: https://github.com/facebook/mysql-5.6/commit/3b47dae Reference patch: https://github.com/facebook/mysql-5.6/commit/87e0683 Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/948 Reviewed By: lloyd Differential Revision: D13941932 (https://github.com/facebook/mysql-5.6/commit/06b23751db85d0fa1a81702609309e4bc390cfcf) Pulled By: lth fbshipit-source-id: aa806c17f27

view details

Satya Bodapati

commit sha e5cdb5b3d64cd1d870e402f96e0eaa7cbbcb47d8

FB8-52: Slowing down transaction log init (#931) (#931) Summary: JIRA: https://jira.percona.com/browse/FB8-52 Reference Patch: https://github.com/facebook/mysql-5.6/commit/80ca845 Reference Patch: https://github.com/facebook/mysql-5.6/commit/0fb428b Patch introduces new variable 'innodb_txlog_init_rate' to limit the IO rate of InnoDB redo log file initialization. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/931 Reviewed By: lloyd Differential Revision: D13789095 (https://github.com/facebook/mysql-5.6/commit/48615c444dd7ccaf0bc387017ca44b3dda7764ae) Pulled By: lth fbshipit-source-id: d667be91978

view details

Zsolt Parragi

commit sha 99d1e08720508c3e5d6439ef277438da853534ee

FB8-54, FB8-55, FB8-70, FB8-101: Expose more information to audit plugin (#934) (#934) Summary: JIRA: https://jira.percona.com/browse/FB8-54 JIRA: https://jira.percona.com/browse/FB8-55 JIRA: https://jira.percona.com/browse/FB8-70 JIRA: https://jira.percona.com/browse/FB8-101 This commit adds the following fields to the generic event in audit log: * query_id * database * affected_rows * connection_certificate Reference Patch: https://github.com/facebook/mysql-5.6/commit/1def6b7 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ce95a09 Reference Patch: https://github.com/facebook/mysql-5.6/commit/588be34 Reference Patch: https://github.com/facebook/mysql-5.6/commit/ba03c70 Reference Patch: https://github.com/facebook/mysql-5.6/commit/be8c587 Reference Patch: https://github.com/facebook/mysql-5.6/commit/22b2508 We need some extra info for the shadowing and security logging. This is a simple first step of info that MariaDB actually also exposes. Now we would have the `query_id` and the database name for general events. Making as few changes as possible to accomplish it, so I'm just taking the information from the TDH and exposing it through `mysql_event_general` struct and as a argument to disconnect. Forward the connection certificate to the audit plugin. The connection certificate can then be parsed by the audit plugin and handled appropriately. It made more sense for the certificate to live in the connection events, since they generally don't change between every general event, so the move was done. This is done by caching a BUF_MEM struct on the THD object. Since it's not possible to change certificates on the same connection, this caching should be correct. The BUF_MEM is released on THD::release_resources. If upstream bumps the MYSQL_AUDIT_INTERFACE_VERSION, we should bump ours to be greater or equal to it. Expose the port current mysqld is running on for the audit plugin. If no port, 0 is used. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/934 Reviewed By: lloyd Differential Revision: D13874133 (https://github.com/facebook/mysql-5.6/commit/309222564ff6400059ecd8202791d214ae21835a) Pulled By: lth fbshipit-source-id: 55383042dd8

view details

Xuli Liu

commit sha 2a2518445b81bde52e24352f875b3eef42232e8f

FB8-93: output symbolic errno for 'SHOW SLAVE STATUS' (#940) (#940) Summary: JIRA: https://jira.percona.com/browse/FB8-93 Reference Patch: https://github.com/facebook/mysql-5.6/commit/a2b0c32 Reference Patch: https://github.com/facebook/mysql-5.6/commit/e82ab5a Instead of just showing the errno when running SHOW SLAVE STATUS command, also printing the error name so that user won't have misunderstanding Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/940 Reviewed By: lloyd Differential Revision: D13924591 (https://github.com/facebook/mysql-5.6/commit/e04518828b51cfbf2adbd1ab75ec20510ad774e6) Pulled By: lth fbshipit-source-id: 119d4674eca

view details

Satya Bodapati

commit sha 00e34237ea9eb482d4b52a605ec413ba9be1ab87

FB8-47 : Increase max user name length to 80 (#937) (#937) Summary: https://jira.percona.com/browse/FB8-47 The max user name length is increased to 80 chars (utf8mb3) Since the system tables are InnoDB tables and because Primary key is on the full username (80 *3 = 240 bytes) and other columns, InnoDb initialization with 4k page size fails. This is new limitation. Reference Patch: https://github.com/facebook/mysql-5.6/commit/3dbbcfc This mitigates upstream bug: http://bugs.mysql.com/3083 User name length limit of 16 was too short, so made it 80. 80 is effectively the limit without making additional changes because of MyISAM's 1000 byte limit on key length and the primary key of mysql.columns_priv. To compute the size of the key for that table, the lengths of the char fields are summed and then multipled by 3, resulting in a limit on the max size of user name lengths being 81 characters. Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/937 Reviewed By: lloyd Differential Revision: D13890369 (https://github.com/facebook/mysql-5.6/commit/1075e5717f04d69102592bdd5c3a352c25f5e49c) Pulled By: lth fbshipit-source-id: 2ec67345fb8

view details

Zsolt Parragi

commit sha b3003ebe953a89dbdf2d543d5d55f2cd9874a6fb

FB8-120: Added db_metadata options to create database command (#972) (#972) Summary: JIRA ticket: https://jira.percona.com/browse/FB8-120 Reference commit: https://github.com/facebook/mysql-5.6/commit/2f08e3a Reference commit: https://github.com/facebook/mysql-5.6/commit/a21e84f Reference commit: https://github.com/facebook/mysql-5.6/commit/5983ba3181a Reference commit: https://github.com/facebook/mysql-5.6/commit/414870abeee Added new option db_metadata to the create database `CREATE DATABASE [IF NOT EXISTS] db_name DB_METADATA [=] metadata` The new option is stored in the db.opt file along with other database options like charset and read_only. The options accepts any string value. If not specified, it writes empty string as the default value in the db.opt file. To retrieve the option value for a particular database, the show command can be used as follows: `SHOW CREATE DATABASE db_name` Example Usage: create database test2 charset utf8 read_only = true db_metadata '{\'shard_name\':\'myshard_for_test2\'}'; show create database test2; Output: +----------+--------------------------------------------------------------------------------------------------------------------------+ | Database | Create Database | +----------+--------------------------------------------------------------------------------------------------------------------------+ | test2 | CREATE DATABASE `test2` /*!40100 DEFAULT CHARACTER SET utf8 READ_ONLY DB_METADATA {'shard_name':'myshard_for_test2'} */ | +----------+--------------------------------------------------------------------------------------------------------------------------+ Pull Request resolved: https://github.com/facebook/mysql-5.6/pull/972 Differential Revision: D14883632 (https://github.com/facebook/mysql-5.6/commit/1cae2d47620e8ecad3ae6f6496c88070efe2fc03) Pulled By: yizhang82 fbshipit-source-id: ced7e589419

view details

Abhinav Sharma

commit sha 7927087e78f03b123f9716f44a1cdb98ef3feeb2

Handle rows_query events correctly when filters are specified Summary: When database or table filter was specified along with verbose mode, mysqlbinlog would print rows_query events of filtered transactions. Fixed that behavior in this change by buffering the rows_query event until the filtering decision is taken in the following rows or query event. Note that the "# at <pos>" comment in the output of mysqlbinlog was always broken for buffered events (like int_var and user_var events). This diff does not fix that behavior for rows_query. The result is that the "at" comment for buffered events are printed consecutively before the actual events. For 8.0: the binlog_gtid.binlog_rows_query_with_filter requires the --binlog-trx-meta-data patch to be ported, so it will currently fail. Reference Patch: https://github.com/facebook/mysql-5.6/commit/b45040ad5a Differential Revision: D10219146 (https://github.com/facebook/mysql-5.6/commit/3431ef0a46b2844b8480a5da59ee35d2c93877c1) fbshipit-source-id: 64e7130f58d

view details

push time in 11 days

delete branch inikep/percona-server

delete branch : PS-7767-8.0

delete time in 12 days

push eventpercona/percona-server

Rahul Kumar

commit sha f4ea4d4e651a2d805675fc7ff1cf40e76744bec0

Adds capability to set different BottommostLevelCompaction option for MyRocks Manual Compaction Upstream commit ID : fb-mysql-5.6.35/49e8df3e1ce20d06b41f14ad9d471be9e0235a55 PS-7767 : Merge percona-202103 Summary: Currently BottommostLevelCompaction is set to default kForceOptimized for manual compaction. This is fine usually. However, for privacy triggered compactions, it does not make much sense, because it compacts Lmax SST files that do not have any overlapping key updates at all. This adds a session variable to specify the option for bottommost level compaction before triggering manual compaction. Reviewed By: yoshinorim, yizhang82 Differential Revision: D27665464 fbshipit-source-id: 2bbdf04ec18

view details

Herman Lee

commit sha f1b87d3e600fa8575cf189967e2c99d021f756e3

Increase timeout for shutting down the server in rocksdb tests to 60s Upstream commit ID : fb-mysql-5.6.35/f1404cd548c82d2363c249eeb22717be3429aba9 PS-7767 : Merge percona-202103 Summary: Valgrind tests were failing for some rocksdb tests because shutdown was hitting the 10 second timeout. Increase this timeout to 60 to allow shutdown to complete within the timeout period. Reviewed By: Pushapgl Differential Revision: D27865611 fbshipit-source-id: b3bbcd79b44

view details

Herman Lee

commit sha d780da58ba03e42b97d770383007b1e09c824c84

Updating RocksDB submodule to 6.20.2 for prod202103 Upstream commit ID : fb-mysql-5.6.35/67c2a98018b80586f5910ce5b908812e3f421e49 PS-7767 : Merge percona-202103 Summary: Updating RocksDB submodule in 8.0 to the latest. Rebase multiple cf MTR due to memtable size difference. update-submodule: rocksdb Reviewed By: luqun Differential Revision: D28479203 fbshipit-source-id: 076fdaee925

view details

Yi Zhang

commit sha 65f17f0399161f63d8cbe3ca240bf283d320ea6a

Fix SELECT COUNT(*) FOR UPDATE Upstream commit ID : fb-mysql-5.6.35/b60ebcb6e34ef97c44ee41034ea7c16b3e3610f3 PS-7767 : Merge percona-202103 Summary: SELECT COUNT(*) in 8.0 goes through ha_rocksdb::records / records_from_index and it incorrectly skips locking entirely. So SELECT COUNT(*) FOR UPDATE / etc wouldn't work correctly. Given that such case is uncommon, the easiest fix is to skip setting m_iteration_only to go through "slow path". Thanks Manuel for finding this issue. Reviewed By: lth Differential Revision: D28464889 fbshipit-source-id: a7f6ac21200

view details

Manuel Ung

commit sha c64c28cec6e1ecf809f963ec6fb1fefd35740489

Refactor index_first_intern and index_last_intern into one function Upstream commit ID : fb-mysql-5.6.35/6de27e35d23470045e4bce33667801b64e65bc68 PS-7767 : Merge percona-202103 Summary: These two functions share a lot of code, with the only difference between whether to start at the beginning or the end, and the direction to move in. Reviewed By: luqun Differential Revision: D23300063 fbshipit-source-id: 73bed490869

view details

Manuel Ung

commit sha 220f304ff1e758abb072ed52a2b55caf80e12ab9

Consolidate rnd_init/rnd_next/rnd_end into index_* functions Upstream commit ID : fb-mysql-5.6.35/e55e072359398b67c2456ff8c8a53e0e820ab30f PS-7767 : Merge percona-202103 Summary: Instead of maintaining separate code paths for full table scans, merge them into full index scan code paths. Full table scans are just full index scans on the primary key anyway. Reviewed By: hermanlee Differential Revision: D23358422 fbshipit-source-id: 6996eeb304b

view details

Manuel Ung

commit sha 5bf0c1575451db73e85ecc7d631651a774f27cd7

Merge index scan with index reads Upstream commit ID : fb-mysql-5.6.35/8c07f1896649995a82ec391bfa890f5643d2a576 PS-7767 : Merge percona-202103 Summary: This merges index scans with index lookups. This is so that the codepaths to perform retry with snapshot refresh are consolidated into one location. Also, in `position_to_correct_key`, I save a redundant call to `pack_index_tuple` since we just packed it earlier. To do this, the packed tuple is saved in `m_sk_match_prefix_buf`. Reviewed By: hermanlee Differential Revision: D23358423 fbshipit-source-id: ac84bdac747

view details

Yoshinori Matsunobu

commit sha ee339b483906943cd85d2a1954bbfa8064abc275

Aborting process on all RDB_IO_ERROR_TX_COMMIT errors on commit() Upstream commit ID : fb-mysql-5.6.35/5862e8828ca2ba4da2947cc8b3acf83792bd5c8b PS-7767 : Merge percona-202103 Summary: When hitting some I/O errors in RocksDB (e.g. out of space), current RocksDB may return error codes other than kIOError at engine commit phase, while not returning any error at prepare phase, depending on how it hits errors. Our expectation is any engine commit failure of the two phase commit protocol should abort the process, since binlog commit is already done. This diff makes sure to call abort() to terminate the process on RDB_IO_ERROR_TX_COMMIT at commit phase, not just RDB_IO_ERROR_DICT_COMMIT. This prevents from a case that a transaction gets written to binlog (and replicated) but it is rolled back in MyRocks engine due to commit failure, which is an inconsistent state. With this diff, binlog and engine will be consistent based on binlog state (either rolled forward or rolled back, aligned to binlog state). This diff makes another change that it no longer aborts on systems errors on MyRocks Prepare. Prior to this diff, kIOError at Prepare() terminated the process, but it was not necessary since transactions could be safely rolled back on Prepare() error. Clients hitting errors at Prepare() will get ER_INTERNAL_ERROR with RocksDB error status. Default ER_ERROR_DURING_COMMIT is not returned. This was because ER_ERROR_DURING_COMMIT does not take string arguments so it is difficult for users to understand why it failed. This was a port from fb-5.6 1b0b6893f302dfe64ff77060a1de9c6ca4ac6a6a Reviewed By: hermanlee Differential Revision: D28610172 fbshipit-source-id: c544aa682ac

view details

Manuel Ung

commit sha 34d61ddfd3fd1673be71e231ed039a002ee79755

Combine rnd_next/index_next/position_to_correct_key codepaths Upstream commit ID : fb-mysql-5.6.35/2b8ff6dcc11945e94b8f9814df35073bf5bbd75a PS-7767 : Merge percona-202103 Summary: This combines the iteration codepaths so that we consolidate the places where we perform the snapshot refresh loop/bloomfilter calculations/ttl/icp into a single location. Roughly, I'm doing the following: - `rnd_next`/`index_next` as well as `index_read_map` are now all calling into `index_next_with_direction_intern`, which will perform the required iteration. - In `index_read_map` filtering logic was split between `position_to_correct_key` and `read_row_from_secondary_key`. The first function did some TTL checks, and the second function did ICP. Instead `position_to_correct_key` is now solely responsible for seeking to the correct key without regard for ttl (or any other filtering) logic. With better rocksdb primitives (ie. prefix seek), we wouldn't even need these functions. All filtering logic is now consolidated into `index_next_with_direction_intern`. - There are checks to see if the current key is in the current index id, as well as checks against `m_sk_match_prefix_buf` to see if the current key is within the current check. Conceptually, these two checks are doing the same thing, and can be merged together. This probably saves some memcmps. Reviewed By: hermanlee Differential Revision: D23399596 fbshipit-source-id: 642388720fd

view details

Manuel Ung

commit sha 83cab4a0cd0dd21b736e73f9367afd8d0c7c1377

Reset iterator after point lookup Upstream commit ID : fb-mysql-5.6.35/0fda5b43b292ac5274997408e32351f06184ca77 PS-7767 : Merge percona-202103 Summary: Although I did not find an example of this happening today, it seems like the SQL layer can "index_next" after a point lookup. However, since we don't position our iterator for point lookups, we would be potentially returning incorrect results (eg. if we had a previous valid iterator from before the point lookup). It seems safer to just reset the iterator after a point lookup. Reviewed By: hermanlee Differential Revision: D23399585 fbshipit-source-id: 8b74cc5f89c

view details

Przemyslaw Skibinski

commit sha c02962e1964cbf94edf377070c9ac39a6a80d195

Merge pull request #4386 from inikep/PS-7767-8.0 PS-7767 [8.0]: Merge percona-202103 tag from fb-mysql-8.0.20

view details

push time in 12 days

PR merged percona/percona-server

PS-7767 [8.0]: Merge percona-202103 tag from fb-mysql-8.0.20
  1. Update rocksdb submodule to v6.20.2 c56ad3c60a61bc39159c6fa1a112f6301cd86c89 Update HISTORY.md and bump version for 6.20.2

  2. Add new MyRocks variable rocksdb_manual_compaction_bottommost_level.

  3. Cherry-pick the following commits from percona-202103:

  1. Skipped upstream RocksDB commits:
  1. Skipped upstream commits:
+1287 -977

1 comment

27 changed files

inikep

pr closed time in 12 days

push eventinikep/sh

Przemyslaw Skibinski

commit sha 33fd7ad6a1472da38509829e9802c1b4e2690f14

update

view details

push time in 12 days

push eventinikep/sh

Luis Donoso

commit sha b86e336fefea51060ea9955fc150158f87a48938

Use the same filter to decide if a commit is skipped as gen_merge_plan.sh

view details

Przemyslaw Skibinski

commit sha 92df4971a692552e00e835d0930429cba7eea29d

Merge pull request #1 from ldonoso/master Use the same filter to decide if a commit is skipped as

view details

push time in 12 days

PR merged inikep/sh

Use the same filter to decide if a commit is skipped as

gen_merge_plan.sh

+1 -1

0 comment

1 changed file

ldonoso

pr closed time in 12 days

PullRequestReviewEvent