profile
viewpoint

cockroachdb/cockroach 22593

CockroachDB - the open source, cloud-native distributed SQL database.

nicktrav/Duplicates 4

Script to find duplicate files

nicktrav/Gmail-Classifier 2

Classify my gmail documents

nicktrav/airflow 1

Airflow is a system to programmatically author, schedule and monitor data pipelines.

nicktrav/cockroach 1

CockroachDB - the open source, cloud-native distributed SQL database.

nicktrav/dotfiles 1

My .dotfiles ... yeah, original

nicktrav/iTunes-Ratings 1

Terminal app to rate tracks and show stats

nicktrav/babushka 0

Test-driven sysadmin.

push eventnicktrav/pebble

Jackson Owens

commit sha 4b595de954b1c9034a1a518c37f1db35431ab676

compaction: use sstable writer for detecting user key changes The `userKeyChangeSplitter` is responsible for avoiding splits within a user key during flushes. Previously, it worked by recording the current user key when a split is requested. This introduced a delay in splitting. When the split is requested, the current key might already be different than the last key written to the sstable. This change alters the `userKeyChangeSplitter` to read the last written point key from the current sstable writer and the last written range key from the range deletion fragmenter. This has a couple advantages: a) It avoids an extra copy of a user key. b) It may split sooner, closer to the target file size. This is a practical concern in writing unit tests that involves flushes and small target file sizes. If we prevent splitting user keys across outputs in compactions too, it will become a practical concern there too. c) It exposes the previous point key to the broader compaction loop, which is necessary for narrowing the conditions during which the compaction loop must flush all/additional range tombstones. Currently, we ignore the splitters' suggested split point during flushes because we may have already output a key with the user key to the current sstable. Future commits will be able to read this previous point key and use the splitters' suggestion if the previous point key's user key is not equal to the splitters' suggestion.

view details

push time in 3 days

PullRequestReviewEvent

push eventnicktrav/pebble

Nick Travers

commit sha afaa43dba0260a9ebc9f853f9c8083e343203248

mkbench: emit and parse write amplification in write benchmarks To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval. Update `mkbench` to parse the write amplification and expose in the output files. Make `(*Metrics).Total()` return a `LevelMetrics` pointer, to allow calling `WriteAmp()`, which has a pointer receiver.

view details

push time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventcockroachdb/pebble

Nick Travers

commit sha afaa43dba0260a9ebc9f853f9c8083e343203248

mkbench: emit and parse write amplification in write benchmarks To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval. Update `mkbench` to parse the write amplification and expose in the output files. Make `(*Metrics).Total()` return a `LevelMetrics` pointer, to allow calling `WriteAmp()`, which has a pointer receiver.

view details

push time in 5 days

delete branch nicktrav/pebble

delete branch : nickt.write-throughput-write-amp

delete time in 5 days

PR merged cockroachdb/pebble

Reviewers
mkbench: emit and parse write amplification in write benchmarks

To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval.

Update mkbench to parse the write amplification and expose in the output files.

Make (*Metrics).Total() return a LevelMetrics pointer, to allow calling WriteAmp(), which has a pointer receiver.

+53 -30

1 comment

9 changed files

nicktrav

pr closed time in 5 days

PullRequestReviewEvent

push eventnicktrav/pebble

Nick Travers

commit sha 805b8a0a97d18a8e8c9d2f5913503758c10223bc

mkbench: emit and parse write amplification in write benchmarks To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval. Update `mkbench` to parse the write amplification and expose in the output files. Make `(*Metrics).Total()` return a `LevelMetrics` pointer, to allow calling `WriteAmp()`, which has a pointer receiver.

view details

push time in 5 days

push eventnicktrav/cockroach

Eric Harmeling

commit sha 7a6a109e8debcb199a2e31385d6d08363d60ccba

Updated GRANT SQL syntax diagram Release note: None

view details

Eric Harmeling

commit sha 5f23e79b61ef0ca54e7286f0b285f42264d872d4

Updated REVOKE SQL syntax diagram Release note: None

view details

Andrei Matei

commit sha c57a51014d5fc5489391034830f6b29534aea4fc

kvnemesis: improve tracing Before this patch, each operation was traced using a new tracer, but then injected into the cluster, which uses a different tracer. This kind of mixing of tracers in a single trace is fairly broken, and will not be supported any more. This patch fixes it by doing the tracing with the traces of the server performing the respective operation. Release note: None

view details

Andrei Matei

commit sha 386a6476efbf34c0ec97e4b3d7ab2b9d0dc77e30

tracing: remove SetOperationName This method was barely used. Because of it, the span's name had to be treated as mutable and protected by the span's mutex. That's a complication that's not worth it. In particular, I want to start exporting the span's name and use it in assertions in places where it's unclear whether that mutex can be taken. I'd rather make the name immutable and avoid headaches. Release note: None

view details

Michael Butler

commit sha 55aa786dc4f9b200c1b8969ef44a6bd00a987b9f

backupccl: export relations with null values to parquet Previously, EXPORT PARQUET could only export relations without null values. Now EXPORT PARQUET can handle relations with null values for column types currently supported by EXPORT PARQUET. Release note (sql change): users can now export relations with null values to parquet files. Informs: #67710

view details

Darius Asri Rezaei

commit sha 18b95cbb23c005e816b3b3f8747b43cc311509ca

kvprober: add setting to bypass admission queue Currently kvprober is bypassing the kv admission queue unconditionally. This is fine for dedicated clusters, since the user owns capacity planning. But it is a problem for serverless clusters because the SRE team owns capacity scaling. When hardware resources are overloaded in serverless clusters, the probes are expected to fail. This commit adds a new cluster setting for kvprober that determines if the admission control should be bypassed. The cluster setting works by adjusting the admission source header in kvprober's transaction requests: - When set, it uses a txn sourced from OTHER which is going to bypass the admission queue. - When not set, it uses a txn sourced from ROOT_KV which is not going to bypass admission queue. Control over what the txn admission header is set to is provided through kv.DB and kv.Txn. Release note: None

view details

Ricky Stewart

commit sha 30524f23bfecc882ac07d2fab9907c0a83ad37c1

ci: add bazel roachtest weekly job Closes #67155. Release note: None

view details

Alex Sarkesian

commit sha 7b4a23377705bb731d06ae0392338e9acee56530

roachtest: fix flags used in multitenant-upgrade for versions >= v21.2 Previously, the `multitenant-upgrade` test had several workarounds and branches for dealing with predecessor versions less than v21.2, particularly the fact that these versions did not support the `--store` flag passed on the `mt start-sql` CLI command. This was introduced in #71040, and subsequently incorporated into the roachtest (including workarounds) in #71604. Now that the current version is v22.1, and thus the predecessor version supports the `--store` flag, it is necessary to remove the workaround in order to ensure proper functionality of the test. Additionally, this change cleans up an even earlier workaround for validating the cluster version for predecessor versions less than v21.1.2. Fixes #72971. Release note: None

view details

irfan sharif

commit sha ee68771412fb4a6a2e3fd34c77071d9a32cbe423

spanconfig/sqlwatcher: speed up tests Using a more aggressive closed timestamp target duration brings the runtime for TestSQLWatcherReactsToUpdate down from 45s+ to single digits. To speed it up, we also just re-use the same sqlwatcher instead of creating a new one for every test case. Other tests in the package benefit from a similar treatment. Using the default target duration in fact masked a buggy test; in a future commit we end up rewrite that test so it's skipped for now. Release note: None

view details

irfan sharif

commit sha 46c7597875cc01414aa0384c2e5f1e299961211a

spanconfig/sqlwatcher: hide the factory pattern The only mutable state across concurrent WatchForSQLUpdate calls was the internal buffer, which does not need to hang off the surrounding struct. This obviates the need for the factory pattern we were using -- callers can set up multiple SQLWatchers concurrently as is (see TestSQLWatcherMultiple). This commit simply hides the factory under the package boundary; in a future commit we'll shed it altogether. This has the benefit of reducing the number of symbols in pkg/spanconfig and making it symmetric with the other spanconfig dependencies typically found together (KVAccessor, SQLTranslator). It's also every-so-slightly less verbose to use in the upcoming spanconfig.Reconciler (#71994). Release note: None

view details

irfan sharif

commit sha 6fdc4752ff5c0dd5d1d4c042c2ee999051338c22

spanconfig/sqlwatcher: remove internal factory pattern A previous commit obviated the need for a factory altogether, but went only as far hiding it within the package boundary. This commit gets rid of it entirely. Release note: None

view details

Rail Aliiev

commit sha b6380b26d438d32cf20263e601b1a37f359d15a7

gceworker: update ubuntu to 20.04 Previously, gceworkers used Ubuntu 18.04, while the docker build image and roachprod switched to Ubuntu 20.04. This patch upgrades gceworkers to Ubuntu 20.04, upgrades nodejs to 16.x and fixes an issue with cmake installation. Release note: None

view details

Rail Aliiev

commit sha 11c674c45ba9dcce0aa41c123687d64bdcfb224c

gceworker: install bazelisk This PR ensures that we install `bazelisk` on the GCE workers as a part of the bootstrap process. Release note: None

view details

Nathan VanBenschoten

commit sha 534e4511f84b9bc518a6eacff5629dc664383581

kv: rename observedts package to uncertainty This commit renames the `kvserver/observedts` package to `kvserver/uncertainty`, in preparation for its role increasing in scope.

view details

Marcus Gartner

commit sha 206f95610d0ce0a8ac195a8b60eea857804bf308

tree: add missing casts to and from UUIDs Automatic IO conversion casts are possible from the UUID type to string types, and from string types to the UUID type. This commit adds these missing casts to `castMap` which lists all valid casts. Release note: None

view details

Marcus Gartner

commit sha 6be015325c5f10ec3a916b1044d73c7ff06317dd

sql: allow cast from a type T to T in all contexts This commit updates `tree.ValidCast` so that it returns true when the given types are identical for any cast context. Casts from a type without modifiers (like width modifiers) to the same type have been removed from the cast map because they are no longer necessary. Release note: None

view details

Marcus Gartner

commit sha a803d88981bfcde9c9ae2ccd05cfbb4704e8c2a4

sql: remove UNKNOWN from castMap The UNKNOWN type, which is given to expressions that statically evaluate to NULL, can be cast to any type. Codifying all possible casts from with an UNKNOWN source type in the `castMap` is cumbersome and it is easy to accidentally omit some target types. This commit removes the UNKNOWN casts from the `castMap` and adds logic in `ValidCast` that allows UNKNOWN types to be cast to any other type in all contexts. Release note: None

view details

Yahor Yuzefovich

commit sha d739b06eec252deb0e75ef78eb22b8f3d5be5b98

sql: convert mem_limit logic test into a unit test This commit converts the `mem_limit` logic test into a corresponding unit test and expands it. It is written under the assumption that the Streamer API has already been implemented to support the index joins, so there are now two test cases. The index join will soon be powered by the Streamer and requires some concurrency to hit the memory limit whereas the new case that is doing a simple read (and will not be powered by the Streamer) can get away with no concurrency. Release note: None

view details

Yahor Yuzefovich

commit sha af9c5f89e138eab0f8fb676c015e5596500de96d

roachpb: add Spans.String Release note: None

view details

Rafi Shamim

commit sha a0f4a2a741cfa3ddd71f0da6ad0e508b9f63fe49

sql: reorder serial_normalization enum The commit that added this changed the order of this enum, which is not backwards compatible if an upgraded cluster is trying to read the cluster setting. There's no release note since this fixes an issue in a non-released version. Release note: None

view details

push time in 5 days

push eventnicktrav/pebble

Nick Travers

commit sha f6167e302c23f17c288d7a598997b510708316f2

rangekey: fix formatting Update formatting in a file that landed after the previous formatting change.

view details

Nick Travers

commit sha ca4328708983f7feccbe102770c98c05fa11891b

mkbench: emit and parse write amplification in write benchmarks To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval. Update `mkbench` to parse the write amplification and expose in the output files. Make `(*Metrics).Total()` return a `LevelMetrics` pointer, to allow calling `WriteAmp()`, which has a pointer receiver.

view details

push time in 5 days

push eventnicktrav/pebble

Nick Travers

commit sha f6167e302c23f17c288d7a598997b510708316f2

rangekey: fix formatting Update formatting in a file that landed after the previous formatting change.

view details

push time in 5 days

pull request commentcockroachdb/pebble

rangekey: fix formatting

I think this was just a matter of two in-flight PRs that landed without knowledge of the other.

PRs from here on out should fail if there's any formatting issue :crossed_fingers:

nicktrav

comment created time in 5 days

push eventcockroachdb/pebble

Nick Travers

commit sha f6167e302c23f17c288d7a598997b510708316f2

rangekey: fix formatting Update formatting in a file that landed after the previous formatting change.

view details

push time in 5 days

delete branch nicktrav/pebble

delete branch : nickt.format

delete time in 5 days

PR merged cockroachdb/pebble

rangekey: fix formatting

Update formatting in a file that landed after the previous formatting change.

+1 -1

1 comment

1 changed file

nicktrav

pr closed time in 5 days

PullRequestReviewEvent

PR opened cockroachdb/pebble

Reviewers
mkbench: emit and parse write amplification in write benchmarks

To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval.

Update mkbench to parse the write amplification and expose in the output files.

Make (*Metrics).Total() return a LevelMetrics pointer, to allow calling WriteAmp(), which has a pointer receiver.

+54 -32

0 comment

10 changed files

pr created time in 5 days

push eventnicktrav/pebble

Nick Travers

commit sha 9535b173b2e3d63e8038b0c9415aac9a98fe0f52

mkbench: emit and parse write amplification in write benchmarks To better understand the write amplification over the duration of a write-throughput benchmark, emit the write amp after each interval. Update `mkbench` to parse the write amplification and expose in the output files. Make `(*Metrics).Total()` return a `LevelMetrics` pointer, to allow calling `WriteAmp()`, which has a pointer receiver.

view details

push time in 5 days

create barnchnicktrav/pebble

branch : nickt.write-throughput-write-amp

created branch time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentcockroachdb/pebble

rangekey: fix formatting

 func TestUnsetValue_Roundtrip(t *testing.T) { 		{ 			endKey: []byte("hello world"), 			suffixes: [][]byte{-				[]byte{},+				{},

Bit of a dubious change, but probably harder than it's worth to try and skip over this particular alteration.

nicktrav

comment created time in 6 days

PullRequestReviewEvent

PR opened cockroachdb/pebble

Reviewers
rangekey: fix formatting

Update formatting in a file that landed after the previous formatting change.

+1 -1

0 comment

1 changed file

pr created time in 6 days

create barnchnicktrav/pebble

branch : nickt.format

created branch time in 6 days

push eventnicktrav/pebble

Jackson Owens

commit sha 5d99779ccb8da71056dcd753a263b9a59a3b89b2

batch: add RangeKey{Set,Unset,Delete} methods Add key kinds for range key sets, unsets and deletes. Add corresponding methods to the Batch, behind an experimental type. With this commit, these keys may be encoded into a batch but are discarded when committed.

view details

Jackson Owens

commit sha 08c5d46c75722b9527c360ca1e7069c0d2286e59

internal/base: document new Split requirements Document the new Split requirements necessary for range keys support. See #1341.

view details

Nick Travers

commit sha fc3ffab07caa66f2fa453b5be7e79619f5fd1bbc

ci: add gofmt linter Check that files are formatted according to `gofmt`.

view details

Nick Travers

commit sha 7fb375f7d8e3d911e65259593a11e48cbd002cc2

*: format files according to gofmt Run `gofmt -s -w` across all files, excluding the `vendor` directory using the `make format` target. Most of these changes are adding the equivalent `//go:build` lines to source files that are built conditionally (per the [Go 1.17 release notes][1]). [1]: https://go.dev/doc/go1.17

view details

push time in 6 days

more