profile
viewpoint

siying/china-indie-podcasts 4

发现与推荐高质量的中文独立播客

siying/rocksdb 1

A library that provides an embeddable, persistent key-value store for fast storage.

siying/cassandra 0

Mirror of Apache Cassandra

siying/mysql-5.6 0

Facebook's branch of the Oracle MySQL v5.6 database. This includes MyRocks.

PR opened facebook/rocksdb

Reviewers
File temperature information should be preserved when restart the DB

Summary: Fix a bug that causes file temperature not preserved after DB is restarted, or options.max_manifest_file_size is hit. Also, pass temperature information to NewRandomAccessFile() to allow users to hack a solution where they don't preserve tiering information.

Test Plan: Add a unit test that would fail without the fix.

+213 -182

0 comment

17 changed files

pr created time in 3 days

create barnchsiying/rocksdb

branch : tiering_restart

created branch time in 3 days

PR opened facebook/rocksdb

Reviewers
Reduce build-linux parallelism

Summary: CircleCI main branch tests occasionally fail with "C++ exception with description "Resource temporarily unavailable" thrown in the test body." on TwoWriteQueues/SnapshotConcurrentAccessTest

Not sure what's the reason, but try to reduce parallelism a little bit and see whether it still happens. Right now the test usually runs for 18 minutes, while ASAN runs for 33 minutes, so have quite some room to reduce parallelism without slowing down all CI progress.

Test Plan: Watch CI

+1 -1

0 comment

1 changed file

pr created time in 13 days

create barnchsiying/rocksdb

branch : circleci_res

created branch time in 13 days

issue commentfacebook/rocksdb

The LRUCache::DisownData() drops data in case code is compiled with ASAN

I assume you mean running ASAN with your program, rather than RocksDB unit tests? We don't observe it in RocksDB unit tests' ASAN run. If you have a proper fix, feel free to submit it.

oleksandr-kachan

comment created time in 13 days

PullRequestReviewEvent
PullRequestReviewEvent

PR opened facebook/rocksdb

Reviewers
Fix flaky DBTest2.RateLimitedCompactionReads

Summary: DBTest2.RateLimitedCompactionReads sometime shows following failure:

what(): db/db_test2.cc:3976: Failure Expected equality of these values: i + 1 Which is: 4 NumTableFilesAtLevel(0) Which is: 0

The assertion itself doesn't appear to be correct. Fix it.

Test Plan: Removing an assertion shouldn't break anything.

+3 -1

0 comment

1 changed file

pr created time in 17 days

create barnchsiying/rocksdb

branch : db_test_wait

created branch time in 17 days

created tagfacebook/rocksdb

tagv6.26.1

A library that provides an embeddable, persistent key-value store for fast storage.

created time in 18 days

release facebook/rocksdb

v6.26.1

released time in 18 days

push eventfacebook/rocksdb

sdong

commit sha 4d57a393a8f1563c0dff1ff468756c23d89e3601

Improve HISTORY.md

view details

push time in 18 days

push eventfacebook/rocksdb

sdong

commit sha 224da7f494fb22209aa6aa63a2936bc04231564a

Release 6.26.1

view details

push time in 18 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventfacebook/rocksdb

Adam Retter

commit sha 9674b843c2ee70c5534f0a77be238decb24f2d26

Fix an error on GCC 4.8.5 where -Werror=unused-parameter fails (#9145)

view details

push time in 20 days

PR merged facebook/rocksdb

Reviewers
6.26.fb - Fix an error on GCC 4.8.5 where -Werror=unused-parameter fails bug CLA Signed arm ppc64le

Backport of https://github.com/facebook/rocksdb/pull/9144 to the 6.26.fb branch


Before this fix compilation with GCC 4.8.5 20150623 (Red Hat 4.8.5-36) would fail with the following error:

CC       jls/db/db_impl/db_impl.o
In file included from ./env/file_system_tracer.h:8:0,
                 from ./file/random_access_file_reader.h:15,
                 from ./file/file_prefetch_buffer.h:15,
                 from ./table/format.h:13,
                 from ./table/internal_iterator.h:14,
                 from ./db/pinned_iterators_manager.h:12,
                 from ./db/range_tombstone_fragmenter.h:15,
                 from ./db/memtable.h:22,
                 from ./db/memtable_list.h:16,
                 from ./db/column_family.h:17,
                 from ./db/db_impl/db_impl.h:22,
                 from db/db_impl/db_impl.cc:9:
./include/rocksdb/file_system.h:108:8: error: unused parameter 'opts'
[-Werror=unused-parameter]
 struct FileOptions : EnvOptions {
        ^
db/db_impl/db_impl.cc: In member function 'virtual rocksdb::Status
rocksdb::DBImpl::SetDBOptions(const
std::unordered_map<std::basic_string<char>, std::basic_string<char>
>&)':
db/db_impl/db_impl.cc:1230:36: note: synthesized method
'rocksdb::FileOptions& rocksdb::FileOptions::operator=(const
rocksdb::FileOptions&)' first required here
       file_options_for_compaction_ = FileOptions(new_db_options);
                                    ^
  CC       jls/db/db_impl/db_impl_compaction_flush.o
cc1plus: all warnings being treated as errors
make[1]: *** [jls/db/db_impl/db_impl.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/rocksdb-local-build'
make: *** [rocksdbjavastatic] Error 2
Makefile:2222: recipe for target 'rocksdbjavastaticdockerarm64v8' failed
make: *** [rocksdbjavastaticdockerarm64v8] Error 2

This was detected on both ppc64le and arm64v8, however it does not seem to appear in the same GCC 4.8 version we use for x64 in CircleCI - https://app.circleci.com/pipelines/github/facebook/rocksdb/9691/workflows/c2a94367-14f3-4039-be95-325c34643d41/jobs/227906

+1 -1

0 comment

1 changed file

adamretter

pr closed time in 20 days

Pull request review commentfacebook/rocksdb

Add tiered storage related read bytes stats to Statistic

 const std::vector<std::pair<Tickers, std::string>> TickersNameMap = {     {BACKUP_WRITE_BYTES, "rocksdb.backup.write.bytes"},     {REMOTE_COMPACT_READ_BYTES, "rocksdb.remote.compact.read.bytes"},     {REMOTE_COMPACT_WRITE_BYTES, "rocksdb.remote.compact.write.bytes"},+    {HOT_FILE_READ_BYTES, "rocksdb.hot.file.read.bytes"},+    {WARM_FILE_READ_BYTES, "rocksdb.warm.file.read.bytes"},+    {COLD_FILE_READ_BYTES, "rocksdb.cold.file.read.bytes"},

It might be good to have bytes read, but it might be even more important to get read counts. For HDD, read counts are much more important than total bytes read.

zhichao-cao

comment created time in 24 days

PullRequestReviewEvent
PullRequestReviewEvent
GollumEvent

release facebook/rocksdb

v6.26.0

released time in a month

created tagfacebook/rocksdb

tagv6.26.0

A library that provides an embeddable, persistent key-value store for fast storage.

created time in a month

push eventfacebook/rocksdb

Adam Retter

commit sha 7448618ac866ed80cbbd859a48a1718420069238

Remove invalid RocksJava native entry (#9147) Summary: It seems that an incorrect native source file entry was introduced in https://github.com/facebook/rocksdb/pull/8999. For some reason it appears that CI was not run against that PR, and so the problem was not detected. This PR fixes the problem by removing the invalid entry, allowing RocksJava to build correctly again. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9147 Reviewed By: pdillinger Differential Revision: D32300976 fbshipit-source-id: dbd763b806bacf0fc08f4deaf07c63d0a266c4cf

view details

push time in a month

PullRequestReviewEvent

push eventfacebook/rocksdb

Adam Retter

commit sha 52626f10b106c86774ff8d64b466f7dc658f0bb2

Restore Java 7 Compatibility (#9103) Summary: RocksDB should still compile on Java 7. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9103 Reviewed By: pdillinger Differential Revision: D32067561 Pulled By: mrambacher fbshipit-source-id: bbe9c18c8007ab3e113de4add56a84c9bde61c8e

view details

push time in a month

PullRequestReviewEvent

push eventsiying/rocksdb

Andrew Kryczka

commit sha 4217d1bce72d3ff02b5eba76bfb418b7448daa8e

Support `GetMapProperty()` with "rocksdb.dbstats" (#9057) Summary: This PR supports querying `GetMapProperty()` with "rocksdb.dbstats" to get the DB-level stats in a map format. It only reports cumulative stats over the DB lifetime and, as such, does not update the baseline for interval stats. Like other map properties, the string keys are not (yet) exposed in the public API. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9057 Test Plan: new unit test Reviewed By: zhichao-cao Differential Revision: D31781495 Pulled By: ajkr fbshipit-source-id: 6f77d3aee8b4b1a015061b8c260a123859ceaf9b

view details

leipeng

commit sha 0c53b41856a8f2d683df545555d069ad7babfcef

db_impl_write.cc: use stats_ instead of immutable_db_options_.stats (#9053) Summary: This PR has no semantic changes, just to make code shorter. `stats_` has value same with `immutable_db_options_.stats`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9053 Reviewed By: zhichao-cao Differential Revision: D31801603 Pulled By: anand1976 fbshipit-source-id: cbd8fe478d3e90ae078ace49b4f2eb9bb028ccf6

view details

leipeng

commit sha 0a73ada7b57c4e69a231793776bee30706ae862b

remove unused local obj and simpilify comple code (#9052) Summary: This PR does not change code sematics, it just changes for: 1. local obj `nonmem_w` and `lfile` are unused 2. null check for `delete ptr` is unnecessary 3. use `unique_ptr::reset` instead of `release` + `delete` Pull Request resolved: https://github.com/facebook/rocksdb/pull/9052 Reviewed By: zhichao-cao Differential Revision: D31801661 Pulled By: anand1976 fbshipit-source-id: 16a77d45da8c8833bf5bf3bce546bb3711b335df

view details

sdong

commit sha 633f069c29a59abcaab78c1f7046239c701a2c46

Update Release Version to 6.26 (#9059) Summary: Before cutting release branch 6.26, update version.h and release notes Pull Request resolved: https://github.com/facebook/rocksdb/pull/9059 Reviewed By: ajkr Differential Revision: D31805126 fbshipit-source-id: ae85ccf06ec756fa21163161f53fd0b728e6e32e

view details

Yanqin Jin

commit sha f72fd5856585774063ac3fc8926f70626963d488

Fix atomic flush waiting forever for MANIFEST write (#9034) Summary: In atomic flush, concurrent background flush threads will commit to the MANIFEST one by one, in the order of the IDs of their picked memtables for all included column families. Each time, a background flush thread decides whether to wait based on two criteria: - Is db stopped? If so, don't wait. - Am I the one to commit the currently earliest memtable? If so, don't wait and ready to go. When atomic flush was implemented, error writing to or syncing the MANIFEST would cause the db to be stopped. Therefore, this background thread does not have to check for the background error while waiting. If there has been such an error, `DBStopped()` would have been true, and this thread will **not** wait forever. After we improved error handling, RocksDB may map an IOError while writing to MANIFEST to a soft error, if there is no WAL. This requires the background threads to check for background error while waiting. Otherwise, a background flush thread may wait forever. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9034 Test Plan: make check Reviewed By: zhichao-cao Differential Revision: D31639225 Pulled By: riversand963 fbshipit-source-id: e9ab07c4d8f2eade238adeefe3e42dd9a5a3ebbd

view details

Jonathan Albrecht

commit sha e970248602f9efd193a5d2d65427003766dd56d0

Add support for building on s390x platform (#8962) Summary: This PR adds support for building on s390x including updating travis CI. It uses the previous work in https://github.com/facebook/rocksdb/pull/6168 and adds some more changes to get all current tests (make check and jni tests) to pass. The tests were run with snappy, lz4, bzip2 and zstd all compiled in. There are a few pieces still needed to get the travis build working that I don't think I can do. adamretter is this something you could help with? 1. A prebuilt https://rocksdb-deps.s3-us-west-2.amazonaws.com/cmake/cmake-3.14.5-Linux-s390x.deb package 2. A https://hub.docker.com/r/evolvedbinary/rocksjava s390x image Not sure if there is more required for travis. Happy to help in any way I can. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8962 Reviewed By: mrambacher Differential Revision: D31802198 Pulled By: pdillinger fbshipit-source-id: 683511466fa6b505f85ba5a9964a268c6151f0c2

view details

myasuka

commit sha dc00e4b1206cba9995fe042ec976df54d0f6e76f

Introduce allowStall option for write buffer manager constructor (#9076) Summary: https://github.com/facebook/rocksdb/pull/7898 enable write buffer manager to stall write when memory_usage exceeds buffer_size, this is really useful for container running case to limit the memory usage. However, this feature is not visiable for rocksJava yet. This PR targets to introduce this feature for rocksJava. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9076 Reviewed By: akankshamahajan15 Differential Revision: D31931092 Pulled By: anand1976 fbshipit-source-id: 5531c16a87598663a02368c07b5e13a503164578

view details

Peter Dillinger

commit sha 4ec31dc8ac208f46baabe175be2ad7d25697d0e2

Make format-diff.sh locale-independent (#9079) Summary: Force POSIX locale for calls to 'git remote' that might have locale-dependent formatting, as shown in https://github.com/facebook/rocksdb/issues/8731 comment Pull Request resolved: https://github.com/facebook/rocksdb/pull/9079 Test Plan: manual (haven't tried on a machine with non-english default locale) Reviewed By: ltamasi Differential Revision: D31943092 Pulled By: pdillinger fbshipit-source-id: 7dbe5915824f39f73b412cc3d1a86a2521cf76c1

view details

Calin Culianu

commit sha 82846f41d3cd245b8dd6f2dc7e8782dff96424b2

Fix incorrect order of comments in win_thread.cc (#9033) Summary: The comments in the `#endif` section at the end of the file were in the wrong order. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9033 Reviewed By: mrambacher Differential Revision: D31935856 Pulled By: ajkr fbshipit-source-id: 24aca039993d6e27022cfe8d6434e90f2934c87c

view details

Peter Dillinger

commit sha 5bf9a7d5ee367b542626f9e58041886e25d1650b

Clarify caching behavior for index and filter partitions (#9068) Summary: Somewhat confusingly, index and filter partition blocks are never owned by table readers, even with cache_index_and_filter_blocks=false. They still go into block cache (possibly pinned by table reader) if there is a block cache. If no block cache, they are only loaded transiently on demand. This PR primarily clarifies the options APIs and some internal code comments. Also, this closes a hypothetical data corruption vulnerability where some but not all index partitions are pinned. I haven't been able to reproduce a case where it can happen (the failure seems to propagate to abort table open) but it's worth patching nonetheless. Fixes https://github.com/facebook/rocksdb/issues/8979 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9068 Test Plan: existing tests :-/ I could cover the new code using sync points, but then I'd have to very carefully relax my `assert(false)` Reviewed By: ajkr Differential Revision: D31898284 Pulled By: pdillinger fbshipit-source-id: f2511a7d3a36bc04b627935d8e6cfea6422f98be

view details

Andrew Kryczka

commit sha f24c39ab3d1536285d87e57696718226d1469b50

Prevent corruption with parallel manual compactions and `change_level == true` (#9077) Summary: The bug can impact the following scenario. There must be two `CompactRange()`s, call them A and B. Compaction A must have `change_level=true`. Compactions A and B must run in parallel, and new data must be added while they run as well. Now, on to the details of the race condition. Compaction A must reach the refitting phase while B's next step is to trivial move new data (i.e., data that has been inserted behind A) down to the same level that A's refit targets (`CompactRangeOptions::target_level`). B must be unregistered (i.e., has not yet called `AddManualCompaction()` for the current `RunManualCompaction()`) while A invokes `DisableManualCompaction()`s to prepare for refitting. In the old code, B could still proceed to register a manual compaction, while A had disabled manual compaction. The next part of the race condition is B picks and schedules a trivial move while A has released the lock in refitting phase in order to persist the LSM state change (i.e., the log phase of `LogAndApply()`). That way, B does not see the refitted data when picking a trivial-move compaction. So it is susceptible to picking one that overlaps. Finally, B executes the picked trivial-move compaction. Trivial-move compactions are special in that they never check whether manual compaction is disabled. So the picked compaction causing overlap ends up being applied, leading to LSM corruption if `force_consistency_checks=false`, or entering read-only mode with `Status::Corruption` if `force_consistency_checks=true` (the default). The fix is just to prevent B from registering itself in `RunManualCompaction()` while manual compactions are disabled, consequently preventing any trivial move or other compaction from being picked/scheduled. Thanks to siying for finding the bug. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9077 Test Plan: The test does not go all the way in exposing the bug because it requires a compaction to be picked/scheduled while logging LSM state change for RefitLevel(). But the fix is to make such a compaction not picked/scheduled in the first place, so any repro of that scenario would end up hanging RefitLevel() logging. So instead I just verified no such compaction is registered in the scenario where `RefitLevel()` disables manual compactions. Reviewed By: siying Differential Revision: D31921908 Pulled By: ajkr fbshipit-source-id: 9bb5d0e847ad428211227f40830c685c209fbecb

view details

Peter Dillinger

commit sha a7d4bea43aaa2dba3af04bcf9a76ea2f7ad917e6

Implement XXH3 block checksum type (#9069) Summary: XXH3 - latest hash function that is extremely fast on large data, easily faster than crc32c on most any x86_64 hardware. In integrating this hash function, I have handled the compression type byte in a non-standard way to avoid using the streaming API (extra data movement and active code size because of hash function complexity). This approach got a thumbs-up from Yann Collet. Existing functionality change: * reject bad ChecksumType in options with InvalidArgument This change split off from https://github.com/facebook/rocksdb/issues/9058 because context-aware checksum is likely to be handled through different configuration than ChecksumType. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9069 Test Plan: tests updated, and substantially expanded. Unit tests now check that we don't accidentally change the values generated by the checksum algorithms ("schema test") and that we properly handle invalid/unrecognized checksum types in options or in file footer. DBTestBase::ChangeOptions (etc.) updated from two to one configuration changing from default CRC32c ChecksumType. The point of this test code is to detect possible interactions among features, and the likelihood of some bad interaction being detected by including configurations other than XXH3 and CRC32c--and then not detected by stress/crash test--is extremely low. Stress/crash test also updated (manual run long enough to see it accepts new checksum type). db_bench also updated for microbenchmarking checksums. ### Performance microbenchmark (PORTABLE=0 DEBUG_LEVEL=0, Broadwell processor) ./db_bench -benchmarks=crc32c,xxhash,xxhash64,xxh3,crc32c,xxhash,xxhash64,xxh3,crc32c,xxhash,xxhash64,xxh3 crc32c : 0.200 micros/op 5005220 ops/sec; 19551.6 MB/s (4096 per op) xxhash : 0.807 micros/op 1238408 ops/sec; 4837.5 MB/s (4096 per op) xxhash64 : 0.421 micros/op 2376514 ops/sec; 9283.3 MB/s (4096 per op) xxh3 : 0.171 micros/op 5858391 ops/sec; 22884.3 MB/s (4096 per op) crc32c : 0.206 micros/op 4859566 ops/sec; 18982.7 MB/s (4096 per op) xxhash : 0.793 micros/op 1260850 ops/sec; 4925.2 MB/s (4096 per op) xxhash64 : 0.410 micros/op 2439182 ops/sec; 9528.1 MB/s (4096 per op) xxh3 : 0.161 micros/op 6202872 ops/sec; 24230.0 MB/s (4096 per op) crc32c : 0.203 micros/op 4924686 ops/sec; 19237.1 MB/s (4096 per op) xxhash : 0.839 micros/op 1192388 ops/sec; 4657.8 MB/s (4096 per op) xxhash64 : 0.424 micros/op 2357391 ops/sec; 9208.6 MB/s (4096 per op) xxh3 : 0.162 micros/op 6182678 ops/sec; 24151.1 MB/s (4096 per op) As you can see, especially once warmed up, xxh3 is fastest. ### Performance macrobenchmark (PORTABLE=0 DEBUG_LEVEL=0, Broadwell processor) Test for I in `seq 1 50`; do for CHK in 0 1 2 3 4; do TEST_TMPDIR=/dev/shm/rocksdb$CHK ./db_bench -benchmarks=fillseq -memtablerep=vector -allow_concurrent_memtable_write=false -num=30000000 -checksum_type=$CHK 2>&1 | grep 'micros/op' | tee -a results-$CHK & done; wait; done Results (ops/sec) for FILE in results*; do echo -n "$FILE "; awk '{ s += $5; c++; } END { print 1.0 * s / c; }' < $FILE; done results-0 252118 # kNoChecksum results-1 251588 # kCRC32c results-2 251863 # kxxHash results-3 252016 # kxxHash64 results-4 252038 # kXXH3 Reviewed By: mrambacher Differential Revision: D31905249 Pulled By: pdillinger fbshipit-source-id: cb9b998ebe2523fc7c400eedf62124a78bf4b4d1

view details

Peter Dillinger

commit sha 92e2399669df3fdc6c8573f9a4adf09e50a2796f

Fix EnvLibrados and add to CI (#9088) Summary: This feature was not part of any common or CI build, so no surprise it broke. Now we can at least ensure compilation. I don't know how to run the test successfully (missing config file) so it is bypassed for now. Fixes https://github.com/facebook/rocksdb/issues/9078 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9088 Test Plan: CI Reviewed By: mrambacher Differential Revision: D32009467 Pulled By: pdillinger fbshipit-source-id: 3e0d1e5fde7f0ece703d48a81479e1cc7392c25c

view details

Jonathan Albrecht

commit sha f2d11b3fdceb0886304a5a7c6302121610ebf39f

Temporarily disable s390x+cmake* Travis jobs (#9095) Summary: Temporarily disable s390x+cmake* jobs until a cmake-3.14.5-Linux-s390x.deb can be installed to https://rocksdb-deps.s3-us-west-2.amazonaws.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9095 Reviewed By: akankshamahajan15 Differential Revision: D32025417 Pulled By: riversand963 fbshipit-source-id: eefb9737937987c7d9273482a89e4d2266cd5375

view details

Yanqin Jin

commit sha fdf2a0d7eb4fb363075712cdf0a4e5c4ac2688be

Fix a compaction bug for write-prepared txn (#9061) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/9061 In write-prepared txn, checking a sequence's visibility in a released (old) snapshot may return "Snapshot released". Suppose we have two snapshots: ``` earliest_snap < earliest_write_conflict_snap ``` If we release `earliest_write_conflict_snap` but keep `earliest_snap` during bottommost level compaction, then it is possible that certain sequence of events can lead to a PUT being seq-zeroed followed by a SingleDelete of the same key. This violates the ascending order of keys, and will cause data inconsistency. Reviewed By: ltamasi Differential Revision: D31813017 fbshipit-source-id: dc68ba2541d1228489b93cf3edda5f37ed06f285

view details

Levi Tamasi

commit sha 44d04582cbcbf1af87e04c077f2eacca2fe13ae5

Aggregate blob file related changes in VersionBuilder as VersionEdits are applied (#9085) Summary: The current VersionBuilder code on mainline keeps track of blob file related changes ("delta") induced by a series of `VersionEdit`s in the form of `BlobFileMetaDataDelta` objects. Specifically, `BlobFileMetaDataDelta` contains the amount of additional garbage generated by compactions, as well as the set of newly linked/unlinked SSTs. This is very handy for detecting trivial moves, since in that case the newly linked and unlinked SSTs cancel each other out. However, this representation does not allow us to easily tell whether a certain blob file is obsolete after applying a set of `VersionEdit`s or not. In order to solve this issue, the patch introduces `MutableBlobFileMetaData`, which, in addition to the delta, also contains the materialized state after applying a set of version edits (i.e. the total amount of garbage and the resulting set of linked SSTs). This will enable us to add further consistency checks and to improve certain pieces of functionality where knowing up front which blob files get obsoleted is beneficial. (Note: this patch is just the refactoring part; I plan to create separate PRs for the enhancements.) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9085 Test Plan: Ran `make check` and the stress tests in BlobDB mode. Reviewed By: riversand963 Differential Revision: D31980867 Pulled By: ltamasi fbshipit-source-id: cc4286778b10900af720423d6b772c77f28a93e3

view details

CodemodService Bot

commit sha 49af999954c0c130fefdb5f4bafc919c18341521

internal_repo_rocksdb/repo Reviewed By: DrMarcII Differential Revision: D32033741 fbshipit-source-id: af12d9d72f109a4a2837cb64e02fa0dbc9175711

view details

Yanqin Jin

commit sha 8f4f302316017fd89a2dbf4f965b797b8a85210e

Attempt to deflake DBFlushTest.FireOnFlushCompletedAfterCommittedResult (#9083) Summary: DBFlushTest.FireOnFlushCompletedAfterCommittedResult uses test sync points to coordinate interleaving of different threads. Before this PR, the test writes some data to memtable, triggers a manual flush, and triggers a second manual flush after a first bg flush thread starts executing. Though unlikely, it is possible for the second bg flush thread to run faster than the first bg flush thread and deques flush queue first. In this case, the original test will fail. The fix is to wait until the first bg flush thread deques the flush queue before triggering second manual flush. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9083 Test Plan: ./db_flush_test --gtest_filter=DBFlushTest.FireOnFlushCompletedAfterCommittedResult Reviewed By: jay-zhuang Differential Revision: D31951239 Pulled By: riversand963 fbshipit-source-id: f32d7cdabe6ad6808fd18e54e663936dc0a9edb4

view details

Yanqin Jin

commit sha 8e59a1dc9ace6b3033f8b2082bf41a5a16305040

Attempt to deflake ListenerTest.MultiCF (#9084) Summary: EventListenerTest.MultiCF uses TestFlushListener which has members flushed_dbs_ and flushed_column_family_names_ that are not protected by locks. This implicitly indicates that we need to ensure the methods accessing these data structures in a single threaded way. In other tests, e.g. MultiDBMultiListeners, we use TEST_WaitForFlushMemtable() to wait until all memtables of a given column family are flushed, hence no pending flush threads will concurrently call OnFlushCompleted() and cause data race for flushed_dbs_. To fix a test failure, we should do the same for MultiCF. Example data race stack traces reported by TSAN ``` Read of size 8 at 0x7b6000002840 by main thread: #0 std::vector<rocksdb::DB*, std::allocator<rocksdb::DB*> >::size() const /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/stl_vector.h:655:40 https://github.com/facebook/rocksdb/issues/1 rocksdb::EventListenerTest_MultiCF_Test::TestBody() /home/circleci/project/db/listener_test.cc:380:7 Previous write of size 8 at 0x7b6000002840 by thread T2: #0 void std::vector<rocksdb::DB*, std::allocator<rocksdb::DB*> >::_M_emplace_back_aux<rocksdb::DB* const&>(rocksdb::DB* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/vector.tcc:442:26 https://github.com/facebook/rocksdb/issues/1 std::vector<rocksdb::DB*, std::allocator<rocksdb::DB*> >::push_back(rocksdb::DB* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/bits/stl_vector.h:923:4 https://github.com/facebook/rocksdb/issues/2 rocksdb::TestFlushListener::OnFlushCompleted(rocksdb::DB*, rocksdb::FlushJobInfo const&) /home/circleci/project/db/listener_test.cc:255:18 ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/9084 Test Plan: ./listener_test --gtest_filter=EventListenerTest.MultiCF Reviewed By: jay-zhuang Differential Revision: D31952259 Pulled By: riversand963 fbshipit-source-id: 94a7f29e4e9466ead42418944eb2247fc32bd499

view details

Yanqin Jin

commit sha d263505417a6dcb273e19c5afb2e7d449175a465

Avoid div-by-zero error in db_stress (#9086) Summary: If a column family has 0 levels, then existing `TestCompactFiles(...)` may hit divide-by-zero. To fix, return early if the cf is empty. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9086 Test Plan: TBD Reviewed By: ajkr Differential Revision: D31986799 Pulled By: riversand963 fbshipit-source-id: 48f7dfb2b2b47cfc1315cb71ca80eb230d947f17

view details

push time in a month

PullRequestReviewEvent

PR opened facebook/rocksdb

Reviewers
Add options.manual_wal_flush to db_bench

Summary: It is useful to add options.manual_wal_flush to db_bench

Test Plan: Run the benchamrk with the option.

+4 -0

0 comment

1 changed file

pr created time in a month

more