• Updated the roadmap as of January 2021.
• Added a changelog
• Moved roadmap notes to the bottom, after the roadmap table.
• Added a link to the YT board.
• Added a TOC.

I would like to pitch for some random sampling methods such as

fun <E> List<E>.sampleRandom(random: Random = DefaultRandom): E
fun <E> List<E>.sampleRandomList(size: Int, random: Random = DefaultRandom): List<E>
fun <E> List<E>.sampleRandomSet(size: Int, random: Random = DefaultRandom): Set<E>
fun <E> List<E>.sampleRandomSet(size: Int, weights: List<Int> = LazyList(this.size) {i -> 1}): Set<E>

The benefit of having such functions in the stdlib is preventing developers from implementing naive-but-incorrect implementations themselves. For example, for sampleRandomList it could be tempting to do something like the following, which is incorrect because the result has a different distribution than initial Random.nextInt()

fun <E> List<E>.sampleRandomList(sampleSize: Int): List<E> {
    val selectedIndices = mutableSetOf<Int>()

    repeat(sampleSize) {
        do {
            val i: Int = Random.nextInt(indices)
            val wasNew = selectedIndices.add(i)
        } while(!wasNew)
    }

    return this.slice(selectedIndices)
}

Another inefficient approach could be the implementation below, which is very easy to implement with the current proposal, and thus even more probable to be a source of problems.

fun <E> List<E>.sampleRandomList(sampleSize: Int): List<E> = shuffled().take(sampleSize)

As far as I understand, one of the correct and efficient algorithms for sampling is "Vose's Alias" algorithm, of which I am frankly totally ignorant, but you can read about it here http://www.keithschwarz.com/darts-dice-coins/

Can you please explain, why the two methods you present for taking random samples are "naive-but-incorrect"? "Vose's Alias" which you cited is an algorithm for getting random numbers with a biased (non-uniform) distribution. I cannot see how this fits into your example.

Is there any chance to reproduce with some self-contained code?

Fixes #2464

👍 Nice clarification. Thanks.

Interesting. I actually haven't considered that as I initially dismissed that dispatcher thinking that Unconfined refers to unconfined thread pool, i.e. a dispatcher that creates threads as needed instead of using a thread pool. Now that you pointed this out, I read the docs and I can see it's something completely different. This might be exactly the solution I was looking for.

Have you considered explicitly using rxSingle(Dispatchers.Unconfined) in order to let rxSingle continue using the thread provided by the Rx Scheduler?

I'm trying to interop Kotlin Flow with RxJava callers using kotlin-coroutines-rx2. All my consumers of the API are currently RxJava chains, which I'm planning to slowly migrate to coroutines. However, my data is exposed as Flow<T> that needs to be consumed by these Rx chains. The data is coming from Jetpack DataStore. I'm using coroutines-rx2 rxSingle to wrap the flow into a Single in order to satisfy the caller dependencies. I end up with a function like this:

override fun getData(): Single<Data> = rxSingle { dataStore.data.first() }

Which gets called from chains like this:

 repository.getData()
        .flatMap { someNetworkCall(it) }
        .subscribeOn(schedulerProvider.io())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribe({ processResult(it) }, { processError(it) })

The threading of this looks like:

1. getData() is called on a background thread from Schedulers IO
2. rxSingle doesn't have a coroutine context and therefore the dispatcher defaults to DefaultDispatcher
3. someNetworkCall now runs on the DefaultDispatcher instead of the IO, which would be expected by just reading the Rx chain

In my app, all the threading is at the top layer, where the subscribe is called. This thread switching that rxSingle does is not obvious to whoever's working with the API, and the thread injected and used in bottom layer where getData exists, might not satisfy the remaining of the Rx chain.

I'm not sure why rxSingle switches the thread context instead of using the one currently specified by the Rx scheduler, there's probably a good reason for it, but as I see it, my options for this are:

1. Each time I'm using rxSingle to wrap a Flow, I can finish with switching the thread using .observeOn(). This doesn't really satisfy the requirement of being transparent to the caller of the function, as I will just default to an arbitrary thread i decide on, which most likely going to be observeOn(schedulers.io()) to be safe. So the same getData will look like override fun getData(): Single<Data> = rxSingle(backgroundDispatcher) { dataStore.data.first() }.observeOn(schedulers.io())

2. Or I pass down a dispatcher to use from the Rx chain caller like repository.getData(schedulers.io().asCoroutineDispatcher()) and pass it to my rxSingle wrapping function: override fun getData(dispatcher: Dispatcher): Single<Data> = rxSingle(dispatcher) { dataStore.data.first() }

Both options are not great so I'm looking for some advice, maybe I'm missing something here?

Checklist:

• [X] Is this pull request against the branch main?

Resolves https://github.com/Kotlin/kotlinx.coroutines/issues/2283

First commit includes a test case that fails in both macosx64 background & main tests.

Resolves #2322

While 1.4.30-M1 fixed Worker.execute() leaking ObjC autoreleasing objects when running native code on a K/N Worker thread (KT-42822), it does not address the issue at KoltinX Coroutines framework level.

More specifically, when tasks on an EventLoop call into native code, any autoreleasing object — especially many ObjC method return values — is effectively leaked ^[1]. This is because the first available autoreleasepool in the call stack is in workerMain(), and it wraps around a most likely never-returning ^[2] runEventLoop().

This PR introduced a failing test case of this bug, and fixed it by wrapping all Runnable.run() calls in EventLoopImplBase with an autoreleasepool (or nothing for non-Darwin targets). This is equivalent to the DISPATCH_AUTORELEASE_FREQUENCY_WORK_ITEM^[3] in GCD.

An alternative to the per-item wrapping is integrating pool push/pop into runEventLoop(), with heruistics based on the park time returned by processNextEvent(). Not familiar enough about the particulars to try this out though.

^[1] ... unless an autoreleasepool is declared in user code explicitly, which is a rare practice except for memory optimization. ^[2] Dispatchers.Default, for example. ^[3] ... which is not the default behaviour. GCD threads inherit autorelease pool management from NSRunLoop, which recreates pools for each runloop iteration.