profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/luqmana/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

luqmana/rhythmbox-plugins 58

Porting various Rhythmbox plugins to work with the new plugin api.

luqmana/gdax-client 37

Rust client library for Coinbase Exchange (GDAX).

luqmana/rray 8

Raytracer written in rust.

luqmana/mcchat 7

Fake client to get chat messages from minecraft (out-of-date).

luqmana/Pronto 7

Links freedesktop notifications to iOS app prowl.

luqmana/RemoteJoyPlus 6

Based on the original RemoteJoy by TyRaNiD with more stuff!

luqmana/llvm 2

Temporary fork of LLVM for Rust

luqmana/NowPlayingFile 1

A Rhythmbox plugin which simply outputs the currently playing track's information to a file. Based off of NowPlayingXML by kflorence. Updated to work with new plugin api.

AaronMorais/MarkovsMix 0

Generating music using Markov chains

luqmana/asuswrt-merlin 0

Enhanced version of Asus's router firmware (Asuswrt) (legacy code base)

delete branch oxidecomputer/propolis

delete branch : libc-bump

delete time in 8 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha 6c59195c44d1a74e8bf681e2ba2cccdfad753e64

Switch back to crates.io libc now that 0.2.102 has been published. (#48) Remove TODO now that the crates.io libc has Illumos preadv/pwritev bindings. **NOTE:** Run `cargo update -p libc` if you run into an error mentioning "cannot find function `preadv` in crate `libc`" or similar.

view details

push time in 8 days

PR merged oxidecomputer/propolis

Switch back to crates.io libc now that 0.2.102 has been published.

Remove TODO now that the crates.io libc has Illumos preadv/pwritev bindings.

NOTE: Run cargo update -p libc if you run into an error mentioning "cannot find function preadv in crate libc" or similar.

+1 -4

0 comment

1 changed file

luqmana

pr closed time in 8 days

PR opened oxidecomputer/propolis

Switch back to crates.io libc now that 0.2.102 has been published.

Remove TODO now that the crates.io libc has Illumos preadv/pwritev bindings.

NOTE: Run cargo update -p libc if you run into an error mentioning "cannot find function preadv in crate libc" or similar.

+1 -4

0 comment

1 changed file

pr created time in 8 days

create barnchoxidecomputer/propolis

branch : libc-bump

created branch time in 8 days

delete branch luqmana/libc

delete branch : ver-102

delete time in 9 days

PullRequestReviewEvent

Pull request review commentoxidecomputer/propolis

Integrate tokio runtime into propolis dispatcher

-use std::collections::VecDeque; use std::fs;-use std::io::{ErrorKind, Read, Result, Write};-use std::os::unix::io::AsRawFd;-use std::os::unix::net::{UnixListener, UnixStream};+use std::io::{ErrorKind, Result};+use std::os::unix::net::UnixListener as StdUnixListener; use std::path::Path; use std::sync::{Arc, Condvar, Mutex, Weak}; -use crate::dispatch::events::{Event, EventTarget, FdEvents, Resource, Token};-use crate::dispatch::DispCtx;-use crate::util::self_arc::*;+use crate::chardev::{Sink, Source};+use crate::dispatch::Dispatcher; -use super::{Sink, Source};+use tokio::net::unix::SocketAddr;+use tokio::net::{UnixListener, UnixStream};+use tokio::sync::Notify; -pub struct UDSock {-    socks: Mutex<Socks>,-    sink_driver: Mutex<SinkDriver>,-    source_driver: Mutex<SourceDriver>,-    cv: Condvar,-    sa_cell: SelfArcCell<Self>,-}-struct Socks {-    state: SockState,-    server: UnixListener,-    client: Option<UnixStream>,-    client_token_fd: Option<Token>,-}+#[derive(Default)] struct SinkDriver {-    sink: Option<Arc<dyn Sink<DispCtx>>>,-    buf: VecDeque<u8>,+    sink: Mutex<Option<Arc<dyn Sink>>>,+    notify: Notify, }-struct SourceDriver {-    source: Option<Arc<dyn Source<DispCtx>>>,-    buf: VecDeque<u8>,-}- impl SinkDriver {-    fn new(bufsz: usize) -> Self {-        assert!(bufsz > 0);-        Self { sink: None, buf: VecDeque::with_capacity(bufsz) }-    }-}-impl SourceDriver {-    fn new(bufsz: usize) -> Self {-        assert!(bufsz > 0);-        Self { source: None, buf: VecDeque::with_capacity(bufsz) }-    }-}--#[derive(Copy, Clone, PartialEq, Eq, Debug)]-enum BufState {-    Steady,-    ProcessCapable,-    ProcessRequired,-}--trait BufDriver {-    fn drive(&mut self);-    fn buffer_state(&self) -> BufState;--    fn drive_transfer(&mut self) -> Option<BufState> {-        let old = self.buffer_state();-        self.drive();-        let new = self.buffer_state();--        if new != old {-            Some(new)-        } else {-            None+    async fn drive(&self, client: &UnixStream) -> Result<()> {+        let sink = {+            let guard = self.sink.lock().unwrap();+            guard.as_ref().map(|s| Arc::clone(s))+        };+        if sink.is_none() {+            return Ok(());         }-    }-}--impl BufDriver for SinkDriver {-    fn drive(&mut self) {-        if let Some(sink) = self.sink.as_ref() {-            while let Some(b) = self.buf.pop_front() {-                if !sink.sink_write(b) {-                    self.buf.push_front(b);-                    break;+        let sink = sink.unwrap();+        loop {+            let mut buf = [0u8];+            loop {

Ah, that needs &mut to client, right.

pfmooney

comment created time in 9 days

Pull request review commentoxidecomputer/propolis

Integrate tokio runtime into propolis dispatcher

-use std::collections::VecDeque; use std::fs;-use std::io::{ErrorKind, Read, Result, Write};-use std::os::unix::io::AsRawFd;-use std::os::unix::net::{UnixListener, UnixStream};+use std::io::{ErrorKind, Result};+use std::os::unix::net::UnixListener as StdUnixListener; use std::path::Path; use std::sync::{Arc, Condvar, Mutex, Weak}; -use crate::dispatch::events::{Event, EventTarget, FdEvents, Resource, Token};-use crate::dispatch::DispCtx;-use crate::util::self_arc::*;+use crate::chardev::{Sink, Source};+use crate::dispatch::Dispatcher; -use super::{Sink, Source};+use tokio::net::unix::SocketAddr;+use tokio::net::{UnixListener, UnixStream};+use tokio::sync::Notify; -pub struct UDSock {-    socks: Mutex<Socks>,-    sink_driver: Mutex<SinkDriver>,-    source_driver: Mutex<SourceDriver>,-    cv: Condvar,-    sa_cell: SelfArcCell<Self>,-}-struct Socks {-    state: SockState,-    server: UnixListener,-    client: Option<UnixStream>,-    client_token_fd: Option<Token>,-}+#[derive(Default)] struct SinkDriver {-    sink: Option<Arc<dyn Sink<DispCtx>>>,-    buf: VecDeque<u8>,+    sink: Mutex<Option<Arc<dyn Sink>>>,+    notify: Notify, }-struct SourceDriver {-    source: Option<Arc<dyn Source<DispCtx>>>,-    buf: VecDeque<u8>,-}- impl SinkDriver {-    fn new(bufsz: usize) -> Self {-        assert!(bufsz > 0);-        Self { sink: None, buf: VecDeque::with_capacity(bufsz) }-    }-}-impl SourceDriver {-    fn new(bufsz: usize) -> Self {-        assert!(bufsz > 0);-        Self { source: None, buf: VecDeque::with_capacity(bufsz) }-    }-}--#[derive(Copy, Clone, PartialEq, Eq, Debug)]-enum BufState {-    Steady,-    ProcessCapable,-    ProcessRequired,-}--trait BufDriver {-    fn drive(&mut self);-    fn buffer_state(&self) -> BufState;--    fn drive_transfer(&mut self) -> Option<BufState> {-        let old = self.buffer_state();-        self.drive();-        let new = self.buffer_state();--        if new != old {-            Some(new)-        } else {-            None+    async fn drive(&self, client: &UnixStream) -> Result<()> {+        let sink = {+            let guard = self.sink.lock().unwrap();+            guard.as_ref().map(|s| Arc::clone(s))+        };+        if sink.is_none() {+            return Ok(());         }-    }-}--impl BufDriver for SinkDriver {-    fn drive(&mut self) {-        if let Some(sink) = self.sink.as_ref() {-            while let Some(b) = self.buf.pop_front() {-                if !sink.sink_write(b) {-                    self.buf.push_front(b);-                    break;+        let sink = sink.unwrap();+        loop {+            let mut buf = [0u8];+            loop {

Since we're already in an async context, can't we replace the nested loop with a AsyncReadExt::read call:

use tokio::io::AsyncReadExt;
...
if client.read(&mut buf).await? == 0 {
    return Ok(());
}
pfmooney

comment created time in 9 days

PullRequestReviewEvent

Pull request review commentoxidecomputer/propolis

Integrate tokio runtime into propolis dispatcher

 impl Dispatcher {     /// - `vcpu_fn`: A function, which will be invoked by the dispatcher,     /// to run the CPU. This function is responsible for yielding control     /// back to the dispatcher when requested.-    pub fn create(vm: &Arc<Machine>, vcpu_fn: VcpuRunFunc) -> Result<Self> {-        let mut workers = BTreeMap::new();-        let mctx = MachineCtx::new(vm);-        let event_dispatch = Arc::new(EventDispatch::new());-        let disp_inner = Arc::new(DispInner::default());--        // Spawn event dispatch thread.-        let evt_ctrl = WorkerCtrl::create(true);-        let mut evt_ctx = DispCtx::new(-            mctx.clone(),-            Arc::clone(&event_dispatch),-            Some(Arc::clone(&evt_ctrl)),-            Arc::clone(&disp_inner),-        );-        let evt_edisp = Arc::clone(&event_dispatch);-        let evt_join = Builder::new()-            .name("event-dispatch".to_string())-            .spawn(move || {-                if evt_ctx.check_yield() {-                    return;-                }-                events::event_loop(evt_edisp, &mut evt_ctx);-            })-            .unwrap();--        let wake_edisp = Arc::downgrade(&event_dispatch);-        workers.insert(-            Worker::Events,-            WorkerState {-                join: Some(evt_join),-                ctrl: evt_ctrl,-                wake: Some(Box::new(move |_ctx: &DispCtx| {-                    if let Some(edisp) = Weak::upgrade(&wake_edisp) {-                        edisp.notify();-                    }-                })),-            },-        );--        // Spawn vCPU threads-        for id in 0..mctx.max_cpus() {-            let ctrl = WorkerCtrl::create(true);-            let mut ctx = DispCtx::new(-                mctx.clone(),-                Arc::clone(&event_dispatch),-                Some(Arc::clone(&ctrl)),-                Arc::clone(&disp_inner),-            );-            let name = format!("vcpu-{}", id);-            let vcpu = VcpuHdl::new(vm.get_hdl(), id as i32);-            let hdl = Builder::new()-                .name(name)-                .spawn(move || {-                    // wait at dispatch hold point until start-                    if ctx.check_yield() {-                        return;-                    }--                    vcpu_fn(vcpu, &mut ctx);-                })-                .unwrap();--            workers.insert(-                Worker::Vcpu(id),-                WorkerState {-                    join: Some(hdl),-                    ctrl,-                    wake: Some(Box::new(move |ctx: &DispCtx| {-                        let _ = ctx.mctx.vcpu(id).barrier();-                    })),-                },-            );+    pub fn new(vm: &Arc<Machine>) -> Arc<Self> {+        let mut this = Arc::new(Self {+            async_disp: AsyncDispatch::new(),+            sync_disp: SyncDispatch::new(),+            machine: Arc::clone(vm),+            parent: ParentRef::new(),+            sa_cell: SelfArcCell::new(),+        });+        SelfArc::self_arc_init(&mut this);+        this+    }++    /// Perform final setup tasks on the dispatcher, including spawning of+    /// threads for running the instance vCPUs.+    pub(crate) fn finalize(+        &self,+        inst: &Arc<instance::Instance>,+        vcpu_fn: VcpuRunFunc,+    ) {+        self.parent.set(inst);+        let mctx = MachineCtx::new(&self.machine);+        for vcpu in mctx.vcpus() {+            let shared = SharedCtx::create(self);+            self.sync_disp.spawn_vcpu(shared, vcpu, vcpu_fn);         }--        // Release the events thread to run immediately-        workers.get(&Worker::Events).unwrap().ctrl.release();--        let this = Self {-            mctx,-            event_dispatch,-            workers: Mutex::new(workers),-            inner: disp_inner,-        };-        Ok(this)-    }--    /// Associates an instance with a dispatcher.-    ///-    /// # Panics-    ///-    /// Panics if the dispatcher has already been associated with-    /// an instance.-    pub(crate) fn assoc_instance(&self, inst: Weak<instance::Instance>) {-        let res = self.inner.inst.lock().unwrap().replace(inst);-        assert!(res.is_none());     }      /// Spawns a new dedicated worker thread named `name` which invokes     /// `func` on `data`.

Nit: update comment as we don't take a data anymore

pfmooney

comment created time in 9 days

PullRequestReviewEvent

issue commentrust-lang/rust

Tracking Issue for linking modifiers for native libraries

@petrochenkov Is there anything left before at least stabilizing the modifiers syntax (the native_link_modifiers feature)? If not I can help move that forward. It's definitely something we need to do before we stabilize any of the specific formats.

petrochenkov

comment created time in 14 days

pull request commentoxidecomputer/propolis

Integrate tokio runtime into propolis dispatcher

If this were gerrit, it would have only the latest commit, but I'm not versed enough in GH these days to know how to do that (or if it's even possible)

You could change the base branch from master to lifecycle and that should give just the changes on top.

pfmooney

comment created time in 15 days

delete branch oxidecomputer/propolis

delete branch : register-cleanup

delete time in 17 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha c77c43ad2ab3a0d39759061948cbf2234a04287e

NVMe: Use bitstruct for defining/manipulating controller "registers" and other structures. (#45) Pulled in the [`bitstruct`](https://github.com/dancrossnyc/rust-bitstruct) crate to define the controller register structures. Now instead of one-off fields we've added in an ad-hoc way, we can manipulate the corresponding registers in a way that follows directly from the spec. Makes manipulating the parts we need a lot more clear than twiddling the individual bits. Also fixed a few spots where I got it wrong previously. As a new example, the last commit also adds a first stab at handling shutdown notifications (aka linux won't hang a bit anymore during shutdown as the driver fruitlessly spun waiting for us to say we finished with shutdown processing).

view details

push time in 17 days

PR merged oxidecomputer/propolis

NVMe: Use bitstruct for defining/manipulating controller "registers" and other structures.

Pulled in the bitstruct crate to define the controller register structures. Now instead of one-off fields we've added in an ad-hoc way, we can manipulate the corresponding registers in a way that follows directly from the spec. Makes manipulating the parts we need a lot more clear than twiddling the individual bits. Also fixed a few spots where I got it wrong previously.

As a new example, the last commit also adds a first stab at handling shutdown notifications (aka linux won't hang a bit anymore during shutdown as the driver fruitlessly spun waiting for us to say we finished with shutdown processing).

+447 -128

1 comment

5 changed files

luqmana

pr closed time in 17 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha a3e6696605e7144c4bc898030b91e53cf3274540

Address review comments.

view details

push time in 17 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha 2d6ba31931a4ab07a6d7fadf85c8576f74ef5411

fmt

view details

push time in 17 days

PR opened oxidecomputer/propolis

NVMe: Use bitstruct for defining/manipulating controller "registers" and other structures.

Pulled in the bitstruct crate to define the controller register structures. Now instead of one-off fields we've added in an ad-hoc way, we can manipulate the corresponding registers in a way that follows directly from the spec. Makes manipulating the parts we need a lot more clear than twiddling the individual bits. Also fixed a few spots where I got it wrong previously.

As an example, the last commit also adds a first stab at handling shutdown notifications (aka linux won't hang a bit anymore during shutdown as the driver fruitlessly spun waiting for us to saw we finished with shutdown processing).

+451 -128

0 comment

5 changed files

pr created time in 17 days

create barnchoxidecomputer/propolis

branch : register-cleanup

created branch time in 17 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

delete branch oxidecomputer/propolis

delete branch : multi-mappings

delete time in 21 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha f8fd93c21326442bf8558623d08ba8f6774c18d8

Use preadv/pwritev to efficiently handle multiple mappings at once. (#40) Illumos, like linux, has vectored versions of pread/pwrite which we can use here instead of calling pread/pwrite in a loop. Using preadv/pwritev saves us a couple of roundtrips.

view details

push time in 21 days

PR merged oxidecomputer/propolis

Use preadv/pwritev to efficiently handle multiple mappings at once.

Illumos, like linux, has vectored versions of pread/pwrite which we can use here instead of calling pread/pwrite in a loop. Using preadv/pwritev saves us a couple of roundtrips.

In fact, with #39 we can see the improvement in latency just simply booting/shutting down a guest off an file-backed nvme storage:

<details> <summary>Before:</summary>

  write (us)                                            
           value  ------------- Distribution ------------- count    
               8 |                                         0        
              16 |@@@@@@@                                  25       
              32 |@@@@@@                                   21       
              64 |@@@@@@@@@@@@@@@@@@@@@@                   73       
             128 |@@@@                                     13       
             256 |@                                        3        
             512 |                                         0        

  read (us)                                            
           value  ------------- Distribution ------------- count    
               4 |                                         0        
               8 |@@@@@                                    197      
              16 |@@@@@@@@@@@                              405      
              32 |@@@@@@@@@@@@@@@@@@@                      713      
              64 |@@                                       96       
             128 |@@                                       77       
             256 |@                                        39       
             512 |                                         11       
            1024 |                                         1        
            2048 |                                         0        
            4096 |                                         2        
            8192 |                                         0

</details>

<details> <summary>After:</summary>

  write (us)                                            
           value  ------------- Distribution ------------- count    
               8 |                                         0        
              16 |@@@@@@@@@@@@@@@@@@@@@@@@@@               89       
              32 |@@@@@@                                   20       
              64 |@@@@@@@                                  24       
             128 |@                                        3        
             256 |                                         0        

  read (us)                                             
           value  ------------- Distribution ------------- count    
               4 |                                         0        
               8 |@@@@@@@@@@@@@@@                          570      
              16 |@@@@@@@@@@@@@@@@@                        659      
              32 |@@@@@                                    180      
              64 |@@@                                      105      
             128 |                                         19       
             256 |                                         6        
             512 |                                         1        
            1024 |                                         1        
            2048 |                                         0

</details>

+96 -38

0 comment

4 changed files

luqmana

pr closed time in 21 days

push eventoxidecomputer/propolis

Luqman Aden

commit sha 80a5c456c49962c444784c3096b9770c8705e57c

Use specific git rev for libc while we wait for a new release.

view details

push time in 21 days

PullRequestReviewEvent

Pull request review commentoxidecomputer/propolis

Overhaul VM lifecycle handling

 provider propolis {  	/* vm_exit(vcpuid, rip, code) */ 	probe vm_exit(uint32_t, uint64_t, uint32_t);++	/* virtio_vq_notify(virtio_dev_addr, virtqueue_id) */+	probe virtio_vq_notify(uint64_t, uint16_t);+	/* virtio_vq_pop(vq_addr, avail_idx) */+	probe virtio_vq_pop(uint64_t, uint16_t);+	/* virtio_vq_pop(vq_addr, used_idx, used_len) */
	/* virtio_vq_push(vq_addr, used_idx, used_len) */
pfmooney

comment created time in 21 days