profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/maxux/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Maxime Daniel maxux @threefoldtech Gentoo, Liège, Belgium https://www.maxux.net

Jumpscale/prefab9 1

was called cuisine before

maxux/boot 1

asm boot experiments

Jumpscale/jscockpit 0

private neutral management environment for g8 based clouds

issue commentthreefoldtech/zos

qsfs mounted on /var/lib/docker not working as expected

https://github.com/threefoldtech/0-db-fs/issues/29 Again it's not zdbfs issue, but any fuse fs not supporting this.

sameh-farouk

comment created time in 7 days

issue commentthreefoldtech/0-db-fs

overlayfs: investigate if overlayfs can use zdbfs as layer

It's not supported by FUSE, not even related to zdbfs. No. Nothing possible yet.

maxux

comment created time in 8 days

issue commentthreefoldtech/0-db-fs

QSFS mounted not working as expected

Please show output of cat /proc/mounts

mhost39

comment created time in 9 days

issue commentthreefoldtech/zos

qsfs mounted on /var/lib/docker not working as expected

statfs is implemented.

Docker use special driver for each filesystem. It make sens for me that it doesn't support fuse stuff. Why would you do that anyway ?!

It could maybe works by forcing some legacy driver ? Anyway this is not a zdbfs issue but docker issue.

sameh-farouk

comment created time in 9 days

push eventthreefoldtech/0-bootstrap

Maxime Daniel

commit sha 5b8e12a26ad2ee05209ee8a6dc81083708b5883a

generate: make farm instruction more generic

view details

push time in 13 days

push eventthreefoldtech/0-bootstrap

Maxime Daniel

commit sha ba831875909597bbc7fcf8fe8e640b631efbaf38

bootstrap: add kernel extra argument for usb

view details

Maxime Daniel

commit sha 0d62f15094d93f5f4fd79cb32962ab5b157932f0

bootstrap: add kernel extra argument for uefimg

view details

Maxime Daniel

commit sha 91459759c57684ee9ff71ed0b645bd76158c7ae2

bootstrap: fix kernel argument for usb

view details

Maxime Daniel

commit sha 008acfe87f57d969354b9fc47395009e1331728d

Merge branch 'development' into development-v3

view details

Maxime Daniel

commit sha 59cf4f2f8355b840d5f0abd141b47ca4cfd721d4

generate: replace farm documentation url

view details

Maxime Daniel

commit sha a317dae89c8733595a2ec5c9e727bd7438b2bb65

generate: enable testnet

view details

push time in 13 days

push eventthreefoldtech/0-bootstrap

Maxime Daniel

commit sha 91459759c57684ee9ff71ed0b645bd76158c7ae2

bootstrap: fix kernel argument for usb

view details

push time in 15 days

push eventthreefoldtech/0-bootstrap

Maxime Daniel

commit sha ba831875909597bbc7fcf8fe8e640b631efbaf38

bootstrap: add kernel extra argument for usb

view details

Maxime Daniel

commit sha 0d62f15094d93f5f4fd79cb32962ab5b157932f0

bootstrap: add kernel extra argument for uefimg

view details

push time in 15 days

issue closedthreefoldtech/0-flist

improve zero-hub with new flist implementation

Description

The 0-hub code, in production, use python and jumpscale to handle the complete stack of flist management (creation, modification, merging, listing, ...)

Some random crashes occurs related to capnp and python, the hub is not responding and memory leaks have been detected after long time running.

The fact is, the implementation in jumpscale is not scalable at all when running for days or weeks and is not performant enough to be placed on concurrent requests.

Goal

Make a more stable (code, memory usage, cpu usage, code consistancy) flist management.

The idea is building an external tool dedicated to flist management. This command line tool will be invoked from the python, as new process. This fix the memory leak by running a short-live process each time. Writing it in a compiled language will fix the time and performance needed to perform actions.

Requirement

Building an flist is basically a mix of:

  • capnp structure
  • rocksdb database operations
  • system calls for directory transversal and walking operation

Python is not the idea language to build this kind of stuff.

Capnp and Rocksdb are both written in C++ and the interface they provide for C++ is really well-made and easy. The best approach would be C++. Any other language always use binding on top of the C++ library them self, which make any language dependent of the C++ code/library anyway (that true for Python and Go, at least).

For a personal issue, I (Maxux) can't do C++ properly. Rocksdb provide upstream C interface. One C port of Capnp exists. The binary can be built in static mode without dependencies at the end. Using another language don't fix anything since you always need platform specific external libraries.

Deliverables

A command line tool, which can easily produce JSON response, doing:

  • [x] List flist contents
  • [x] Create a flist
  • [x] Merge multiple flists together (no need to merge files, when 2 files same location -> die)
  • [x] Checking backend integrity

Port the 0-hub to use the new tool:

  • [x] Port staging hub to use flister to list flist contents
  • [x] Port staging hub to use flister to create and upload flists
  • [x] Port staging hub to use flister to convert dockers
  • [x] Port staging hub to use flister to upload existing flists
  • [x] Port staging hub to use flister to merge multiple flists
  • [x] Port production hub to use flister (list, create, docker)
  • [x] Port production hub to use flister (merge, existing flist)
  • [ ] Benchmark and comparaison with current implementation

And of course:

  • [ ] Writing test-case
  • [ ] Write exact and precise documentation about how flist works internally

Stakeholders

  • @despiegk

closed time in 21 days

despiegk

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha de00145b51ecea0b711de26908ec7dde47928218

wireguard: disable kernel patching

view details

Maxime Daniel

commit sha 16f18b3df6c73399e7c6d89eda524e669725f911

firmware: bump to 20210716

view details

Maxime Daniel

commit sha 8c40167c2be6b600203f5253b9ec234e49e9d581

kernel: bump to 5.10.55

view details

Maxime Daniel

commit sha 37517d33da2dea7e0474c59aa2b710ee802a9fd0

initramfs: avoid include lib32 for libresolv

view details

Maxime Daniel

commit sha 5d4c045dbce62be3309c26e64592c699cf1e86dc

autobuild: stick rust version to 1.46.0 for zinit

view details

Maxime Daniel

commit sha 89aefb247867258a3e624be3f794a9b0319e8954

Merge branch 'development-zos-v2-kernel-5.10.55' into development-zos-v2-logs

view details

push time in 22 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 8bad1badb907f63f96d4c0886f5d0d21de02b348

zos: switch to v2-logs branch

view details

push time in 22 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha b5931b6197179919b195dd25682fe7149cdc527d

libzdb: fix printf compilation type warning

view details

push time in 23 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 612f27a544c65d0a338a93f70177c151f4eec841

code: fix compilation warning on some type

view details

push time in 23 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 38c9009d1173bfce0c0627dfb1000b12fb04796c

autobuild: stick rust version to 1.46.0 for zinit

view details

Maxime Daniel

commit sha 0e60ea67bd285929b156f71bc3d9ec6dc867c9bd

initramfs: avoid include lib32 for libresolv

view details

push time in 23 days

push eventmaxux/gps-server

Maxime Daniel

commit sha f9c70fc5a3c9aa04f2e92c0105c5de16a75af65c

support new ios push system

view details

push time in 23 days

create barnchthreefoldtech/0-initramfs

branch : development-zos-v2-logs

created branch time in 23 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha e301d326668b68723dcb8f297e0935c183277d21

crc32: fix segmentation fault on small input

view details

push time in 23 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 58213dcb7e0a913e3914c1de43c47359b9a4bc25

libzdb: remove x86intrin still included by default

view details

push time in 23 days

create barnchthreefoldtech/0-db

branch : development-v2-crc32

created branch time in 23 days

issue openedthreefoldtech/0-db

architechture: support arm64

created time in 23 days

push eventthreefoldtech/0-bootstrap

Maxime Daniel

commit sha 40a233e0fcbd9c263faae39c1d2d92a104bc2728

kernel: use v3 kernel by default

view details

push time in 24 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha db8820c82a618ae56d64653cd0134f8a32a737c1

zos: use main branch for v3

view details

push time in a month

push eventfreeflowuniverse/vkube

Maxime Daniel

commit sha a57721b989c1e6dd8560192eec86980c37813407

json: use new sum type from v for template

view details

push time in a month

push eventmaxux/hookhub

Maxime Daniel

commit sha a14a7e0a5e0d087b532d96d336e1e2b61dccc1f3

make fail

view details

push time in a month

push eventthreefoldtech/0-flist

Maxime Daniel

commit sha 0d32ce1b71db00961db06e57d044ce59e48a9af3

zflist: hub: return an error on api call failure

view details

push time in a month

push eventmaxux/hookhub

Maxime Daniel

commit sha a0651a8d78bb747de719f0855338b2360d21849b

update current dir target

view details

push time in a month

push eventmaxux/hookhub

Maxime Daniel

commit sha f79e1da266270bc899ebeb72bd36512e817cbc2c

updte token

view details

push time in a month

push eventmaxux/hookhub

Maxime Daniel

commit sha 3affe5cb7113bf7ae532663437cc0b138e25dd16

add hook test

view details

push time in a month