profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jart/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

grpc/grpc-web 6013

gRPC for Web Clients

defunkt/coffee-mode 572

Emacs Major Mode for CoffeeScript

jart/bestline 131

ANSI Standard X3.64 Teletypewriter Command Session Library

jart/asterisk-voicechanger 23

Asterisk module for adjusting pitch of voices

jart/coding-interview-university 6

A complete computer science study plan to become a software engineer.

jart/asterisk-parrot 4

Annoying robot that repeats what you say on the phone

jart/coffee-mode 2

Emacs Major Mode for CoffeeScript

jart/chessy 1

A happy toy chess engine, just for fun :) If you want a proper chess engine, check out Stockfish.

jart/closurecheatsheet 1

Content for http://www.closurecheatsheet.com

jart/aggregate-framework 0

Aggregate Framework是基于DDD和CQRS思想而开发的一个领域驱动框架。其主要目标是方便开发人员运用DDD和CQRS思想来构建复杂的、可扩展的应用系统。该框架提供了最核心的构建块的实现,比如Aggregate、Repository和Event。此外,该框架支持与Spring集成,提供使用annotation的方式让开发人员方便地注册事件及定义事件处理,使用Spring事务管理器管理事务时,支持Unit Of Work模式存储数据。

issue commentjart/cosmopolitan

Compiling Python (2.7.18/3.6.14)

Oh wow we're the fastest at pickling.

  1. Could you run the benchmark suite again under MODE=optlinux?
  2. Could we add a benchmark for audioop.add() so we can lay claim to 10x improvements?

The new optlinux mode should give you the boost you were expecting, because it enables red zone and disables frame pointers. For example, here's Musl Libc:

$ time python3.6 -m test.test_pickle
real    0m2.730s
user    0m2.655s
sys     0m0.074s

Here's MODE=opt:

$ time o/opt/third_party/python/pythontester.com -m test.test_pickle
real    0m2.535s
user    0m2.455s
sys     0m0.080s

Here's MODE=optlinux:

$ time o/optlinux/third_party/python/pythontester.com -m test.test_pickle
real    0m2.353s
user    0m2.270s
sys     0m0.082s

Please be advised that disabling frame pointers isn't worth the performance gain in practice, since it takes away things like backtraces. Speaking of which, @pkulchenko we now have more solid code for stack unwinding and such, which shouldn't ever crash. The recommended algorithm is this:

  size_t gi = __garbage->i;
  const struct StackFrame *frame = __builtin_frame_address(0);
  for (frame = bp; frame; frame = frame->next) {
    if (!IsValidStackFramePointer(frame)) {
      __printf("%p corrupt frame pointer\n", frame);
      break;
    }
    addr = frame->addr;
    if (addr == (intptr_t)&__gc) {
      do --gi;
      while ((addr = garbage->p[gi].ret) == (intptr_t)&__gc);
    }
    __printf("%p %p %s\n", frame, addr, __get_symbol_by_addr(addr));
  }

If our Python takes 10% longer to startup then that's impressive if you consider APE binaries are shell scripts and it has to DEFLATE all .pyc / .py files it loads from the ZIP structure. APE loading is going to be more important as apps grow larger, because bloated Python code spends most of its time crawling the filesystem if you strace it. With the ZIP central directory we can make that an effectively O(1) pure userspace operation. Right now the LIBC_ZIPOS code isn't as optimized as it can be, but to give you a basic idea of how important a boost this can be, consider this benchmark:

 *     stat syscall        l:       892𝑐       288𝑛𝑠   m:     1,024𝑐       331𝑛𝑠
 *     stat() fs           l:       915𝑐       296𝑛𝑠   m:       981𝑐       317𝑛𝑠
 *     stat() zipos        l:       135𝑐        44𝑛𝑠   m:       169𝑐        55𝑛𝑠
     1,732⏰     1,109⏳   1,100k       0iop o/rel/test/libc/calls/stat_test.com -b

This is basically what Java does because enterprise apps are always humongous, so having .jar files designed as .zip files was a really brilliant move on Java's part that allowed it to meet the requirements of big companies. So in many ways you could think of Actually Portable Python as an Enterprise Python even though we're a scrappy open source project. There should ideally be some way for the benchmarks to capture that, once we optimize it more. I also spotted an issue that helps us save 100 cycles on read() and write() function calls. So i/o should be a little bit better now.

PGO is something I want. I looked into doing it and came to the conclusion that I'd prefer to kick that can down the road. We can do what PGO does manually for the time being, by using the COUNTBRANCH() macro. You can wrap any expression with that, and it'll cause the linker to generate code that prints the percentage of times the branch was taken at the end of the program. If it ends up being 99% yes or 1% no then you would then add LIKELY() or UNLIKELY() macros around the expression. It makes a measurable difference because the Python codebase is written in such a way that error handling code usually blocks the critical path, and everything counts in small amounts.

ahgamut

comment created time in 7 days

push eventjart/cosmopolitan

Justine Tunney

commit sha 67b5200a0b8b42f1b06d43ac91161e9c19853e9a

Add MODE=optlinux build mode (#141)

view details

push time in 7 days

issue closedjart/cosmopolitan

Cannot build cosmopolitan.a

I can no longer build cosmopolitan.a using WSL on the latest commits, I get the error: Timed out after 64 seconds!

closed time in 8 days

Keithcat1

issue commentjart/cosmopolitan

Cannot build cosmopolitan.a

OK I believe this is fixed. If it isn't, let us know and I'll reopen!

Keithcat1

comment created time in 8 days

push eventjart/cosmopolitan

Justine Tunney

commit sha 226aaf354786505e05283adfd52b43b668807336

Improve memory safety This commit makes numerous refinements to cosmopolitan memory handling. The default stack size has been reduced from 2mb to 128kb. A new macro is now provided so you can easily reconfigure the stack size to be any value you want. Work around the breaking change by adding to your main: STATIC_STACK_SIZE(0x00200000); // 2mb stack If you're not sure how much stack you need, then you can use: STATIC_YOINK("stack_usage_logging"); After which you can `sort -nr o/$MODE/stack.log`. Based on the unit test suite, nothing in the Cosmopolitan repository (except for Python) needs a stack size greater than 30kb. There are also new macros for detecting the size and address of the stack at runtime, e.g. GetStackAddr(). We also now support sigaltstack() so if you want to see nice looking crash reports whenever a stack overflow happens, you can put this in main(): ShowCrashReports(); Under `make MODE=dbg` and `make MODE=asan` the unit testing framework will now automatically print backtraces of memory allocations when things like memory leaks happen. Bugs are now fixed in ASAN global variable overrun detection. The memtrack and asan runtimes also handle edge cases now. The new tools helped to identify a few memory leaks, which are fixed by this change. This change should fix an issue reported in #288 with ARG_MAX limits. Fixing this doubled the performance of MKDEPS.COM and AR.COM yet again.

view details

push time in 8 days

issue commentjart/cosmopolitan

Cannot build cosmopolitan.a

It looks like the command line for ar.com to build cosmopolitan.a is now 509kb. So chances are we're brushing up against whatever ARG_MAX is under WSL. This can be fixed right now by locally modifying to say:

o/$(MODE)/cosmopolitan.a: $(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_A_OBJS))
        $(file >$@.args) $(foreach x,$^,$(file >>$@.args,$(x)))
        @$(COMPILE) -AARCHIVE -T$@ $(AR) $(ARFLAGS) $@ @$@.args

I'll do that in a follow-up change shortly.

Keithcat1

comment created time in 8 days

issue commentjart/cosmopolitan

redbean: adding ZIP64 support.

redbean should have zip64 support but it's possible that it regressed since there needs to be unit tests for it. I'll write some as soon as I get a chance, because zip64 is something I absolutely intend to support.

joshnecanter

comment created time in 8 days

push eventjart/cosmopolitan

Florian Lemaitre

commit sha a0b39f886cb9480bfe7b4707fafb06aeb0cec464

[WIP] Threading (#282) * Thread creation * Proper thread creation and exit * Join/Detach protocol * Added semaphore with futex (hopefully fast)

view details

push time in 8 days

PR merged jart/cosmopolitan

[WIP] Threading

This is the PR associated to #277.

The goal is to implement the pthread interface (or at least similar) for cosmopolitan. The key observation that makes it possible to consider such a daunting task is the following: It seems that there are really few part of the code that is OS dependent, namely OS thread creation, OS thread destruction, futex, and TLS.

The complex parts are built on top of those, and on top of atomics which are OS independent.

The goal of this PR is to focus on the portable abstraction, and make it works on Linux (because that's the platform I know).

Status:

  • [x] thread creation
  • [x] thread exit
  • [x] detach
  • [x] join
  • [ ] kill
  • [ ] cancel
  • [ ] mutex
  • [ ] rw_lock
  • [ ] semaphore
  • [ ] condition variable
  • [x] TLS setup
  • [ ] TLS allocation
  • [ ] TLS for main thread
  • [ ] dynamic TLS
  • [x] custom stack size
  • [ ] custom stack
  • [ ] thread-safe libc
  • [ ] actual pthread interface
  • [ ] anything I forgot
+792 -12

3 comments

30 changed files

lemaitre

pr closed time in 8 days

PullRequestReviewEvent

pull request commentjart/cosmopolitan

[WIP] Threading

Thanks for adding futexes!

I'd like to move forward with merging since even though it's a work in progress it's a step in the right direction. I can provide some assistance and collaboration on this effort. You may see me making follow up changes to what you've contributed. Simple things like polyfills, style, and integration so that your project has a good home in this codebase. Think of it as me volunteering for your project. However I want it to be your project and I want to be able to tell anyone who asks about Cosmopolitan Threads that you're leading effort.

lemaitre

comment created time in 8 days

pull request commentjart/cosmopolitan

Fix build from docker build with alpine from macOS

Thank you for spotting this and taking the time to send a PR.

Also, welcome as a new contributor. This change looks good so far. We need to go through one quick hurdle before I can merge. Could you please email Justine Tunney jtunney@gmail.com and say that you intend to assign her the copyright to the changes you contribute to Cosmopolitan? See CONTRIBUTING.md for further details. We make the ask because it helps us ship the tiniest most legally permissive prim and proper binaries. You also only need to do it one time, so any future changes you send us can be quickly merged.

Thanks!

Et7f3

comment created time in 8 days

push eventjart/cosmopolitan

Gautham

commit sha d852640a1e1067b34cf8312c0107592814601f6a

Add Python ftrace contextmanager (#285)

view details

push time in 8 days

PR merged jart/cosmopolitan

contextmanager for ftrace

Python likes to make a bunch of function calls for anything, so ftrace-ing a Python script from start to end produces a TON of info. cosmo.ftrace() + cosmo.exit1() are nice, but I have to still structure my Python script to exit right after a particular line.

As mentioned here, This commit adds a simple contextmanager wrapper to cosmo.ftrace() so instead I can do things like this:

import cosmo

with cosmo.ftrace(): # everything inside this will be logged to stderr
    a = 2 + 3

# logging disabled, do other things without having to terminate
print("a =", a)

the above snippet produces 32 lines of ftrace in stderr, which is much easier to examine:

+       sigprocmask 15481145
+ meth_dealloc 11490
+ PyFrame_BlockSetup 29
+ object_dealloc 389
+ PyObject_Free 61
+ _PyObject_Free.isra.0 14
+ PyDict_SetItem 372
+ insertdict 72
+   lookdict_unicode_nodummy 141
+   dictresize 133
+     new_keys_object 198
+       PyObject_Malloc 99
+       PyMem_Malloc 14
+       _PyObject_Alloc.isra.0 14
+         dlmalloc 54
+           dlmalloc_impl 14
+             tmalloc_large 78
+       memset 372
+       memset_avx 44
+       bzero 43
+       bzero_avx 38
+     PyObject_Free 197
+     _PyObject_Free.isra.0 10
+   find_empty_slot.isra.0 448
+ PyFrame_BlockPop 96
+ PyObject_CallFunctionObjArgs 252
+   objargs_mkstack.constprop.0 31
+   _PyObject_FastCallDict 55
+     _PyCFunction_FastCallDict 40
+       PyTuple_New 116
+       TracerObject_exit 150
a =  5
+82 -20

2 comments

3 changed files

ahgamut

pr closed time in 8 days

Pull request review commentjart/cosmopolitan

contextmanager for ftrace

 cosmo_exit1(PyObject *self, PyObject *args)     _Exit(1); } +static bool ftrace_installed = 0;++typedef struct {+    PyObject_HEAD+} TracerObject;++static int TracerObject_init(PyObject* self, PyObject *args, PyObject *kwargs)+{+    if (!ftrace_installed) {+        ftrace_install();+        ftrace_installed = 1;+        ftrace_enabled = 0;+    }+    return 0;+}++static PyObject* TracerObject_enter(PyObject *self, PyObject *Py_UNUSED(ignored))+{+    ftrace_enabled = 1;

I'm assuming you chose not to use g_ftrace because some libraries routines might unintentionally turn it back on? We really need a function to remove __hook() after it's been installed, to rewrite the binary in memory to restore the NOPs. I'll probably do that as a follow-up soon. Please be warned that until we fix that, installing ftrace means that, even after you turn it off, there's going to be a nontrivial performance segregation throughout the remaining lifecycle of the process.

ahgamut

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentjart/cosmopolitan

contextmanager for ftrace

@pkulchenko Absolutely. Lua doesn't require parens in function calls, and it lets you pass code to a function, so we could probably define the syntax as follows:

ftrace {
  some_code()
}
ahgamut

comment created time in 8 days

push eventjart/sectorlisp

Justine Tunney

commit sha 4233210a863dde3619426608c5eb7e81addbef4c

Remove old code and update documentation

view details

push time in 12 days

push eventjart/sectorlisp

Justine Tunney

commit sha e994e80d0b89b2f4e4c69fcc1267ca70af0a0309

Remove old code and update documentation

view details

push time in 12 days

push eventjart/sectorlisp

Justine Tunney

commit sha 2ef4bdea598bbf303e91fb5a96106c052206d79b

Remove old code and update documentation

view details

push time in 12 days

push eventjart/cosmopolitan

Justine Tunney

commit sha 425a57080d4c8fa32b6e080edf867757eadd2905

Fix minor issues

view details

push time in 12 days

push eventjart/bestline

Justine Tunney

commit sha 1291efa0ff75007021f44bb04bf98375d1f8db0a

Fix bug with pasting

view details

push time in 13 days

push eventjart/cosmopolitan

Justine Tunney

commit sha bba144246a2eb90d4c6ed51701ea3398f2706761

Improve Das Blinkenlights It's now possible to scroll panels is the TUI while the display is blocked on input. INT 16h now translates UTF-8 to CP-437 and displays unmappable characters using a lambda symbol. Bracketed paste mode guards will also be filtered out.

view details

push time in 13 days

push eventjart/cosmopolitan

Justine Tunney

commit sha 7061c79c22cf31769e2480cfe86045b597ad6206

Make fixes, improvements, and chibicc python bindings - python now mixes audio 10x faster - python octal notation is restored - chibicc now builds code 3x faster - chibicc now has help documentation - chibicc can now generate basic python bindings - linenoise now supports some paredit-like features See #141

view details

push time in 13 days

issue commentjart/cosmopolitan

Compiling Python (2.7.18/3.6.14)

No only the compiler would run as a subprocess. (At least until we can retool chibicc to not rely on _exit() to free() its memory). It would use the same ABI for all platforms. One compilation. Otherwise it's not build once run anywhere.

ahgamut

comment created time in 14 days

issue commentjart/cosmopolitan

cross platform GUI / 2D software rendering

All you have to do is put the platform local binaries inside the zip structure of the binary. You have to build the binary with a STATIC_YOINK("zip_uri_support"); statement. Then you can say if (IsWindows()) open("/zip/opengl-driver.exe", O_RDONLY); etc. etc.to move the binary to a local file system folder. Then you can call pipe(), fork(), and exec() to communicate with it as a subprocess.

nikhedonia

comment created time in 14 days

push eventjart/bestline

Justine Tunney

commit sha bdbf480a400970092a76aa65435b8ccac68fa4c3

Add paren awareness and quoted insert

view details

push time in 14 days

pull request commentjart/cosmopolitan

[WIP] Threading

Nsync was handwritten by the guy who built all the locking infrastructure at Google. Before that he invented Altavista. Before that he invented Bzip2. This guy really doesn't evangelize his work so not many people know about it.

OS dependencies are toilsome but it isn't something I fear, since that's our bread and butter. Cosmopolitan libraries let users to do things like copy a file twice as fast as their system's cp command because we did all the hard work of figuring out and abstracting the optimum systems integrations. On Linux I'm reasonably certain that for mutexes that entails futex. Even if we only support just Linux threads for the present we can iterate on it and improve. I just want to be sure that whatever we do choose to do, is the best.

lemaitre

comment created time in 17 days

push eventjart/cosmopolitan

Justine Tunney

commit sha 28997f3acb4b00ff4fc83e5e5ba54c0900c5da87

Make mkdeps.com go faster This program usually runs once at the begininng of each GNU Make invocation. It generates an o//depend file with 170,000 lines of Makefile code to define source -> headers relationships. This change makes that take 650 milliseconds rather than 1,100ms by improving the performance of strstr(), using longsort(), plus migrating to the new append library.

view details

push time in 17 days