profile
viewpoint

edahlseng/eff-javascript 9

An extensible effect monad

edahlseng/atom-kak-mode 0

Kakoune text editor emulator for Atom

edahlseng/automerge 0

A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.

edahlseng/bugwarrior 0

Pull github, bitbucket, and trac issues into taskwarrior

edahlseng/cipm 0

standalone ci-oriented package installer for npm projects

edahlseng/circleci-images 0

Scripts to generate images for building projects on CircleCI 2.0

pull request commentautomerge/automerge

Add cevitxe to readme under 'Sending and receiving changes'

Great, thank you!

HerbCaudill

comment created time in 14 hours

PR merged automerge/automerge

Add cevitxe to readme under 'Sending and receiving changes'

see https://github.com/automerge/automerge/pull/284#issuecomment-734817546

+10 -5

0 comment

1 changed file

HerbCaudill

pr closed time in 14 hours

push eventautomerge/automerge

HerbCaudill

commit sha 089e84fc6103f01cf9e97135ac15a974e951ab5b

add cevitxe to readme under 'Sending and receiving changes'

view details

Martin Kleppmann

commit sha 08f884026d3e73d3c50ae7fbad212d3fe74da930

Revert unrelated changes

view details

Martin Kleppmann

commit sha fed28a58ceebe9ea6e3469e496c1de4ed942e6cc

Merge pull request #289 from HerbCaudill/readme-cevitxe add cevitxe to readme under 'Sending and receiving changes'

view details

push time in 14 hours

push eventautomerge/automerge

Martin Kleppmann

commit sha 2009189a9ff1f8659e720c0403e9a675b258b3f6

Remove transit dependency

view details

Martin Kleppmann

commit sha d70083ab43a0f91232836ff5f7abd8acf6cc79f9

Make BackendDoc.applyChanges match Backend.applyChanges

view details

Martin Kleppmann

commit sha f3743200b5f3c68ee4dc1c60004e17d81f548f25

Remove the undo/redo feature It's too much work to support for now, and it was never really well thought-out. Better to remove it for now, and bring back in well-conceived form in the future.

view details

Martin Kleppmann

commit sha f4102e6d3acf3a3ae72ed42b11428246363b2c21

Remove Backend.getChangesForActor()

view details

push time in 15 hours

pull request commentautomerge/automerge

Allow hooking into patch application

Thanks @ept that is very useful for my use case. It will simplify the code receiving remote changes and applying them both to the Automerge document and the user facing representation of the document.

ept

comment created time in 17 hours

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

@HerbCaudill By the way, I just realised that Cevitxe isn't listed as one of the data sync layers in the README. I've put a brief one-sentence description in here: 6679330 — does this look ok? Feel free to change it to be better.

I'd like to tweak the wording a bit - why don't I just make a separate PR, since it's not really related to this one.

josharian

comment created time in 19 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I'm with @HerbCaudill here. I think this kind of options hash is common in JavaScript, since it gives us something a bit like named parameters in the absence of language support, and it allows us to add further options in the future without breaking existing code. Moreover, lots of existing functions in the Automerge API (init, from, load, change, etc.) already take an options hash. I don't see a problem with passing a function as part of an options hash: in JS, functions are values like any other.

ept

comment created time in 19 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I agree that readability would improve on the call side. But I think that's caused by missing labeled arguments in JavaScript which we should not try to fix.

When just reading function applyChanges(doc, changes, options = {}) I would expect to pass a document, some changes and a map which only contains key value pair, which help configuring the change method. I would not expect to pass references into options which provide me with the resulting patches.

On the other hand when reading function applyChanges(doc, changes, patchCallback) it is clear to me that I can pass a document, changes and a callback which provides some kind of patches. (It is also future prove to just add another parameter)

This is not really a strong opinion, so I am happy to go with options map as well :)

ept

comment created time in 19 hours

pull request commentautomerge/automerge

Allow hooking into patch application

On second thought I do see that this is a little spooky since you're getting a new Automerge document every time.

const state = Automerge.init({}, { patchCallback }) 
const state1 = Automerge.change(state, s => s.foo = 42)
const state2 = Automerge.change(state, s => s.boo = 'pizza') // maybe it is weird for patchCallback to fire here
ept

comment created time in 20 hours

pull request commentautomerge/automerge

Allow hooking into patch application

Overall this seems like a step in the right direction. It always seemed odd to me that you could watch for changes on a DocSet but not on an individual document.

An alternative API would be to pass in a callback function once when initialising the document, and automatically calling it on all subsequent updates. But I thought that had a bit of a "spooky action at a distance" feel, while the callback on every change/applyChanges call was more explicit. Though it could also be annoying for app developers to have to add the option to every single Automerge.change call, which might be scattered around an application. Any thoughts on this?

I'd imagine most developers would find the idea of a change listener to be totally natural, and not spooky at all.

Is there a reason why we can't have both?

const patchCallback = patch => alert(patch)
const changeFn = doc => { doc.foo = 42 }

// this works
const state = Automerge.init({}, { patchCallback }) // patchCallback will fire on every change
const newState = Automerge.change(state, changeFn)

// this works too
const state = Automerge.init({})
const newState = Automerge.change(state, changeFn, { patchCallback }) // patchCallback only fires on this change
ept

comment created time in 20 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I prefer the interface @ept suggests here - the options map makes it easier to add the change API further in the future. It also makes the code calling this function more readable.

ept

comment created time in 21 hours

issue commentautomerge/automerge

Python Implementation of the Automerge Server?

Is it completely automerge agnostic? I would expect the interest in a topic resided in the content of the automerge CRDT messages?

No, the "topic" in this sense generally represents one Automerge document or a set of Automerge documents (e.g. a DocSet, or what Hypermerge and Cevitxe both refer to as a repository).

echarles

comment created time in 21 hours

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Thanks for doing clean-up. :)

josharian

comment created time in a day

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

@HerbCaudill By the way, I just realised that Cevitxe isn't listed as one of the data sync layers in the README. I've put a brief one-sentence description in here: 66793306c9a6d0536c086f56de4bc7d1ef8e8c84 — does this look ok? Feel free to change it to be better.

josharian

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

why is there an options map needed. Could we just pass the callback directly?

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function docFromChanges(options, changes) {   const doc = init(options)   const [state, _] = Backend.applyChanges(Backend.init(), changes)   const patch = Backend.getPatch(state)+  if (options && options.patchCallback) options.patchCallback(Object.assign({}, patch))

Same as above

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {   const oldState = Frontend.getBackendState(doc)   const [newState, patch] = Backend.applyChanges(oldState, changes)+  if (options.patchCallback) options.patchCallback(Object.assign({}, patch))

I would prefer a operate line + indentation, which improves readability, safety and debuggablility.

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function makeChange(doc, requestType, context, options) {    if (doc[OPTIONS].backend) {     const [backendState, patch] = doc[OPTIONS].backend.applyLocalChange(state.backendState, request)+    if (options && options.patchCallback) options.patchCallback(patch)

I would prefer a operate line + indentation, which improves readability, safety and debuggablility.

ept

comment created time in 2 days

push eventautomerge/automerge

Martin Kleppmann

commit sha 7942fed63a152949f78be5818741cf7d5e381941

Add patchCallback to Automerge.load as well

view details

push time in 2 days

PR opened automerge/automerge

Allow hooking into patch application

Here is a simple but potentially very important feature: allowing applications to hook into Automerge's process for applying patches to a document.

The use case is when you have some data outside of Automerge that you want to keep consistent with the state of an Automerge document. This could be a user interface (especially a complicated widget like a text editor), or a cache, a search index, or suchlike. Such an application needs a mechanism to get notified when an Automerge document changes, so that it can apply the appropriate update to the external state.

At the moment, Automerge only really supports the React-style approach where you pass the entire document to the render function, and the renderer works out which bits of the document have actually changed, and updates those, while leaving the unchanged parts untouched. But in other circumstances, it would be really useful to have a description of the change that occurred, not just the before-and-after document. Moreover, this process should be the same, regardless of whether a change is generated by the local user or was received over the network from a remote user.

Automerge's existing concept of a patch is probably the best form we have for describing what change occurred. It's easier for applications to consume than the raw change format because it includes more context, and e.g. for list updates it includes the index of the list position being changed, not just the list element ID. This patch format is currently used for updates sent from the backend to the frontend, and used by the frontend to update the document.

This PR adds the ability for applications to get a callback for every patch that is applied to a document. When making a local change, you can add a callback like this:

newState = Automerge.change(oldState, {patchCallback: (patch) => {...}}, doc => {
  // mutate doc as usual
})

And when applying a remote change, you can add a callback like this:

newState = Automerge.applyChanges(oldState, changes, {patchCallback: (patch) => {...}})

The patch format is the same in both cases. The patch format used by the currently released version of Automerge is documented in INTERNALS.md (in the "Frontend-backend protocol" section). This format has changed considerably on the performance branch, and the new format is documented in BINARY_FORMAT.md on that branch. The new format is intended to be easier to consume.

Note that this API is not intended to allow applications to modify the patches before they get applied. The patch callback should treat the patch as read-only.

An alternative API would be to pass in a callback function once when initialising the document, and automatically calling it on all subsequent updates. But I thought that had a bit of a "spooky action at a distance" feel, while the callback on every change/applyChanges call was more explicit. Though it could also be annoying for app developers to have to add the option to every single Automerge.change call, which might be scattered around an application. Any thoughts on this?

This feature is especially for @cklokmose, but I welcome feedback from others as well. @HerbCaudill what do you think?

+60 -1

0 comment

3 changed files

pr created time in 2 days

create barnchautomerge/automerge

branch : patch-callback

created branch time in 2 days

issue commentautomerge/automerge

Python Implementation of the Automerge Server?

Relaying

Agree. We have that notion in place in our experiments https://github.com/jupyterlab/rtc/tree/main/packages/relay

All it needs to do is establish that Alice and Bob are interested in the same topic,

Is it completely automerge agnostic? I would expect the interest in a topic resided in the content of the automerge CRDT messages?

Availability

Agree also with your explanations. On top of availability, server also allows to ensure persistence (in jupyter case, the persistence of the notebooks).

Authentication

+1

echarles

comment created time in 2 days

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Beat me to it! 😆 I've had this TODO in my code for over a year:

image

https://github.com/DevResults/cevitxe/blob/master/packages/cevitxe/src/clocks.ts#L28-L41

josharian

comment created time in 2 days

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Hi @josharian, this is a great contribution, thank you! I have made a few changes in an additional commit, in particular I kept the argument order as getMissingChanges(doc, clock) rather than the other way round. I understand your reasoning for having the clock first, as it more closely mirrors the getChanges(oldDoc, newDoc), but there's already an existing backend function called getMissingChanges that puts the doc first, and I don't want to change it since that would be a breaking API change. Also, the Automerge API functions has the convention that the document is generally the first argument.

@orionz and/or @pvh, could you take a look at this? Especially @orionz as the Rust/wasm backend used to have a getClock function at some point (did it get removed again?). Do you think this is a good API to support for Automerge 1.0 going forward? Here are the docs.

josharian

comment created time in 2 days

issue closedtc39/proposal-optional-chaining

Typo (missing the dot operator)

https://github.com/tc39/proposal-optional-chaining/blob/8461024a040d9c41ec039e2d4291fe2dfda36574/README.md#L58

The following constructs are missing the dot . operator: a?[b] or a?(b)

It should be: a?.[b] or a?.(b)

closed time in 3 days

RinatValiullov

pull request commentapple/swift-evolution

Add an `adjacentPairs` algorithm to Sequence

slidingWindows from Swift Algorithms requires Collection, so it's not entirely the same thing.

mpangburn

comment created time in 3 days

issue commenttc39/proposal-optional-chaining

Typo (missing the dot operator)

No, because this sentence refers to the languages mentioned in the “Prior Art” section, not to ECMAScript.

RinatValiullov

comment created time in 3 days

issue openedtc39/proposal-optional-chaining

Typo (missing the dot operator)

https://github.com/tc39/proposal-optional-chaining/blob/8461024a040d9c41ec039e2d4291fe2dfda36574/README.md#L58

The following constructs are missing the dot . operator: a?[b] or a?(b)

It should be: a?.[b] or a?.(b)

created time in 3 days

issue commentautomerge/automerge

Python Implementation of the Automerge Server?

@echarles I think it's helpful to break down the services that a server traditionally provides and rethink how you might meet those needs in a peer-to-peer context.

  • Relaying: One big advantage of a server is simply that it has a stable public IP address, so you never have any trouble making a connection with it from anywhere on the internet. It's theoretically possible for Alice's laptop to communicate directly over the internet with Bob's phone, but with the standards we have today (WebRTC etc.) it's really hard to pull off reliably in practice. It's a lot easier for Alice and Bob both to connect if there's a known, stable, public endpoint that they can each connect to, which can then act as an intermediary. As @ept's example above shows, this kind of "server" can be exceedingly simple - it's not really a server, and it needn't know anything about Automerge. All it needs to do is establish that Alice and Bob are interested in the same topic, and then pipe their sockets together and let them chat away. In Cevitxe we call this a "signal server" - perhaps "relay server" would be more descriptive.

  • Availability: Another advantage of a server is that it's always turned on and always online, Alice's laptop and Bob's phone may very well be switched off or offline at any given time, and if they can only synchronize when they're both online, it can be hard to work asynchronously. Rather than create a whole new server codebase to address this need, I like the idea of creating an always-on version of your client that you can deploy somewhere, and that can reliably persist its state using a traditional database or something. I haven't actually done this - it's on the roadmap for Cevitxe. I'd be curious to know if anyone else has actually done this.

  • Authentication: The relay server approach described above can provide a very basic sort of security, in the sense that it will only connect two peers if they both request the same topic (a.k.a. channel or document key). This can be good enough for many purposes, especially if that key is long and randomly generated. But it doesn't give the kind of fine-grained permissions control that a lot of applications require, and there's not an obvious remedy if it's compromised. One solution would be to add basic OAuth authentication to the signal server; I sketched out a possible design for that here. I eventually decided against that approach in favor of a completely decentralized solution. I've been fleshing this out for the past few months in a separate project called taco-js.

echarles

comment created time in 4 days

more