profile
viewpoint
Jeff Peterson jeffpeterson @inkandswitch San Francisco https://yak.sh

automerge/automerge 9194

A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.

automerge/hypermerge 868

Build p2p collaborative applications without any server infrastructure in Node.js

automerge/pushpin 365

A collaborative corkboard app

jeffpeterson/chloroform 9

knock yourself out finding color in stuff with javascript

jeffpeterson/backbone-form_builder 6

Rails-style forms for Backbone with CoffeeScript.

jeffpeterson/backbone-computed_attributes 3

Keep computed model attributes up to date.

jeffpeterson/arsinh-js 2

area hyperbolic sine

jeffpeterson/circulation 1

Get your heart pumping with this simple io irc framework.

jeffpeterson/idb 1

A sweet wrapper for indexedDB. It doesn't work yet.

jeffpeterson/io-extensions 1

Some extensions to make io more succinct.

startedloadimpact/k6

started time in 3 hours

fork hunterloftis/git

Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. Please follow Documentation/SubmittingPatches procedure for any of your improvements.

fork in 11 hours

startedomaralbeik/UserDefaultsStore

started time in 14 hours

pull request commentautomerge/automerge

Add cevitxe to readme under 'Sending and receiving changes'

Great, thank you!

HerbCaudill

comment created time in 15 hours

PR merged automerge/automerge

Add cevitxe to readme under 'Sending and receiving changes'

see https://github.com/automerge/automerge/pull/284#issuecomment-734817546

+10 -5

0 comment

1 changed file

HerbCaudill

pr closed time in 15 hours

push eventautomerge/automerge

HerbCaudill

commit sha 089e84fc6103f01cf9e97135ac15a974e951ab5b

add cevitxe to readme under 'Sending and receiving changes'

view details

Martin Kleppmann

commit sha 08f884026d3e73d3c50ae7fbad212d3fe74da930

Revert unrelated changes

view details

Martin Kleppmann

commit sha fed28a58ceebe9ea6e3469e496c1de4ed942e6cc

Merge pull request #289 from HerbCaudill/readme-cevitxe add cevitxe to readme under 'Sending and receiving changes'

view details

push time in 15 hours

push eventautomerge/automerge

Martin Kleppmann

commit sha 2009189a9ff1f8659e720c0403e9a675b258b3f6

Remove transit dependency

view details

Martin Kleppmann

commit sha d70083ab43a0f91232836ff5f7abd8acf6cc79f9

Make BackendDoc.applyChanges match Backend.applyChanges

view details

Martin Kleppmann

commit sha f3743200b5f3c68ee4dc1c60004e17d81f548f25

Remove the undo/redo feature It's too much work to support for now, and it was never really well thought-out. Better to remove it for now, and bring back in well-conceived form in the future.

view details

Martin Kleppmann

commit sha f4102e6d3acf3a3ae72ed42b11428246363b2c21

Remove Backend.getChangesForActor()

view details

push time in 15 hours

pull request commentautomerge/automerge

Allow hooking into patch application

Thanks @ept that is very useful for my use case. It will simplify the code receiving remote changes and applying them both to the Automerge document and the user facing representation of the document.

ept

comment created time in 18 hours

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

@HerbCaudill By the way, I just realised that Cevitxe isn't listed as one of the data sync layers in the README. I've put a brief one-sentence description in here: 6679330 — does this look ok? Feel free to change it to be better.

I'd like to tweak the wording a bit - why don't I just make a separate PR, since it's not really related to this one.

josharian

comment created time in 19 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I'm with @HerbCaudill here. I think this kind of options hash is common in JavaScript, since it gives us something a bit like named parameters in the absence of language support, and it allows us to add further options in the future without breaking existing code. Moreover, lots of existing functions in the Automerge API (init, from, load, change, etc.) already take an options hash. I don't see a problem with passing a function as part of an options hash: in JS, functions are values like any other.

ept

comment created time in 19 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I agree that readability would improve on the call side. But I think that's caused by missing labeled arguments in JavaScript which we should not try to fix.

When just reading function applyChanges(doc, changes, options = {}) I would expect to pass a document, some changes and a map which only contains key value pair, which help configuring the change method. I would not expect to pass references into options which provide me with the resulting patches.

On the other hand when reading function applyChanges(doc, changes, patchCallback) it is clear to me that I can pass a document, changes and a callback which provides some kind of patches. (It is also future prove to just add another parameter)

This is not really a strong opinion, so I am happy to go with options map as well :)

ept

comment created time in 20 hours

pull request commentautomerge/automerge

Allow hooking into patch application

On second thought I do see that this is a little spooky since you're getting a new Automerge document every time.

const state = Automerge.init({}, { patchCallback }) 
const state1 = Automerge.change(state, s => s.foo = 42)
const state2 = Automerge.change(state, s => s.boo = 'pizza') // maybe it is weird for patchCallback to fire here
ept

comment created time in 21 hours

pull request commentautomerge/automerge

Allow hooking into patch application

Overall this seems like a step in the right direction. It always seemed odd to me that you could watch for changes on a DocSet but not on an individual document.

An alternative API would be to pass in a callback function once when initialising the document, and automatically calling it on all subsequent updates. But I thought that had a bit of a "spooky action at a distance" feel, while the callback on every change/applyChanges call was more explicit. Though it could also be annoying for app developers to have to add the option to every single Automerge.change call, which might be scattered around an application. Any thoughts on this?

I'd imagine most developers would find the idea of a change listener to be totally natural, and not spooky at all.

Is there a reason why we can't have both?

const patchCallback = patch => alert(patch)
const changeFn = doc => { doc.foo = 42 }

// this works
const state = Automerge.init({}, { patchCallback }) // patchCallback will fire on every change
const newState = Automerge.change(state, changeFn)

// this works too
const state = Automerge.init({})
const newState = Automerge.change(state, changeFn, { patchCallback }) // patchCallback only fires on this change
ept

comment created time in 21 hours

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

I prefer the interface @ept suggests here - the options map makes it easier to add the change API further in the future. It also makes the code calling this function more readable.

ept

comment created time in 21 hours

issue commentautomerge/automerge

Python Implementation of the Automerge Server?

Is it completely automerge agnostic? I would expect the interest in a topic resided in the content of the automerge CRDT messages?

No, the "topic" in this sense generally represents one Automerge document or a set of Automerge documents (e.g. a DocSet, or what Hypermerge and Cevitxe both refer to as a repository).

echarles

comment created time in 21 hours

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Thanks for doing clean-up. :)

josharian

comment created time in a day

created repositoryjimpick/lotus-utils

Bits extracted from Filecoin Lotus for use in standalone projects

created time in a day

created repositoryjimpick/lotus-query-ask-api-daemon

A Lotus-based daemon that only supports ClientQueryAsk

created time in a day

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

@HerbCaudill By the way, I just realised that Cevitxe isn't listed as one of the data sync layers in the README. I've put a brief one-sentence description in here: 66793306c9a6d0536c086f56de4bc7d1ef8e8c84 — does this look ok? Feel free to change it to be better.

josharian

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {

why is there an options map needed. Could we just pass the callback directly?

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function getAllChanges(doc) {   return getChanges(init(), doc) } -function applyChanges(doc, changes) {+function applyChanges(doc, changes, options = {}) {   const oldState = Frontend.getBackendState(doc)   const [newState, patch] = Backend.applyChanges(oldState, changes)+  if (options.patchCallback) options.patchCallback(Object.assign({}, patch))

I would prefer a operate line + indentation, which improves readability, safety and debuggablility.

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function docFromChanges(options, changes) {   const doc = init(options)   const [state, _] = Backend.applyChanges(Backend.init(), changes)   const patch = Backend.getPatch(state)+  if (options && options.patchCallback) options.patchCallback(Object.assign({}, patch))

Same as above

ept

comment created time in 2 days

Pull request review commentautomerge/automerge

Allow hooking into patch application

 function makeChange(doc, requestType, context, options) {    if (doc[OPTIONS].backend) {     const [backendState, patch] = doc[OPTIONS].backend.applyLocalChange(state.backendState, request)+    if (options && options.patchCallback) options.patchCallback(patch)

I would prefer a operate line + indentation, which improves readability, safety and debuggablility.

ept

comment created time in 2 days

push eventautomerge/automerge

Martin Kleppmann

commit sha 7942fed63a152949f78be5818741cf7d5e381941

Add patchCallback to Automerge.load as well

view details

push time in 2 days

PR opened automerge/automerge

Allow hooking into patch application

Here is a simple but potentially very important feature: allowing applications to hook into Automerge's process for applying patches to a document.

The use case is when you have some data outside of Automerge that you want to keep consistent with the state of an Automerge document. This could be a user interface (especially a complicated widget like a text editor), or a cache, a search index, or suchlike. Such an application needs a mechanism to get notified when an Automerge document changes, so that it can apply the appropriate update to the external state.

At the moment, Automerge only really supports the React-style approach where you pass the entire document to the render function, and the renderer works out which bits of the document have actually changed, and updates those, while leaving the unchanged parts untouched. But in other circumstances, it would be really useful to have a description of the change that occurred, not just the before-and-after document. Moreover, this process should be the same, regardless of whether a change is generated by the local user or was received over the network from a remote user.

Automerge's existing concept of a patch is probably the best form we have for describing what change occurred. It's easier for applications to consume than the raw change format because it includes more context, and e.g. for list updates it includes the index of the list position being changed, not just the list element ID. This patch format is currently used for updates sent from the backend to the frontend, and used by the frontend to update the document.

This PR adds the ability for applications to get a callback for every patch that is applied to a document. When making a local change, you can add a callback like this:

newState = Automerge.change(oldState, {patchCallback: (patch) => {...}}, doc => {
  // mutate doc as usual
})

And when applying a remote change, you can add a callback like this:

newState = Automerge.applyChanges(oldState, changes, {patchCallback: (patch) => {...}})

The patch format is the same in both cases. The patch format used by the currently released version of Automerge is documented in INTERNALS.md (in the "Frontend-backend protocol" section). This format has changed considerably on the performance branch, and the new format is documented in BINARY_FORMAT.md on that branch. The new format is intended to be easier to consume.

Note that this API is not intended to allow applications to modify the patches before they get applied. The patch callback should treat the patch as read-only.

An alternative API would be to pass in a callback function once when initialising the document, and automatically calling it on all subsequent updates. But I thought that had a bit of a "spooky action at a distance" feel, while the callback on every change/applyChanges call was more explicit. Though it could also be annoying for app developers to have to add the option to every single Automerge.change call, which might be scattered around an application. Any thoughts on this?

This feature is especially for @cklokmose, but I welcome feedback from others as well. @HerbCaudill what do you think?

+60 -1

0 comment

3 changed files

pr created time in 2 days

create barnchautomerge/automerge

branch : patch-callback

created branch time in 2 days

issue commentautomerge/automerge

Python Implementation of the Automerge Server?

Relaying

Agree. We have that notion in place in our experiments https://github.com/jupyterlab/rtc/tree/main/packages/relay

All it needs to do is establish that Alice and Bob are interested in the same topic,

Is it completely automerge agnostic? I would expect the interest in a topic resided in the content of the automerge CRDT messages?

Availability

Agree also with your explanations. On top of availability, server also allows to ensure persistence (in jupyter case, the persistence of the notebooks).

Authentication

+1

echarles

comment created time in 2 days

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Beat me to it! 😆 I've had this TODO in my code for over a year:

image

https://github.com/DevResults/cevitxe/blob/master/packages/cevitxe/src/clocks.ts#L28-L41

josharian

comment created time in 2 days

pull request commentautomerge/automerge

add Automerge.getClock and Automerge.getMissingChanges

Hi @josharian, this is a great contribution, thank you! I have made a few changes in an additional commit, in particular I kept the argument order as getMissingChanges(doc, clock) rather than the other way round. I understand your reasoning for having the clock first, as it more closely mirrors the getChanges(oldDoc, newDoc), but there's already an existing backend function called getMissingChanges that puts the doc first, and I don't want to change it since that would be a breaking API change. Also, the Automerge API functions has the convention that the document is generally the first argument.

@orionz and/or @pvh, could you take a look at this? Especially @orionz as the Rust/wasm backend used to have a getClock function at some point (did it get removed again?). Do you think this is a good API to support for Automerge 1.0 going forward? Here are the docs.

josharian

comment created time in 2 days

more