profile
viewpoint

sipa/minisketch 161

Minisketch: an optimized library for BCH-based set reconciliation

bbuenz/BulletProofLib 132

Library for generating non-interactive zero knowledge proofs without trusted setup (Bulletproofs)

sipa/bitcoin-net-simul 17

Bitcoin network simulator

gmaxwell/safegcd-bounds 0

Bounds on divsteps iterations in safegcd

issue commentmozilla/mozjpeg

Can this be used in php ?

I sadly don't think there's currently a way to pass through a scan script when using the (much) more efficient link-with-MozJPEG approach.

Scan scripts appear to have originated with libjpeg-turbo (unsure if they existed in original libjpeg).

PHP's main image manipulation library, libgd, doesn't provide any way to plumb this sort of functionality through to scripts.

Imagick is relatively straightforward to install (via package management) but I don't think it provides a passthrough either.

Making a libjpeg-turbo PHP extension wouldn't solve this, because it would have to become Yet Another Image Manipulation Library in the process of exposing all of libjpeg's capabilities to scripts in a useful way.

Others with actual experience might be able to constructively comment more usefully here, but I personally suspect that extending libgd and/or ImageMagick/GraphicsMagick to provide these kinds of capabilities would also suggest fine control over other aspects of image development, which these libraries don't generally do (particularly GD).

TL;DR: What sounds the most viable to me is a PHP extension that can be handed PNG/PPM/JPEG images and then express extremely fine control over the encoding stages. PHP also has a reputation for "importing" objects between extensions where it makes sense to do so (eg, socket->stream, DOMDocument<->SimpleXML), so maybe the extension could "import" GD objects directly.

I personally reckon it would definitely be worth it to make the effort to figure out how to make this work, so I filed https://github.com/libgd/libgd/issues/663 to put this on their radar at a high level, in case there is interest.

kressly

comment created time in 8 hours

push eventsipa/minisketch

Pieter Wuille

commit sha fa795710a7cd3a0d164728e907173a03f4306663

Add Python implementation of Minisketch

view details

push time in 11 hours

pull request commentbitcoin/bips

BIP-0078: fix typo

Hi @NicolasDorier , could you check this?

rage-proof

comment created time in 13 hours

push eventsipa/minisketch

Pieter Wuille

commit sha 168505975b90b87f7a9d7e1b5f39502ba861c3fc

Add Python implementation of Minisketch

view details

push time in a day

push eventsipa/minisketch

Pieter Wuille

commit sha 70e7baf10c1c894d6495c9d5946a706b51f8d931

Add Python implementation of Minisketch

view details

push time in a day

PR opened sipa/minisketch

Add Python implementation of Minisketch

This adds a reimplementation of the Minisketch library interface in pure Python.

It can serve as documentation of the algorithms. It may also be useful to generate unit test cases, to test directly against the C API using FFI, or to incorporate in functional tests of higher-level applications.

+443 -0

0 comment

1 changed file

pr created time in a day

push eventsipa/minisketch

Pieter Wuille

commit sha d131a430aa22fee8960f0e6f6d53b77511ce70e0

Add Python implementation of Minisketch

view details

push time in a day

create barnchsipa/minisketch

branch : 202011_pyminisketch

created branch time in a day

issue commentxiph/vorbis

Cannot build tests on MSVC

And obviously we should add the building and running the tests to the appveyor config!

SegaraRai

comment created time in 2 days

issue commentxiph/vorbis

Cannot build tests on MSVC

I see. M_PI is also referenced by several files in lib but the library source generally includes os.h which has a fallback definition of M_PI for MSVC. The tone utility should have the same problem, but it looks like the CMake build doesn't try to compile it.

gcc enables M_PI by default as a gnu extension (part of X/Open) In my editor, clang-complete also complains about the missing definition.

So defining _USE_MATH_DEFINES here, possibly protected by an #ifdef MSVC is probably the correct fix, but it might be better to change the line in os.h similarly. tone.c can just include os.h since it's internal to the library source.

SegaraRai

comment created time in 2 days

issue openedxiph/vorbis

Cannot build tests on MSVC

MSVC displayed following error when I tried to build vorbis with CMake.

vorbis\test\util.c(37,41): error C2065: "M_PI": undeclared identifier

MSVC seems to require #define _USE_MATH_DEFINES before #include <math.h> to use M_PI. Build succeeded once I add it to L16.

created time in 2 days

push eventElementsProject/dicemix

Ján Sáreník

commit sha 7e3b5e952e7163606bb0130d95767f938d53762f

nit: doc/protocol.md: remove superfluous the Caught my eye. Have not read it all yet. This can be squashed if there are more.

view details

push time in 2 days

PR merged ElementsProject/dicemix

nit: doc/protocol.md: remove superfluous the

Caught my eye. Have not read it all yet. This can be squashed if there are more.

+1 -1

1 comment

1 changed file

jsarenik

pr closed time in 2 days

pull request commentElementsProject/dicemix

nit: doc/protocol.md: remove superfluous the

Thanks.

jsarenik

comment created time in 2 days

push eventmozilla/mozjpeg

Kornel

commit sha c302c77d17fa390eb1d69114ed8ebd53a669f3e8

Mark mozjpeg's output as acceptable

view details

Kornel

commit sha 1a7384c79076bec20a70085e6dcae17a3268f72b

Run turbo's tests with turbo's settings Fixes #384

view details

push time in 3 days

issue closedmozilla/mozjpeg

Multiple failures when running make test on v4.0.0

After building tag v4.0.0, I ran make test and found that 31 of the 151 failed. Is this expected? Perhaps I am missing a dependency?

Build system is Arch Linux building in a clean buildroot.

cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=/usr/lib \
-DENABLE_STATIC=FALSE -DPNG_SUPPORTED=TRUE -DWITH_JPEG8=TRUE
make
  • Log of cmake and make output.
  • Log of make test output.

closed time in 3 days

graysky2

issue commentmozilla/mozjpeg

Multiple failures when running make test on v4.0.0

I'm seeing the same failures. These failures are sort-of expected, because the tests come from libjpeg-turbo. They are just md5 hashes of output files, so they are very sensitive to the smallest changes of the implementation.

I've checked the results manually, and all the differences that are reported are so tiny they're imperceptible. Probably just rounding errors or some float shenanigans.

I'll just update expected hashes to match the files that are being generated.

graysky2

comment created time in 3 days

pull request commentxiph/speexdsp

Fix incorrect macro names in arch.h

Cool. I hadn't realized there was now a xiph gitlab. I'm messing with this project (and specifically its autoconf setup), so I might find more things to make pull/merge requests for. Is the preferred location for that the gitlab repo?

Yeah exactly.

LRFLEW

comment created time in 3 days

pull request commentxiph/speexdsp

Fix incorrect macro names in arch.h

Cool. I hadn't realized there was now a xiph gitlab. I'm messing with this project (and specifically its autoconf setup), so I might find more things to make pull/merge requests for. Is the preferred location for that the gitlab repo?

LRFLEW

comment created time in 3 days

pull request commentxiph/opus

CI - generator script for cmake and github actions

Looks like github actions needs to be activated, they run fine as pr in my fork.

xnorpx

comment created time in 3 days

pull request commentxiph/opus

CI - generator script for cmake and github actions

Not sure, why the actions is not showing up here. They might need to be activated. Here is actions running on my fork.

https://github.com/xnorpx/opus/actions/runs/376994457

@rillian @mark4o

xnorpx

comment created time in 3 days

PR opened xiph/opus

CI - generator script for cmake and github actions

First iteration of proposal of CI generator script for cmake and github actions.

The script will generate a yaml file for github actions, currently different combinations of build settings and desktop x86|x64 is built.

There is plenty of todo that is not considered in this commit:

  • autotools support
  • meson support
  • gitlab support
  • ios, android, arm
  • custom scripts (whitespace check etc.) etc.
+1282 -0

0 comment

5 changed files

pr created time in 3 days

push eventxiph/speexdsp

Tristan Matthews

commit sha 2f852101483cefd8db46d127b73734988d8bb224

CI: add basic gitlab-ci This is largely based off of speex's.

view details

Karl Tomlinson

commit sha dc6394e975ec00618a90a38bfd95b28b8e48405b

fixed-point: Remove unused MULT16_32_Q1[1-4] macros and inlines

view details

Karl Tomlinson

commit sha 9617cea1d4d41199a36c664cc268a830aa0f4ba7

fixed-point: don't truncate 32-bit arg to MULT16_32_Q15 The 32-bit arg and return value have one bit more than required to represent +/-32767 with 15 fractional bits. Keeping the most significant bit allows saturation for 16-bit conversion to sometimes be delayed until after these operations.

view details

Karl Tomlinson

commit sha 00d2e621c53bd5ca6dabd12f97d1224b3705645a

fixed-point resample: remove 1-bit shift right before interpolation This was added in https://gitlab.xiph.org/xiph/speexdsp/-/commit/0dd7bfebe55abcac7e9acbca9d2ac2eddeee2b6f#5424d940076d3b85a3fc2d9f2dcf0d905c43abf0_466_472 to address truncation reported at http://lists.xiph.org/pipermail/speex-dev/2009-June/007277.html Truncation of the most-signficant bit was occuring on conversion to spx_word16_t in MULT16_32_Q15(). Changes to MULT16_32_Q15() mean this will no longer occur. The associated adjustment to the bit-shift before saturation was accidentally removed in https://gitlab.xiph.org/xiph/speexdsp/-/commit/0e5d424fdba2fd1c132428da38add0c0845b4178#6479ffd77de750bc70cf92af3f9b8a4c1e15a98a_473_478 which caused the output to have half the intended amplitude when the interpolating resampler was used. Reported by Andreas Pehrson.

view details

Karl Tomlinson

commit sha 06181733a6c304bd152caadfc5cd2aaf492016b2

fixed-point: introduce MULT16_32_32 to handle unexpected types in MULT16_32_Q15

view details

LRFLEW

commit sha 095fd36e189554bbcbfd9884630a53d7792409dc

Fix incorrect macro names in arch.h Signed-off-by: Tristan Matthews <tmatth@videolan.org>

view details

push time in 3 days

PR closed xiph/speexdsp

Fix incorrect macro names in arch.h

I found what appears to be a few typos in libspeexdsp/arch.h in the option combination tests.

  • One check looks for FIXED_POINT_DEBUG, but this is the only time a macro by this name is ever used in the project (or in libspeex). This seems like it was meant to be FIXED_DEBUG based on the error message, so I changed it.

  • The check for the ARM optimization macros has (defined (ARM4_ASM)||defined (ARM4_ASM)), which is redundant. Based on the context, I assume this was meant to be (defined (ARM4_ASM)||defined (ARM5E_ASM)). I ended up replacing the entire check with a simpler version that utilizes the fact that the sum of boolean comparisons is the count of true values.

  • There were a few cases of inconsistent whitespace usage in this file. In the other files in the project, defined() did not have a space before the parenthesis, but spaces were inconsistently added in this file. I went ahead and removed all these spaces to make the file consistent with the rest of the project.

There was something else I was unable to figure out what was going on with. There are a few instances of the macos CONFIG_TI_C54X, CONFIG_TI_C55X, and CONFIG_TI_C6X in the project, but not in the configuration. The configure.ac file does have an option that defines the macro TI_C55X, but this macro isn't used anywhere in the project (or in libspeex). I'm not sure what's going on here, so I've left it alone, but this probably needs to be looked into.

+6 -6

1 comment

1 changed file

LRFLEW

pr closed time in 3 days

pull request commentxiph/speexdsp

Fix incorrect macro names in arch.h

Thanks, applied here: https://gitlab.xiph.org/xiph/speexdsp/-/commit/095fd36e189554bbcbfd9884630a53d7792409dc

LRFLEW

comment created time in 3 days

PR opened xiph/speexdsp

Fix incorrect macro names in arch.h

I found what appears to be a few typos in libspeexdsp/arch.h in the option combination tests.

  • One check looks for FIXED_POINT_DEBUG, but this is the only time a macro by this name is ever used in the project (or in libspeex). This seems like it was meant to be FIXED_DEBUG based on the error message, so I changed it.

  • The check for the ARM optimization macros has (defined (ARM4_ASM)||defined (ARM4_ASM)), which is redundant. Based on the context, I assume this was meant to be (defined (ARM4_ASM)||defined (ARM5E_ASM)). I ended up replacing the entire check with a simpler version that utilizes the fact that the sum of boolean comparisons is the count of true values.

  • There were a few cases of inconsistent whitespace usage in this file. In the other files in the project, defined() did not have a space before the parenthesis, but spaces were inconsistently added in this file. I went ahead and removed all these spaces to make the file consistent with the rest of the project.

There was something else I was unable to figure out what was going on with. There are a few instances of the macos CONFIG_TI_C54X, CONFIG_TI_C55X, and CONFIG_TI_C6X in the project, but not in the configuration. The configure.ac file does have an option that defines the macro TI_C55X, but this macro isn't used anywhere in the project (or in libspeex). I'm not sure what's going on here, so I've left it alone, but this probably needs to be looked into.

+6 -6

0 comment

1 changed file

pr created time in 4 days

Pull request review commentbitcoin/bips

BIP 330: Transaction announcements reconciliation

+<pre>+  BIP: 330+  Layer: Peer Services+  Title: Transaction announcements reconciliation+  Author: Gleb Naumenko <naumenko.gs@gmail.com>+          Pieter Wuille <pieter.wuille@gmail.com>+  Comments-Summary: No comments yet.+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0330+  Status: Draft+  Type: Standards Track+  Created: 2019-09-25+  License: CC0-1.0+  License-Code: MIT+</pre>++==Abstract==++This document specifies a P2P protocol extension for reconciliation of transaction announcements <b>between 2 nodes</b>, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost.++==Motivation==++Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''.++A <b>reconciliation-based protocol</b> which uses the technique suggested in this document can have better scaling properties than INV-based flooding.++Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes.++===Erlay===++[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency.++Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions.+Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections.+Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network.++All transactions not propagated through flooding are propagated through efficient set reconciliation.+To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared.+A more detailed description of a set reconciliation round and other implementation details can be found in the paper.++Erlay allows us to:+* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019.+* achieve similar latency+* increase network connectivity for almost no bandwidth or latency cost+* improves privacy as a side-effect++This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay.++==Specification==++===New data structures===++Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay.++====32-bit short transaction IDs====++During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners.++Short IDs are computed as follows:+* Let ''salt<sub>1</sub>'' and ''salt<sub>2</sub>'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged.+* Sort the two salts such that ''salt<sub>1</sub> &le; salt<sub>2</sub>'' (which side sent what doesn't matter).+* Compute ''h = SHA256("Tx Relay Salting" || salt<sub>1</sub> || salt<sub>2</sub>)'', where the two salts are encoded in 64-bit little-endian byte order.+* Let ''k<sub>0</sub>'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order.+* Let ''k<sub>1</sub>'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order.+* Let ''s = SipHash-2-4((k<sub>0</sub>,k<sub>1</sub>),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141.+* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''.++This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details.

brackets [] usually mean inclusive?

naumenkogs

comment created time in 4 days

pull request commentmozilla/mozjpeg

Make mozjpeg as its own lib.

@Jehan say:

It's not about opinion.

It is in it. The goal determines the direction of development.

Jehan

comment created time in 4 days

pull request commentxiph/flac

Add Aarch64 support

@nnghuy Cool, Thanks for testing this PR on the new M1 chip! Glad to hear it compiles and runs.

This PR basically only adds one optimization to the FLAC project. It would be interesting if we could beat Rosetta's score with more optimizations.

coreyjjames

comment created time in 4 days

pull request commentmozilla/mozjpeg

Make mozjpeg as its own lib.

This is your and only your opinion.

It's not about opinion. It allows the lib "libmozjpeg" to always work as being mozjpeg (unequivocally). And optionally we can have also have it named libjpeg in a generic way for other cases (e.g. if some distributions were to actually provide an option for this, etc.). The possibility of having an always-available non-ambiguous meaning is a strength.

Jehan

comment created time in 4 days

more