profile
viewpoint
Jon Mease jonmmease Plotly Maryland https://www.linkedin.com/in/jon-mease/ Chief Scientist at Plotly

holoviz/datashader 2319

Quickly and accurately render even the largest data.

holoviz/holoviews 1664

With Holoviews, your data visualizes itself.

holoviz/panel 824

A high-level app and dashboarding solution for Python

holoviz/spatialpandas 86

Pandas extension arrays for spatial/geometric operations

jonmmease/jupyterlab_delux 12

Proof-of-concept of a conda package that includes JupyterLab with preinstalled extensions.

jonmmease/ipyplotly 9

[WIP] Towards an ipywidget Plotly library

jonmmease/hilbert_frame 3

Experimental spatially-partitioned data structure for use with the Datashader

jonmmease/plotly.py_release_notebooks 3

Repository of demo notebooks for new releases of plotly.py

issue commentplotly/Kaleido

ValueError: Transform failed with error code 1: Failed to serialize document: Uncaught

Hi @wdecoster, thanks for the report. There is an open issue for figures that are "too large" (https://github.com/plotly/Kaleido/issues/19) but that results in a particular error that's different that what you're seeing.

My first question would be whether this happens with even small simple figures.


Here's one guess. When you build the PlotlyScope using the default constructor, Kaleido uses an online CDN location for the plotly.js bundle. So if the user doesn't have internet access (or is behind a proxy/firewall) then this kind of error could occur.

The plotlyjs argument/property to PlotlyScope can be used to point Kaleido to a local plotly.js bundle, which is what plotly.py does. If you're comfortable requiring plotly >= 4.9, then I would actually recommend letting plotly.py take care of interacting with kaleido, replacing what you have with

import plotly.io as pio
pio.write_image(fig, "output.png", engine="kaleido")

If you need to support older versions of plotly.py, then you could copy the kaleido initialization logic from

https://github.com/plotly/plotly.py/blob/8289eb1c85125e147c4ce14b56272ed50b99ef20/packages/python/plotly/plotly/io/_kaleido.py#L9-L17

which will configure the PlotlyScope to use the plotly.js bundle that is included with plotly.py.

wdecoster

comment created time in 16 hours

issue commentplotly/Kaleido

Error with Pyinstaller .exe file (Windows 10)

Glad you got things working! I'm assuming --add-data is a PyInstaller option? It does make sense that this directory would be needed. Are you aware of anything that could be done in the Kaleido Python package itself to help PyInstaller figure this out automatically?

DouglasDL28

comment created time in 17 hours

issue openedholoviz/holoviews

Website request: Support permalinks to specific backends in (Reference) Gallery

Describe the solution you'd like

I'd like to be able to reference the HoloViews Gallery and Reference Gallery for a specific backend (Plotly in my case).

Right now the URLs for the Gallery (http://holoviews.org/gallery/) and Reference Gallery (http://holoviews.org/reference/) pages always default to showing the Bokeh backend. You can switch to the Matplotlib and Plotly backends using the buttons at the top, but this doesn't change the URL path so it doesn't look like there is currently a way to use a URL to land and the page for the Matplotlib or Plotly backends.

I'm not familiar with how the documentation is structured, but would there be a way to assign a separate subpath or anchor to the different backends?

I'd like to be able to reference the Plotly elements from some future Dash documentation.

created time in 18 hours

issue commentplotly/Kaleido

[Bug] Plots dont contain graphs after static image export

Ok, thanks for the trying that out. So to boil it down, it looks like this is a problem with WebGL plots on WSL 1.

klnrdknt

comment created time in 21 hours

pull request commentconda-forge/staged-recipes

Adding Kaleido

Ok, here's a proposal of things I understand how to do that we can discuss when we meet up.

  • Update the kaleido CI job that compiles kaleido (embedding chromium) to produce a minimal archive that doesn't vendor anything. This would amount to removing the lib directory of shared libraries and removing the font files.
  • Create a conda-forge kaleido/kaleido-core/kaleido-bin package that:
    • Declares conda-forge depedencies for these shared libraries and some basic fonts (maybe fonts-conda-forge?)
    • Downloads the minimal kaleido executable from the appropriate GitHub release page.
    • On activation, sets some new special KALEIDO_EXEC_PATH environment variable to the path of the kaleido executable.
  • Update the Python portion of the kaleido library to look for KALEIDO_EXEC_PATH and, if found, use this path instead of assuming kaleido is vendored in the python package.
  • Update the python-kaleido conda-forge package to be noarch: python and depend on the kaleido package above. This package would checkout the kaleido repo at the appropriate tag, and include only the pure Python portion of kaleido.

I am comfortable with all of these changes. I see the value of them and, after making these updates, I don't expect there to be much added maintenance burden. We could either keep our current PYPI pipeline as-is, or update that to pull from the conda pipeline.

I'm less comfortable with the idea of trying to build the core kaleido executable from within the conda forge ecosystem. It's an intimidatingly complex build system (at least to me) and I'd be very concerned that it would be very fragile and break whenever we wanted to update chromium. Given that the architecture shouldn't actually cause any compatibility problems with the rest of conda-forge, I'd much rather stick with the official chromium build instructions for maintainability.

That said, if someone really want to try to figure this out at some point, I'd recommend experimenting with the headless_shell example described here (https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md#usage-as-a-c_library). In terms of architecture and dependencies, Kaleido is just a variation of the headless_example/headless_shell examples here.

jonmmease

comment created time in 21 hours

issue commentplotly/Kaleido

[Bug] Plots dont contain graphs after static image export

Thanks for the reproducible example! This example has enough points that plotly express will switch to webgl rendering, so just to double check, does this example work properly for you?

import plotly.graph_objects as go
import numpy as np

N = 100000
fig = go.Figure(data=go.Scattergl(
    x = np.random.randn(N),
    y = np.random.randn(N),
    mode='markers',
    marker=dict(
        color=np.random.randn(N),
        colorscale='Viridis',
        line_width=1
    )
))

fig.show()

Also, to double check, when you mentioned that things work from a regular ipython terminal, is that terminal running in the exact same WSL environment as the Jupyter notebook?

Might be relevant: I'm using it within poetry virtualenv on WSL

The poetry environment probably doesn't make a difference but WSL very well could. We've had one other report of a different issue on WSL 1 at https://github.com/plotly/Kaleido/issues/38, and it's not something we've tested ourselves yet.

Are you on WSL 1 or WSL 2? I have a hunch that things will work more reliably over all on WSL 2 since it's based on a real Linux VM that's just a hunch at this point.

klnrdknt

comment created time in a day

pull request commentconda-forge/staged-recipes

Adding Kaleido

With the executable being independent of the Python version, it would actually be preferrable to build them separately on conda-forge. It seems like the Python part is solely pure Python, so the Python package could be a noarch: python package that would only be built once and not for the full matrix "platform x Python version". The binary part could be in a kaleido-bin package that is just build per platform without the need to expand the matrix by the Python version.

This would take a small bit of rework in kaleido itself, but yeah I definitely see the advantages and would be happy to make this change.

Still, the vendored shared libraries are only suboptimal we should not bundle them.

Ok, yeah, it looks all of the shared library files that we vendor on linux are available across the expat, nspr, nss, and sqlite conda-forge packages. So, would we make these packages runtime dependencies of the core kaleido package? Does conda set RPATH to point to these libraries? Or does it rely on binary relocation (and the step I'm skipping here :smiley:)

The only exceptions are the swiftshader shared libraries. These seem to actually be produced by the chromium build process, and they are a bit odd in the fact that they don't show up in ldd and they seem to need to be located in a particular relative path from the executable. So as long as it's ok to consider these as part of chromium itself, then it looks like nothing else would need to be vendored.

Also, these swiftshader files are the only shared libraries that are currently bundled on non-linux platforms.

If we go this way, I think we'd still need to produce the vendored wheels for use on PyPI, but maybe we change to pulling these shared libraries from the conda packages. Is this the kind of thing that conda-press was designed to help with?

jonmmease

comment created time in a day

issue commentplotly/Kaleido

[Bug] Plots dont contain graphs after static image export

Thanks for the report, and update, @klnrdknt.

This is certainly an odd combination of behavior! I don't know of any reason why kaleido would act differently when called from ipython and from within a Jupyter notebook. Also, Kaleido is using the same image export pathway as the save button in the toolbar, but it's using it's own embedded chromium browser, not the browser displaying the figure.

Are you able to reproduce this behavior with any of the scatter plots from the docs at https://plotly.com/python/line-and-scatter/? We'll need something that we can fully reproduce to dig in further. In particular, try both the plain scatter and scattergl variations in case this has something to do with WebGL accelerated figures.

Thanks!

klnrdknt

comment created time in a day

pull request commentconda-forge/staged-recipes

Adding Kaleido

First off, thanks very much to the conda-forge team for taking the time to discuss this, and for reaching out. I'm looking forward to discussing this in a higher bandwidth meeting soon.

@xhochy, thanks for the detailed description of considerations/concerns. I totally understand that these are very important for libraries like pyarrow and qt to avoid binary incompatibility nightmares, but I wanted to emphasize what I believe is an important distinction in kaleido's architecture:

Both ABIs should be able to co-exist in the same process but due to peculiarities of the devtoolset compilers in the manylinux Docker images ...

Kaleido doesn't load anything into a shared process. The native portion of kaleido (the custom chromium executable and vendored shared libraries) is a standalone command-line executable ("standalone" architecturally, but only really intended to be used programmatically). The Python portion of kaleido is pure Python (nothing compiled, OS independent) that launches the native kaleido executable in a separate process using the subprocess module and communicates with it by writing JSON to standard in, and reading JSON responses from standard out.

So the wheels for the different operating systems all contain identical *.py files, they just bundle a different version of the compiled executable.

My understanding is that since the sub process doesn't have a shared memory space with the parent process, there is not risk of introducing a binary incompatibility. This was a large part of my motivation for adopting this unorthodox architecture, but please let me know if I'm not understanding something properly here. Thanks!

jonmmease

comment created time in a day

issue commentholoviz/holoviews

Support widgets in link_selections

Yeah, I like that.

Are these the usages you're picturing?

link_selections.filter_by_value(dim1=3)
link_selections.filter_by_value(dim1=param_obj.param.value)

link_selections.filter_by_range(dim1=(0, 10))
link_selections.filter_by_range(dim1=param_obj.param.value)

link_selections.filter_by_expr((dim("dim1") + dim("dim2")) > 0)
link_selections.filter_by_expr(param_obj.param.selection_expr)

One thing about using the value directly is that I don't see a good way to remove the filter later. e.g.

link_selections.filter_by_value(dim1=3)
link_selections.filter_by_value(dim1=4)

would result in an empty selection unless we added some sort of overwrite logic. This might be fine, but also might be surprising.

jonmmease

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentconda-forge/staged-recipes

Adding Kaleido

-max_py_ver: '37'-max_r_ver: '35'+max_py_ver: '38'+max_r_ver: '36'

Thanks, done in https://github.com/conda-forge/staged-recipes/pull/12093/commits/6d503160575bfaa094611e0fb957fcde31b6d6e7

jonmmease

comment created time in 3 days

pull request commentconda-forge/staged-recipes

Adding Kaleido

Thanks for taking a look @xhochy. Let me know if there's anything that would be helpful from me for discussion with the core team.

From my understanding, there are two large challenges here.

  1. To build chromium from source in the conda-forge ecosystem would require gn and the ability to reproduce all of the dependencies in https://github.com/chromium/chromium/blob/master/build/install-build-deps.sh#L309-L346 using conda packages. Although, these probably aren't all strictly required when building chromium in headless mode on Linux (which is what kaleido does). It would probably also be desirable to separately package depot_tools (https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md#install), which is used to actually download the chromium source tree and handle things like submodules for the current build configuration. Also, expect this to take > 5 hours on CI.

  2. After completing (1), it would be possible to package the full (or headless) Chromium browser in conda-forge. But there's still the question of how the kaleido package would make use of this because there's no upstream-supported way to build chromium as a shared library to link into other applications. As I noted above, Electron use to have a custom downstream patchset for doing this (https://github.com/electron/libchromiumcontent), but they deprecated it a few years ago after facing a lot of challenges in keeping up with upstream.

jonmmease

comment created time in 3 days

pull request commentconda-forge/staged-recipes

Adding Kaleido

Well, how is this wheel getting created is my question? I am mostly on board with not building chromium here and doing a repack. However, we shouldn't repack a wheel. We should go from the python source and rebuild the package on CI

Right now, the wheel is built in the same CI job where chromium is compiled. Here's an alternative approach that I could do.

  • Clone the kaleido repo at the tag that corresponds to the recipe version. e.g. for 0.0.3, use https://github.com/plotly/Kaleido/tree/v0.0.3.
  • wget the kaleido binary for the current OS from the GitHub release. e.g. for 0.0.3, download the binary artifact from https://github.com/plotly/Kaleido/releases/tag/v0.0.3.
  • Build the wheel with python setup.py bdist_wheel ...

Would this be preferable? If so, I'm happy to update the recipe with this approach.

Thanks for the help on this!

jonmmease

comment created time in 3 days

issue commentholoviz/holoviews

Support widgets in link_selections

Actually maybe we'd just allow kwargs mapping from dim name to parameters, which we do allow elsewhere.

Would this be something like the following?

link_selections.add_value_selection(dim1=parameter_obj.param.value)

In the case of link_selections.add_expression_selection, there wouldn't be a single dim so I guess we'd still use a single arg.

link_selections.add_expr_selection(param_obj.param.selection_expr)
jonmmease

comment created time in 3 days

push eventjonmmease/staged-recipes

Jon Mease

commit sha 6c11e26fafb85c59ade09ae24dde8c4508602d4b

Update missing_dso_whitelist for osx with /System/Library/Frameworks/*

view details

push time in 3 days

push eventjonmmease/staged-recipes

Jon Mease

commit sha 6d503160575bfaa094611e0fb957fcde31b6d6e7

Revert changes to conda-forge.yml

view details

Jon Mease

commit sha 6ce5bf5779f5a5aed49b738dcdf07cfa5d8a90e1

Merge remote-tracking branch 'origin/kaleido' into kaleido

view details

push time in 3 days

issue commentholoviz/holoviews

Support widgets in link_selections

Couldn't it just be any parameter?

Yeah, it would be more general than just widgets. The part that felt widget specific was the specific interpretation of the value param as a selection expression. e.g. If value is a scalar number the use this value exactly, if a range then convert to an interval selection.

Maybe there should be a separate method for each "interpretation" of value. e.g.

  • link_selections.add_value_selection(dim, param_obj): param_obj.value expected to be a scalar value and we build the expression dim == param_obj.value.
  • link_selections.add_range_selection(dim, param_obj): param_obj.value expected to be an interval and we build the expression (param_obj.value[0] <= dim) & (dim <= param_obj.value[1]).
  • link_selections.add_expr_selection(param_obj): param_obj.value expected to be a predicate expression that should be used as-is.
jonmmease

comment created time in 3 days

pull request commentconda-forge/staged-recipes

Adding Kaleido

Is there a way to build the non-chromium parts of Kaleido here?

Not that I know of. The instructions for using Chromium headless as a C++ library are to build the application from within the Chromium source tree. Kaleido basically works just like the headless_example app from https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md#usage-as-a-c_library.

Here are C++ source files for kaleido: https://github.com/plotly/Kaleido/tree/master/repos/kaleido/cc. It's basically one entry point .cc file, like headless_example, with a handful of .h header files that all have includes that reference chromium namespaces.

jonmmease

comment created time in 4 days

push eventholoviz/holoviews

Jon Mease

commit sha 5b76560f46de938d67d4b3039e3e100199b12d14

fix flakes

view details

push time in 4 days

issue openedholoviz/holoviews

Support widgets in link_selections

Motivation

This is an idea for extending the link_selections framework to support performing selections using widgets. For example

  • A range slider could be used to create a selection similarly to how a selection can be performed on a histogram element.
  • A dropdown or multi-select could be used to create a selection on a categorical dimension.

The top section of the Dash Oil and Gas demo shows an example of the basic idea:

Screenshot_20200914_071626

API

Here's an idea for the API. We could add a new method to the link_selections function object called something like add_widget/add_selection_widget. This method would expect a dimension name and a Parameterized object that contains a param named value (with an option to override this name).

from holoviews.selection import link_selections
from ??? import Slider

lnk_sel = link_selections.instance()
hvobj_linked = lnk_sel(hvobj)

slider = Slider(start=0, end=10, value=6)
lnk_sel.add_widget("x0", slider)

If slider.value is a scalar, then the widget is used to select this exact value. If an interval, then widget is used to select a range of values.

The add_widget method would wrap this input object in a Param stream, then build a Derived stream that would convert this value and the dimension into a selection expression, and this selection expression would then be treated just like a selection expression coming from a figure element.

Usage

The main initial usage would be in the context of a Panel app, and in this case a Panel widget could be passed as the argument to add_widget. But for use in non-panel contexts (like Dash), it would be nice to have a set of parameterized objects in HoloViews that could be used instead.

These could live somewhere like a holoviews.widget_models module, and they could have identical names to corresponding Panel widgets, but only contains a subset of the properties. For example, holoviews.widget_models.Slider would only have value, start, end, and step params.

With this approach, a user constructing a Dash app would pass widget_models.Slider to add_widget and then the holoviews_to_dash function (see https://github.com/holoviz/holoviews/pull/4605) would transform the Slider model into the appropriate Dash widget.

We may want to hold off on introducing these widget models until we work on the 2.0 refactor because we may need to do something similar to remove the hard dependency on Panel from the HoloViews core.

cc @philippjfr @jlstevens @jbednar

created time in 4 days

pull request commentconda-forge/staged-recipes

Adding Kaleido

Thanks for the bump @h-vetinari, and for weighing in @scopatz.

Yeah, we have allowed binary repacks of things like firefox in the past. I think that would/could be acceptable for chromium.

Thanks, in the short term we'd very much appreciate it!

However, it would be best really if there was a chromium-static package that this build of kaleido and other projects could build off of. It should be possible to split kaleido from its deps and not redistribute a kaleido binary, and instead build kaleido here. We'd really want to minimize repackaging of dependencies to just chromium, if at all possible.

Yeah, I totally agree in principle here. It's just a bit beyond my packaging ability at this point. The Chromium project itself doesn't support building as a static library so it would require developing an unofficial process for this. The closest thing I've found is the now-deprecated libchromiumcontent that was formerly used by the Electron project. There is some additional info on how this approached used to work at https://www.electronjs.org/blog/electron-internals-building-chromium-as-a-library.

If someone is interested in taking this on and developing a chromium-static package down the road, I'd definitely be open to porting the kaleido package to build against it.

jonmmease

comment created time in 5 days

push eventholoviz/holoviews

Philipp Rudiger

commit sha 09bed5d69a1cc9fa9530b9e165f76a08f5b8c181

Fixed categorical handling in Geom plot types (#4575) * Fixed categorical handling in Geom plot types * Correctly compute factors in overlay * Fixed flakes * Fix handling of categorical factors * Handle elements without len

view details

Philipp Rudiger

commit sha 60c96aa03cedd1ba507ad59c568078a1d391d0fe

Do not attempt linking on annotations (#4584)

view details

Philipp Rudiger

commit sha 90bd54b93e256ae7a8f9cdd743d08656f4f7c2e0

Reset RangeXY when framewise is set (#4585) * Reset RangeXY when framewise is set * Test RangeXY stream reset

view details

Philipp Rudiger

commit sha 40645534d074480e7ed7866f5cc373f4ed0e72bf

Add automatic collate for Overlay of AdjointLayouts (#4586)

view details

Philipp Rudiger

commit sha 24de9a2c80f9e7fc830cfe2dd47fc9b945277213

Fixed color-ranging after box select on side histogram (#4587)

view details

Philipp Rudiger

commit sha 1f4c3184931c1026cea698a3338d994e9c80ddad

Use HTTPS throughout on homepage (#4588)

view details

Philipp Rudiger

commit sha 8168fa13f95743361c9570085770f95dc795a058

Fix tests on builds (#4590)

view details

Philipp Rudiger

commit sha 9072acfb8edfac9a19bf7157545fcaf00285f2e0

Use python 3.7 for pip builds

view details

Philipp Rudiger

commit sha c108277cba92d04afcc8cf8ffb1946c3e84faad1

Ensure pip build environment is activated

view details

Philipp Rudiger

commit sha 712d6b8d4d6e94a3d2dec0af832e8028190f5467

Add conda-forge to doc build channels [doc-build]

view details

Philipp Rudiger

commit sha bec812b0a6f4bce06f658899d77bd2af7015a3c6

Fixed bug colormapping Path (#4593) * Fixed bug colormapping Path * Add test

view details

Philipp Rudiger

commit sha 1d3c25c97d1bfe55b80b5bc1f128c97e143da237

Expose sizing_mode on Div element (#4594)

view details

Philipp Rudiger

commit sha f66aec5604032be2703ffa2bac5f8b933f7b53b5

Add option to link streams on link_selections input (#4595)

view details

Philipp Rudiger

commit sha 67662672ca6a86aaf12e9004223d80c5a1731ed9

Fix bug in linked_selections table rendering (#4597)

view details

Philipp Rudiger

commit sha 1e70d928fa263389ed630f412ba91b823111ce3f

Ensure dynspread handles zero width/height Image/RBG (#4600)

view details

Philipp Rudiger

commit sha f0b1272c5fb85507f133727857181c7cdd1857c4

Add CHANGELOG for 1.13.4 (#4599)

view details

Philipp Rudiger

commit sha a7ba2d9852d75e80b0cc2a5576540969df235575

Add selenium to doc env

view details

Philipp Rudiger

commit sha f0066369c1c8f31e9ba02e8abe88d8826525bf9d

Pin bokeh in the doc build

view details

Philipp Rudiger

commit sha da0d1efc5ffc59e0ef64f572f8ec0ae64baea163

Avoid using Parameterized repr in callable_name (#4601) * Avoid using Parameterized repr in callable_name * Small fix

view details

Philipp Rudiger

commit sha 1da5906edd98451d655232076ec7fe6566070522

Readd firefox and geckodriver to doc build env

view details

push time in 7 days

PR opened holoviz/holoviews

Add Plotly Dash support

This PR adds support for displaying dynamic HoloViews objects using Plotly Dash.

The new logic is localized to the single holoviews/plotting/plotly/dash.py file which provides a new holoviews_to_dash function.

A new section has been added to all of the dynamic Plotly reference guide notebooks that includes the code for displaying the example in a Dash app. This is a good place to look at the example usage for the holoviews_to_dash function.

This PR is dependent on some of the fixes in https://github.com/holoviz/holoviews/pull/4572, and so it should be rebased on master after #4572 is merged.

+1837 -210

0 comment

20 changed files

pr created time in 7 days

create barnchholoviz/holoviews

branch : dash_support

created branch time in 7 days

issue commentplotly/Kaleido

Error with Pyinstaller .exe file (Windows 10)

Hi @DouglasDL28, thanks for the report.

Would you be willing to create a minimal PyInstaller example that produces this error? We don't have much experience using PyInstaller, so there's a much better chance of someone taking a look if we have this as a starting point. Might be easiest to do this in a small standalone repo, including a README for how to package it, and link it here.

Thanks!

DouglasDL28

comment created time in 7 days

pull request commentholoviz/datashader

Drop ds.geo and ds.spatial modules

Wanted to cross reference https://github.com/pyviz-topics/examples/pull/97. This was my old WIP PR to port the SpatialPointsFrame demo notebook to spatialpandas. As I recall, at that time there was still some work needed in HoloViews to get spatialpandas DaskGeoDataFrame working with datashader.

thuydotm

comment created time in 12 days

Pull request review commentholoviz/datashader

Drop ds.geo and ds.spatial modules

 def points(self, source, x=None, y=None, agg=None, geometry=None):         if agg is None:             agg = count_rdn() -        # Handle down-selecting of SpatialPointsFrame         if geometry is None:-            import sys-            if 'datashader.spatial.points' in sys.modules:-                from datashader.spatial.points import SpatialPointsFrame-                if (isinstance(source, SpatialPointsFrame) and-                        source.spatial is not None and-                        source.spatial.x == x and source.spatial.y == y and-                        self.x_range is not None and self.y_range is not None):--                    source = source.spatial_query(-                        x_range=self.x_range, y_range=self.y_range)             glyph = Point(x, y)

To use the points glyph as an example, the Dask partition-level spatial indexing is done at

https://github.com/holoviz/datashader/blob/dd8f634eb9c61887255f8be80e2354cba55dcef6/datashader/core.py#L209-L215

Here DaskGeoDataFrame.cx_partitions is basically equivalent to SpatialPointsFrame.spatial_query

The within partition spatial indexing is done in https://github.com/holoviz/datashader/blob/dd8f634eb9c61887255f8be80e2354cba55dcef6/datashader/glyphs/points.py#L269-L272

thuydotm

comment created time in 12 days

PullRequestReviewEvent

Pull request review commentholoviz/datashader

Drop ds.geo and ds.spatial modules

 def points(self, source, x=None, y=None, agg=None, geometry=None):         if agg is None:             agg = count_rdn() -        # Handle down-selecting of SpatialPointsFrame         if geometry is None:-            import sys-            if 'datashader.spatial.points' in sys.modules:-                from datashader.spatial.points import SpatialPointsFrame-                if (isinstance(source, SpatialPointsFrame) and-                        source.spatial is not None and-                        source.spatial.x == x and source.spatial.y == y and-                        self.x_range is not None and self.y_range is not None):--                    source = source.spatial_query(-                        x_range=self.x_range, y_range=self.y_range)             glyph = Point(x, y)

This logic is specific to the old spatial points frame, so it's not needed when using spatialpandas.

thuydotm

comment created time in 13 days

PullRequestReviewEvent

issue commentplotly/Kaleido

Export to image on Ubuntu under Windows works only the first time

Hi @cesgarma, thanks for the report.

The next thing you could try would be to call the kaleido native executable from the command line. There are some instructions for that in https://github.com/plotly/Kaleido/issues/26#issuecomment-664948907. These were written for Windows, but the approach is the same for Linux.

Since you're export is working the first time, I expect everything will work up until the first time you enter {"data":{"layout":{}}}, so do this part multiple times. It will probably hang the second time, but hopefully chromium will output some error info. Please copy all of the terminal output that you see.

All that said, it looks like folks have had trouble with using Chromium-based projects under WSL 1. See https://github.com/microsoft/WSL/issues/648 and https://github.com/microsoft/WSL/issues/2760. So it might not be possible to get this working reliably. I expect things will work better on WSL 2 since this is based on a true Linux VM instead of a Linux-to-windows translation layer.

cesgarma

comment created time in 13 days

issue openedholoviz/holoviews

Spatial indexing for linked selections

When performing linked selections on large tabular datasets, it would sometimes be useful to maintain spatial indices for subsets of columns in the dataset.

  • For pandas DataFrames, these could be RTree indices from spatialpandas (https://github.com/holoviz/spatialpandas).
  • For cudf, these could be quad-tree indices from cuspatial (https://docs.rapids.ai/api/cuspatial/stable/api.html#indexing)

These would make it possible to accelerate multi-dimensional range selections, and 2-dimensional geometric selections.

I'm picturing that we could create these indices on demand when selections are requested. But, what I'm trying to think through is, where could we store/cache them? They would be associated with a specific DataFrame, so it would be nice to somehow store them alongside the DataFrame, or in the Dataset. We might also want to extend the tabular data interface(s) to include spatial selection operations.

created time in 14 days

push eventplotly/plotly.py

Tarek Allam Jr

commit sha e8b862b7f3c145a66d246586cce723f2d72b4e32

Refine div formatting with `to_html()` + full_html=False (#2469) Remove newlines and double quote strings for better Markdown compatibility

view details

push time in 15 days

issue closedplotly/plotly.py

`.to_html()` method produces incorrectly formatted `<div>` element string for `full_html=False` option

When one invokes plotly.io.to_html(fig, include_plotlyjs=False, full_html=False) the following example output is produced:

'<div>\n        \n        \n            <div id="8a9f01d7-9e37-401b-a602-15b3bbdac943"
class="plotly-graph-div" style="height:100%; width:100%;"></div>\n            <script
type="text/javascript">\n                \n                    window.PLOTLYENV=window.PLOTLYENV ||
{};\n                    \n                if
(document.getElementById("8a9f01d7-9e37-401b-a602-15b3bbdac943")) {\n
Plotly.newPlot(\n                        \'8a9f01d7-9e37-401b-a602-15b3bbdac943\',\n
...

Note the \n characters as well as a single quoted id.

The resulting <div> element appears like this when placed in a markdown document, i.e no plot showing up: image

Note I have ensured to include:

<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>

above the <div> element.

Removing the \n characters and replacing the single quoted id with double quotes seems to resolve this issue.

See PR #2469

closed time in 15 days

tallamjr

PR merged plotly/plotly.py

div formatting with `to_html()` + full_html=False

Code PR

  • [x] I have read through the contributing notes and understand the structure of the package. In particular, if my PR modifies code of plotly.graph_objects, my modifications concern the codegen files and not generated files.

The inclusion of newlines "\n" as well as the plot id tag being referenced by single quotes instead of double quotes results in the following output of a figure:

This has been resolved by ensuring now newline characters are included in the formatted string for and base_url_line now encapsulates plotly_platform_url in double quotes.

This has been tested in a markdown setting:

Before: image

After: image

Fixes #2468

+21 -22

1 comment

1 changed file

tallamjr

pr closed time in 15 days

pull request commentplotly/plotly.py

div formatting with `to_html()` + full_html=False

Looks fine to me. Thanks @tallamjr!

tallamjr

comment created time in 15 days

issue commentholoviz/holoviews

Selection Expression Serialization

FWIW, the pickle serialization problem is probably what I was seeing in https://github.com/holoviz/holoviews/issues/4581, which was fixed on master in https://github.com/holoviz/holoviews/pull/4582.

That said, it would also be nice to a human and machine readable text representation.

c-conner

comment created time in 15 days

pull request commentplotly/plotly.js

Bump mapbox-gl v1.12.0

Which tests are failing locally?

archmoj

comment created time in 15 days

issue commentholoviz/holoviews

Add method to unlink link_selections

Do you have in mind what this would mean? Are you picturing trying to fully invert the link_selections transformation, or keeping the element selectable, but severing it from being linked to other elements?

The former case doesn't feel well defined at first thought. But I think the latter case is something that we could reason about on top of https://github.com/holoviz/holoviews/pull/4572.

philippjfr

comment created time in 15 days

pull request commentplotly/plotly.py

Fix selection behaviour of histograms

BTW, https://github.com/meffmadd/pandas-visual-analysis looks really cool!

meffmadd

comment created time in 15 days

push eventplotly/plotly.py

meffmadd

commit sha f5bff696c83e8c00c96c687b69e1b8ecb7f00718

Fix FigureWidget selection behaviour of histograms (#2711) * fixes selection behaviour of histogram plots * non ES6 reduce and every * moved variable assignemnt of flatPointIndex inside of if statement * removed line at the end of the last if statement * sorting of point indices

view details

push time in 15 days

PR merged plotly/plotly.py

Reviewers
Fix selection behaviour of histograms

This change addresses the selection behaviour in histograms.

A selection now returns the point indices in a single list like scatter plots or others. In the past, the selection was just a list of empty objects due to histograms having a different structure of the selected points. More information can be found in my comment on the original issue.

Now the indices of the selection are passed correctly: histogram selection

Fixes #2698.

+38 -9

3 comments

1 changed file

meffmadd

pr closed time in 15 days

issue closedplotly/plotly.py

The selectedpoints attribute for Histogram plots is being set incorrectly

I am working in a Jupyter notebook and when accessing the selectedpoints attribute of a go.Histogram or the points list in the on_selection callback, the list does not contain integer indices of the selected points but instead just a list of objects.

import plotly.graph_objects as go
import numpy as np

np.random.seed(1)
x = np.random.randn(500)

def on_selection_print(obj, points, selector):
    print(points.point_inds)

figure_widget = go.FigureWidget(data=[go.Histogram(x=x)])
figure_widget.update_layout(dragmode="select")    
figure_widget.data[0].on_selection(on_selection_print)
figure_widget

When selecting points in the histogram, the output looks something like: [<object object at 0x7ff062c497f0>, <object object at 0x7ff062c497f0>, <object object at 0x7ff062c497f0>, <object object at 0x7ff062c497f0>, <object object at 0x7ff062c497f0>, <object object at 0x7ff062c497f0>]

A call to figure_widget.data[0].selectedpoints returns the same list.

The expected output would be something like: [0, 12, 13, 26, 31, ..., 2534]

As you can see, all objects are identical (memory address 0x7ff062c497f0) and the objects in the list do not have any attributes. The length of the list corresponds to the number of bars selected if that helps.

I am working with plotly version 4.9.0, ipywidgets version 7.5.1 and notebook version 6.0.1.

UPDATE: I checked how Plotly.js handles selection for histograms and found that it creates an object for each selected bar, which in turn contains all the points for that bar. e.g.

{
    points: [
        0: {..., pointIndices: [1,2,3,4], ...},
        1: {..., pointIndices: [5,6,7,8,9], ...},
        ...
    ],
    range: [...]
}

Codepen can be found here: https://codepen.io/meffmadd/pen/zYqGmaw

So the problem seems to be that the points are not correctly serialised for histograms. Without understanding the whole codebase I think there seem to be two options:

  1. Update the setter to _js2py_pointsCallback when the figure is a histogram in https://github.com/plotly/plotly.py/blob/efc66509526bdde772e8a8caaf1bcee865ccec3a/packages/javascript/plotlywidget/src/Figure.js#L1134
  2. Updating the model for the histogram on the Python side to reflect the behaviour of the JavaScript implementation. I don't know how exactly this would work yet.

Any pointers in the right direction are highly appreciated!

closed time in 15 days

meffmadd

pull request commentplotly/plotly.py

Fix selection behaviour of histograms

Looks and works great @meffmadd, thanks!

meffmadd

comment created time in 15 days

pull request commentplotly/plotly.py

Fix selection behaviour of histograms

Hi @meffmadd, thanks for the PR. I'll look it over soon. Sorry it slipped through cracks initially!

meffmadd

comment created time in 15 days

pull request commentplotly/plotly.py

full_figure_for_development

OK, all done! I had to add .decode("utf-8") to get bytes into strings for the Python 3.5 optional tests to pass ... normal? that'll work on 2.7 also right?

Not sure about the Python 3.5 thing specifically, but decoding explicitly to utf-8 is the right thing to do.

nicolaskruchten

comment created time in 16 days

Pull request review commentplotly/plotly.py

full_figure_for_development

 def write_image(         file.write(img_data)  -__all__ = ["to_image", "write_image", "scope"]+def full_figure_for_development(fig, warn=True, as_dict=False):+    """+    Compute default values for all attributes not specified in the input figure and+    returns the output as a "full" figure. This function calls Plotly.js via Kaleido+    to populate unspecified attributes. This function is intended for interactive use+    during development to learn more about how Plotly.js computes default values and is+    not generally necessary or recommended for production use.++    Parameters+    ----------+    fig:+        Figure object or dict representing a figure++    warn: bool+        If False, suppress warnings about not using this in production.++    as_dict: bool+        If True, output is a dict with some keys that go.Figure can't parse.+        If False, output is a go.Figure with unparseable keys skipped.++    Returns+    -------+    plotly.graph_objects.Figure or dict+        The full figure+    """++    # Raise informative error message if Kaleido is not installed+    if scope is None:+        raise ValueError(+            """+Image export using the "kaleido" engine requires the kaleido package,

Should probably refer to full_figure_for_development instead of Image export here

nicolaskruchten

comment created time in 17 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventholoviz/holoviews

Philipp Rudiger

commit sha 517ff243a6b6522ef872a23899ba34213a80968a

Fix deprecation warning for matplotlib cmaps (#4573)

view details

Philipp Rudiger

commit sha d4ba973a9fbe71775bb50441f4e42ba1d4a6a192

Ensure propagated options are correctly applied in batched mode (#4574) * Ensure propagated options are correctly applied in batched mode * Fixed flake

view details

Philipp Rudiger

commit sha 380a70cfff6c28fcb0a9e838bd3709a6f72ce55c

Simplify test env (#4576) * Simplify test env * Attempt to drop conda-forge

view details

Philipp Rudiger

commit sha c3c01f73c86b628fede38688073330b850fef166

Ensure plotly TriSurfacePlot raises SkipRendering error if SciPy is missing

view details

Philipp Rudiger

commit sha 27f13ec8ed964b1bf5b80e128339b5d8a0c329a4

Allow rendering to pgf in matplotlib (#4577)

view details

Philipp Rudiger

commit sha b7b580636feb9b7c4408bf621571afa3a724c6c1

Add ability to supply transforms for all dimensions (#4578)

view details

Philipp Rudiger

commit sha d584e1ba16e9b90d03cde8793bd3c04b7c7c1864

Fix aspect if width/height has been constrained (#4579) * Fix aspect if width/height has been constrained * Update holoviews/plotting/bokeh/element.py

view details

Jon Mease

commit sha 5798606a7a92700f56b1f5aca9339287a00a9f38

Add __getstate__ / __setstate__ method to dim to fix RecursionError (#4582)

view details

Philipp Rudiger

commit sha b4fb89cc9d14ab9f77a250bc61092b7ed5462fd4

Handle rcParam deprecations in matplotlib (#4583)

view details

Jon Mease

commit sha 01874dba4cc99b2bdc0348674e029902684faa89

Treat kdims with the same name as identical during decollation Handles case where dims of the outer objects in an object tree have labels and dims for inner objects don't.

view details

Jon Mease

commit sha c530ee65150ae09814bc8e195eaf46e8516c66dd

Merge branch 'master' into link_selections_refactor

view details

Jon Mease

commit sha 7555653b43cf2ae35325716e323576a62008dcaf

Merge remote-tracking branch 'origin/master' into link_selections_refactor

view details

push time in 17 days

pull request commentholoviz/holoviews

Link selections streams refactor

In 3492f0a, 64b6f9c, and 4547b36 I added a few Plotly-specific changes related to linked selections, but not directly needed by the core link_selections refactor here.

Happy to move these to a separate PR if that would make it easier to review.

jonmmease

comment created time in 17 days

push eventholoviz/holoviews

Jon Mease

commit sha 64b6f9c694ce415ddcc6aa24f09d3034c6b66408

hide Plotly HVLine/HVSpan if coords are None Plotly chooses default values automatically in this case, so easier to just hide the shapes until there is valid data

view details

Jon Mease

commit sha 4547b362f7514fa8edc9930b892a7e864aafd0e3

Handle Dash-style axis range event data e.g. {'xaxis.range[0]': 1, 'xaxis.range[1]': 4} instead of {'xaxis.range': [1, 4]}

view details

push time in 17 days

issue closedplotly/Kaleido

[SOLVED] Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working because of selinux policy [EDITED]

Hi! Thank you for the hardwork,

I have a question regarding Kaleido implementation of Centos - Nginx - Gunicorn - Django - Kaleido on a Digital Ocean Droplet using Cloudflare SSL.

It works seamlessly on my local development env, Ubuntu/Windows, but not in the mentioned environment (staging).

As seen from the gunicorn status below, the worker(s) are exiting and rebooting when I try exporting plots using fig.write_image function.

● gunicorn.service - Gunicorn Django Daemon
   Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2020-08-26 16:47:48 UTC; 56s ago
 Main PID: 288482 (gunicorn)
    Tasks: 37 (limit: 11328)
   Memory: 276.9M
   CGroup: /system.slice/gunicorn.service
           ├─288482 /home/lqophe/pheEnv/bin/python /home/lqophe/pheEnv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/lqophe/pheLQO/pheLQO.sock pheLQO.wsgi:application
           ├─288485 /home/lqophe/pheEnv/bin/python /home/lqophe/pheEnv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/lqophe/pheLQO/pheLQO.sock pheLQO.wsgi:application
           ├─288488 /home/lqophe/pheEnv/bin/python /home/lqophe/pheEnv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/lqophe/pheLQO/pheLQO.sock pheLQO.wsgi:application
           ├─288506 /bin/bash /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/kaleido plotly --disable-gpu --plotlyjs='/home/lqophe/pheEnv/lib/python3.6/site-packages/plotly/package_data/plotly.min.js' --mathjax='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js'
           ├─288511 ./bin/kaleido --no-sandbox --allow-file-access-from-files --disable-breakpad plotly --disable-gpu --plotlyjs='/home/lqophe/pheEnv/lib/python3.6/site-packages/plotly/package_data/plotly.min.js' --mathjax='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js'
           ├─288513 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=zygote --no-zygote-sandbox --no-sandbox --headless --headless
           ├─288514 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=zygote --no-sandbox --headless --headless
           ├─288527 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=gpu-process --field-trial-handle=7383061888025097530,4985302504621741626,131072 --no-sandbox --disable-breakpad --headless --ozone-platform=headless --headless --gpu-preferences=OAAAAAAAAAAgAAAgAAAAAAAAAAAAAAAAAABgAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAA== --use-gl=swiftshader-webgl --override-use-software-gl-for-tests --shared-files
           ├─288529 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=utility --field-trial-handle=7383061888025097530,4985302504621741626,131072 --lang=en-US --service-sandbox-type=network --no-sandbox --use-gl=swiftshader-webgl --headless --shared-files
           └─288562 /home/lqophe/pheEnv/bin/python /home/lqophe/pheEnv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/lqophe/pheLQO/pheLQO.sock pheLQO.wsgi:application

Agu 26 16:47:48 staging-centos-s-2vcpu-2gb-sgp1-01 systemd[1]: Started Gunicorn Django Daemon.
Agu 26 16:47:48 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:48 +0000] [288482] [INFO] Starting gunicorn 20.0.4
Agu 26 16:47:48 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:48 +0000] [288482] [INFO] Listening at: unix:/home/lqophe/phe>
Agu 26 16:47:48 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:48 +0000] [288482] [INFO] Using worker: sync
Agu 26 16:47:48 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:48 +0000] [288485] [INFO] Booting worker with pid: 288485
Agu 26 16:47:49 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:48 +0000] [288487] [INFO] Booting worker with pid: 288487
Agu 26 16:47:49 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:47:49 +0000] [288488] [INFO] Booting worker with pid: 288488
Agu 26 16:48:33 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:48:33 +0000] [288482] [CRITICAL] WORKER TIMEOUT (pid:288487)
Agu 26 16:48:33 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 23:48:33 +0700] [288487] [INFO] Worker exiting (pid: 288487)
Agu 26 16:48:33 staging-centos-s-2vcpu-2gb-sgp1-01 gunicorn[288482]: [2020-08-26 16:48:33 +0000] [288562] [INFO] Booting worker with pid: 288562

closed time in 17 days

irwanOyong

PR opened holoviz/holoviews

Add __getstate__ / __setstate__ method to dim to fix RecursionError

Closes https://github.com/holoviz/holoviews/issues/4581

The Recursion error was being caused by bouncing between the __dir__ and __getattr__ methods while unpickling. This adds __getstate__/__setstate__ methods to explicitly implement pickling.

+13 -1

0 comment

2 changed files

pr created time in 17 days

create barnchholoviz/holoviews

branch : dim_serialize

created branch time in 17 days

issue openedholoviz/holoviews

unpickling dim object causes RecursionError

When unpickling a holoviews.util.transform.dim object, a RecursionError is raised.

from holoviews.util.transform import dim
import pickle
pickle.loads(pickle.dumps(dim("a")))
...
    ns = self._ns
  File "/home/jmmease/PyDev/repos/holoviews/holoviews/util/transform.py", line 271, in __getattr__
    if attr in dir(self):
  File "/home/jmmease/PyDev/repos/holoviews/holoviews/util/transform.py", line 277, in __dir__
    ns = self._ns
  File "/home/jmmease/PyDev/repos/holoviews/holoviews/util/transform.py", line 271, in __getattr__
    if attr in dir(self):
RecursionError: maximum recursion depth exceeded

PR on the way

created time in 17 days

push eventholoviz/holoviews

Jon Mease

commit sha 3492f0a14025dd47a803be3124ceef5e1a480b81

Refactor Plotly callback classes to expose stateless get_event_data_from_property_update

view details

push time in 17 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Hi @irwanOyong, great! Yeah, I wasn't recommending disabling selinux permanently, just to test things out. I haven't worked with selinux much, but there must be some way to allow the execution of individual executables.

The native executable will be located in a directory under

/path/to/site-packages/kaleido/executable/

So the trick will be working out how to tell selinux to allow execution from this directory. If you work out a solution for allowing this, please add it to the issue here! Also, please let us know if you come across information on anything that we could do on our end to avoid getting flagged.

irwanOyong

comment created time in 18 days

issue commentplotly/plotly.py

[BUG] Plotly Express (px) pandas datetime with timezone converted to UTC

I would favor fixing this since it is a bug with respect to the v4 change log.

m0wer

comment created time in 18 days

pull request commentconda-forge/staged-recipes

Adding Kaleido

Another friendly ping for feedback on https://github.com/conda-forge/staged-recipes/pull/12093#issuecomment-672991369 @conda-forge/help-python-c @conda-forge/help-python. Thanks!

jonmmease

comment created time in 18 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Thanks @irwanOyong, this error message is helpful.

After a little searching, I've seen a few references to this kind of error being causes by selinux security policies. e.g. https://forums.gentoo.org/viewtopic-t-1114806-start-0.html.

I'm not very familiar with selinux, but it would be worth checking if this is installed on the droplet and to try disabling it if so: https://www.tecmint.com/disable-selinux-in-centos-rhel-fedora/.

irwanOyong

comment created time in 18 days

push eventholoviz/holoviews

Jon Mease

commit sha 33a0b3790e6a705e69feb78e21ed6dccd7a82289

Remove stray type hint

view details

push time in 20 days

PR opened holoviz/holoviews

Link selections streams refactor

Description of Changes

This PR refactors the link_selections transformation to fully rely on chained streams converting linked stream interactions (BoundsXY and Lasso) into linked selection interactions.

This is done by extracting much of link_selections logic into two new stream classes:

  • SelectionExprSequence represents the selection produced by a sequence of selection operations on a single plot (e.g. when selection_mode is "intersect"). SelectionExprSequence is a Derived stream subclass that uses as input stream a History stream wrapping a SelectionExpr stream. It is configured by a mode property that corresponds to the link_selections.selection_mode parameter. It produces a selection_expr and region_element params that represent the series of selections on the plot, combined according to the current mode.

  • CrossFilterSet represents the combined selection across all linked plots. CrossFilterSet is a Derived stream subclass that uses as input one SelectionExprSequence stream per linked plot. It produces a combined selection_expr param. It is supports a mode argument that corresponds to the link_selections.cross_filter_mode param.

The SelectionDisplay for each linked plot now gets the combined selection_expr from the CrossFilterSet stream, and it gets the region element from the SelectionExprSequence stream associated with the plot.

Motivation

The link_selections class still registers callbacks on it's own params to update values of non-interactive streams (e.g. setting the selected_color param pushes an update to the internal _Cmap stream), but the linked stream updates are handled using Derived and History streams. This makes it possible to introspect the Dynamic object produced by link_selections and find the linked streams that are driving it. Along with the new decollate transformation logic, this will make it possible to add support for displaying selection linked plots using Dash.

Testing

The current link_selections test suite has been updated, and I ran through the documentation notebook and everything looks like it's working the same way to me. But it would be good for someone else to run through that notebook and play with it by hand as well to check for regressions.

Thanks!

+377 -158

0 comment

5 changed files

pr created time in 20 days

push eventholoviz/holoviews

Jon Mease

commit sha 180d021f1769307fcd9be3e44d0c5e304f814232

Remove PlotReset stream from SelectionExprSequence. This is handled by `link_selection` by clearing history

view details

push time in 20 days

create barnchholoviz/holoviews

branch : link_selections_refactor

created branch time in 21 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Ok, so it sounds like it specifically a problem under gunicorn. And again, to double check, running with gunicorn on your local development machine work properly?

If it were a problem with gunicorn everywhere, then I'd suspect it has something to do with gunicorn's process forking model, but if it's only a problem on specific os configuration, that doesn't make as much sense.

irwanOyong

comment created time in 21 days

pull request commentholoviz/holoviews

Add decollate transformation

Thanks!

jonmmease

comment created time in 21 days

pull request commentholoviz/holoviews

Add decollate transformation

Not sure what's going on with the tests. But the same things are failing here as on master (https://github.com/holoviz/holoviews/commit/1a366bcb0440a8fdb50e583f4acefa83dcc87d31).

jonmmease

comment created time in 22 days

push eventholoviz/holoviews

Jon Mease

commit sha 19a9715af0c884db65e9b5e06ea0deaa92102ed9

Remove unused internal function

view details

Jon Mease

commit sha 5d70107aedd3875e4345b388bc76095eb9f9d787

Add stream clone method

view details

push time in 22 days

pull request commentholoviz/holoviews

Add decollate transformation

Thanks @jlstevens!

Got some weird CI errors, so I merged in master and pushing again :crossed_fingers:

jonmmease

comment created time in 22 days

push eventholoviz/holoviews

Justin Bois

commit sha 3d07924ac67ec1f5a94f4bae61fd8d4f4135288e

Fixed whiskers for box-whisker so that they never point inwards. (#4548)

view details

Jean-Luc Stevens

commit sha 89be3069a48f6dc311e70ee97f9e180635302ac4

Updated spreading operation to support aggregate arrays (#4562)

view details

Andrew

commit sha 79b8624776b462d51407236489929e4c5ddca08a

legend_opts implemented (#4558)

view details

Philipp Rudiger

commit sha 265fa84cd5999e7a76d2a6819cf54dc5bb14abf9

Compatibility with bokeh 2.2 for CDSCallback (#4568) * Compatibility with bokeh 2.2 for CDSCallback * Fix flake

view details

Jon Mease

commit sha cf57b8874043ed016adacb66e14dc76c00980da1

Add History stream (#4554) * Add History stream and tests * Python 2 compat * more Python 2 compat

view details

Andrew

commit sha 4673965509b62a14bb4b1c9413255aa7064082aa

Hover mode (#4527) * Allow vline/hline in tools * Add doc * Change to list comprehension Co-authored-by: ahuang11 <ahuang11@illinois.edu>

view details

Philipp Rudiger

commit sha 1a366bcb0440a8fdb50e583f4acefa83dcc87d31

Fix issues with boomeranging events when aspect is set (#4569) * Fix issues with boomeranging events when aspect is set * Fixed test

view details

Jon Mease

commit sha 3744e078c127ace42e2981e6670a0dbbb68668aa

Merge remote-tracking branch 'origin/master' into uncollate

view details

push time in 22 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Yes, I can assure that it happens right on this line of code fig.write_image("repository/bubble-pie/well_{}.png".format(well), engine="kaleido").

Ok, thanks for confirming.

Also to clarify, do we need GPU to finish the write_image with kaleido? I see --disable-gpu argument here but just to make sure.

No, a GPU is not required.

I don't have any other ideas off hand. Do you have any other logging from your app itself? Can you tell if the write_image call is hanging indefinitely vs. raising an exception? Have you tried with a single gunicorn worker?

irwanOyong

comment created time in 22 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Just to clarify, everything works when you comment out the write_image call?

irwanOyong

comment created time in 22 days

pull request commentholoviz/holoviews

Add decollate transformation

Thanks @jlstevens, I renamed HoloViewsExpr to Expr but I'll wait for @philippjfr input on whether to add some lower-level tests for the to_expr_extract_streams function itself. I think the functionality is covered pretty well by the decollate tests, but I'm happy to add some lower level tests as well.

jonmmease

comment created time in 22 days

Pull request review commentholoviz/holoviews

Add decollate transformation

+import param+from .operation import OperationCallable+from .. import (+    Layout, DynamicMap, Element, Callable, Overlay, GridSpace, NdOverlay, HoloMap+)+from . import ViewableTree+from collections import namedtuple+from ..streams import Stream, Derived++from ..plotting.util import initialize_dynamic++HoloViewsExpr = namedtuple("HoloviewsExpr", ["fn", "args", "kwargs"])

Done in https://github.com/holoviz/holoviews/pull/4551/commits/1e0aeb67d938e7342ef7bca3f948833f4c5cdd4e

jonmmease

comment created time in 22 days

PullRequestReviewEvent

Pull request review commentholoviz/holoviews

Add decollate transformation

 def transform_function(cls, stream_values, constants):         v = sum([val["v"] for val in stream_values if val["v"]])         return dict(v=v + constants['base']) -Val = Stream.define("Val", v=0.0)++class Val(Stream):+    v = param.Number(constant=True)

Actually, this does work. Not sure what I was running into during development. In https://github.com/holoviz/holoviews/pull/4551/commits/711f082e81b13e17824e054c55ac994b71047156 I reverted some of these to use Stream.define so that there is a mix of both in the tests.

jonmmease

comment created time in 22 days

PullRequestReviewEvent

push eventholoviz/holoviews

Jon Mease

commit sha 1e0aeb67d938e7342ef7bca3f948833f4c5cdd4e

Renmae HoloViewsExpr -> Expr

view details

Jon Mease

commit sha 711f082e81b13e17824e054c55ac994b71047156

Use mix of Stream subclasses and Stream.define to make sure both work

view details

push time in 22 days

Pull request review commentholoviz/holoviews

Add decollate transformation

 def transform_function(cls, stream_values, constants):         v = sum([val["v"] for val in stream_values if val["v"]])         return dict(v=v + constants['base']) -Val = Stream.define("Val", v=0.0)++class Val(Stream):+    v = param.Number(constant=True)

The reason I changed that was that I was cloning streams using:

cloned_stream = type(stream)(**stream.contents)

And this wasn't working for streams build with Stream.define. Is there a better way to clone streams? Maybe we should look at adding a Stream.clone method.

jonmmease

comment created time in 22 days

PullRequestReviewEvent

Pull request review commentholoviz/holoviews

Add decollate transformation

+import param+from .operation import OperationCallable+from .. import (+    Layout, DynamicMap, Element, Callable, Overlay, GridSpace, NdOverlay, HoloMap+)+from . import ViewableTree+from collections import namedtuple+from ..streams import Stream, Derived++from ..plotting.util import initialize_dynamic++HoloViewsExpr = namedtuple("HoloviewsExpr", ["fn", "args", "kwargs"])

sure!

jonmmease

comment created time in 22 days

PullRequestReviewEvent

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

I think those lines also come from the Chromium, no?

They are flags that are accepted by Chromium, but I'm not clear on how they are being set. Here are the flags that we set when calling the kaleido executable, which passes them on to Chromium: https://github.com/plotly/Kaleido/blob/master/repos/linux_scripts/launch_script. I guess these may be subprocesses that chromium launches internally.

Are you using the --preload gunicorn flag? If so, could you try calling gunicorn without it? And if not, try calling gunicorn with it?

irwanOyong

comment created time in 22 days

issue commentplotly/Kaleido

Centos - Nginx - Gunicorn - Django - Kaleido on Digital Ocean Droplet not working as expected

Hi @irwanOyong,

Can you SSH into the droplet and try exporting an image from a python/ipython session? This will help narrow in on where things are going wrong.

Also, FWIW, these lines look odd

           ├─288511 ./bin/kaleido --no-sandbox --allow-file-access-from-files --disable-breakpad plotly --disable-gpu --plotlyjs='/home/lqophe/pheEnv/lib/python3.6/site-packages/plotly/package_data/plotly.min.js' --mathjax='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js'
           ├─288513 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=zygote --no-zygote-sandbox --no-sandbox --headless --headless
           ├─288514 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=zygote --no-sandbox --headless --headless
           ├─288527 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=gpu-process --field-trial-handle=7383061888025097530,4985302504621741626,131072 --no-sandbox --disable-breakpad --headless --ozone-platform=headless --headless --gpu-preferences=OAAAAAAAAAAgAAAgAAAAAAAAAAAAAAAAAABgAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAA== --use-gl=swiftshader-webgl --override-use-software-gl-for-tests --shared-files
           ├─288529 /home/lqophe/pheEnv/lib/python3.6/site-packages/kaleido/executable/bin/kaleido --type=utility --field-trial-handle=7383061888025097530,4985302504621741626,131072 --lang=en-US --service-sandbox-type=network --no-sandbox --use-gl=swiftshader-webgl --headless --shared-files

The top line lists the chromium command line arguments that kaleido uses. But the following lines list a bunch of command line arguments that aren't specified by kaleido (e.g. --type=zygote, --headless, etc.). Do you know where these arguments are coming from?

irwanOyong

comment created time in 22 days

IssuesEvent

issue closedplotly/Kaleido

Support additional architectures

Right now we are building Kaleido on 64-bit MacOS, Linux, and Windows. Folks may eventually be interested in builds for other architectures.

In addition to those above, it looks like Chromium also supports 32-bit Windows and Linux, and 32-bit ARM.

If you're interested in helping out with adding support for an additional architecture, please chime in here!

closed time in 24 days

jonmmease

issue commentplotly/Kaleido

Support additional architectures

Update: As of 0.0.3.post1, win-32 and linux-arm64 builds are available in addition to win-64, linux-64, and macos-64.

Some other architectures that could potentially be supported in the future:

  • win-arm64: https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md
  • linux-arm (32-bit I think): https://chromium.googlesource.com/chromium/src.git/+/master/docs/linux/chromium_arm.md
jonmmease

comment created time in 24 days

issue closedplotly/Kaleido

How to package the kaleido executable

Hi there @jonmmease , thank you for reaching out, i really appreciate it!

Adding a lightweight F# wrapper for kaleido seems to be pretty straightforward, but how exactly should one package the kaleido executable?

To incorporate Kaleido in the .NET ecosystem one would have to provide a Nuget package. Is it better to include the executable in the package (am i even allowed to do that license wise?), or will you/the plotly org host the kaleido executable as standalone package that can be consumed from the wrapper?

closed time in 24 days

kMutagene

issue commentplotly/Kaleido

How to package the kaleido executable

I think this is established for the time being. Releases are included as zip files in GitHub releases https://github.com/plotly/Kaleido/releases. Going to close, but happy to reopen if there is anything actionable that would improve the situation for folks.

kMutagene

comment created time in 24 days

issue closedplotly/Kaleido

Windows pipenv install error

I'm having difficulty installing kaleido. I've tried pipenv, pip and pip3 to no avail.

I'm a relative newbie to Python so I'm sure it's my issue :)

py --version gives Python 3.8.1

# pipenv install kaleido
Installing kaleido…
Error:  An error occurred while installing kaleido!
Error text:
ERROR: Could not find a version that satisfies the requirement kaleido (from -r c:\users\...\pipenv-cgfgya4v-requirements\pipenv-9c77lr8r-requirement.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for kaleido (from -r c:\users\...\pipenv-cgfgya4v-requirements\pipenv-9c77lr8r-requirement.txt (line 1))

Installation Failed
# pip (and pip3) install kaleido
ERROR: Could not find a version that satisfies the requirement kaleido (from versions: none)
ERROR: No matching distribution found for kaleido
WARNING: You are using pip version 20.1.1; however, version 20.2.1 is available.
You should consider upgrading via the 'c:\users\...\python37-32\python.exe -m pip install --upgrade pip' command.

closed time in 24 days

Dynatrace-Adam-Gardner

issue commentplotly/Kaleido

Windows pipenv install error

Win32 build released to PyPI in 0.0.3.post1

Dynatrace-Adam-Gardner

comment created time in 24 days

pull request commentholoviz/holoviews

Add decollate transformation

Renamed to decollate. This should be ready for review.

jonmmease

comment created time in 24 days

push eventholoviz/holoviews

Jon Mease

commit sha 9d4d15d79c91a90739a7beca5aa965856f5dcf66

rename uncollate -> decollate

view details

push time in 24 days

pull request commentholoviz/holoviews

Add History stream

It's ready on my end. Thanks @jlstevens

jonmmease

comment created time in 24 days

startedlife4/deal

started time in a month

created tagplotly/Kaleido

tagv0.0.3.post1

Fast static image export for web-based visualization libraries with zero dependencies

created time in a month

more