Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

optimized array subsampling #721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Apr 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
9b02b9b
add 4D case with a single z-plane
FlynnOConnell Feb 6, 2025
a52f9fa
add subsample_array function
FlynnOConnell Feb 22, 2025
a34d4fe
use new subsample_array function
FlynnOConnell Feb 22, 2025
d98c14f
remove redundant target_elements from subsample
FlynnOConnell Feb 22, 2025
a67326f
import utils from __all__
FlynnOConnell Feb 22, 2025
c7e0102
rename subsample vars using standard numpy names
FlynnOConnell Feb 22, 2025
991b544
subsample max_items -> max_size
FlynnOConnell Feb 22, 2025
6d212bc
largest bytesize -> largest array size, lowercase errrrything
FlynnOConnell Feb 22, 2025
20333ab
use subsample_array in quick_min_max
FlynnOConnell Feb 22, 2025
4450195
make 1e6 int
FlynnOConnell Feb 22, 2025
f65d9eb
update png from artifact (imgui-screenshots)
FlynnOConnell Feb 28, 2025
fc996a1
fix subsample array
FlynnOConnell Apr 6, 2025
4be2eaa
bring back original image_widget_grid
FlynnOConnell Apr 7, 2025
6fb06ab
Update functions.py
kushalkolar Apr 7, 2025
1311e21
revert image_widget_grid from regenerate
FlynnOConnell Apr 8, 2025
608545d
hopefully the correct screenshot
FlynnOConnell Apr 8, 2025
ea74963
Merge branch 'main' into hlut-4d
FlynnOConnell Apr 8, 2025
af9051e
black
FlynnOConnell Apr 8, 2025
b2b4a81
replace iw-zfish-grid-qreplace nb-iw-zfish-grid-init-mw5 to see
FlynnOConnell Apr 8, 2025
66371a9
replace nb-image-widget screenshots
FlynnOConnell Apr 8, 2025
eb37f25
force git to refresh
FlynnOConnell Apr 9, 2025
4c245d3
nb-iw from regen screenshots
FlynnOConnell Apr 10, 2025
8a785c1
plzplzplzplzplz work
FlynnOConnell Apr 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 3 additions & 2 deletions 5 examples/screenshots/image_widget_grid.png
25 changes: 4 additions & 21 deletions 25 fastplotlib/tools/_histogram_lut.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import pygfx

from ..utils import subsample_array
from ..graphics import LineGraphic, ImageGraphic, TextGraphic
from ..graphics.utils import pause_events
from ..graphics._base import Graphic
Expand Down Expand Up @@ -193,28 +194,10 @@ def _fpl_add_plot_area_hook(self, plot_area):
self._plot_area.controller.enabled = True

def _calculate_histogram(self, data):
if data.ndim > 2:
# subsample to max of 500 x 100 x 100,
# np.histogram takes ~30ms with this size on a 8 core Ryzen laptop
# dim0 is usually time, allow max of 500 timepoints
ss0 = max(1, int(data.shape[0] / 500)) # max to prevent step = 0
# allow max of 100 for x and y if ndim > 2
ss1 = max(1, int(data.shape[1] / 100))
ss2 = max(1, int(data.shape[2] / 100))

data_ss = data[::ss0, ::ss1, ::ss2]

hist, edges = np.histogram(data_ss, bins=self._nbins)

else:
# allow max of 1000 x 1000
# this takes ~4ms on a 8 core Ryzen laptop
ss0 = max(1, int(data.shape[0] / 1_000))
ss1 = max(1, int(data.shape[1] / 1_000))

data_ss = data[::ss0, ::ss1]

hist, edges = np.histogram(data_ss, bins=self._nbins)
# get a subsampled view of this array
data_ss = subsample_array(data, max_size=int(1e6)) # 1e6 is default
hist, edges = np.histogram(data_ss, bins=self._nbins)

# used if data ptp <= 10 because event things get weird
# with tiny world objects due to floating point error
Expand Down
70 changes: 64 additions & 6 deletions 70 fastplotlib/utils/functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ def make_colors_dict(labels: Sequence, cmap: str, **kwargs) -> OrderedDict:
return OrderedDict(zip(labels, colors))


def quick_min_max(data: np.ndarray) -> tuple[float, float]:
def quick_min_max(data: np.ndarray, max_size=1e6) -> tuple[float, float]:
"""
Adapted from pyqtgraph.ImageView.
Estimate the min/max values of *data* by subsampling.
Expand All @@ -276,6 +276,9 @@ def quick_min_max(data: np.ndarray) -> tuple[float, float]:
----------
data: np.ndarray or array-like with `min` and `max` attributes

max_size : int, optional
largest array size allowed in the subsampled array. Default is 1e6.

Returns
-------
(float, float)
Expand All @@ -289,11 +292,7 @@ def quick_min_max(data: np.ndarray) -> tuple[float, float]:
):
return data.min, data.max

while np.prod(data.shape) > 1e6:
ax = np.argmax(data.shape)
sl = [slice(None)] * data.ndim
sl[ax] = slice(None, None, 2)
data = data[tuple(sl)]
data = subsample_array(data, max_size=max_size)

return float(np.nanmin(data)), float(np.nanmax(data))

Expand Down Expand Up @@ -405,3 +404,62 @@ def parse_cmap_values(
colors = np.vstack([colormap[val] for val in norm_cmap_values])

return colors


def subsample_array(arr: np.ndarray, max_size: int = 1e6):
"""
Subsamples an input array while preserving its relative dimensional proportions.

The dimensions (shape) of the array can be represented as:

.. math::

[d_1, d_2, \\dots d_n]

The product of the dimensions can be represented as:

.. math::

\\prod_{i=1}^{n} d_i

To find the factor ``f`` by which to divide the size of each dimension in order to
get max_size ``s`` we must solve for ``f`` in the following expression:

.. math::

\\prod_{i=1}^{n} \\frac{d_i}{\\mathbf{f}} = \\mathbf{s}

The solution for ``f`` is is simply the nth root of the product of the dims divided by the max_size
where n is the number of dimensions

.. math::

\\mathbf{f} = \\sqrt[n]{\\frac{\\prod_{i=1}^{n} d_i}{\\mathbf{s}}}

Parameters
----------
arr: np.ndarray
input array of any dimensionality to be subsampled.

max_size: int, default 1e6
maximum number of elements in subsampled array

Returns
-------
np.ndarray
subsample of the input array
"""
if np.prod(arr.shape) <= max_size:
return arr # no need to subsample if already below the threshold

# get factor by which to divide all dims
f = np.power((np.prod(arr.shape) / max_size), 1.0 / arr.ndim)

# new shape for subsampled array
ns = np.floor(np.array(arr.shape) / f).clip(min=1)

# get the step size for the slices
slices = tuple(
slice(None, None, int(s)) for s in np.floor(arr.shape / ns).astype(int)
)
return np.asarray(arr[slices])
Morty Proxy This is a proxified and sanitized view of the page, visit original site.