prysm v0.20#

Summary#

Version 20 of prysm is the largest breaking release the library has ever had. Your programs will be more a bit verbose when written in this style, but they will be more clear, contain fewer bugs, and run faster. This version marks prysm transitioning from an extremely object oriented style to a data oriented style. The result is that code is more direct, and there is less of it. Side benefits are that by deferring the caches that used to help keep prysm fast to the user level, the user is in control over their program’s memory usage. A new high level object oriented API may be produced at some point, likely in a separate package.

This version will produce one more zero point release (0.21) for cleanup after longer experience in this style, after which version 1 will be released. In addition to the breaking changes, this release brings landmark additions:

  • 2D-Q polynomials also known as Forbes polynomials, Chebyshev, Legendre, and Hopkins polynomials,

  • Sophistocated, highly optimized tools for segmented apertures.

  • Tilted plane projections for DMs and other oblique elements

  • Realistic detector noise modeling

  • Bayer focal plane routines

As perhaps a motivational comment, the official model of the Low Order Wavefront Sensing and Control (LOWFS/C) system on the Roman Coronagraph Instrument was ported from prysm v0.19 to v0.20, and runs 12x faster on CPU and 6x faster on GPU. A total of two new lines of code were gained in aggregate. The port took approximately two person-hours. The model now runs in 430 microseconds per wavelength through the 7-plane model; over twice faster than the actual realtime WFSC system!

The remainder of this page will be divided by logical unit of function, then sub-divided between breaking changes and new features.

Changes#

bayer#

This is a new submodule, for working with bayer imaging systems. It provides a complete toolkit for both forward modeling and processing of bayer images, real or synthetic. The following functions are included:

  • wb_prescale() for performing white-balance pre-scaling to mosaiced data in-place.

  • wb_scale() for performing white-balance scaling to RGB data in-place.

  • composite_bayer() for compositing dense color plane data into a bayer mosaic. This function is used to synthesize “raw” bayer imagery in a forward model.

  • decomposite_bayer() for “sifting” bayer subplanes from a mosaiced image.

  • recomposite_bayer() the inverse operation of decomposite_bayer, for taking bayer subplanes and re-mosaicing them. composite_bayer works with fully dense data with (m, n) pixels per color plane. recomposite_bayer works with sparse data with (m/2, n/2) pixels per color plane.

  • demosaic_malvar() for performing Malvar-He-Cutler demosaicing.

conf#

  • All Labels related code has been removed. There is no substitute.

  • Unit munging has been removed; wavelengths are no longer astropy units but are floats with units of microns again.

  • The following parameters have been removed from Config:

    • Q

    • wavelength

    • interpolation

    • unit_format

    • show_units

    • phase_xy_unit

    • phase_z_unit

    • image_xy_unit

    • image_z_unit

    • mtf_xy_unit

    • mtf_z_unit

    • ptf_xy_unit

    • ptf_z_unit

    • pupil_labels

    • interferogram_labels

    • convolvable_labels

    • mtf_labels

    • ptf_labels

    • psd_labels

convolution#

This module has been substantially rewritten. Up to version 0.19, a Convolvable object was the key to the convolution API, which was capable of forming prototypical FFT based convolution, as well as convolution with various analytic blurs, and convolution of datasets which were not equally sampled. The API has been significantly simplified and disentangled in this version.

Breaking:

  • Convolvable no longer exists.

  • the deconv method for Wiener-Helstrom deconvolution no longer exists

The new API is comprised of:

coordinates#

  • GridCache and its variable transformation functions have been deleted. The functionality is deferred to the user, who can quite naturally write code that reuses grids.

  • make_xy_grid() has had its signature changed from (samples_x, samples_y, radius=1) to (shape, *, dx, diameter, grid=True). shape auto-broadcasts to 2D and dx/diameter are keyword only. grid controls returning vectors or a meshgrid. make_xy_grid is now FFT-aligned (always containing a zero sample).

  • make_rho_phi_grid() has been removed, combine make_xy_grid() with cart_to_polar().

  • New warping function suite used to work with non-normal incidence beams (e.g., DMs, OAPs)

degredations#

  • The Smear class has been removed, and replaced with smear_ft()

  • The Jitter class has been removed, and replaced with jitter_ft()

detector#

  • The Detector class has been reworked, and its purpose changed. Previously, it existed to impart blur into a system as would be experienced given a particular pixel design. It now exists to model noise. Expect no API compatibility between v0.19 and v0.20.

  • The OLPF class has been removed, and replaced with olpf_ft()

  • The PixelAperture class has been removed, and replaced with pixel_ft()

  • bindown_with_units() was removed.

  • bindown() will now error if the array dimensions are not an integer multiple of the binning factor. It now supports ND data, with possible unique factors per dimension.

  • tile() has been added, which is the adjoint operation to bindown. It replicates the elements of an array factor times, and has the same ND support bindown now does.

fttools#

  • The matrix DFT executor was mildly reworked. There is no more norm option. The code was modified such that a forward-reverse calculation that goes to any domain containing the majority of the spectrum of the signal and returns to the same input domain will be energy conserving automatically. This means that idft2(dft2(x)) ~= x

geometry#

The geometry module was rewritten. The object oriented mask interface and MaskCache have been removed. All functions now take x, y or r, t args as appropriate, instead of samples. The arguments now all have consistent units.

  • Higher side count regular polygon functions have been removed, use regular_polygon() directly:

    • heptagon()

    • octagon()

    • nonagon()

    • decagon()

    • hendecagon()

    • dodecagon()

    • trisdecagon()

  • inverted_circle() was removed, call ~circle(...) for equivalent output.

io#

  • write_zygo_ascii() no longer takes a high_phase_res parameter. It did not do anything before and has been removed, as it is not likely prysm needs to support ancient version of MetroPro that are incompatible with that convention.

  • the dat and datx readers no longer flip the phase and intensity data upside down. They used to do this due to prysm explicitly having an origin in lower left convention, but v0.20 has no enforced convention for array orientation, and the flipud is an unexpected behavior in this paradigm.

mathops#

The several quasi-identical classes to shim over numpy and scipy were removed and replaced with a single BackendShim type. The engine variable no longer exists. Users should overwrite prysm.backend.np._srcmodule, as well as the same for fft, ndimage, etc.

interferogram#

The interferogram module is largely unchanged. With the removal of astropy units, the user must manage their own units. Phase is loaded from dat/datx files in units of nm.

  • prysm.interferogram.Interferogram.fit_zernikes() was removed, use lstsq from the polynomials submodule with Interferogram.data, Interferogram.x, Interferogram.y, Interferogram.r, Interferogram.t directly, minding spatial axis normalization.

  • prysm.interferogram.Interferogram.remove_piston_tiptilt_power() and prysm.interferogram.Interferogram.remove_piston_tiptilt() have been removed, call remove_piston(), etc, in sequence.

  • prysm.interferogram.Interferogram.mask() now accepts arrays only.

  • filter() has returned to stay and uses a new 2D filter design method behind the scenes. The out-of-band rejection is approximately 50dB higher for typical sized arrays.

jacobi#

See the new polynomials module.

objects#

The changes to this module are similar to geometry. Functions no longer take a samples argument, but take x/y or r,t grids directly. Objects which have analytic fourier transforms retain functions to compute those.

otf#

The OTF module was maed data oriented instead of object oriented, in line with the rest of the changes to prysm in this release. Note that the three functions below accept both arrays, and RichData-like objects with data and dx attributes, and return RichData objects.

polynomials#

prysm’s support of polynomials has been unified under a single package. The polynomials package is now the fastest known for the supported polynomials, e.g. beating POPPY by more than a factor of 100 on large collections of Zernike polynomials. This speed introduces mild complexity into the API, which must be appreciated. For a complete tutorial see Ins and Outs of Polynomials.

  • prysm.polynomials/ - top level routines, common to any basis set:

    • lstsq() for least-squares fitting of 2D basis functions to data

    • sum_of_2d_modes() for (weighted) summing 2D modes. This function does what zernike_compose or zernike_sum does in other packages, once the user has the basis set in hand.

  • sum_of_xy_modes() some polynomial bases, like the Legendre and Chebyshev polynomials, are separable in the x, y dimensions. This function reflects that, and reduces the time complexity from (m*n) per mode to (m+n) per mode. This can bring, for example, a 1000x speedup for 1024x1024 arrays.

    • separable_2d_sequence() for computing a set of separable polynomials, such as the Legendre or Chebyshev polynomials, in 2D, with optimal time complexity.

    • /zernike for Zernike polynomials. These functions are all re-exported at the root of polynomials/:

      • Stand-alone functions for the first few terms have been removed, use zernike_nm with one of the naming convention functions to replace the behavior:

        • piston()

        • tip()

        • tilt()

        • defocus()

        • primary_astigmatism_00()

        • primary_astigmatism_45()

        • primary_coma_y()

        • primary_coma_x()

        • primary_spherical()

        • primary_trefoil_x()

        • primary_trefoil_y()

      • e.g., for primary_coma_y, either zernike_nm(3, 1, ...) or zernike_nm(*zernike_noll_to_nm(7), ...)

      • classes FringeZernike, NollZernike, ANSI1TermZernike, ANSI2TermZernike have been removed. Combine zernike_nm() with one of the naming functions to replace the phase synthesis behavior.

      • zernike_norm() for computing the norm of a given Zernike polynomial, given the ANSI order (n, m).

      • zero_separation() for computing the minimum zero separation on the domain [0,1] for a Zernike polynomial, given the ANSI order (n, m).

      • zernike_nm() for computing a Zernike polynomial, given the ANSI order (n, m).

      • zernike_nm_sequence() – use to compute a series of Zernike polynomials. Much faster than zernike_nm() in a loop. Returns a generator, which you may want to exhaust into a list or into a list, then an array.

      • nm_to_fringe() for converting ANSI (n, m) indices to FRINGE indices, which begin with Z1 for piston.

      • nm_to_ansi_j() for converting ANSI (n, m) indices to ANSI j indices (dual to mono index).

      • noll_to_nm() for converting the Noll indexing scheme to ANSI (n, m).

      • fringe_to_nm() for converting the FRINGE indexing scheme to ANSI (n, m).

      • zernikes_to_magnitude_angle_nmkey() for converting a sequence of [(n1, m1, coef1), ...] to a dictionary keyed by (n, |m|) with the magnitude and angle as the value. This basically converts the “Cartesian” Zernike polynomials to a polar representation.

      • zernikes_to_magnitude_angle() for doing the same as zernike_to_magnitude_angle_nmkey, but with dict keys of the form “Primary Coma” and so on.

      • nm_to_name() for converting ANSI (n, m) indices to a friendly name like “Primary Trefoil”.

      • top_n() for identifying the largest N coefficients in a Zernike series.

      • barplot() for making a barplot of Zernike polynomials, based on their mono index (Z1..Zn)

      • barplot_magnitudes() for doing the same as barplot, but with labels of “Tilt”, “Power”, and so on.

    • /cheby for Chebyshev polynomials. These functions are all re-exported at the root of polynomials/:

      • cheby1(), the Chebyshev polynomial of the first kind of order n

      • cheby2(), the Chebyshev polynomial of the second kind of order n

      • cheby1_sequence(), a sequence of Chebyshev polynomials of the first kind of orders ns; much faster than cheby1 in a loop.

      • cheby2_sequence(), a sequence of Chebyshev polynomials of the second kind of orders ns; much faster than cheby2 in a loop.

    • /legendre for Legendre polynomials. These functions are all re-exported at the root of polynomials/:

      • legendre_sequence(), a sequence of Legendre polynomials of orders ns; much faster than legendre in a loop.

    • /jacobi for Jacobi polynomials. These functions are all re-exported at the root of polynomials/:

      • jacobi(), the Jacobi polynomial of order n with weight parameters alpha and beta

      • jacobi_sequence(), a sequence of Jacobi polynomials of orders ns with weight parameters alpha and beta; much faster than jacobi in a loop.

    • /qpoly for Q (Forbes) polynomials. These functions are all re-exported at the root of polynomials/:

      • Qbfs(), the Q best fit sphere polynomial of order n, at normalized radius x.

      • Qbfs_sequence(), the Q best fit sphere polynomials of orders ns, at normalized radius x. Much faster than Qbfs in a loop.

      • Qcon(), the Q best fit sphere polynomial of order n, at normalized radius x.

      • Qcon_sequence(), the Q conic polynomials of orders ns, at normalized radius x. Much faster than Qcon in a loop.

      • Q2d(), the 2D-Q polynomials of order (n, m). Note that the API is made the same as Zernike by intent, so the sign of m controls whether it is a cosine (+) or sine (-), not a and b coefficients.

      • Q2d_sequence(), the 2D-Q polynomials of orders [(n1, m1), …]. Much faster than Q2d in a loop.

propagation#

  • prop_pupil_plane_to_psf_plane() and prop_pupil_plane_to_psf_plane_units() have been removed, they were deprecated and marked for removal.

  • Any argument which was sample_spacing is now dx.

  • Any coherent argument was removed, all routines now explicitly work with fields (see prysm.propagation.Wavefront.intensity()).

  • Any norm argument was removed.

  • Units are no longer fed through astropy units, but are mm for pupil plane dimensions, um for image plane dimensions, and nm for OPD.

  • Angular spectrum (free space) propagation now allows the transfer function to be computed once and passed as the tf kwarg, accelerating repetitive calculations.

    • See also: ~prysm.propagation.angular_spectrum_transfer_function

  • The focus_units and unfocus_units functions were removed. Since prysm largely bookkeeps dx now, they are superfluous.

psf#

The PSF module has changed from being a core part of propagation usage to a module purely for computing criteria of PSFs, such as fwhm, centroid, etc.

pupil#

  • this entire submodule has been removed. To synthesize pupil functions which have given phase and amplitude, combine prysm.geometry with prysm.polynomials or other phase synthesis code. The function from_amp_and_phase() largely replicates the behavior of the Pupil constructor, with the user generating their own phase and amplitude arrays.

segmented#

This is a new module for working with segmented systems. It contains routines for rasterizing segmented apertures and for working with per-segment phase errors. prysm’s segmented module is considerably faster than anything else in open source, and is approximately constant time in the number of segments. For the TMT aperture, it is more than 100x faster to rasterize the amplitude than POPPY. For more information, see This post. The Notable Telescope Apertures page also contains example usage.

A future update will bring fast per-segment phase errors with a clean API.

qpoly#

See the new polynomials module.

util#

This module is likely to move to prysm.stats in a future release.

wavelengths#

This data-only module has been changed to contain all quantities in units of microns, now that prysm no longer uses astropy.

zernike#

See the new polynomials module.