Microscopy: Differentiable model-based adaptive optics with transmitted and reflected light

Imaging and Microscopy

Ref.-No.: 2022-6511-FG

The invention relates to a method and illumination device for optimizing parameters of a physical light propagation model, in particular a light propagation model in a confocal laser microscope using a raster scanning method. The method can determine aberration corrections that go beyond a predetermined model of aberrations, such as combina-tions of Zernike modes.


  • Improves resolution and intensity of the image in an efficient way
  • Needs a small number of measurements for finding aberration corrections
  • Suitable for epi-detection configurations


  • Laser scanning microscopy
  • Biological imaging

Fig. 1: experimental setup:  light reflected off a spatial light modulator (SLM) passes through an aberration (A) and is focused onto a first (transmission) camera; for experiments in an epi-detection configuration, light reflected off the mirror M at the sample plane is additionally recorded with a second (reflection) camera; also shown is the imaged light distribution recorded by each camera; BS = beam splitter, L = lens.


Machine learning offers novel approaches to correct for aberrations encountered when imaging through scattering materials with transmitted and reflected light. However, for training a neural network large datasets are required, because the parameters of the neural network need to be adjusted i) to work under a wide range of conditions and ii) to learn the underlying image generat-ing process through the optical system. The training dataset can be reduced, if information about the image generation process is a priori known. However, training datasets are often based on combinations of Zernike polynomials and hence might not accurately capture all aspects of the aberrations. Additionally, for strongly scattering samples higher orders of Zernike modes are required, which further increases the training dataset. Therefore, for at least those optical sys-tems for which a priori information of the image generation process is known, an improved meth-od for determining/correcting the aberrations in the optical system is needed.


Fig. 2: a uncorrected first focus recorded in transmission (its distortion due to aberrations is evident); b corrected first focus recorded in transmission (focus is surrounded by a white frame and magnified in the insert); c wavefront correction at SLM obtained after a two-step transmission-based optimization and used for aberration correction of the first focus; g uncorrected second focus recorded in transmission (its distortions due to aberrations is evident); h corrected second focus recorded in transmission (insert shows a magnification of this focus); i wavefront correction at SLM recovered only from reflected light in a two-step optimization and used for aberration correction of the second focus; "max" indicates the maximum of the colorbar, η is the enhancement, and the field of view is 1766 μm by 1766 μm.

The method uses a model of the optical system, which describes the light propagation through the optical system as a differentiable function. The model also includes unknown parameters, which account for aberrations occurring in the system and which are to be determined. For determining the unknown aberrations/parameters a number of output images resulting from corresponding input phase modulations are recorded and the unknown parameters are adjusted so that the input-output relationship of the computational model and the experimental setup match. Once the aberration pattern in the optical system is determined, the aberrations can be corrected by setting a complementary aberration pattern into the excitation path of the optical system; which can be done, for instance, by means of a spatial light modulator.

A microscope, employing the method according to the invention, is depicted in Fig. 1. In the microscope an expanded and collimated laser beam is reflected off a spatial light modulator (SLM) with a beam splitter cube BS1 and a part of the beam is imaged onto a transmission camera over a beam splitter BS3. For experiments with reflected light, the remaining part of the beam is additionally sent to a mirror at the sample plane (having the same focal plane as the transmission camera). The mirror serves as a proxy for a reflecting sample. Light reflected by the mirror is imaged onto a reflection camera with a beam splitter BS2. Aberrations are introduced between BS2 and BS3 by means of a scattering element A. The microscope can determine the aberration A by: a) providing a physical, mathematical differentiable model on a computer; b) irradiating an input light distribution I0 by the collimated laser beam into the excitation path of the microscope; c) scattering the light I0 with the scattering element A, thereby transforming I0 to form the transmission light distribution ITR; d) recording the light ITR by means of the transmission camera; e) transferring the recorded light distribution ITR to the model; and f) calculating transmission distortion parameters VTR of the model based on the recorded distribution ITR, wherein the parameters VTR describe the aberrations introduced by the scattering element A in the excitation path.

Also, the microscope can determine the aberration A by: performing the above-mentioned steps a) to c); reflecting the light distribution ITR by the mirror M; scattering the reflected light ITR with element A to generate the reflection light distribution IRE; recording the distribution IRE with the reflection camera; transferring the recorded IRE to the model; and calculating reflection distortion parameters VRE of the model based on IRE. The microscope can correct the aberrations A by setting a distortion pattern V*TR or a pattern V*RE on the SLM, wherein V*TR and V*RE are based on the determined parameters VTR and VRE, respectively. Examples of such corrections are shown in Fig. 2.

Patent Information

  • DE 10 2020 119 566 A1
  • WO 2022/018002 A1


Ivan Vishniakou and Johannes D. Seelig, Differentiable model-based adaptive optics with transmitted and reflected light, Optics Express, Vol. 28, No. 18 / 31 August 2020 / 26436

PDF Download


Patent- & License Manager

Dr. Franz Gadelmeier


Phone: +49 89 / 29 09 19-13