Reflection Microscopy: Ready to Use Neural Networks for Wavefront Correction in Adaptive Optics


Imaging and Microscopy

Ref.-No.: 2022-6510-FG

The invention relates to a method for wavefront correction in adaptive optics compatible with laser scanning microscopy based on reflected light and deep neural networks. For network training sample aberrations are generated in the excitation and detection path by means of spatial light modulators, and the corresponding reflected focus images are recorded by CMOS cameras. After training, the network can disentangle and independently correct excitation and detection aberrations. The predicted aberration corrections are validated using two photon imaging.

Advantages

  • Improves resolution and intensity of the image in an efficient way
  • Enables modelling of complex optical paths which otherwise are difficult to be modelled
  • Suitable for epi-detection configurations

Applications

  • Laser scanning microscopy, particularly multi-photon microscopy
  • Biological imaging

Fig. 1: setup: excitation and detection pathways are each controlled with a spatial light modulator (SLM); reflected light is imaged in an epi-detection configuration onto three camer-as (CMOS 1-3); fluorescence is observed using two-photon scanning microscopy and a photomultiplier tube (PMT); MO = microscope objective, DC = dichroic mirror, SM = scanning mirrors, PBS = polarizing beam splitter, PH = pinhole.

Background

Imaging in many samples, particularly in biological tissue, with resolution performance at the diffraction limit of the microscope is limited due to aberrations and scattering. In this situation, laser scanning combined with adaptive optics can be used to improve the image resolution; thereby the adaptive optics shapes the wavefront in such a way as to correct for aberrations. In reflectionmode imaging, however, the excitation point spread function (PSF) and the detection PSF can undergo different aberrations, and therefore, for an accurate aberration correction, the contributions to aberrations accumulated in the excitation and reflected detection pathway need to be disentangled. This can be achieved by using matrix methods (for strongly scattering samples) or a wavefront sensor (for weakly scattering samples), which is technically demanding and/or restricted to a combination of sample and detection characteristics. Approaches for wavefront sensing based on deep neural networks, which have been developed so far, work only for configurations that require the correction of a single pass through a scattering medium, and hence cannot be applied for epi-detection detection configurations. Hence, there is a need to extend these approaches to reflection-mode imaging in an epi-detection configuration.

Technology

Fig. 2: aberration correction: a, left: corrected and uncorrected image of fluorescent beads distributed on a reflector surface imaged through a layer of vacuum grease; a, center: white frame in left figure; a, right: axial cross section through lines in the center figure recorded in a z-stack with 1 μ m step size; b, top: uncorrected focus at center of field-of-view in "a"; b, bottom: corrected based on reflected light (colorscale in the corrected image is saturated); c: cross sections for uncorrected (black) and corrected (red) images along the lines indicated in "a, center"; d, left: excitation phase mask; d, right: detection phase mask; all scale bars are 5 μm.

The two-photon laser scanning microscope depicted in Fig. 1 includes spatial light modulators (SLM) and reflected light detection for wavefront sensing and correction. The reflected focal spot is monitored at three different focal planes using cameras. As the microscope has a SLM in the excitation and detection path, it can disentangle the aberrations accumulated in these paths. The microscope corrects a reflection image and/or fluorescence image by: a) radiating a light distribution into an excitation path of the microscope, wherein the excitation path guides the light distribution I0 to a sample and, while entering the sample, I0 is distorted to a light distribution IA,Probe, by scattering; b) reflecting the light distribution IA,Probe at the sample into the detection path, wherein, while exiting the sample, IA,Probe is distorted to form the light distribution ID,Probe; c) recording the reflected light distribution ID,Probe; d) transferring ID,Probe to a mathematical model which models the light propagation in the microscope; e) outputting a 2-tuple (MA, MD), wherein MA describes the distortion of the light distribution I0 while entering the sample and MD describes the distortion of the light distribution IA,Probe while exiting the sample; f) setting the complementary distortion MA* on the SLM in the excitation path and the complementary distortion MD* on the SLM in the detection path. The mathematical model is implemented as a neural network, which is trained by the following steps: a’) radiating a light distribution I in the excitation path; b’) modulating the light I with the modulation MA with an SML to generate the light distribution IA; c’) reflecting the light IA at the sample’s position into the detection path; d’) modulating the reflected light IA with the modulation MD with another SML to generate the light distribution ID; e’) recording the light distribution ID; repeating the steps a’) to e’) n times, thereby generating n tuples (MA, MD, ID); and adapting the neural network, i.e. the mathematical model that describes the light propagation in the microscope, based on the n tuples (MA, MD, ID). Fig. 2 shows the effect of the aberration correction achieved with the microscope described above.

Patent Information

  • Patent DE 10 2020 109 734 B4
  • Application WO 2021/204663 A1

Publications

Ivan Vishniakou and Johannes D. Seelig, Wavefront Correction for adaptive optics with reflected light and deep neural networks, Optics Express, Vol. 28, No. 10 / 11 May 2020 / 15459

PDF Download

Contact

Patent- & License Manager

Dr. Franz Gadelmeier

Physicist

Phone: +49 89 / 29 09 19-13
Email:
gadelmeier@max-planck-innovation.de