Electron Microscopy: Ready to Use AI for Focus and Astigmatism Adjustment


Imaging and Microscopy

Ref.-No.: 0202-6337-BC

The invention relates to a new method for imaging setting optimization for electron microscopy
based on deep learning. The AI is a robust and simple to implement optimization possibility for
direct operation with various types of electron microscopes.

For a wide spectrum of applications automated precision electron microscopy is a powerful
method. Existing auto focusing routines are often based on physical models of the microscope
and are hence lacking general applicability and require high execution time making the detailed
imaging of large samples impractical.

The problem was solved by a method based on deep learning. The versatile AI can be used not
only with different microscopes, but also for the optimization of various parameters. After a
specific learning process, it delivers reliable and accurate adjustment corrections in real time.
Pairs of pictures with known parameter variations around the current settings are the only input
needed for the optimization calculation.

Advantages

Fig.1 Schematic view of the electron beam and a selection of parameters that can be controlled in an electron microscope: Stigmatism in x- and y-direction and working distance.

  • Ready to implement technology
  • Onetime training for specific environment
  • Flexible application to various types, models and
    imaging parameters of electron microscopy
  • Low processing time
  • Quick convergence
  • Applicable also for high-throughput electron
    microscopes
  • Scalable accuracy by variation of input data
    amount
  • Simple to use

Applications

  • Scanning electron microscopy
  • High electron dose imaging
  • Automated (3D) imaging
  • High precision microscopy with large samples

Background

With increasing resolution and sample sizes the auto focusing method is significant for the result of automated electron microscopy. State of the art methods rely on the estimation of wavefront aberrations caused by variations of the relevant settings, hence being microscope specific and involving detailed theory. The lack of applicability to different systems and samples as well us long processing durations make existing solutions insufficient.

Technology

A new auto focusing method based on deep learning has been developed to overcome the aforementioned shortcomings. It was developed and tested with a typical imaging scenario involving the adjustment of the working distance and astigmatism corrections in two dimensions, as shown in figure 1. The basic optimization procedure is visualized in figure 2 for this exemplary context. Two images (2, 3) with known parameter perturbations (4, 5) around the current setting for an flawed image (6) are used as an input. The method according to the invention selects subareas (7, 9) of the first image and identical sections (8, 10) from the second one. The processing unit (11) uses each image patch pair as an input to estimate a correction term (12) to the current working distance and stigmator setting, which are entered to a function calculating the final correction suggestion. The processing unit (11) should at least comprise one processor (13) and a memory (14). After applying the obtained settings, an optimized image (15) can be taken.
The AI can be trained for various sorts of microscopes and is also capable of optimizing other parameters than those presented. It was shown that for a standard initial out-of-focus setting the optimization quickly converges and 3 iterations of the described procedure are sufficient for determining ideal settings.

Fig. 2: Brief overview of the autofocus method. The out-of-focus image is perturbed and patch-pairs are cropped
and processed by the autofocus routine. The independent predictions are used to calculate a correction term ΔF
for each focus parameter (wd: working distance, stig x: stigmator x; stig y: stigmator y).

Patent Information

(Application Filed)

PDF Download

Contact

Senior Patent- & License Manager

Dr. Bernd Ctortecka, M. Phil.

Physicist

Phone: +49 89 / 29 09 19-20
Email:
ctortecka@max-planck-innovation.de