About the demo

Scanning electron microscope (SEM) analysis is a powerful tool that is widely used in materials science, biology, and medicine to perform a detailed examination of objects on extremely small scales. In nanoscience, this technique is among a very few methods that allow understanding of the shape of synthesized particles. And particle shape is an integral characteristic for chiral nanomaterials where left-handed (twisted to the left) and right-handed (twisted to the right) particles usually have opposite properties.

In collaboration with Dr. Anastasia Visheratina and Prof. Nicholas Kotov, we developed an algorithm for synthetic data generation that allows to use 15-20 original images to create thousands of images for training. We used this approach to train a variety of SegFormer models to perform a semantic segmentation of left-handed and right-handed nanoparticles. As a result, we got 94-97% mIoU on the validation dataset depending on the model size. Intrestingly, all trained models demonstrate generalizability - althouth they were trained on only one kind of particles, they can detect other particles that are twisted.

In this demo, you can check out three trained models - B0 (3.8M parameters), B1 (13.7M parameters), and B5 (84.7M parameters). All models were exported to ONNX using standard PyTorch functionality.

How to use the demo:

1. Select the model and load it.
2. Load the image from the device, or select one of example images.
3. Generate segments.
4. You can click on the image to see the class of the object.