Description

# KTM Code the Light

Model can downloaded from the provided link.

## Requirements

To simplify execution, I exported my model to ONNX format. This reduces the installation requirements to:

1. Onnxruntime: `pip install onnxruntime-gpu` or `pip install onnxruntime` for the CPU version (if GPU for some reason does not work). Even better, use [Jetson Zoo](https://elinux.org/Jetson_Zoo#ONNX_Runtime) for JetPack support.

2. (probably already installed) Numpy: `pip install numpy`

3. (probably already installed) Opencv2: `pip install opencv-python`

As a single command `pip install onnxruntime-gpu numpy opencv-python`

Depending on the Python version installed, numpy might need to be downgraded.

I am not sure if onnxruntime-gpu automatically uses AGX Xavier, therefore downloading the appropriate pip wheel and following the instructions from [Jetson Zoo](https://elinux.org/Jetson_Zoo#ONNX_Runtime) is the better option.

The onnx model has to be put into the running directory.

## Algorithm

I trained a segmentation model with PyTorch and exported it to ONNX format, quantized it to INT8 and run it with onnxruntime for better inference portability.

I used static quantization and used all images for calibration.

Furthermore, I used the [segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch) project for simplifying training and [albumentations](https://albumentations.ai/) for training augmentations. I used the FPN model and figured out some hyperparameters with [optuna](https://optuna.org/).

To improve inference performance, I used a small encoder model, MobileNet v2. I also played around with FPN parameters to make it faster.

Another important step I perform is Test Time Augmentation (TTA), basically performing inference twice, once on the input image and another time on its horizontally flipped version and taking the mean prediction of the outputs.

I also added pseudo-labels to the test set.