Panoptic Segmentation Without Bounding Box Proposals Week 7

Panoptic Segmentation Without Bounding Box Proposals Week 7 Audrey Wiseman, Kevin Duarte

The Project • Panoptic Segmentation without Bounding Box Proposals • Using centroids to locate objects in an image • Ability to use information from the entire image

Work So Far • Semantic Segmentation Network • Implemented using Deep. Labv 3 • Evaluated using Io. U Scores Figure 1. Output segmentations from semantic segmentation model

Work So Far (cont. ) • Instance Segmentation Network • Dataloader • Loads Instance Centers and Regressions • Decoder • Model produces segmentation map, instance centers, and instance regressions Figure 1. Ground truth images from dataloader

Work So Far (cont. ) • Evaluated Semantic and Instance Segmentation Networks • Semantic Segmentation Results • Class Average Io. U Score: 0. 632 • Category Average Io. U Score: 0. 810 • Improvement from last week but still slightly low • Instance Segmentation Results • Average AP: 7. 7% • Average AP_50%: 15. 7% • Again, an improvement from last week but still needs work

Qualitative Results Ground Truths Predictions

Qualitative Results Ground Truths Predictions

Current Work • Training using new decoders • Exact decoders from Panoptic Deeplab paper • Previously just an upsampling layer • Expected to improve our scores for both networks

Current Work (cont. ) • Hough Transform • A way to locate the centers of instances • Only requires regressions • Each pixel of instance ‘votes’ for where the center is • In the Hough transform, each edge pixel gets a vote • We are modifying this so each pixel in the entire instance gets a vote

Next Steps • Future Experiments • Improve the instance map creation • Ideally, we will not need the centers, only regression predictions • Further improve segmentation/instance predictions through hyperparameter fine-tuning, data augmentation, etc.