Panoptic Segmentation Without Bounding Box Proposals Week 7
Panoptic Segmentation Without Bounding Box Proposals Week 7 Audrey Wiseman, Kevin Duarte
The Project • Panoptic Segmentation without Bounding Box Proposals • Using centroids to locate objects in an image • Ability to use information from the entire image
Work So Far • Semantic Segmentation Network • Implemented using Deep. Labv 3 • Evaluated using Io. U Scores Figure 1. Output segmentations from semantic segmentation model
Work So Far (cont. ) • Instance Segmentation Network • Dataloader • Loads Instance Centers and Regressions • Decoder • Model produces segmentation map, instance centers, and instance regressions Figure 1. Ground truth images from dataloader
Work So Far (cont. ) • Evaluated Semantic and Instance Segmentation Networks • Semantic Segmentation Results • Class Average Io. U Score: 0. 632 • Category Average Io. U Score: 0. 810 • Improvement from last week but still slightly low • Instance Segmentation Results • Average AP: 7. 7% • Average AP_50%: 15. 7% • Again, an improvement from last week but still needs work
Qualitative Results Ground Truths Predictions
Qualitative Results Ground Truths Predictions
Current Work • Training using new decoders • Exact decoders from Panoptic Deeplab paper • Previously just an upsampling layer • Expected to improve our scores for both networks
Current Work (cont. ) • Hough Transform • A way to locate the centers of instances • Only requires regressions • Each pixel of instance ‘votes’ for where the center is • In the Hough transform, each edge pixel gets a vote • We are modifying this so each pixel in the entire instance gets a vote
Next Steps • Future Experiments • Improve the instance map creation • Ideally, we will not need the centers, only regression predictions • Further improve segmentation/instance predictions through hyperparameter fine-tuning, data augmentation, etc.
- Slides: 10
