Project Implementation for ITCS 4122 Jianping Fan Department

Project Implementation for ITCS 4122 Jianping Fan Department of Computer Science University of North Carolina at Charlotte http: //www. cs. uncc. edu/~jfan

You can select one of them • Semantic image classification & Visual Analytics • Image Clustering & Visual Analytics • Deep Image Segmentation & Visual Analytics • Interactive Data Analytics

1. Build a system for supporting semantic image classification & visual analytics ---you are encouraged to use deep learning for feature extraction System Components: a. Image analysis and feature extraction (1) Image segmentation using JSEG You can download jseg image segmentation source code from: http: //vision. ece. ucsb. edu/segmentation/jseg/ (2) Feature extraction: extract color histogram, color moments, Tamular textures (moments et al. ) from each homogeneous regions (3) Region classifier training You can download software from: http: //www. cs. cornell. edu/people/tj/svm_light/ http: //www. csie. ntu. edu. tw/~cjlin/libsvm/

(4) Region merging After the labels for the homogeneous regions are obtained, the neighboring regions with the same label will be merged as one big region for the relevant salient object (5) Interesting point detection You can download software from: http: //vision. ucla. edu/~vedaldi/code/sift. html Detecting the object via SIFT features (6) Object-Based feature extraction Extract color histogram, color moments, textures, and SIFT from salient objects

(7) Concept classifier training positive samples negative support vectors You can download software from: http: //www. cs. cornell. edu/people/tj/svm_light/ http: //www. csie. ntu. edu. tw/~cjlin/libsvm/ ma rg positive support vectors in negative samples

(8) Visualization for features and classification results (9) User-system interaction for result evaluation and integrating human advises for further data analytics. Deep Learning approach for this project Input Images Deep Networks such as Alex. Net, VGG, Google. Net, Res. Net, … Outputs: Features & List of Object Classes & Concepts Parameter Fine-Tune Visualization of Categorydriven Data Distributions

You need to build classifiers for: • Salient Objects • Tree, grass, red flower, yellow flower, sky water, sand, road, car, building • Image Concepts • Garden, beach

2. Automatic Image Clustering & Visual Analytics ---you are encouraged to use deep learning for feature extraction • You can download 1, 000 images from http: //www. image-net. org/ • You can extract one of the following visual features: color histogram or SIFT features, you can also try CNN features, you can get source code from http: //clickdamage. com/sourcecode/index. php • You can apply AP clustering technique to get an optimal partition of these 1, 000 images according to their visual similarity, you can get AP clustering source code from http: //bonsai. hgc. jp/~mdehoon/software/cluster/software. htm

• Image Query Result Clustering • You can go to Google Images and type in keywords such as ``apple”, ``bank”, ``cloud” et al • Download the returned images (more than 1000 images), total images will be 100, 000. • Extract color histogram or SIFT by using http: //clickdamage. com/sourcecode/index. php • Apply AP clustering to get an optimal partition of the returned images http: //bonsai. hgc. jp/~mdehoon/software/cluster/software. htm

Visualization for features and classification results User-system interaction for result evaluation and integrating human advises for further data analytics. ---you are encouraged to use deep learning for feature extraction Input Images Deep Networks such as Alex. Net, VGG, Google. Net, Res. Net, … Outputs: Features & Image Clusters Parameter Fine-Tune Visualization of Data Distributions

3. Deep Image Segmentation & Analytics • You can use Deep. Lab v 3, U-nets, Mask RCNN, … to do semantic image segmentation • Integrating visualization to evaluate the segmentation results • Leveraging human advises to improve deep image segmentation

Project Requirements • Project implementation: you need to spend time to understand the project and the relevant source code and compile these source codes and make them work in your computer and try to make some revisions. • Penalties: TA will test your system by using his images, so it is easy for us to check whether you really do the project or not. If we find that you did not spend enough time to work on your project, you will receive big penalties on your project grade.

4. Traditional Tools for Interactive Data Visualization Input Data Feature Extraction & Data Clustering Data & Knowledge Visualization Constraint-Driven Data Clustering User Assessment & Feedbacks

Group or not • Each group may have no more than 5 students • We encourage individual project & bonus is expected for individual project • Bonus will be given to good implementation such as improving on algorithms or including nice interface & visualization