========================================================================================
Predicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation(s)

Danna Gurari, Kun He, Bo Xiong, Jianming Zhang, Mehrnoosh Sameki, Suyog Dutt Jain, 
Stan Sclaroff, Margrit Betke, and Kristen Grauman
========================================================================================

Directory structure:
|-- data
|-- extern
|-- models
    |-- VGG16_SalObjSub
    |-- STATIC_ft
    |-- VizWiz_ft


1. Prerequisites

Downloaded the zipped image data, and unzip into the 'data' directory: 
- STATIC.zip: images from the 7 computer vision datasets
- VizWiz.zip: images from VizWiz

Install or symlink Caffe (https://github.com/BVLC/caffe.git) at:
extern/caffe
(Note that our implementation requires Caffe to be compiled with LMDB support 
and MATLAB interface.)

In models/VGG16_SalObjSub, run 'download_caffemodel.sh' to download pretrained
VGG-16 model for salient object subtizing.



2. Prepare images with Caffe

In directory 'data', run:
./convert.sh [dataset] [subset]

where [dataset] is 'STATIC' or 'VizWiz', and [subset] can be one of 
{train, val, trainval, test}. This will convert the images into LMDB format.
For CNN finetuning, subsets 'trainval' and 'test' are needed.



3. CNN finetuning

We finetune the pretrained VGG-16 salient object subtizing network.
In directory 'models/STATIC_ft' or 'models/VizWiz_ft', run:
./run_ft.sh [GPU_ID]

We also provide our finetuned models for easy reference.



4. Evaluating finetuned CNN

After finetuning, model snapshots will be stored in the 'snapshot' directory under
'[dataset]_ft', where [dataset] is 'STATIC' or 'VizWiz'. We provide MATLAB code 
to evaluate finetuned CNN models, in 'demo_cnnft.m'. 

To invoke it, first create a symlink named '[dataset]_ft_trainval.caffemodel'
to the desired snapshot, under 'models/[dataset]_ft'. Then, run in MATLAB:

demo_cnnft( [dataset] )

Note that although finetuning itself doesn't require Caffe to be compiled with 
MATLAB interface, our evaluation code does. Also, our model was trained with an 
older (2015) version of Caffe and the MATLAB interface may have changed since.

