Depth Edge Detection Using Multiflash Imaging

A report by Sheira Ben Haim and Dvir Segal, April 2014.

This project is based on the paper: "Non-photorealistic Camera: Depth Edge and Stylized Rendering using Multi-Flash Imaging" [Ramesh et al. 2004]

w Introduction to the problem

In this project we deal with the problem of directly finding depth edges or create stylized images based on depth discontinuities.

The purpose is being able to convey 3D structure of imaged scene using only 2D camera capabilities.

w Our Approach

We present a rendering approach to capture and convey shape features of real-world scenes. We use a model that mimics a camera with multiple flashes. The use of multiple flashes helps us in casting shadows along depth discontinuities in the scene. The projective-geometric relationship of the camera-flash setup is then exploited to detect depth discontinuities and distinguish them from intensity edges due to material discontinuities.

In addition, we implement a method for utilizing the detected edge features to generate stylized images. The resulting images more clearly convey the 3D structure of the imaged scenes.

This approach of capturing geometric features is different than traditional approaches that require reconstructing a 3D model.

w Algorithm

Creating depth edges image:

Input: 4 images, each captured with a different flash position.

Preparations:

1. Make gray scale images out of the input images.

2. Normalize the intensity for all gray scale images, represented by.

Calculate:

3. Compute the maximum intensity image .

4. For each, compute ratio image, such that .

5. For each image, find the pixels with step edges with negative transition, and mark them as depth edges pixels.

6. Create the Depth Edge Image – by creating an image combined of all four images in last step, taking the maximum value found for each pixel

7. Create 4, 2-level canny-like hysteresis images, and combine them to obtain the final binary edge image.

Outputs:

1. Depth Edge Image

2. Binary Edge Image

Stylizing the image:

Input: the Binary Edge Image

A. Emphasized image

1. Use image morphology in order to improve the edges image:

a. Create a square structure

b. Dilate the edges of the input image according to the structure

c. Ask the user to mark regions in order to fill in holes.

d. Erode back the filled image according to the structure.

2. Create an Emphasized image using the last step output.

B. Styled image

1. Use the input image to find out connected components.

2. Average the color of each component – to eliminate texture.

Outputs:

1. An Emphasized image – based on one of the four input images emphasized by depth edges.

2. A Styled Image - a synthesized image created by abstracting the texture.

w Implementation

1. In order to find the depth edges we are using the Sobel kernel, generated by fspecial, to find the derivative of the intensity, applying it on each by imfilter.

2. For creating the 4 edge images , we use a threshold method implemented by bwlabel in the following manner:

a) Find low and high contours which are higher than the given confidence value.

b) Compute all connected components of both images using bwlabel.

3. In order to enhance and improve and edged image, in the emphasizing part, we are using the strel function, Creating a morphological structuring element.

We are using the structure for dilating and eroding the edged image while filling in areas of holes.

w Results\Examples

Loaded images

Flash Down Flash Left

Flash Right Flash Up

Results

Edge detected Binary Image

Emphasized image Stylized image

Additional Results

Edge detected Binary Image

Emphasized image Stylized image

Edge detected Binary Image

Emphasized image Stylized image

Edge detected Binary Image

Emphasized image Stylized image

w Who wrote this and why?

This Project submitted by Sheira Ben Haim and Dvir Segal in Computational Photography course as part of their B.Sc. in Computer Science at the University of Haifa.

The project was performed under the supervision of Hagit Hel-Or.

w Terms of use

Copyright © 2014 by Sheira Ben Haim and Dvir Segal.

The SOFTWARE ("Depth Edge Detection Using Multiflash Imaging") is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage that may unintentionally be caused through its use.

You may not use the source to SOFTWARE or distribute SOFTWARE in any form, without express written permission of the copyright holders.