Computer Vision Seminar – Forensic Imaging

NOTE: Please send requests for lectures starting from 21:00 on Wednesday (14/11/24). 

Emails arriving before 21:00 will not be considered.

Send at least 3 choices in order of preference.

IMPORTANT: list the Lecture NUMBER + NAME of lecture (preferably send name of pair of students).

Send requests by email to hagit@cs.haifa.ac.il

Course Homepage:
http://cs.haifa.ac.il/~hagit/courses/seminars/ImageForgery/ImageForgery.html

List of references (papers) for the talks can be found here.

PART 1. Image Forgery Detection - OLD SCHOOL

In this talk we will cover classic Image Processing based forgery detection methods.

These detection methods fall into several categories:

* Physics of the scene

* Statistics of Natural Images

* Camera Based

Lecture 1 - Inconsistencies in the physics of the scene.

Laws of physics and nature should be preserved under their projection into the image.
Inconsistencies and breaking of these rules form a basis for forgery detection.

In this talk 3 approaches will be presented from among:
Size Inconsistencies

Reflection Inconsistencies

Inconsistent Lighting

Shadow inconsistencies

Lecture 2 - Statistics of natural images.

Certain image statistics should be observed in natural images.

Forgery Detection is based on breaking/contradicting these statistics of natural images.

First describe the forgery types COPY-MOVE forgery and SPLICING forgery.

Then present  3 approaches for detecting these forgeries from amongst:

Uses PCA

Uses DCT

Uses Sift

Uses Moments

Uses Patch-Match

Uses Wavelets 

    Wavelet coefs - sort lexicographically

    Discrete Wavelet Trans coefs are correlated to find repeated regions.

Lecture 3 - Camera Based Forgery Detection.

The inherent characteristics of the acquisition system, namely the camera, its

components and imaging pipeline used to acquire the images form the basis for forgery detection

In this talk first review the camera components and the camera acquisition mechanism.
Then present 2 or three approaches from among:

Optics based Forgery detection   
Radial Distortion

Vignetting

Chromatic Aberration     

Sensor Based Forgery detection

Demosaicing and Color Filter Array (CFA) based detection 

PART 2. Image Forgery Detection - NEW SCHOOL

Lecture 4 – Machine Learning for Copy-Paste and Splicing detection

Copy-Move and Splicing Detection using Deep Neural Networks

First Explain Neural Networks . Explain Convolution Neural Networks (CNN).

Then choose at least two papers with different approaches.

Approaches include:
end-to-end CNN classifier, boundary of splined region detector, rich feature extraction, patches based,
Source-target disambiguation in copy-paste (which patch is original and which is the copy) and more.

Lecture 5 - Camera/Source Identification   

In this talk first explain the goal – to detect for a given image which camera acquired the image.

Then choose at least 2 papers with different approaches. At least one of the papers should be a
Machine learning based method.

Lecture 6 – Inpainting Detection

In this talk first explain Inpainting, then present 1-2 methods for performing inpainting

From among:

Diffusion based methods (PDE & total variation inpainting)

Exemplar-based image inpainting

Patch based

Machine Learning-Based Methods

Then present at least 2 approaches to Inpainting detection

PART 3. Distinguishing between Generated images and Natural images

Lecture 7 – Computer Graphics (CG) vs Natural image detection

Computer Graphics (CG) apps and algorithms have long been creating images. These may be considered “fake”.

In this talk, present the world of CG, describe a few methods that are used by animators and graphic artists.

Then choose at least 2 approaches to detecting these “fake” images, that is methods for distinguishing between CG images and natural images (images acquired by a camera).

Lecture 8– Generative Adversarial Nets (GAN) image detection

Generative Adversarial Nets (GAN) neural networks can create very good “fake” images.

In this talk explain how GAN networks work. Then choose at least 2 methods for detecting GAN images, that is \

distinguishing between GAN generated images and natural images (images acquired by a camera).

Do NOT talk about face images in this talk as Face images will be dealt with in one of the following talks.

Methods for detecting GAN images:

Pixel-level detection of errors produced by GAN-.

Based on Gan color artifacts

Classic supervised learning classification of GAN vs real images

Features as co-occurrence matrices

Lecture 9– Gan source identification

There are numerous GAN architectures that can create “fake” images. Can one determine

Which GAN created a given image? Similar to Camera source identification, this talk focuses on

Methods for detecting Source GAN of a given image.

Present 2-3 different approaches.

Lecture 10 - Faces (Still Images)

Face images are significant in our world. In this and the following talks we will focus on forged/fake/tampered

face images and video. In this talk you will present detection of Face warping, Face Morphing and

Faces Generated by GAN networks. You may start with detection of warping or morphing of faces then
continue to GAN generated Face images. OR you may focus only on GAN generated images.
In both cases explain GAN generation of  face  images using StyleGAN. Then present 2-3 methods

For distinguishing between GAN face images and natural face images (acquired by a camera).

Lecture 11 - Deepfake: FaceSwap + Face2Face

Deepfake is the term for fake images created using deep neural networks.
FaceSwap and Face2Face (and others) are methods for changing the identity of a face image

While preserving all other characteristics including facial expression and head pose.

In this talk you will present the world of Deepfake. Describe the methods of FaceSwap and Face2Face
(and or other approaches eg NeuralTextures). Then present at least 2 forgery detection approaches

to distinguish between Deepfake face images and natural images (acquired by camera).

Lecture 12 - Moving Portraits, Facial Motion Retargeting

Moving Portraits and Motion retargeting are methods for changing the facial expression, shape and position

of mouth and eyes and changing head pose of a single face image. Still images are usually created

but videos can be produced by creating frame after frame (thus forming facial MOTION and MOVING Portraits).
In his talk you will present the world of Moving Portraits and Motion retargeting. Present methods

of creating them, and then at least 2 forgery detection methods in which these “Fake” images are

distinguished from natural images (acquired by a camera).

Lecture 13 – Forgery detection in Face Videos

Although previous talks introduced methods that can create videos, they primarily focused on still images.

In this talk you will focus on detecting forgery in tampered/forged/fake videos.

You will start with methods of creating fake videos (coordinate with the 2 previous talks to ensure you do

not present the same methods). Then present 2-3 methods of forgery detection in videos (choose at least 1 deep
method) from among:

Video Face Manipulations detection from 3D model fit

Visual Artifacts (diff erenteye color, bad teeth synth)

Eye blink rates

Heart beat

Head pose inconsistencies

Physiologically-based detection of computer generated faces in video,

Face warping artifacts

Facial expression and head movements are correlated. (when deepfake becomes very good)

General approach not specific cues (train on videos)

Use optical flow

Look at boundaries (assume blending in forgery )