This is an archived course. A more recent version may be available at ocw.mit.edu.

Projects

There will be four project assignments and a final project of the students' choosing.

Project Assignments

The four project assignments cover the following topics.

  1. Relighting
  2. One of:

    a. Virtual optical bench

    b. Dual photography

    c. Direct-global photography

  3. Lightfield capture: mask or LCD with programmable aperture
  4. One of:

    a. Interaction with multiflash camera

    b. Strange aperature camera

Assignment 1: Relighting: Warm up Assignment

Combine two photos by mixing the color channels. Take multiple photos by changing lighting and other parameters. Be creative. Then mix and match color channels to relight.

You can use these for inspiration:

MATLAB background:

Create a Web page for all your homework assignments. The Web page for each assignment should at least have well commented source code, all input images, intermediate results and final output. Please include some description below each image.

Assignment 2

Assignment 2 summary slides (PDF)

Assignment 2A. Virtual Optical Bench

Part 1:

  • Use ray-matrix operations and the 'ray' class to create an interface similar to Andrew Adam's software. (Opens Shockwave file, requires Adobe Flash Player.)

Part 2:

  • Create images of 3D and 2D objects. Show effects like depth of field using aperture or capture lightfield by selectively blocking the aperture.

Assignment 2B. Dual Photography

Read all the practical issues carefully in the original paper.

Use a projector and video camera.

Part 1:

  • Use a photodetector; or use a camera and sum up the pixels

Part 2:

  • Use a camera

Extra credit:

  • Create relighting effects

Low resolution results are ok. First try with 50x50 resolution.

Assignment 2C. Direct-Global Photography

Read the original paper carefully to understand the practical issues.

Then look at sample results to realize types of objects/scenes ideal for this method.

Use a projector and camera. Project patterns and take about 16 or 25 photos. Compute direct illumination and global illumination component

Extra credit:

  • Create relighting effects

Low resolution results are ok.

Assignment 3: Lightfield Photography

Read the Lightfield Camera Papers very carefully.

(a) Translate camera and take photos

(b) Show refocussing and see-thru effects

See examples at The (New) Stanford Light Field Archive.

Part 1:

  • Create images with plane of focus at different depth
  • Create images with variable depth of field (just use fewer images)
  • Create see-thru effects (just small depth of field)
  • Find depth using max-contrast operator

Part 2:

Extra credit:

  • Create new bokeh (point spread function)
  • Use high depth complexity, colorful, point specular (sphere) objects.
  • To create multiple camera views, you can also aim at an array of cameras, put the camera on a robot or x-y platform.
  • Be creative with camera configurations, maybe with very large baseline or with microscope. You can also use unstructured positions and use a calibration target (or structure from motion or photosynth software) to find the positions.
  • More projects at Stanford's "Light Fields and Computation Photography" page
  • You can also create lightfield with:

Assignment 4A: Interaction with Multiflash Camera

Track hand or finger using shadows from colored RGB lights, video camera.

References:

Part 1:

  • Turn on 1 LED at a time, take 3 (or 4) photos
  • Find silhouette from shadow
  • Do region filling to indicate (render) hand against textured background on table

Part 2:

  • Turn on 3 LEDs, R, G, B
  • From one photo, decompose 3 photos using RGB channels
  • Find shadows, do region filling and find foreground

Extra credit:

  • Let two hands overlap, find internal silhouettes (i.e. boundary between two hands), indicate the two hands in different color in rendering
  • On table, have printed photos of hand (so ordinary camera will be confused) or other crazy texture

Assignment 4B: Strange Aperture Camera

Add new pattern in camera aperture.

References:

Part 1:

  • On optical bench, crack open a lens, insert an aperture pattern. (We can give you Canon 50mm lens with access to aperture as well)
  • Use a simple aperture with red and blue pin-holes
  • Do simple stereo 3D recon between red and blue channel of camera (code available)
  • Initially use just 2 painting at different depth to test

Part 2:

  • Use a coded aperture. (We can give you the pattern, but request at least 2 days in advance for printing.)
  • Digital refocussing (source code mostly available)

Extra credit:

  • Use rainbow, polarized or dynamic (LCD) etc. aperture patterns

Final Project

The final project is worth 30% of the total course grade. Projects will be evaluated on three criteria, weighted roughly equally: novelty, coolness (+execution), and impact (+evaluation)

There will be three deliverable phases:

  1. Pre-proposal: Meet with me, the TA or Prof. Hiura to discuss at least 3 ideas of interest. Focus on your preferred idea and prepare a brief email on motivation, goal, novelty and prior art, approach, and references. Then prepare a 3 minute presentation for the class on your pre-proposal.
  2. Final proposal: complete initial experiments
  3. In-class presentation with conference-quality paper

Final Project Topic Ideas

User interaction device

  • Camera based
  • Illumination based
  • Photodetector or line-scan camera

Capture the invisible

  • Tomography for internals
  • Structured light for 3D scanning
  • Fluorescence for transparent materials

Cameras in different EM/other spectrum

  • Wifi, audio, magnetic, haptic, capacitive
  • Visible Thermal IR segmentation
  • Thermal IR (emotion detection, motion detector)
  • Multispectral camera, discriminating (camel-sand)

Illumination

  • Multi-flash with lighfield
  • Schielren photography
  • Strobing and colored strobing

External non-imaging sensor

  • Camera with gyro movement sensors, find identity of user
  • Cameras with GPS and online geo-tagged photo collections
  • Interaction between two cameras (with lasers on-board)

Optics

  • Lightfield
  • Coded aperture
  • Bio-inspired vision

Time

  • Time-lapse photos
  • Motion blur