There will be four project assignments and a final project of the students' choosing.
The four project assignments cover the following topics.
- One of:
a. Virtual optical bench
b. Dual photography
c. Direct-global photography
- Lightfield capture: mask or LCD with programmable aperture
- One of:
a. Interaction with multiflash camera
b. Strange aperature camera
Assignment 1: Relighting: Warm up Assignment
Combine two photos by mixing the color channels. Take multiple photos by changing lighting and other parameters. Be creative. Then mix and match color channels to relight.
You can use these for inspiration:
- Haeberli, Paul. Synthetic Lighting for Photography. January, 1992.
- Raskar, Ramesh, Adrian Ilie, and Jingyi Yu. Image Fusion for Context Enhancement and Video Surrealism. Presented at NPAR 2004.
Create a Web page for all your homework assignments. The Web page for each assignment should at least have well commented source code, all input images, intermediate results and final output. Please include some description below each image.
- Here is good example of how to assemble your homework into a Web page.
Assignment 2 summary slides (PDF)
Assignment 2A. Virtual Optical Bench
- Use ray-matrix operations and the 'ray' class to create an interface similar to Andrew Adam's software. (Opens Shockwave file, requires Adobe Flash Player.)
- Create images of 3D and 2D objects. Show effects like depth of field using aperture or capture lightfield by selectively blocking the aperture.
Assignment 2B. Dual Photography
Read all the practical issues carefully in the original paper.
- Sen, P., et al. "Dual Photography." Proceedings of SIGGRAPH, 2005.
Use a projector and video camera.
- Use a photodetector; or use a camera and sum up the pixels
- Use a camera
- Create relighting effects
Low resolution results are ok. First try with 50x50 resolution.
Assignment 2C. Direct-Global Photography
Read the original paper carefully to understand the practical issues.
- Computer Vision Laboratory, Columbia University. "Fast Separation of Direct and Global Images."
Then look at sample results to realize types of objects/scenes ideal for this method.
Use a projector and camera. Project patterns and take about 16 or 25 photos. Compute direct illumination and global illumination component
- Create relighting effects
Low resolution results are ok.
Assignment 3: Lightfield Photography
Read the Lightfield Camera Papers very carefully.
(a) Translate camera and take photos
(b) Show refocussing and see-thru effects
See examples at The (New) Stanford Light Field Archive.
- Create images with plane of focus at different depth
- Create images with variable depth of field (just use fewer images)
- Create see-thru effects (just small depth of field)
- Find depth using max-contrast operator
- Images with slanted plane of focus
- Scheimpflug Principle
- Vaish, V., et al. "Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform." Proceedings of A3DISS 2005 (at CVPR), San Diego, CA.
- See-thru effect by eliminating foreground color pixels
- Create new bokeh (point spread function)
- Use high depth complexity, colorful, point specular (sphere) objects.
- To create multiple camera views, you can also aim at an array of cameras, put the camera on a robot or x-y platform.
- Be creative with camera configurations, maybe with very large baseline or with microscope. You can also use unstructured positions and use a calibration target (or structure from motion or photosynth software) to find the positions.
- More projects at Stanford's "Light Fields and Computation Photography" page
- You can also create lightfield with:
- Flatbed scanner+lenticulars: see Yang, Jason C. "A Light Field Camera for Image Based Rendering." MSEE Thesis, MIT, 2000. (PDF - 1.7 MB)
- Masks: see Veeraraghavan, A., et al. "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperature Refocusing." Proceedings of SIGGRAPH 2007.
Assignment 4A: Interaction with Multiflash Camera
Track hand or finger using shadows from colored RGB lights, video camera.
- Raskar, R., K-H Tan, R. Feris, J. Yu, and M. Turk. "Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging." Proceedings of ACM SIGGRAPH 2004, August 2004. [Includes source code]
- Rogiero Feris' Web site
- Vaquero, D., R. Feris, M. Turk, and R. Raskar. "Multiflash Imaging and Applications."
- Turn on 1 LED at a time, take 3 (or 4) photos
- Find silhouette from shadow
- Do region filling to indicate (render) hand against textured background on table
- Turn on 3 LEDs, R, G, B
- From one photo, decompose 3 photos using RGB channels
- Find shadows, do region filling and find foreground
- Let two hands overlap, find internal silhouettes (i.e. boundary between two hands), indicate the two hands in different color in rendering
- On table, have printed photos of hand (so ordinary camera will be confused) or other crazy texture
Assignment 4B: Strange Aperture Camera
Add new pattern in camera aperture.
- Veeraraghavan, A., R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin. "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing." Proceedings of ACM SIGGRAPH 2007. [Includes source code.]
- Bando, Y., B-Y Chen, and T. Nishita. "Extracting Depth and Matte using a Color-Filtered Aperature." Proceedings of SIGGRAPH Asia 2008.
- On optical bench, crack open a lens, insert an aperture pattern. (We can give you Canon 50mm lens with access to aperture as well)
- Use a simple aperture with red and blue pin-holes
- Do simple stereo 3D recon between red and blue channel of camera (code available)
- Initially use just 2 painting at different depth to test
- Use a coded aperture. (We can give you the pattern, but request at least 2 days in advance for printing.)
- Digital refocussing (source code mostly available)
- Use rainbow, polarized or dynamic (LCD) etc. aperture patterns
The final project is worth 30% of the total course grade. Projects will be evaluated on three criteria, weighted roughly equally: novelty, coolness (+execution), and impact (+evaluation)
There will be three deliverable phases:
- Pre-proposal: Meet with me, the TA or Prof. Hiura to discuss at least 3 ideas of interest. Focus on your preferred idea and prepare a brief email on motivation, goal, novelty and prior art, approach, and references. Then prepare a 3 minute presentation for the class on your pre-proposal.
- Final proposal: complete initial experiments
- In-class presentation with conference-quality paper
Final Project Topic Ideas
User interaction device
- Camera based
- Illumination based
- Photodetector or line-scan camera
Capture the invisible
- Tomography for internals
- Structured light for 3D scanning
- Fluorescence for transparent materials
Cameras in different EM/other spectrum
- Wifi, audio, magnetic, haptic, capacitive
- Visible Thermal IR segmentation
- Thermal IR (emotion detection, motion detector)
- Multispectral camera, discriminating (camel-sand)
- Multi-flash with lighfield
- Schielren photography
- Strobing and colored strobing
External non-imaging sensor
- Camera with gyro movement sensors, find identity of user
- Cameras with GPS and online geo-tagged photo collections
- Interaction between two cameras (with lasers on-board)
- Coded aperture
- Bio-inspired vision
- Time-lapse photos
- Motion blur