Abstract
Various computer vision problems could be extended to deal with Internet data. Suc-cessful applications include 3D reconstruction using multi-view stereo, intrinsic im-age decomposition etc.. As important vision research topics, photometric methods are still generally not viable in unconstrained Internet data because of their restrictive as-sumptions. In this thesis, we focus on two important photometric methods, namely, radiometric calibration and photometric stereo. By generalizing the assumptions used for conventional radiometric calibration and photometric stereo research, our methods provide the potential to apply these techniques in unconstrained Internet data. Radiometric calibration is the first step of photometric analysis. The purpose of radiometric calibration is to estimate the camera (inverse) response function, which is used to recover linear data from image intensity values that are usually nonlinearly mapped from scene irradiance by commercial cameras for compressing the dynamic range and aesthetic purpose. Such a calibration is needed because many computer vi-sion problems require input images to be radiometrically linearized. Traditional radio-metric calibration approaches use a set of images captured in a static scene using one camera and various exposure times as input. This setup is not applicable for calibrating unconstrained image data because images in such a photo collection are captured by uncontrolled cameras using unkown and different settings. In our method, we exploit the observation that the ratio of albedo values for pixels in the same image with the same surface normal should be equal across the image set, only when each image is ap-plied with a correctly calibrated inverse radiometric response function. Based on this observation, a rank minimization framework is developed to estimate inverse response functions for all images in an Internet photo collection simultaneously up to a unified exponential ambiguity. The ambiguity is solvable using single-image constraints. Photometric stereo is a 3D-reconstruction technique that is well-known for its capa-bility to produce detailed 3D shapes when compared to other 3D-reconstruction tech-niques. It takes a set of radiometrically linearized images taken under varying illumi-nation conditions and a fixed-view point as input and performs a per-pixel analysis to generate the surface normal map at its input image resolution. Traditional photometric stereo setup has two assumptions on lighting - directional and calibrated lighting. The former assumption is applied in uncalibrated photometric stereo methods while the latter one requires a mirror sphere to be placed in the input images to calibrate envi-ronment lighting. Neither of the assumptions work for unconstrained lighting setup -uncalibrated natural illumination. In the proposed approach, we found that for a small surface patch with slowly varying normals, the visible hemisphere of environment map also shows smooth changes. Under this assumption, the environment lighting could be approximated as an equivalent directional lighting by summing up all samples on the visible hemisphere for that patch. Based on this observation, we proposed a new light-ing model - equivalent directional lighting model and “divide and conquer" approach to solve for the uncalibrated photometric stereo under natural illumination. Both of the proposed approaches have been evaluated using both synthetic data and real data to show their effectiveness and robustness. The main contributions of the thesis are three folds: We explore the consistency of scene reflectance of corresponding pixels across images in a same Internet photo collection to apply radiometric calibra-tion for Internet photo collections; We inventively propose the equivalent directional lighting model to solve the uncalibrated photometric stereo under natural illumination, which, to the best of our knowledge, is the first method that is able to deal with this challenging problem; The combination of the above proposed approaches cover the whole pipeline of photometric analysis. They greatly relaxed the assumptions used in traditional approaches, which provides the potential to bring these lab applications to unconstrained real-world applications.