About me
Projects
Curriculum Vitae
LinkedIn
GitHub
Description
The task of object insertion is inserting a 3D model into a given image. To make the object appear realistic it should, first of all, interact with the scene in a logical way, such as hiding parts that are occluded to the camera.
Secondly, the object should be rendered in such a manner that the object appears realitic, as if it had always been there. This requires a certain understanding of the scene illumination, as we can use This
illumination to shade the object. IllumiNet is a deep convolutional neural network created for my master's thesis titled "IllumiNet: Realistically inserting objects in RGB-D
images by recovering indoor scene illumination as point lights" that recovers and represents the scene lighting as a set of point lights.
Instead of using RGB-D images directly as input, the images are preprocessed by means of
intrinsic image decomposition that decomposes an image into the albedo (absolute colour of the surfaces) and shading (how the surfaces are shaded) images.
The depth images are used to reconstruct the scene in a 3D space, that then can be used to realistically account for occlusion and recover the surface normals. Using a 3D model, the reconstructed surface normals and
the predicted point lights, the model can be realitically inserted into the image. This is done though a custom-made rendering pipeline that accounts for the objects surface roughness and specularity.
IllumiNet is trained using IllumiSet, my dataset containing intrinsic images from synthetic indoor scenes created using Blender.
Not yet publically available
Because of a potential attempt to publish the research in a scientific journal, the thesis is currently not publically available. When either the paper is accepted or when the publication is no longer persued, the thesis will be made public.
