An Authoring Solution for a Façade-Based AR Platform: Infrastructure, Annotation and Visualization

In the last few years, the emergence of smartphones equipped with Internet access, high resolution cameras, and positioning sensors provides the possibility for augmented reality (AR) applications in urban settings. An important component of such systems will be the ability of non-expert users to add their own AR content to the system. A comprehensive 3D model of a scene is normally required as part of this authoring process. This can be computationally ex- pensive to compute and unnecessary since often the fac ̧ades within the urban scene are the main reference point for the user. This paper describes a method of authoring augmented reality content based on planar façade extraction from a single image. With this method the extracted plane can be used to create a front parallel view which in turn can be used as a frame of reference for authoring 3D AR content. This approach permits a simplified method for integrating augmented content into the view which can highlight and visualize geographically or contextually meaningful information about the scene. Results presented demonstrate an authoring solution which is easy to use for a non-expert user based on the fact that the façade based infrastructure allows for an intuitive frame of reference. Furthermore, although the underlying representation is not a complete 3D representation of the environment, it still allows the user to cre- ate full 3D AR content.

Content authoring is an important stage in the workflow of creating rich augmented reality applications. In this paper we describe a fac ̧ade-based database infrastructure for authoring and storing 3D content for use in urban environments. It provides frames of reference for the environment as well as a mechanism to match new images with the fac ̧ades and thus retrieving associated 3D content. The infrastructure is flexible in that we can add different 3D “layers” of content on top of the façades and hence opens many possibilities for augmented reality applications in urban environments. Furthermore the system provides a representation suitable for both manual and automatic content authoring.

Slides from our presentation at ISMAR 12 Workshop Authoring Solutions for Augmented Reality

An Authoring Solution for a Façade-Based AR Platform: Infrastructure, Annotation and Visualization from Guillaume Gales

Pixel Matching from Stereo Images (Callan Institute Seminar)

Abstract

This talk discusses a number of techniques for correspondence estimation between stereo image pairs, i.e. two images of the same scene taken from different positions. The problem is to identify pairs of pixels in the two images that are the projections of the same scene point. Although the human visual system performs this task with ease, developing algorithms for automatically computing correspondences is a challenging task. In particular, existing algorithms can fail in homogeneous areas, near depth discontinuities and occlusions or with a repetitive texture pattern.

The first part of this talk focuses on seed propagation-based approaches that are a special case of local methods based computing an iterative solution, where the solution is initialised using a sparse set of reliable matches (the seeds). I introduce a reliability measure used by the propagation technique for finding the correct correspondent of a pixel, providing robustness in the context of the above difficulties. This measure takes into account an unambiguity term, a continuity term and a colour consistency term. It has the advantage of taking into account information from the other candidates, and leads, according to our experimental evaluation, to better results when compared to other methods based on a correlation score alone.

In the second part of this talk I will present ongoing work in our group on stereo matching in urban environments. In particular we exploit the fact that images of such environments contain multiple planar elements. I will show how utilising this strong geometrical constraint allows us to automatically segment building facades in single images. Furthermore I show how this technique permits robust pixel matching in wide-baseline stereo pairs. Finally, I will discuss how we intend to apply this technique for the development of augmented reality applications.

Slides

Pixel Matching from Stereo Images (Callan seminar) from Guillaume Gales