Search Disabled |
|
|
|
Search Options | Help | Site Map | Cultivate Web Site | ||||
|
|
|||||
| Home | Current Issue | Index of Back Issues |
| Issue 4 Home | Editorial | Features | Regular Columns | News & Events | Misc. | ||
By Michal Haindl and Josef Kittler - May 2001
Michal Haindl and Josef Kittler provide an overview of the joint research INCO-COPERNICUS project no. 960174 VIRTUOUS (Autonomous Acquisition of Virtual Reality Models from Real World Scenes). The article describes the project objectives, introduces the partners and summarises its main achievements.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
VIRTUOUS [1] (Autonomous Acquisition of Virtual Reality Models from Real World Scenes) was an international research project financed (1997 - 1999) by the Commission of the European Communities in frame of the INCO-COPERNICUS scheme.
Virtual reality systems can be used for a variety of applications in entertainment, medicine and manufacturing. Thus producing detailed models is of generic interest. Unfortunately the customary manual creation of virtual reality models of real world scenes is tedious and error-prone, particularly for scenes of high complexity. Any automation that can substantially reduce the laboriousness and consequently the cost of the whole process would be very beneficial. In this context visual sensors offer the ideal route to automation, especially when range and vision sensors are already common and their mutual registration can be accomplished using either standard photogrammetric techniques or an appropriate sensor setup.
The objectives of this 3-year project were to build detailed texture mapped surface models of complex real world objects, to develop efficient ways of processing colour textures and to use these models in a robot arm trainer and simulator. The core of the project was to capture virtual reality models of real world robot cell scenes automatically, without interaction with a human observer and then to validate these models in a Virtual Reality Robot Arm Trainer application. To get a lifelike simulation of the manufacturing process, it was necessary to capture 3D graphic information about all objects located in the robot workcell and make it available to the trainer in a suitable form. The aim was to automate this process as much as possible and to avoid any errors. The key approach is to combine range and visual sensor data to build object and scene models. The models are processed by a scene properties extractor and used by the trainer.
VIRTUOUS was a joint research project between the University of Surrey, Guildford, United Kingdom, Instituto Superior Tecnico, Lisboa, Portugal, Institute of Information Theory and Automation, Prague, Czech Republic, and the Institute of Control Theory and Robotics, Bratislava, Slovakia.
The University of Surrey (UoS) [2] was the coordinator of the whole project. Apart of the project management the work at UoS was primarily aimed at the building of detailed surface models of complex real world objects from range images.
The Instituto Superior Técnico (IST) [3] used computer vision techniques to acquire scene models from video sequences taken from a mobile platform. Although the objective was the same as that persued by the University of Surrey, this was a more challenging task. The advantage of scene reconstruction from video sequences is the low cost of the sensor. However, the software processing is considerably more complex.
The objectives of the Institute of Information Theory and Automation (UTIA) [4] part in the VIRTUOUS project were to segment colour and range images of a single scene, to develop algorithms for analysis of real textures found in this scene and to resynthesize these textures in an efficient way using appropriate mathematical models. Synthetic textures were finally fused with shape data and mapped to corresponding virtual objects faces.
The Institute of Control Theory and Robotics (ICTR) [5] addressed the main project application - development of a Virtual Reality Robot Arm Trainer, which also provided a mechanism to validate the scene models.
A tool for registering partial surface fragments prior to fusion into a single model was developed [UoS]. Several algorithms [6], were developed for improving quality of registered and fused 3D data based on surface refitting, surface decimation and data recalibration. Further improvements were achieved using newly developed methods for n-views registration [7] and joint centers extraction [8]. This approach significantly decreased error accumulated by the traditional pair wise registration alternative. Because single real world objects have moving parts, work has been done on the extraction of joint centers. A novel technique based solely on the marker measurements was developed [8].
A technique for building 3D models from a sequence of uncalibrated images was developed by [IST]. A correspondence analysis method [9] has been developed based on robust matching criteria. The method has a breakdown point of 50% outliers. It allows both the integration of successive images into a mosaic and 3D reconstruction, which is accomplished either using a novel Maximum likelihood estimation algorithm [10], [11] for recovering jointly the structure, camera motion and camera intrinsic parameters or an approximate method which is much faster. As the approximate reconstruction method is sensitive to missing data, an algorithm has been devised for segmenting input data into subsets in which a set of features is visible in all images. The reconstruction results obtained for the different image subsets are then fussed to obtained a single model. Another method was described in [12] which computes a dense disparity or velocity field between two images captured with different viewpoints.
Three novel range image segmentation algorithms [13], [14], [15] and two algorithms [16], [17] for colour texture segmentation were published [UTIA]. One of range image segmentation algorithms [13], [14] is based on a combination of recursive adaptive regression model prediction for detecting range image step discontinuities and of a region growing on surface lines. The algorithm [14] assumes scene objects with planar surfaces but its segmentation quality is higher on noisy range data while keeping the numerical efficiency of the simpler method [13] published in 1997. This algorithm outperforms most of the existing range image segmentation algorithms of its category.
 |
|
 |
| Figure 1: Range image and its segmentation. | ||
Colour texture segmentation methods are based on underlying Markov random field models. One of them uses uses a novel recursive maximum pseudo-likelihood Gaussian Markov random field parameter estimation method [17]. Due to this new estimator the method is significantly faster then a similar method recently published in IEEE PAMI.
|
|
| Figure 2: Natural texture mosaic (marble, sand, grass, stone) and its segmentation. | |
Several multiscale colour Markov random fields - based texture models [18] were derived in the project. The main advantage of these models is the possibility to synthesize texture data using fast non-iterative computations. At the same time the models are flexible enough to model a large set of natural colour textures. The models assume spectral factorization of the original colour texture data space into an orthogonal Karhunen - Loeve space, where each spectral component can be independently modelled by its dedicated 2D (mono-spectral) multi-scale MRF. Multiple resolution decomposition is based on the Laplacian pyramid technique. The resulting band-pass mono-spectral factors can be efficiently modelled with lower order MRF models.
 |
| Figure 3: Natural textures (upper row) and their synthetic counterparts. |
Finally the trainer [19], which consists of a PC family computer running a real-time robot control software, was connected to a workstation used as a scene viewer. Virtual reality models acquired using the above mentioned algorithms are displayed by a dynamic viewer providing a high quality real-time visualization of the robotics scene. The more advanced is the robot workcell or other environment displayed on the workstation monitor, the more realistic impression is experienced by the robot user.
The Virtuous project was concern with the development of the technology for building detailed texture mapped surface models. During the project we have developed an advanced methodology for 3D surface registration, a method for 3D object model acquisition from video sequences, several techniques for colour texture modelling and synthesis, a feedback control strategy for registering 3D surface and texture models and finally a robot trainer has been developed.
 |
 |
 |
 |
| Figure 4: Original colour scene, range image, and its virtual model in the original and upsidedown rotated view directions. | |||
The project research resulted in more than 20 publications apart from project research reports. These achievements have been accomplished with EU project funds but also with a significant contribution of funding from complimentary sources made available at each partner home institution.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dr. Michal Haindl
Senior Researcher
Institute of Information Theory and Automation
Academy of Sciences of the Czech Republic
18208 Prague
Czech Republic
haindl@utia.cas.cz
<http://www.utia.cas.cz/>
Phone: +420 2 66052350
Dr. Michal Haindl is employed
as a Senior Researcher at UTIA (Institute of Information
Theory and Automation, Prague). From 1990 to 1992, he was
visiting researcher at University of Newcastle, Newcastle;
Rutherford Appleton Laboratory, Didcot; Centre for Mathematics
and Computer Science, Amsterdam and Institute National de
Recherche en Informatique et en Automatique, Rocquencourt working
on several image analysis and pattern recognition projects. From
1992 to 1995, he joined the Centre for Mathematics and Computer
Science,Amsterdam to work on a multimedia ESPRIT project. His
present research interest concern random fields applications in
pattern recognition and image processing. He holds Ph.D. and
Doctor of Science degrees and he is the author of about 140
papers published in books, journals and conference
proceedings.
Professor Josef Kittler
Director of the Centre for Vision, Speech, and Signal
Processing
University of Surrey
Guildford
GU2 7XH
United Kingdom
j.kittler@surrey.ac.uk
<http://www.ee.surrey.ac.uk/Research/VSSP/>
Phone: +44 1483 879294
Professor
Josef Kittler (Ph.D., ScD) is the director of the Centre
for Vision, Speech and Signal Processing of the University of
Surrey. He has been a Research Assistant in the Engineering
Department of Cambridge University (1973--75), SERC Research
Fellow at the University of Southampton (1975-77), Royal Society
European Research Fellow, Ecole Nationale Superieure des
Telecommuninations, Paris (1977--78), IBM Research Fellow,
Balliol College, Oxford (1978--80), Principal Research Associate,
SERC Rutherford Appleton Laboratory (1980--84) and Principal
Scientific Officer, SERC Rutherford Appleton Laboratory (1985).
His current research interests include Pattern Recognition,
Neural Networks, Image Processing and Computer Vision. He has
co-authored a book with the title `Pattern Recognition: a
statistical approach' published by Prentice-Hall. He has
published more than 200 papers. He is a member of the Editorial
Boards of IEEE Transactions on Pattern Analysis and Machine
Intelligence, Pattern Recognition Journal, Image and Vision
Computing, Pattern Recognition Letters, Pattern Recognition and
Artificial Intelligence. He has served as the President of the
International Association for Pattern Recognition (IAPR).
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
For citation purposes:
Haindl, M. and Kittler, K. "Autonomous Acquisition of Virtual Reality Models from Real World Scenes", Cultivate Interactive, issue
4, 7 May 2001
URL: <http://www.cultivate-int.org/issue4/virtuous/>
|
|