REPEAT Project Overview

This project concerns autonomous perceiving and understanding of the environment, as well as image-based localization. These constitute key challenges for any autonomous or augmented reality system, especially when these functions are performed under low-quality sensor data. The project combines machine learning, computer vision and computer engineering to develop new methods for data-driven 3D visual computing and image based localization, while considering the computational resource limitations of autonomous systems. We put special interest in advancing the state-of-the-art in vision and sensor fusion methods, and in developing methods capable of inferring location, pose, and semantics from visual data. This is a cross-layer approach over of the fields of computer vision, machine learning, and computer engineering with the aim at renewing the view on how these can be combined.

Visualization of the Aalto odometry benchmark dataset. The data was captured in a mall, recorded by an iPhone and by a Google Tango device while walking, using an elevator, and an escalator. Middle image: the motion trajectory (red), and the point cloud produced by the Tango device. Right: the true trajectories determined by iPhone inertial sensors and manually placed landmarks. A related video available at https://github.com/AaltoVision/ADVIO.

Project Consortium

The REPEAT project is an Academy of Finland consortium project of

Tampere University (Prof. Esa Rahtu, esa.rahtu@tuni.fi; Google Scholar page, consortium lead)

Aalto University (Prof. Juho Kannala, juho.kannala@aalto.fi; Google Scholar page)

University of Vaasa (Prof. Jani Boutellier, jani.boutellier@univaasa.fi; Google Scholar page).

Timeline

REPEAT starts on January 1, 2020

Information about the Partner Universities

The Computer Science Department at Aalto provides world-class research and education in modern computer science to foster future science, engineering and society. The work combines fundamental research with innovative applications. The department is routinely ranked among the top 10 CS departments in Europe and in the top 100 globally.

The Department of Signal Processing at Tampere University has 170 members of which 30-40% are of foreign origin. The department has held the prestiguous status of a Center of Excellence in Research (CoE) elected by the Finnish Academy of Sciences. Core areas of research include image, video and audio signal processing and analysis as well as machine learning related topics.

The University of Vaasa (UVA) is an international research institution with more than 25% of the research and teaching staff coming from outside of Finland. UVA has recently invested 10 M€ in hiring new research faculty, and together with the co-located 200 M€ industry-driven Wärtsilä Smart Technology Hub (to be opened in 2020), the city of Vaasa region is an inspiring location for high-impact research.

Video clips on previous works

DGC-Net (Iaroslav Melekhov, Aleksei Tiulpin, Torsten Sattler, Marc Pollefeys, Esa Rahtu, and Juho Kannala)

ADVIO (Santiago Cortés, Arno Solin, Esa Rahtu, Juho Kannala)

References

  1. A. Solin, S. Cortes, E. Rahtu, and J. Kannala, “Inertial odometry on handheld smartphones,” in FUSION, 2018
  2. S. Cortés, A. Solin, and J. Kannala, “Deep learning based speed estimation for constraining strapdown inertial navigation on smartphones,” in MLSP, 2018.
  3. A. Solin, S. Cortes, E. Rahtu, and J. Kannala, “PIVO: Probabilistic inertial-visual odometry for occlusion-robust navigation,” in WACV, 2018.
  4. S. Cortés, A. Solin, E. Rahtu, and J. Kannala, “ADVIO: An authentic dataset for visual-inertial odometry,” in ECCV, 2018.
  5. A. Meirhaeghe, J. Boutellier, and J. Collin, “The direction cosine matrix algorithm in fixed-point: Implementation and analysis,” in ICASSP, 2019.
  6. I. Melekhov, A. Tiulpin, T. Sattler, M. Pollefeys, E. Rahtu, and J. Kannala, “DGC-Net: Dense geometric correspondence network,” in WACV, 2019.
  7. Z. Laskar, I. Melekhov, H. R. Tavakoli, J. Ylioinas, and J. Kannala, “Geometric image correspondence verification by dense pixel matching,” arxiv.org/abs/1904.06882, 2019.
  8. Z. Laskar, S. Huttunen, D. Herrera, E. Rahtu, and J. Kannala, “Robust loop closures for scene reconstruction by combining odometry and visual correspondences,” in ICIP, 2016.
  9. Y. Hou, J. Kannala, and A. Solin, “Multi-view stereo by temporal nonparametric
    fusion,” arxiv.org/abs/1904.06397, 2019.
  10. X. Li, J. Ylioinas, J. Verbeek, and J. Kannala, “Scene coordinate regression with angle-based reprojection loss for camera relocalization,” in Geometry Meets Deep Learning Workshop, 2018.
  11. X. Li, J. Ylioinas, and J. Kannala, “Full-frame scene coordinate regression for image-based localization,” in RSS, 2018.
  12. M. Linna, J. Kannala, and E. Rahtu, “Real-time human pose estimation with convolutional neural networks,” in VISIGRAPP, 2018.
  13. M. Khan, H. Huttunen, and J. Boutellier, “Binarized convolutional neural networks for efficient inference on GPUs,” in EUSIPCO, 2018.
  14. J. Boutellier, J. Wu, H. Huttunen, and S. S. Bhattacharyya, “PRUNE: Dynamic and decidable dataflow for signal processing on heterogeneous platforms,” IEEE Transactions on Signal Processing, 2018.
  15. I. Melekhov, J. Kannala, and E. Rahtu, “Siamese networks features for image matching,” in ICPR, 2016.
  16. P. Rantalankila, J. Kannala, and E. Rahtu, “Generating object segmentation proposals using global and local search,” in CVPR, 2014.

News

Oct 1, 2019

Project page opened.