Robotic Vision Scene Understanding Challenge

Overview

The Robotic Vision Scene Understanding Challenge evaluates how well a robotic vision system can understand the semantic and geometric aspects of its environment. There are two tasks in this challenge: Object-based Semantic Mapping / SLAM, and Scene Change Detection.

Key features of the challenge:

Watch the video below to learn more:

Challenge Participation

Competition Server

Our challenge is currently live, and available here on the EvalAI website. Please create an account, sign in, & click the “Participate” tab to enter our challenge. Full details on how to participate, the available software framework, and submission requirements are provided on the site.

The challenge is open until the 1st of September 2020, and a total of $2,500 USD will be awarded to high-performing competitors.

Our 2020 competition is also held in conjunction with our ICRA 2020 Workshop “Scene Understanding and Semantic SLAM: Progress, Applications, and Limitations”.

Note: Workshop details will be updated due to the COVID-19 Pandemic.

The BenchBot Software stack

Our Scene Understanding Challenges all use our new software stack called BenchBot. The BenchBot software stack provides user-friendly interfaces (helper scripts & a simple API manages simulation , robot movement, challenge rule enforcement, and evaluation.

To check out the framework and get started on our scene understanding problems, check out the github page at benchbot.org.

The Challenge Tasks

Task 1: Object-based Semantic Mapping / SLAM

For this task we evaluate how well participants can build a map of the environment that contains all objects of interest. The evaluation metric rewards accurate pose, shape, and object semantics.

This task can be done in one of two modes (Active or Passive) and two Streams (with or without groundtruth camera pose). In Active mode, the user can control the robot’s motion to explore the environment. In Passive mode, the user has no control over the robot. In both modes, the user code has access to the data from the robot’s RGB-D camera.

Task 2: Scene Change Detection

The goal of this task is to identify all objects that disappeared, appeared, or moved in an environment from one day to another. The robot can explore the environment on both days, but has to spot all the differences.

This task can be done in one of two modes (Active or Passive) and two Streams (with or without groundtruth camera pose). In Active mode, the user can control the robot’s motion to explore the environment. In Passive mode, the user has no control over the robot. In both modes, the user code has access to the data from the robot’s RGB-D camera.

Watch the videos below Some objects disappeared, some new objects appeared. Can you spot the differences between both days? Which objects are new, which are gone? Can you write an algorithm to solve this problem?

Day 1
Day 2

Organisers, Support, and Acknowledgements

Stay in touch and follow us on Twitter for news and announcements: @robVisChallenge.

Niko Sünderhauf
Queensland University of Technology
Feras Dayoub
Queensland University of Technology
David Hall
Queensland University of Technology
Haoyang Zhang
Queensland University of Technology
Ben Talbot
Queensland University of Technology
Suman Bista
Queensland University of Technology



The Robotic Vision Challenges organisers are with the Australian Centre for Robotic Vision at Queensland University of Technology (QUT) in Brisbane, Australia.