Set Up Replicable Real World Test Bed
Create a real-world test bed for Monty with instructions on how to replicate it.
Currently we predominantly test Monty in simulation. However, working in simulation can miss important factors encountered in the real world. We occasionally venture out to test Monty with real sensors and robotics hardware (see project showcase for some examples), but these projects are not easily replicable or repeatable on a regular basis.
It would be great to have a robotic setup that can easily be replicated by anyone. This means, all parts should be listed and obtainable, ideally at low cost. There should also be precisely specified experimental setups and evaluation pipelines (similar to our benchmarks in simulation).
For some help on using Monty in a robotic application, see our Monty for robotics tutorial
Some inspiration and existing test beds:
- An example robotic setup using a RaspberryPi, LEGOs, and cameras, can be found here: https://github.com/thousandbrainsproject/everything_is_awesome However, this was set up using a variety of LEGO pieces bought at different stores and is therefore not easily replicable. One idea could be to put together a similar setup with 3D printed parts instead, where CAD models can be shared openly.
- We have some "offline" real-world Monty benchmark experiments where we collected data with real sensors and share the dataset and experiments. Those are Monty Meets World (using the iPad depth camera) and the Ultrasound dataset (containing images collected with the Butterfly mobile ultrasound probe). One way to build on these would be to have a setup where the sensor explicitly moves in space (as it did in the ultrasound dataset), but where the input is RGB-D or stereoscopic RGB inputs (which would motivate exploring alternative depth-measurement methods). It may be possible to identify an already existing dataset of this kind in the literature, but note that it is essential that the sensor's pose is accurately tracked and stored as part of the dataset.
Help Us Make This Page Better
All our docs are open-source. If something is wrong or unclear, submit a PR to fix it!
Updated 4 days ago