Model-Free Distant Agent Exploration Policy
A policy for the distant agent to rapidly visit salient regions, then systematically explore.
We would like a model-free strategy for the distant agent to first rapidly explore salient regions of the visual input. In the absence of model-based signals, it should then begin systematically exploring the visual space.
As a starting point, the distant agent SMs can leverage a salience-based goal generator. This generates goals that rapidly move the sensor to salient regions. This goal generation can also leverage an inhibition of return strategy that reduces the likelihood of revisiting locations.
The proposal is to implement a decay in the salience map such that we initially saccade to salient regions, but that these slowly become less interesting. As the salience decays, the policy will converge to a pure inhibition of return strategy. This would capture the intuitive practice of initially looking at interesting things, then exploring more uniformly, and this could continue until a model-based goal is received to move to a particular location.
This policy could be used at both learning and inference, giving us sparse (and hence computationally more efficient), models that still generalize across modalities. In particular, one limitation of a salience-only strategy during learning and inference is that other modalities which do not follow the same policy at inference (such as the surface agent) will visit under-represented regions. At the same time, the policy proposed here could ensure that salient regions are more robustly represented at both learning and inference. This is particularly important in naturalistic settings, where relevant objects are often sparsely distributed (e.g. objects scattered around a room).
Ideally there would also be a model-free reset for the salience map. For example, if we look at an entirely different part of the room, or the object being studied rotates, then the saliency map should become interesting again.
An aim would be to update the benchmarks to leverage sparser models than the current ones learned by the surface agent, even if that means an initial drop in accuracy. The key requirement is that the learned models should look reasonable (i.e. well sampled).
Help Us Make This Page Better
All our docs are open-source. If something is wrong or unclear, submit a PR to fix it!
Updated 4 days ago