Do As You Teach: A Multi-Teacher Approach to Self-Play in Deep Reinforcement Learning | AAMAS 2023

This collaborative work was presented at the 22nd International Conference on Autonomous Agents and Multiagent Systems in May 2023 (AAMAS).

Further information about AIR’s presence at AAMAS 2023 are available in this blog post.

Abstract

A long-running challenge in the reinforcement learning (RL) community has been to train a goal-conditioned agent in sparse reward environment such that it also generalizes to unseen goals. We propose a novel goal-conditioned RL algorithm; Multi-Teacher Asymmetric Self-Play, which allows 1+ agents (i.e., the teachers) to create a successful curriculum for another agent (i.e., the student) and empirically demonstrate its effectiveness on domains like Fetch-Reach and a novel driving simulator designed for goal-conditioned RL. Surprisingly, results also show that training with multiple teachers actually helps the student learn faster by better covering the state space. Moreover, results show that completely new students can learn offline from the goals generated by teachers trained with a previous student. This is crucial in the context of application domains where repeatedly training a teacher agent is expensive or even infeasible.

Cite

@inproceedings{doasyouteach_2023,
    title={
        Do As You Teach: A Multi-Teacher Approach to Self-Play in Deep Reinforcement Learning
    },
    author={
        Kharyal, Chaitanya and 
        Sinha, Tanmay and 
        Gottipati, Sai Krishna and 
        Abdollahi, Fatemeh and 
        Das, Srijita and 
        Taylor, Matthew E
    },
    booktitle={
        Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems
    },
    pages={2457--2459},
    year={2023}
}