We are interested in understanding the mechanisms that enable agents to operate in a complex, changing, and uncertain world. To what extent do agents explore versus exploit? How do they balance internal versus external computation? What kind of dynamics emerge when agents have multiple competing or cooperating learning systems?
In one strand, we are interested in using modular reinforcement learning to study the idea of having “multiple selves”, in an attempt to synthesize longstanding observations of psychological conflict (i.e. psychodynamic theories) with modern computational principles. We evolved to satisfy many ongoing needs; the more independent and possibly conflicting these needs are, the more we expect a modular system would be better suited to balance them. The computational benefits of modularity in such environments, like improvements in exploration and learning, provide a normative basis for the internal tug-of-war people tend to experience when making decisions.