A deep reinforcement learning model based on deterministic policy gradient for collective neural crest cell migration
Modeling cell interactions such as co-attraction and contact-inhibition of locomotion is essential for understanding collective cell migration. Here, we propose a novel deep reinforcement learning model for collective neural crest cell migration. We apply the deep deterministic policy gradient algorithm in association with a particle dynamics simulation environment to train agents to determine the migration path. Because of the different migration mechanisms of leader and follower neural crest cells, we train two types of agents (leaders and followers) to learn the collective cell migration behavior. For a leader agent, we consider a linear combination of a global task, resulting in the shortest path to the target source, and a local task, resulting in a coordinated motion along the local chemoattractant gradient. For a follower agent, we consider only the local task. First, we show that the self-driven forces learned by the leader cell point approximately to the placode, which means that the agent is able to learn to follow the shortest path to the target. To validate our method, we compare the total time elapsed for agents to reach the placode computed using the proposed method and the time computed using an agent-based model. The distributions of the migration time intervals calculated using the two methods are shown to not differ significantly. We then study the effect of co-attraction and contact-inhibition of locomotion to the collective leader cell migration. We show that the overall leader cell migration for the case with co-attraction is slower because the co-attraction mitigates the source-driven effect. In addition, we find that the leader and follower agents learn to follow a similar migration behavior as in experimental observations. Overall, our proposed method provides useful insight on how to apply reinforcement learning techniques to simulate collective cell migration.