You should in general look for simple implementations of dqn and ppo to learn from. Pytorch code is much easier to read for RL because it’s easy to run the environment simulation on cpu and the agent training on gpu. For classes, I recommend David Silver’s UCL class on deep reinforcement learning. He’s very good at explaining the fundamentals. https://github.com/seungeunrho/minimalRL