I don’t know about you, but to me ‘evolving robots inside of a computer’ sounds pretty cool. Especially to undergraduate me, who was a double major in computer science and biological sciences. This class was the perfect combination of my interests, which is exactly why I ended up taking it.
For the project we had two complete two tasks:
- Evolve a quadrupedal robot that ‘walks’ away from its origin
- Expand on this robot by tacking another problem of our choosing
For the second project I decided to build a robot that avoided unstable ground. To do this the robot keeps track of its yaw, pitch, and roll, and tries to minimize them as it moves throughout the landscape.
To train this robot I also had to develop a ‘hill lined’ environment, with a path for the robot to learn.
The project was structured so that the robot was built using the Bullet Physics Engine. To construct the environment and the robots we had to use C++.
In the environment there would be certain variables we wanted to keep track of. These included the location of the robot, each of the joints actuations, and later the pitch, roll, and yaw. All of these environmental variables were passed to python.
In python we had written loss functions and a neural network to control the joints for each of the robots. Random ‘mutations’ (perturbations) were applied to the neural network at each run, and the simulation was run with the neural net in control.
Based on the environmental variables passed back these mutations were either kept in the network or removed. Over time the robot then learned to minimize the loss function we had written for it.
If you want to see more details on the milestones of the project or you want to build a robot yourself, the whole course is actually a MOOC available on the Ludobots Reddit page. My final project milestones can be found on the ‘Evolving a Visually Guided Robot to Avoid “Unstable” Paths’ Wiki Page.