Task: Build Reinforcement Learning models that can generalize across environments with perceptual differences in color, lighting, texture and track dimensions.

Nov 12 – Dec 11 (AOE time): Phase I: Perception challenge

Dec 12: Top 10 winners who advance to Phase II ** Announced **

Winners of the perception challenge have been announced , all their models will be tested at the AI Driving Olympics-3 at NeurIPS 2019 at 11am on Dec 13

1. abaykov1@gmail.com — 8:08 sec
2. chris@veeta.tv — 11:68 sec
kgalev@yahoo.com — 11:23 sec
4. ray.chang@kakamobility.com — 19:21 sec
5. fonanona93@gmail.com — 9:30 sec
6. jochemlugtenburg@gmail.com — N/A
7. tomasz.ptak@gmail.com — N/A
8. serendip811@gmail.com — N/A
9. d.broderick01@gmail.com — 9:78 sec
10. estrella0625@yahoo.com — 15:20 sec

Dec 12-13: Phase II: Simulation to Reality challenge

Dec 13 Friday: Competition Day. Top 3 winners to be announced

1. abaykov1@gmail.com — 8:08 sec
5. fonanona93@gmail.com — 9:30 sec
9. d.broderick01@gmail.com — 9:78 sec


**  The winner will receive a AWS DeepRacer Evo (details will be emailed) **

AWS DeepRacer provides hardware and cloud-based service for end-to-end experimentation with reinforcement learning (RL) and can be used to systematically investigate the key challenges in building intelligent control systems in real-world. As an educational and experimental tool for RL, it is the first successful customizable large-scale deployment of deep reinforcement learning (DRL) on a vision-based robotic control agent, 20 AWS Summit Races thus far with 1600 participants (that is on leaderboard so all can see) and in console, also visible to all, we have had 4500 participants. The robocar uses only raw camera image as observations and a model-free learning method to perform robust path planning.

One of the major challenges for deployment in the real world of RL agents is in the simulation to real world transfer (sim2real). In the context of DeepRacer, we can divide the sim2real problem into two parts – i) Perception ii) Dynamics. Simulators may not have the visual fidelity as we do in the real world and not be able to capture the physics of the real world. Both of these factors can affect the representing the real environment in the simulator effectively. As a result, sometimes even after obtaining desired objective in simulation environment, when running the model in the real world, we may experience failures. In DeepRacer, we realize that the simulation fidelity is not an exact match of the real world and the physics engine uses approximations of the friction coefficients and other inertial parameters. But the beauty of DRL is that we do not require everything to be perfect. To mitigate large perceptual change affecting the car, we make two major assumptions – a) instead of using an RGB image, we grayscale the image to make the perceptual differences between simulator and the real-world narrower b) we intentionally use a shallow image feature embedder, i.e. we only use a few CNN layers, this helps the network not learn the simulation environment entirely but enough to adjust to small variations in environment. By default the DeepRacer model uses a clipped PPO algorithm that runs in asynchronous mode. Using these parameters, over 5000 developers around the world have built models in the simulator and successfully navigated a racing track in the real world.

In this challenge, we’ll take robustness in the real world to a new level. While the models so far have been trained in the simulator and tested on visually similar tracks in the real world, we introduce the following modifications

  1. Visual changes to the track – partial track to be made with tape
  2. Track dimensions to vary
  3. Variations in lighting

The competition will have two phases:

Phase I: Perceptual Sim2Real challenge

In this challenge, you will train a model which will be evaluated on a undisclosed test track, which will be significantly different from the tracks that will be available in the simulator. Each model will be evaluated on the test track, where it will have five attempts to complete a lap and he fastest completed lap will be recorded as the submission.

We close submissions on Dec 1, at which point the top 25 ranked models will be selected to be tested on a previously unseen simulation track world track. We will announce the top 10 winners on Dec 6 for the Phase 1. The 10 winners will advance to Phase II which will take place on Dec 11 at NeurIPS in Vancouver. The models for the top 10 winners will be tested on a real world track during live competition. The top 3 winner will be announced on Dec 11.

Phase II: Advanced Perceptual and Dynamics Sim2Real Challenge at Neurips

In phase II, we’ll focus on both perception and dynamics. We’ll introduce lighting variations in the track environment such as directed head lights and led light array. In addition to test model robustness in dynamics, we’ll inject random actions at inference time on the DeepRacer car. A good model is expected to overcome advanced perceptual changes, and random perturbations in the action space and navigate the track successfully. Each model will have three attempts to clock successful lap times, with the fastest lap being considered for the submission.

Available Resources

  1. Github code for Amazon SageMaker notebook environment
  2. Documentation on Amazon SageMaker DeepRacer notebook
  3. DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
  4. Tutorial and documentation on DeepRacer
  5. Github code for AWS RoboMaker DeepRacer simulation application and documentation
  6. Track Image Dataset

Potential avenues to explore

  1. Optimize PPO algorithm
  2. Explore other RL algorithms
  3. Data Augmentation outside of the simulation
  4. Modifying the assets in simulation
  5. Regularization and other hyper parameter optimization
  6. Split the learning process ( Learn to see; learn to act )

How to submit models and artifacts

Follow the instructions at AWS DeepRacer Github repository for submissions. You will need to prepare a list of model and log files to prepare your submission and participate in the challenge.

For questions, please check: AWS DeepRacer Slack Community