Car-racing game “vcracing” tutorial

vcracing

Overview

You can drive it manually, and also with machine learning. “vcracing” is a car-racing game. Unlike OpenAI Gym’s “CarRacing”, vcracing is designed for machine learning as below.

• Course status (environment) are observable.
• Easy save and duplicate of environment.

We tried modifying CarRacing of OpenAI Gym (→the original）for machine learning（→the article）, but I became totally different. So we released it by own.

Since the original could only observe “step reward” and “episode end flag”, autonomous driving with machine learning was almost impossible. vcracing can watch course information and car status, which allows generalized learning for random made courses.

In addition, the original is definitely unable to “save” and “duplicate” environments, so it cannot branch off in the middle to do distributed searches, resulting in waste of learning time. vcracing can save and replicate environments with copy.deepcopy() to respond to a variety of learning strategies.

Installing

pip install vcracing

Sample code

from vcracing import vcracing

# Load game
env = vcracing()

# See starting image
env.render()

# Drive
for i in range(30):
    # Chose action
    # [Steer, Accel] where Steer in [-1, +1], Accel in [-1, +1]
    # Note that Steer>0 turns LEFT, Accel<0 reverse gear
    action = [0.1, 1] #[Steer, Accel]
    
    # Step
    state, reward, over, done = env.step(action)
    #env.render() # See every step image

# See current image
env.render()

Instant playing manual

Straight acceleration with [0, 1] and slightly turn left with [0.2, 1]. Course consists of n peace of road panels, and each panel you step on gives you (100/n) points of reward. state[‘rewards’]==100 or done==True means complete.

env=vcracing(track_seed=1) allows you to create deterministic course. state=reset() to reset the environment without changing course.

Note that sharp turning will greatly reduce speed, so gentle turning is essential to achieve good record.

Example of manual driving

You can explore play() one by one as below. By checking the last two frames to adjust play(), you may easily reach the goal even manually.

# Deterministic course
env = vcracing(track_len=200, track_seed=5, car=None)

# Stepping function from "action" and "steps"
def play(action, step):
    for i in range(step):
        _, _, _, _ = env.step(action)
        #env.render()

# Play one by one
play([0, 1], 10)
play([0.35, 1], 8)
play([0, 1], 20)
play([0.8, 1], 12)
play([0, 1], 75)
play([0.35, 1], 65)
play([0, 1], 95)
play([0.35, 1], 60)
play([0, 1], 24)

# Checking the last 2 frames
for i in range(2):
    action = [0, 1]
    _, _, _, done = env.step(action)
    print(done)
    env.render()

Specification

Load game and reset

This is where the course is automatically generated and the car image selected. You can specify an approximate length of the course and a random seed.

When you reset game without changing the course , we recommend reset() as next section.

env = vcracing(track_len=200, track_seed=None, car='BT46')

Reset game

Reset game without changing the course. There is no option. The return value is an array of dictionary types filled with variety of information (see env.step() below).

state = env.reset()

Choose action

Consists of two values, steer and accel.

action = [0, 0] #[Steer, Accel]

Step

Enter the action to step the game by one frame. There are four return values. Especially the state is full of information that can be used for machine learning.

state, reward, over, done = env.step(action)

“state” consists of the following.

Rendering an image

You can see the entire course at the moment. Due to the slow processing speed, it is recommended to check only once every few frames.

env.render(mode='plt', dpi=100)

An example of an image.

Saving and duplicating environments

You can easily duplicate an environment using copy.deepcopy() as follows.

from vcracing import vcracing
from copy import deepcopy

# Generate course
env = vcracing(track_len=200, track_seed=5)

# 20 frames go forward
for i in range(20):
    action = [0, 1]
    _, _, _, _ = env.step(action)

# Check image
print('env')
env.render()

# Copy env as "env2"
env2 = deepcopy(env)

# Turn left in env
for i in range(10):
    action = [1, 1]
    _, _, _, _ = env.step(action)

# Turn right in env2
for i in range(10):
    action = [-1, 1]
    _, _, _, _ = env2.step(action)

# Check images
print('env')
env.render()
print('env2')
env2.render()

(For developers) Obtaining driving records

After finish, you can get the driving records as follows.

record = env.get_record()

# All inputs
print(record['input_all'])
# All car positions
print(record['position_all'])
# All car radians
print(record['radian_all'])
# Lap time
print(record['lap_time'])

License

Reference source

This package has been developed with reference to OpenAI Gym CarRacing-v0. We would like to express my respect and appreciation for the efforts of all those involved.

https://gym.openai.com/envs/CarRacing-v0/

gym.openai.com

License

Private Use, Commercial Use are permitted. Don’t forget Copyright notice (e.g. https://vigne-cla.com/vcracing-tutorial-en/) in any social outputs. Thank you.

For Educational Use, Copyright notice is NOT necessary.

Required: Copyright notice in any social outputs.
Permitted: Private Use, Commercial Use, Educational Use
Forbidden: Sublicense, Modifications, Distribution without pip install, Patent Grant, Use Trademark, Hold Liable.

Disclaimer

We will not compensate you for any damage caused by the use of vcracing.

Coming update

render() improvement

Update