Simple Neuroevolution Cartpole

100%

Simple Neuroevolution Cartpole

✓ Published0🌍 Public

Last edited Jun 23, 2016

Created on Jun 22, 2016

This example shows a neuroevolution system training agents to solve the CartPole-v0 balancing task using OpenAI Gym. A population of 100 neural networks evolves over 100 epochs, with each network’s fitness determined by its average score across 5 episodes. The Keras models use densely connected layers with ReLU activation, while the evolution process selects and mutates networks based on their performance. Agent scores are collected and then the top performers are used to generate the next generation, gradually improving the population’s ability to keep the pole upright.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 7814a13dbae9fde1771606fe6081513d

This interactive visualization tracks the training progress of a Dueling Double Deep Q-Network (Duel DDQN) agent within the OpenAI Gym CartPole-v0 environment. It plots the reward per episode over time, showing how the agent's performance improves as it learns to balance the pole. The code uses the Python libraries `gym` for the environment and `torch` for the neural network, with command-line arguments specifying hyperparameters like `--gamma` and `--batch_size`. The chart updates in real time, rendering the learning curve as the agent iterates through episodes.

SSohojoe

67% match

Loading thumbnail…

Gist 36fa85de37aeb50756ec65aa6548b25f

This example implements an artificial neural network where individual neurons are drawn as circles on a canvas. Each neuron connects to a set of previous-layer inputs, multiplying their output values by random weights, then passing the summed result through a sigmoid activation function. The code renders each neuron’s fill color based on its output value, using the Processing API’s `ellipse()` and `fill()` functions to visualize activation levels. The neural activity propagates layer by layer, producing a dynamic, self-contained simulation of a feedforward network without external data inputs.

CCharlesFr

54% match

Loading thumbnail…

Gist c9d1f7872865db4e5bfc0991636376c8

This example visualizes the training progress of a Dueling Double Deep Q-Network (Duel DDQN) agent across 10,000 episodes in the MountainCar-v0 environment. It uses the Python library `duel.py` from the Duel_DDQN repository, invoking it via a command-line call with the `--gym_record` flag to capture the agent’s performance over time. The chart presents episode-by-episode reward trends, showing how the agent’s policy improves as training advances, with the underlying data generated by OpenAI Gym’s environment simulation.

SSohojoe

54% match

Loading thumbnail…

OpenAi.Gym.Baselines.DeepQ Restoring checkpoint in next training

This example demonstrates a continuous training loop for OpenAI Gym’s CartPole-v0 using the Baselines DeepQ algorithm. It shows how a model checkpoint is saved and restored across alternating training and evaluation sessions, allowing training to resume seamlessly. The code uses `deepq.learn` and `deepq.load` from Baselines, with a custom `ActWrapper` that serializes the model via dill and TensorFlow’s `U.save_state`. A shell script alternates between `train_worker.py` and `enjoy_worker.py`, pausing training to render episodes and report rewards before resuming.

SSohojoe

52% match

Loading thumbnail…

kerlym pong.py does not work with default settings

This visualization compares the training progress of DQN and DDQN agents over 55,000 episodes of the Atari Pong-v0 environment, showing that neither algorithm converges to a winning strategy. The plot uses Gym’s evaluation API to fetch reward-per-episode data from two public evaluation records. The chart displays raw episode rewards as thin gray lines with a bold moving average overlay, directly exposing the high variance and lack of improvement in both agents’ performance. The code uses Python with matplotlib for plotting and requests to load JSON data from the Gym evaluation endpoints.

SSohojoe

50% match

Loading thumbnail…

dyna-maze based on https://github.com/thehawkgriffith/dyna-maze

This example animates a reinforcement-learning agent navigating a 6×9 grid maze toward a goal state, using the Dyna-Q algorithm to balance exploration and planning. The visualization shows the agent’s current position as a marker, a live policy heatmap of action values, and the cumulative reward per episode. Built with Matplotlib and NumPy, the scene renders in a terminal via `Environment.render()`, while the agent’s state and action history drive the displayed trajectories. The maze layout, obstacles, and reward structure come from the `dyna-maze` repository’s hardcoded environment.

SSohojoe

49% match

Loading thumbnail…

Gist d6e2a2307801a1a2d367875fe3cf671a

This example demonstrates a three-layer neural network learning to classify patterns through supervised backpropagation. The Processing sketch uses a custom `Neuron` class with a training loop that adjusts connection weights based on error signals. The code shows the core training mathematics, including a hyperbolic tangent activation function, delta computation, and weight updates scaled by a fixed learning rate. The visualization itself is not present in this code fragment, which focuses solely on the algorithmic update logic for each neuron’s error and weight adjustment.

CCharlesFr

48% match

Loading thumbnail…

Training Performance

The visualization shows training progress across four reinforcement-learning environments—Hopper, Walker, Humanoid, and Ant—by comparing their total training steps, wall-clock time, and throughput. It uses a table-style layout rendered with plain HTML and CSS, drawing data from a TSV file. Each environment row displays its metrics, with the steps-per-second value visually encoded as a horizontal bar, making the relative performance of each task immediately comparable. The code loads the data via `d3.tsv` and constructs the bar chart using SVG `rect` elements scaled by the parsed numeric values.

SSohojoe

48% match