worker.collector.base_serial_evaluator

base_serial_evaluator

Please Reference ding/worker/collector/base_serial_evaluator.py for usage

BaseSerialEvaluator

VectorEvalMonitor

class ding.worker.collector.base_serial_evaluator.VectorEvalMonitor(env_num: int, n_episode: int)

Overview:

In some cases, different environment in evaluator may collect different length episode. For example, suppose we want to collect 12 episodes in evaluator but only have 5 environments, if we didn’t do any thing, it is likely that we will get more short episodes than long episodes. As a result, our average reward will have a bias and may not be accurate. we use VectorEvalMonitor to solve the problem.

Interfaces:

__init__, is_finished, update_info, update_reward, get_episode_reward, get_latest_reward, get_current_episode, get_episode_info

__init__(env_num: int, n_episode: int) → None

Overview:

Init method. According to the number of episodes and the number of environments, determine how many episodes need to be opened for each environment, and initialize the reward, info and other information

Arguments:

• env_num (int): the number of episodes need to be open

• n_episode (int): the number of environments

get_current_episode() → int

Overview:

Get the current episode. We can know which episode our evaluator is executing now.

get_episode_info() → dict

Overview:

Get all episode information, such as total reward of one episode.

get_episode_reward() → list

Overview:

Get the total reward of one episode.

get_latest_reward(env_id: int) → int

Overview:

Get the latest reward of a certain environment.

Arguments:

• env_id: (int): the id of the environment we need to get reward.

is_finished() → bool

Overview:

Determine whether the evaluator has completed the work.

Return:

• result: (bool): whether the evaluator has completed the work

update_info(env_id: int, info: Any) → None

Overview:

Update the information of the environment indicated by env_id.

Arguments:

• env_id: (int): the id of the environment we need to update information

• info: (Any): the information we need to update

update_reward(env_id: int, reward: Any) → None

Overview:

Update the reward indicated by env_id.

Arguments:

• env_id: (int): the id of the environment we need to update the reward

• reward: (Any): the reward we need to update