base

BaseAgent & its config.

class cneuromax.fitting.neuroevolution.agent.base.BaseAgentConfig(env_transfer='${config.env_transfer}', fit_transfer='${config.fit_transfer}', mem_transfer='${config.mem_transfer}')

Bases: object

Holds BaseAgent config values.

Parameters:

• env_transfer (bool, default: '${config.env_transfer}') – See env_transfer.

• fit_transfer (bool, default: '${config.fit_transfer}') – See fit_transfer.

• mem_transfer (bool, default: '${config.mem_transfer}') – See mem_transfer.

class cneuromax.fitting.neuroevolution.agent.base.BaseAgent(config, pop_idx, *, pops_are_merged)

Bases: ABC

Root Neuroevolution agent class.

From an algorithmic perspective, we make use of 50% truncation selection, meaning that the top 50% of agents in terms of fitness score are selected and will produce two children agents each.

From an implementation perspective, pop_size instances of this class will be created upon initialization. Whenever an agent is selected, a copy of this object will be created and sent to a MPI process in possession of a non-selected agent. Both this original instance and the copy sent to the other process will be mutated in-place (meaning no new instance will be created).

It might therefore be useful to sometimes consider this class as an AgentContainer class rather than an Agent class.

Parameters:

• config (BaseAgentConfig)

• pop_idx (int) – The agent’s population index. An index of 0 means that the agent is in the generator population while an index of 1 means that the agent is in the discriminator population.

• pops_are_merged (bool) – See pop_merge.

config

Type:

BaseAgentConfig

role

The agent’s role. Can be either "generator" or "discriminator".

Type:

str

is_other_role_in_other_pop

Whether the agent is the other role in the other population. If the two populations are merged (see pops_are_merged), then an agent is both a generator and a discriminator. It is a generator/discriminator in this population while it is a discriminator/generator in the other population. Such type of agent needs to accomodate this property through its network architecture.

Type:

bool

total_num_steps

The total number of steps taken by the agent and all of its predecessors.

Type:

int

curr_eval_score

The score obtained by the agent during the current evaluation.

Type:

float

curr_eval_num_steps

The number of steps taken by the agent during the current evaluation.

Type:

int

saved_env

The torchrl environment instance to resume from (only set if env_transfer is True).

Type:

torchrl.envs.EnvBase

saved_env_out

The latest output from the environment to resume from (only set if env_transfer is True).

Type:

tensordict.Tensordict

curr_episode_score

The current episode score (only set if env_transfer is True).

Type:

float

curr_episode_num_steps

The number of steps taken in the current episode (only set if env_transfer is True).

Type:

int

continual_fitness

The agent’s fitness in addition to all of its predecessors’ fitnesses (only set if fit_transfer is True).

Type:

float

initialize_eval_attributes()

Initializes attributes used during evaluation.

Return type:

None

abstract mutate()

Applies random mutation(s) to the agent.

Return type:

None

abstract reset()

Resets the agent’s memory state.

Return type:

None