def eval_fitness(genome_id,genome,config,time_steps=400):
maze_env = copy.deepcopy(trialSim.orig_maze_environment)
control_net = neat.nn.FeedForwardNetwork.create(genome,config)
fitness = maze.maze_simulation_evaluate(
env=maze_env,net=control_net,time_steps=time_steps
)
record = agent.AgentRecord(
generation=trialSim.population.generation,
agent_id=genome_id
)
record.fitness = fitness
record.x = maze_env.agent.location.x
record.y = maze_env.agent.location.y
record.hit_exit = maze_env.exit_found
record.species_id = trialSim.population.species.get_species_id(genome_id)
record.species_age = record.generation - \
trialSim.population.species.get_species(genome_id).created
trialSim.record_store.add_record(record)
return fitness
该代码首先创建原始迷宫环境的副本，以避免评估运行之间的干扰。之后，它使用提供的NEAT配置从指定的基因组创建ANN，并在给定数量的时间步长下开始迷宫仿真。然后，将返回的智能体适应度得分以及其他统计信息存储到特定的AgentRecord实例中，并将其添加到记录存储中。
将收集的记录将保存到输出目录中的data.pickle文件中，并用于可视化所有评估智能体的性能。
智能体记录可视化
在神经进化过程中收集了所有智能体的评估记录后，可视化记录的数据以观察性能。可视化应包括所有智能体的最终位置，并允许设置物种适应度的阈值，以控制将物种添加到相应的绘图中。
该目标函数基于迷宫求解器的适应度得分的估算，该迷宫求解器通过在执行400个模拟步骤后测量其最终位置与迷宫出口之间的距离来进行估算。因此，目标函数是面向目标的，并且仅取决于实验的最终目标：到达迷宫出口区域。
该实验中使用的目标函数确定如下。首先，需要将损失函数定义为模拟结束时智能体的最终位置与迷宫出口的位置之间的欧几里得距离：
L = ∑ i = 1 2 ( a i − b i ) 2 \mathcal L=\sqrt{\sum_{i=1}^2(a_i-b_i)^2} L=i=1∑2(ai−bi)2
L \mathcal L L是损失函数， a a a是智能体的最终位置坐标， b b b是迷宫出口的坐标。
使用上述损失函数，定义适应度函数：
F = { 1.0 , L ≤ R e x i t F n , otherwise \mathcal F = \begin{cases} 1.0, & \text{} \\ \mathcal F_n, & \text{otherwise} \end{cases} F={1.0,Fn,L≤Rexitotherwise
R e x i t R_{exit} Rexit是迷宫出口点周围出口区域的半径， F n \mathcal F_n Fn是归一化适应度得分。归一化的适应度评分如下：
F n = L − D i n i t D i n i t \mathcal F_n = {\frac {\mathcal L-D_{init}}{D_{init}}} Fn=DinitL−Dinit
D i n i t D_{init} Dinit是导航仿真开始时从智能体到迷宫出口的初始距离。
该公式将适应性评分标准化为 ( 0 , 1 ] (0,1] (0,1]，但在极少数情况下，当智能体的最终位置远离其初始位置并且回合结束时，可能会导致负值。将应用标准化的适应度评分来避免出现负值：
F n = { 0.01 , F n ≤ 0 F n , otherwise \mathcal F_n = \begin{cases} 0.01, & \text{} \\ \mathcal F_n, & \text{otherwise} \end{cases} Fn={0.01,Fn,Fn≤0otherwise
当适应度得分小于或等于0.01时，将为其分配支持的最小适应度得分值（0.01）；否则，将按原样使用。使最小适应度得分高于零，以使每个基因组都有繁殖的机会。
Python中的以下代码实现了面向目标的目标函数：
Calculate the fitness score based on distance from exit
fitness = env.agent_distance_to_exit()
if fitness <= self.exit_range:
fitness = 1.0
else:
Normalize fitness to range (0,1]
fitness = (env.initial_distance - fitness) / \
env.initial_distance
if fitness <= 0.01:
fitness = 0.01
下图显示了用于此实验的迷宫：
图中的迷宫有两个特定的位置，左上方的圆圈表示迷宫导航智能体的起始位置。右下角的圆圈标记迷宫出口的确切位置。迷宫求解器(智能体)需要到达迷宫出口的附近，该迷宫出口由其周围的特定出口范围区域表示，以完成任务。
NEAT-Python超参数选择
根据目标函数定义，可以到达迷宫出口区域而获得的导航智能体适应度得分的最大值为1.0。将人口规模设置为250。
[NEAT]
fitness_criterion = max
fitness_threshold = 1.0
pop_size = 250
reset_on_extinction = False
表型ANN的初始配置包括10个输入节点，2个输出节点和1个隐藏节点。输入和输出节点对应于输入传感器和控制信号输出。提供隐藏节点是为了从神经进化过程的开始就引入非线性，ANN配置如下：
num_hidden = 1
num_inputs = 10
num_outputs = 2
为了扩展解的搜索范围，需要增加种群的种类，以在有限的世代内尝试不同的基因组构型。这可以通过降低相容性阈值或通过增加用于执行基因组相容性分数计算的系数的值来完成。：
[NEAT]
compatibility_disjoint_coefficient = 1.1
[DefaultSpeciesSet]
compatibility_threshold = 3.0
我们的目标是创建具有最少数量的隐藏节点和连接的迷宫求解器控件ANN。最佳的ANN配置在通过神经进化过程进行训练期间以及在迷宫求解模拟器中进行推理的过程中，在计算上的花费较少。可以通过减少添加新节点的概率来产生最佳的ANN配置：
node_add_prob = 0.1
node_detele_prob = 0.1
最后，允许神经进化过程不仅利用具有前馈连接的ANN配置，而且还利用循环结构。通过使用循环连接，使ANN拥有内存并成为状态机成为可能：
feed_forward = False
迷宫环境配置文件
实验的迷宫环境配置以纯文本格式提供。配置文件的内容如下：
11
30 22
0
270 100
5 5 295 5
295 5 295 135
295 135 5 135
...
迷宫配置文件的格式如下：
-
第一行包含迷宫中的墙壁数量。
-
第二行确定智能体的起始位置(x,y)。
-
第三行表示智能体的初始航向，以度为单位。
-
第四行显示迷宫出口位置(x,y)。
-
接下来几行定义了迷宫的墙壁。
迷宫墙由线段表示，前两个数字定义起始端点的坐标，后两个数字确定结束端点的坐标。智能体的起始位置和迷宫出口指示二维空间中某个点的x和y坐标。
实验启动程序
实验启动程序是在maze_experiment.py文件中实现的。提供了以下功能：读取命令行参数，配置和启动神经进化过程，以及呈现实验结果。此外，它还包括回调函数的实现，以评估基因组的适应度。这些回调函数将在初始化期间提供给NEAT-Python库环境。
- 首先初始化迷宫模拟环境：
maze_env_config = os.path.join(local_dir,'%_maze.txt' % args.maze)
maze_env = maze.reade_environment(maze_env_config)
args.maze指的是用户在启动Python脚本时提供的命令行参数，指向迷宫环境配置文件。
- 创建NEAT配置对象，并使用创建的配置对象创建neat.Population对象：
config = neat.Config(neat.DefaultGenome,neat.DefaultReproduction,
neat.DefaultSpeciesSet,neat.DefaultStagnation,config_file)
p = neat.Population(config)
- 创建迷宫模拟环境并将其存储为全局变量，以简化适应度评估回调函数对其的访问：
global trialSim
trialSim = MazeSimulationTrial(maze_env=maze_env,population=p)
MazeSimulationTrial对象包含用于访问原始迷宫模拟环境以及用于保存智能体评估结果的记录存储的字段。在每次对适应度评估回调函数eval_fitness进行调用时，原始迷宫模拟环境将被复制，并由特定的求解器智能体用于迷宫求解模拟。之后，从环境中收集有关智能体的完整统计信息，包括其在迷宫中的最终位置，并将其添加到记录存储中。
- 添加各种统计报告器：
p.add_record(neat.StdOutReporter(True))
stats = neat.StatisticsReporter()
p.add_reporter(stats)
p.add_reporter(neat.Checkpointer(
5,filename_prefix='%s/maze-neat-checkpoint-' % trial_out_dir))
报告器可将神经进化过程的中间结果显示在控制台上，并收集统计信息在程序完成后进行可视化。
- 最后，将神经进化过程运行指定的代数，并检查是否找到了解：
start_time = time.time()
best_genome = p.run(eval_genomes,n=n_generations)
elapsed_time = time.time() - start_time
solution_found = (best_genome.fitness >= config.fitness_threshold)
if solution_found:
print("SUCCESS: The stable maze solver controller was found!!!")
else:
print("FAILURE: Failed to find the stable maze solver controller!!!")
假设，如果NEAT-Python库返回的最佳基因组的适应性得分大于或等于配置文件中设置的适应度阈值，则已经找到解。打印完成该过程所花费的时间。
基因组适应度评估
回调函数用于评估属于特定物种的所有基因组的适应度得分：
def eval_genomes(genomes,config):
for genome_id,genome in genomes:
genome.fitness = eval_fitness(genome_id,genome,config)
运行迷宫导航实验
执行以下命令，运行试验：
$ python3 maze_experiment.py -m medium -g 150
最后，成功的迷宫求解器控制器ANN的基因组配置打印如下：
****** Running generation 218 ******
Maze solved in 376 steps
Population's average fitness: 0.26595 stdev: 0.27894
Best fitness: 1.00000 - size: (2, 5) - species 14 - id 53049
Best individual in generation 218 meets fitness threshold - complexity: (2, 5)
Best genome:
Key: 53049
Fitness: 1.0
Nodes:
0 DefaultNodeGene(key=0, bias=0.19570685490413814, response=1.0, activation=sigmoid, aggregation=sum)
1 DefaultNodeGene(key=1, bias=-1.6913043338626579, response=1.0, activation=sigmoid, aggregation=sum)
Connections:
DefaultConnectionGene(key=(-9, 1), weight=-0.2651164185249115, enabled=True)
DefaultConnectionGene(key=(-8, 1), weight=0.028955000178012114, enabled=True)
DefaultConnectionGene(key=(-7, 1), weight=2.919475204266212, enabled=True)
DefaultConnectionGene(key=(-3, 0), weight=2.9321519677683705, enabled=True)
DefaultConnectionGene(key=(-1, 0), weight=-11.647747360002768, enabled=True)
DefaultConnectionGene(key=(1, 1), weight=-0.6128097813804334, enabled=False)
SUCCESS: The stable maze solver controller was found!!!
在控制台输出中，可以看到成功的迷宫求解器控制器在演化过程中找到，下图显示了控制器ANN的最终配置：
下图显示了历代求解器智能体的适应度得分：
下图显示了种群的物种分布：
智能体记录可视化
可视化各种物种在进化过程中的表现，可以使用以下命令执行：
$ python3 visualize.py -m medium -r out/maze_objective/medium/data.pickle --width 300 --height 150
该命令加载智能体适应度评估的记录，该记录存储在data.pickle文件中。此后，它在迷宫求解模拟结束时在迷宫上绘制智能体的最终位置。每个智能体的最终位置均以彩色圆圈表示。圆圈的颜色编码特定智能体所属的种类。进化过程中产生的每个物种都有唯一的颜色代码。下图显示了此可视化的结果：
为了使可视化更加有用，引入了适应度阈值以筛选出性能最高的物种。顶部子图显示了属于冠军物种的求解器智能体程序的最终位置（适应性得分高于0.8）。
- agent.py
import pickle
class Agent:
"""
This is the maze navigating agent
"""
def init(self, location, heading=0, speed=0, angular_vel=0, radius=8.0, range_finder_range=100.0):
"""
Creates new Agent with specified parameters.
Arguments:
location: The agent initial position within maze
heading: The heading direction in degrees.
speed: The linear velocity of the agent.
angular_vel: The angular velocity of the agent.
radius: The agent's body radius.
range_finder_range: The maximal detection range for range finder sensors.
"""
self.heading = heading
self.speed = speed
self.angular_vel = angular_vel
self.radius = radius
self.range_finder_range = range_finder_range
self.location = location
defining the range finder sensors
self.range_finder_angles = [-90.0, -45.0, 0.0, 45.0, 90.0, -180.0]
defining the radar sensors
self.radar_angles = [(315.0, 405.0), (45.0, 135.0), (135.0, 225.0), (225.0, 315.0)]
the list to hold range finders activations
self.range_finders = [None] * len(self.range_finder_angles)
the list to hold pie-slice radar activations
self.radar = [None] * len(self.radar_angles)
class AgentRecord:
"""
The class to hold results of maze navigation simulation for specific
solver agent. It provides all statistics about the agent at the end
of navigation run.
"""
def init(self, generation, agent_id):
"""
Creates new record for specific agent at the specific generation
of the evolutionary process.
"""
self.generation = generation
self.agent_id = agent_id
initialize agent's properties
self.x = -1
self.y = -1
self.fitness = -1
The flag to indicate whether this agent was able to find maze exit
self.hit_exit = False
The ID of species this agent belongs to
self.species_id = -1
The age of agent's species at the time of recording
self.species_age = -1
class AgentRecordStore:
"""
The class to control agents record store.
"""
def init(self):
"""
Creates new instance.
"""
self.records = []
def add_record(self, record):
"""
The function to add specified record to this store.
Arguments:
record: The record to be added.
"""
self.records.append(record)
def load(self, file):
"""
The function to load records list from the specied file into this class.
Arguments:
file: The path to the file to read agents records from.
"""
with open(file, 'rb') as dump_file:
self.records = pickle.load(dump_file)
def dump(self, file):
"""
The function to dump records list to the specified file from this class.
Arguments:
file: The path to the file to hold data dump.
"""
with open(file, 'wb') as dump_file:
pickle.dump(self.records, dump_file)
- geometry
import math
def deg_to_rad(degrees):
"""
The function to convert degrees to radians.
Arguments:
degrees: The angle in degrees to be converted.
Returns:
The degrees converted to radians.
"""
return degrees / 180.0 * math.pi
def read_point(str):
"""
The function to read Point from specified string. The point
coordinates are in order (x, y) and delimited by space.
Arguments:
str: The string encoding Point coorinates.
Returns:
The Point with coordinates parsed from provided string.
"""
coords = str.split(' ')
assert len(coords) == 2
return Point(float(coords[0]), float(coords[1]))
def read_line(str):
"""
The function to read line segment from provided string. The coordinates
of line end points are in order: x1, y1, x2, y2 and delimited by spaces.
Arguments:
str: The string to read line coordinates from.
Returns:
The parsed line segment.
"""
coords = str.split(' ')
assert len(coords) == 4
a = Point(float(coords[0]), float(coords[1]))
b = Point(float(coords[2]), float(coords[3]))
return Line(a, b)
class Point:
"""
The basic class describing point in the two dimensional Cartesian coordinate
system.
"""
def init(self, x, y):
"""
Creates new point at specified coordinates
"""
self.x = x
self.y = y
def angle(self):
"""
The function to determine angle in degrees of vector drawn from the
center of coordinates to this point. The angle values is in range
from 0 to 360 degrees in anticlockwise direction.
"""
ang = math.atan2(self.y, self.x) / math.pi * 180.0
if (ang < 0.0):
the lower quadrants (3 or 4)
return ang + 360
return ang
def rotate(self, angle, point):
"""
The function to rotate this point around another point with given
angle in degrees.
Arguments:
angle: The rotation angle (degrees)
point: The point - center of rotation
"""
rad = deg_to_rad(angle)
translate to have another point at the center of coordinates
self.x -= point.x
self.y -= point.y
rotate
ox, oy = self.x, self.y
self.x = math.cos(rad) * ox - math.sin(rad) * oy
self.y = math.sin(rad) * ox - math.cos(rad) * oy
restore
self.x += point.x
self.y += point.y
def distance(self, point):
"""
The function to caclulate Euclidean distance between this and given point.
Arguments:
point: The another point
Returns:
The Euclidean distance between this and given point.
"""
dx = self.x - point.x
dy = self.y - point.y
return math.sqrt(dxdx + dydy)
def str(self):
"""
Returns the nicely formatted string representation of this point.
"""
return "Point (%.1f, %.1f)" % (self.x, self.y)
class Line:
"""
The simple line segment between two points. Used to represent maze wals.
"""
def init(self, a, b):
"""
Creates new line segment between two points.
Arguments:
a, b: The end points of the line
"""
self.a = a
self.b = b
def midpoint(self):
"""
The function to find midpoint of this line segment.
Returns:
The midpoint of this line segment.
"""
x = (self.a.x + self.b.x) / 2.0
y = (self.a.y + self.b.y) / 2.0
return Point(x, y)
def intersection(self, line):
"""
The function to find intersection between this line and the given one.
Arguments:
line: The line to test intersection against.
Returns:
The tuple with the first value indicating if intersection was found (True/False)
and the second value holding the intersection Point or None
"""
A, B, C, D = self.a, self.b, line.a, line.b
rTop = (A.y - C.y) * (D.x - C.x) - (A.x - C.x) * (D.y - C.y)
rBot = (B.x - A.x) * (D.y - C.y) - (B.y - A.y) * (D.x - C.x)
sTop = (A.y - C.y) * (B.x - A.x) - (A.x - C.x) * (B.y - A.y)
sBot = (B.x - A.x) * (D.y - C.y) - (B.y - A.y) * (D.x - C.x)
if rBot == 0 or sBot == 0:
lines are parallel
return False, None
r = rTop / rBot
s = sTop / sBot
if r > 0 and r < 1 and s > 0 and s < 1:
x = A.x + r * (B.x - A.x)
y = A.y + r * (B.y - A.y)
return True, Point(x, y)
return False, None
def distance(self, p):
"""
The function to estimate distance to the given point from this line.
Arguments:
p: The point to find distance to.
Returns:
The distance between given point and this line.
"""
utop = (p.x - self.a.x) * (self.b.x - self.a.x) + (p.y - self.a.y) * (self.b.y - self.a.y)
ubot = self.a.distance(self.b)
ubot *= ubot
if ubot == 0.0:
return 0.0
u = utop / ubot
if u < 0 or u > 1:
d1 = self.a.distance(p)
d2 = self.b.distance(p)
if d1 < d2:
return d1
return d2
x = self.a.x + u * (self.b.x - self.a.x)
y = self.a.y + u * (self.b.y - self.a.y)
point = Point(x, y)
return point.distance(p)
def length(self):
"""
The function to calculate the length of this line segment.
Returns:
The length of this line segment as distance between its endpoints.
"""
return self.a.distance(self.b)
def str(self):
"""
Returns the nicely formatted string representation of this line.
"""
return "Line (%.1f, %.1f) -> (%.1f, %.1f)" % (self.a.x, self.a.y, self.b.x, self.b.y)
- maze_config.ini
[NEAT]
fitness_criterion = max
fitness_threshold = 1.0
pop_size = 250
reset_on_extinction = False
[DefaultGenome]
node activation options
activation_default = sigmoid
activation_mutate_rate = 0.0
activation_options = sigmoid
node aggregation options
aggregation_default = sum
aggregation_mutate_rate = 0.0
aggregation_options = sum
node bias options
bias_init_mean = 0.0
bias_init_stdev = 1.0
bias_max_value = 30.0
bias_min_value = -30.0
bias_mutate_power = 0.5
bias_mutate_rate = 0.7
bias_replace_rate = 0.1
genome compatibility options
compatibility_disjoint_coefficient = 1.1
compatibility_weight_coefficient = 0.5
connection add/remove rates
conn_add_prob = 0.5
conn_delete_prob = 0.5
connection enable options
enabled_default = True
enabled_mutate_rate = 0.01
feed_forward = False
initial_connection = partial_direct 0.5
node add/remove rates
node_add_prob = 0.1
node_delete_prob = 0.1
network parameters
num_hidden = 1
num_inputs = 10
num_outputs = 2
node response options
response_init_mean = 1.0
response_init_stdev = 0.0
response_max_value = 30.0
response_min_value = -30.0
response_mutate_power = 0.0
response_mutate_rate = 0.0
response_replace_rate = 0.0
connection weight options
weight_init_mean = 0.0
weight_init_stdev = 1.0
weight_max_value = 30
weight_min_value = -30
weight_mutate_power = 0.5
weight_mutate_rate = 0.8
weight_replace_rate = 0.1
[DefaultSpeciesSet]
compatibility_threshold = 3.0
[DefaultStagnation]
species_fitness_func = max
max_stagnation = 20
species_elitism = 1
[DefaultReproduction]
elitism = 2
survival_threshold = 0.1
min_species_size = 2
- maze_environment.py
import math
import agent
import geometry
The maximal allowed speend for the maze solver agent
MAX_AGENT_SPEED = 3.0
class MazeEnvironment:
"""
This class encapsulates the maze simulation environments.
"""
def init(self, agent, walls, exit_point, exit_range=5.0):
"""
Creates new maze environment with specified walls and exit point.
Arguments:
agent: The maze navigating agent
walls: The maze walls
exit_point: The maze exit point
exit_range: The range arround exit point marking exit area
"""
self.walls = walls
self.exit_point = exit_point
self.exit_range = exit_range
The maze navigating agent
self.agent = agent
The flag to indicate if exit was found
self.exit_found = False
The initial distance of agent from exit
self.initial_distance = self.agent_distance_to_exit()
Update sendors
self.update_rangefinder_sensors()
self.update_radars()
def agent_distance_to_exit(self):
"""
The function to estimate distance from maze solver agent to
the maze exit.
Returns:
The distance from maze solver agent to the maze exit.
"""
return self.agent.location.distance(self.exit_point)
def test_wall_collision(self, loc):
"""
The function to test if agent at specified location collides
with any of the maze walls.
Arguments:
loc: The new agent location to test for collision.
Returns:
The True if agent at new location will collide with any of
the maze walls.
"""
for w in self.walls:
if w.distance(loc) < self.agent.radius:
return True
return False
def create_net_inputs(self):
"""
The function to create the ANN input values from the simulation environment.
Returns:
The list of ANN inputs consist of values get from solver agent sensors.
"""
inputs = []
the range finders
for ri in self.agent.range_finders:
inputs.append(ri)
The radar sensors
for rs in self.agent.radar:
inputs.append(rs)
return inputs
def apply_control_signals(self, control_signals):
"""
The function to apply control signals received from control ANN to the
maze solver agent.
Arguments:
control_signals: The contral signals received from the control ANN
"""
self.agent.angular_vel += (control_signals[0] - 0.5)
self.agent.speed += (control_signals[1] - 0.5)
contrain the speed & angular velocity
if self.agent.speed > MAX_AGENT_SPEED:
self.agent.speed = MAX_AGENT_SPEED
if self.agent.speed < -MAX_AGENT_SPEED:
self.agent.speed = -MAX_AGENT_SPEED
if self.agent.angular_vel > MAX_AGENT_SPEED:
self.agent.angular_vel = MAX_AGENT_SPEED
if self.agent.angular_vel < -MAX_AGENT_SPEED:
self.agent.angular_vel = -MAX_AGENT_SPEED
def update_rangefinder_sensors(self):
"""
The function to update the agent range finder sensors.
"""
for i,angle in enumerate(self.agent.range_finder_angles):
rad = geometry.deg_to_rad(angle)
project a point from agent location outwards
projection_point = geometry.Point(
x = self.agent.location.x + math.cos(rad) * \
self.agent.range_finder_range,
y = self.agent.location.y + math.sin(rad) * \
self.agent.range_finder_range
)
rotate the projection point by the agent's heading
anglet to align it with heading dirction
projection_point.rotate(
self.agent.heading,self.agent.location
)
create the line segment from the agent location to
the projected point
projection_line = geometry.Line(
a = self.agent.location,
b = projection_point
)
set range to maximum detection range
min_range = self.agent.range_finder_range
new test against maze walls to see if projection line
hits any wall and find the closest hit
for wall in self.walls:
found, intersection = wall.intersection(projection_line)
if found:
found_range = intersection.distance(self.agent.location)
we are interested in the closest hit
if found_range < min_range:
min_range = found_range
update sensor value
self.agent.range_finders[i] = min_range
def update_radars(self):
"""
The function to update the agent radar sensors.
"""
target = geometry.Point(self.exit_point.x, self.exit_point.y)
rotate target with respect to the agent's heading to align it with
heading direction
target.rotate(self.agent.heading, self.agent.location)
translate with respect to the agent's location
target.x -= self.agent.location.x
target.y -= self.agent.location.y
the angle between maze eixt point and the agent's heading direction
angle = target.angle()
find the appropriate radar sensor to be fired
for i, r_angles in enumerate(self.agent.radar_angles):
self.agent.radar[i] = 0.0 # reset specific radar
if (angle >= r_angles[0] and angle < r_angles[1]) or \
(angle + 360 >= r_angles[0] and angle + 360 < r_angles[1]):
fire the radar
self.agent.radar[i] = 1.0
def update(self,control_signals):
"""
The function to update solver agent position within maze.
After agent position updated it will be checked to find out if maze
was reached after that.
Arguments:
control_signals: The control signals received from the control ANN
Returns:
The True if maze exit was found after update or exit was
already found in previous simulation cycles.
"""
if self.exit_found:
return True # Maze exit already found
Apply control signals
self.apply_control_signals(control_signals)
get X and Y velocity components
vx = math.cos(geometry.deg_to_rad(self.agent.heading)) * \
self.agent.speed
vy = math.sin(geometry.deg_to_rad(self.agent.heading)) * \
self.agent.speed
update current agent's heading (we consider the simulation time
step size equal to 1s and the angular velocity as per second)
self.agent.heading += self.agent.angular_vel
Enforce angular velocity bounds by wrapping
if self.agent.heading > 360:
self.agent.heading -= 360
elif self.agent.heading < 0:
self.agent.heading += 360
find the next location of the agent
new_loc = geometry.Point(
x = self.agent.location.x + vx,
y = self.agent.location.y + vy
)
if not self.test_wall_collision(new_loc):
self.agent.location = new_loc
update agent's sensors
self.update_rangefinder_sensors()
self.update_radars()
check if agent reached exit point
distance = self.agent_distance_to_exit()
self.exit_found = (distance < self.exit_range)
return self.exit_found
def str(self):
"""
Returns the nicely formatted string representation of this environment.
"""
string = "MAZE\nAgent at: (%.1f, %.1f)" % (self.agent.location.x,self.agent.location.y)
string += "\nExit at: (%.1f, %.1f), exit range: %.1f" % (self.exit_point.x,self.exit_found.y,self.exit_range)
string += "\nWalls [%d]" % len(self.walls)
for w in self.walls:
string += "\n\t%s" % w
return string
def read_environment(file_path):
"""
The function to read maze environment configuration
from provided file.
Arguments:
file_path: The path to the file to read maze configuration from.
Returns:
The initialized environment.
"""
num_lines, index = -1, 0
walls = []
maze_agent, maze_exit = None, None
with open(file_path, 'r') as f:
for line in f.readlines():
line = line.strip()
if len(line) == 0:
skip empty lines
continue
if index == 0:
read the number of line segments
num_lines = int(line)
elif index == 1:
read the agent's position
loc = geometry.read_point(line)
maze_agent = agent.Agent(location=loc)
elif index == 2:
read the agent's initial heading
maze_agent.heading = float(line)
elif index == 3:
read the maze exit location
maze_exit = geometry.read_point(line)
else:
read the walls
wall = geometry.read_line(line)
walls.append(wall)
increment cursor
index += 1
assert len(walls) == num_lines
print("Maze environment configured successfully from the file: %s" % file_path)
create and return the maze environment
return MazeEnvironment(agent=maze_agent,walls=walls,exit_point=maze_exit)
def maze_simulation_evaluate(env,net,time_steps):
"""
The function to evaluate maze simulation for specific environment
and controll ANN provided. The results will be saved into provided
agent record holder.
Arguments:
env: The maze configuration environment.
net: The maze solver agent's control ANN.
time_steps: The number of time steps for maze simulation.
"""
for i in range(time_steps):
if maze_simulation_step(env, net):
print("Maze solved in %d steps" % (i + 1))
return 1.0
Calculate the fitness score based on distance drom exit
fitness = env.agent_distance_to_exit()
Normalize fitness score to range (0,1]
fitness = (env.initial_distance - fitness) / env.initial_distance
if fitness <= 0.01:
fitness = 0.01
return fitness
def maze_simulation_step(env,net):
"""
The function to perform one step of maze simulation.
Arguments:
env: The maze configuration environment.
net: The maze solver agent's control ANN
Returns:
The True if maze agent solved the maze.
"""
create inputs from the current state of the environment
inputs = env.create_net_inputs()
load inputs into controll ANN and get results
output = net.activate(inputs)
apply control signal to the environment and update
return env.update(output)
- maze_experiment.py
import os
import shutil
import math
import random
import time
import copy
import argparse
import neat
import visualize
import utils
import maze_environment as maze
import agent
The current working directory
local_dir = os.path.dirname(file)
The directory to store outputs
out_dir = os.path.join(local_dir,'out')
out_dir = os.path.join(out_dir,'maze_objective')
class MazeSimulationTrial:
"""
The class to hold maze simulator execution parameters and results.
"""
def init(self,maze_env,population):
"""
Creates new instance and initialize fileds.
Arguments:
maze_env: The maze environment as loaded from configuration file.
population: The population for this trial run
"""
The initial maze simulation environment
self.orig_maze_environment = maze_env
The record store for evaluated maze solver agents
self.record_store = agent.AgentRecordStore()
The NEAT population object
self.population = population
trialSim = None
def eval_fitness(genome_id,genome,config,time_steps=400):
"""
Evaluates fitness of the provided genome.
Arguments:
genome_id: The ID of genome.
genome: The genome to evaluate.
config: The NEAT configuration holder.
time_steps: The number of time steps to execute for maze solver
simulation.
Returns:
The phenotype fitness score in range (0,1]
"""
run the simulaion
maze_env = copy.deepcopy(trialSim.orig_maze_environment)
control_net = neat.nn.FeedForwardNetwork.create(genome, config)
fitness = maze.maze_simulation_evaluate(
env=maze_env,net=control_net,time_steps=time_steps
)
Store simulation results into the agent record
record = agent.AgentRecord(
generation=trialSim.population.generation,
agent_id=genome_id
)
record.fitness = fitness
record.x = maze_env.agent.location.x
record.y = maze_env.agent.location.y
record.hit_exit = maze_env.exit_found
record.species_id = trialSim.population.species.get_species_id(genome_id)
record.species_age = record.generation - \
trialSim.population.species.get_species(genome_id).created
add record to the store
trialSim.record_store.add_record(record)
return fitness
def eval_genomes(genomes,config):
"""
The function to evaluate the fitness of each genome in
the genomes list.
Arguments:
genomes: The list of genomes from population in the
current generation
config: The configuration settings with algorithm
hyper-parameters
"""
for genome_id,genome in genomes:
genome.fitness = eval_fitness(genome_id, genome, config)
def run_experiment(
config_file,maze_env,trial_out_dir,args=None,n_generations=100,silent=False):
"""
The function to run the experiment against hyper-parameters defined in the
provided configuration file.
The winner genome will be rendered as a graph as well as the import statictics
of neuroevolution process execution.
Arguments:
config_file: The path to the file with experiment configuration
maze_env: The maze environment to use in simulation.
trial_out_dir: The directory to store outputs for this trial
n_generations: The number of generations to execute.
silent: If True than no intermediary outputs will be presented
until solution is found.
args: The command line argument holder.
"""
seed = 1559231616
random.seed(seed)
Load configuration
config = neat.Config(neat.DefaultGenome,neat.DefaultReproduction,
neat.DefaultSpeciesSet,neat.DefaultStagnation,config_file)
Create the population, which is the top-level object for a NEAT run.
p = neat.Population(config)
create the trial simulation
global trialSim
trialSim = MazeSimulationTrial(maze_env=maze_env, population=p)
Add a stdout reporter to show progress in the terminal.
p.add_reporter(neat.StdOutReporter(True))
stats = neat.StatisticsReporter()
p.add_reporter(stats)
p.add_reporter(neat.Checkpointer(
5,filename_prefix='%s/maze-neat-checkpoint-' % trial_out_dir))
Run for up to N generations.
start_time = time.time()
best_genome = p.run(eval_genomes, n=n_generations)
elapsed_time = time.time() - start_time
Display the best genome among generations.
print('\nBest genome:\n%s' % (best_genome))
solution_found = (best_genome.fitness >= config.fitness_threshold)
if solution_found:
print("SUCCESS: The stable maze solver controller was found!!!")
else:
print("FAILURE: Failed to find the stable maze solver controller!!!")
write the record store data
rs_file = os.path.join(trial_out_dir,'data.pickle')
trialSim.record_store.dump(rs_file)
print("Record store file: %s" % rs_file)
#print("Random seed:",seed)
print("Trial elapsed time: %.3f sec" % (elapsed_time))
Visualize the experiment results
if not silent or solution_found:
node_names = {
-1:'RF_R',-2:'RF_FR',-3:'RF_F',-4:'RF_FL',
-5:'RF_F',-6:'RF_B',-7:'RAD_F',-8:'RAD_F',
-9:'RAD_B',-10:'RAD_R',0:'ANG_VEL',1:'VEL'
}
visualize.draw_net(config,best_genome,True,node_names=node_names,directory=trial_out_dir,fmt='svg')
if args is None:
visualize.draw_maze_records(maze_env,trialSim.record_store.records,view=True)
else:
visualize.draw_maze_records(
maze_env,trialSim.record_store.records,
view=True,width=args.width,height=args.height,
filename=os.path.join(trial_out_dir,'maze_records.svg')
)
visualize.plot_stats(
stats,ylog=False,view=True,
filename=os.path.join(trial_out_dir,'avg_fitness.svg')
)
visualize.plot_species(
stats,view=True,
filename=os.path.join(trial_out_dir,'speciation.svg')
)
return solution_found
if name == 'main':
read command line parameters
parser = argparse.ArgumentParser(description="The maze experiment runner.")
parser.add_argument('-m','--maze',default='medium',help='The maze configuration to use.')
parser.add_argument('-g','--generations',default=500,type=int,help='The number of generations for the evolutionary process')
parser.add_argument('--width',type=int,default=400,help='The width of the records subplot')
parser.add_argument('--height',type=int,default=400,help='The height of the records subplot')
args = parser.parse_args()
if not (args.maze == 'medium' or args.maze == 'hard'):
print("Unsupported maze configuration: %s" % args.maze)
exit(1)
Determine path to configuration file.
config_path = os.path.join(local_dir,'maze_config.ini')
trial_out_dir = os.path.join(out_dir,args.maze)
Clean results of previous run if any or init the output directory
utils.clear_output(trial_out_dir)
run the experiment
maze_env_config = os.path.join(local_dir, '%s_maze.txt' % args.maze)
maze_env = maze.read_environment(maze_env_config)
visualize.draw_maze_records(maze_env,None,view=True)
print("Startting the %s maze experiment" % args.maze)
run_experiment(
config_file=config_path,maze_env=maze_env,
trial_out_dir=trial_out_dir,
n_generations=args.generations,
args=args
)
- medium_maze.txt
11
30 22
0
270 100
5 5 295 5
295 5 295 135
295 135 5 135
5 135 5 5
241 135 58 65
114 5 73 42
130 91 107 46
196 5 139 51
219 125 182 63
267 5 214 63
271 135 237 88
- utils.py
import os
import shutil
def clear_output(out_dir):
"""
Function to clear output directory.
Arguments:
out_dir: The directory to be clear
"""
if os.path.isdir(out_dir):
remove files from previous run
shutil.rmtree(out_dir)
create the output directory
os.makedirs(out_dir,exist_ok=False)
- visualize.py
from future import print_function
import copy
import warnings
import random
import argparse
import os
import graphviz
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
import matplotlib.patches as mpatches
import numpy as np
import geometry
import agent
import maze_environment as maze
def plot_stats(statistics,ylog=False,view=False,filename='avg_fitness.svg'):
"""
Plots the population's average and best fitness.
"""
if plt is None:
warnings.warn("This display is not available due to a missing optional dependency (matplotlib)")
return
generation = range(len(statistics.most_fit_genomes))
best_fitness = [c.fitness for c in statistics.most_fit_genomes]
avg_fitness = np.array(statistics.get_fitness_mean())
stdev_fitness = np.array(statistics.get_fitness_stdev())
plt.plot(generation, avg_fitness, 'b-', label='average')
plt.plot(generation, avg_fitness - stdev_fitness, 'g-.', label='-1 sd')
plt.plot(generation, avg_fitness + stdev_fitness, 'g-', label='+1 sd')
plt.plot(generation, best_fitness, 'r-', label='best')
plt.title("Population's average and best fitness")
plt.xlabel("Generations")
plt.ylabel("Fitness")
plt.grid()
plt.legend(loc="best")
if ylog:
plt.gca().set_yscale('symlog')
plt.savefig(filename)
if view:
plt.show()
plt.close()
def plot_species(statistics,view=False,filename='speciation.svg'):
"""
Visualizes speciation throughout evolution.
"""
if plt is None:
warnings.warn("This display is not available due to a missing optional dependency (matplotlib)")
return
species_sizes = statistics.get_species_sizes()
num_generations = len(species_sizes)
curves = np.array(species_sizes).T
fig, ax = plt.subplots()
ax.stackplot(range(num_generations), *curves)
做了那么多年开发，自学了很多门编程语言，我很明白学习资源对于学一门新语言的重要性，这些年也收藏了不少的Python干货，对我来说这些东西确实已经用不到了，但对于准备自学Python的人来说，或许它就是一个宝藏，可以给你省去很多的时间和精力。
别在网上瞎学了，我最近也做了一些资源的更新，只要你是我的粉丝，这期福利你都可拿走。
我先来介绍一下这些东西怎么用，文末抱走。
（1）Python所有方向的学习路线（新版）
这是我花了几天的时间去把Python所有方向的技术点做的整理，形成各个领域的知识点汇总，它的用处就在于，你可以按照上面的知识点去找对应的学习资源，保证自己学得较为全面。
最近我才对这些路线做了一下新的更新，知识体系更全面了。
（2）Python学习视频
包含了Python入门、爬虫、数据分析和web开发的学习视频，总共100多个，虽然没有那么全面，但是对于入门来说是没问题的，学完这些之后，你可以按照我上面的学习路线去网上找其他的知识资源进行进阶。
（3）100多个练手项目
我们在看视频学习的时候，不能光动眼动脑不动手，比较科学的学习方法是在理解之后运用它们，这时候练手项目就很适合了，只是里面的项目比较多，水平也是参差不齐，大家可以挑自己能做的项目去练练。
（4）200多本电子书
这些年我也收藏了很多电子书，大概200多本，有时候带实体书不方便的话，我就会去打开电子书看看，书籍可不一定比视频教程差，尤其是权威的技术书籍。
基本上主流的和经典的都有，这里我就不放图了，版权问题，个人看看是没有问题的。
（5）Python知识点汇总
知识点汇总有点像学习路线，但与学习路线不同的点就在于，知识点汇总更为细致，里面包含了对具体知识点的简单说明，而我们的学习路线则更为抽象和简单，只是为了方便大家只是某个领域你应该学习哪些技术栈。
（6）其他资料
还有其他的一些东西，比如说我自己出的Python入门图文类教程，没有电脑的时候用手机也可以学习知识，学会了理论之后再去敲代码实践验证，还有Python中文版的库资料、MySQL和HTML标签大全等等，这些都是可以送给粉丝们的东西。
这些都不是什么非常值钱的东西，但对于没有资源或者资源不是很好的学习者来说确实很不错，你要是用得到的话都可以直接抱走，关注过我的人都知道，这些都是可以拿到的。
