PaddlePaddle实现线性回归

在本次实验中我们将使用PaddlePaddle来搭建一个简单的线性回归模型，并利用这一模型预测你的储蓄（在某地区）可以购买多大面积的房子。并且在学习模型搭建的过程中，了解到机器学习的若干重要概念，掌握一个机器学习预测的基本流程。

线性回归的基本概念

线性回归是机器学习中最简单也是最重要的模型之一，其模型建立遵循此流程：获取数据、数据预处理、训练模型、应用模型。

回归模型可以理解为：存在一个点集，用一条曲线去拟合它分布的过程。如果拟合曲线是一条直线，则称为线性回归。如果是一条二次曲线，则被称为二次回归。线性回归是回归模型中最简单的一种。

在线性回归中有几个基本的概念需要掌握：

• 假设函数（Hypothesis Function）
• 损失函数（Loss Function）
• 优化算法（Optimization Algorithm）

假设函数：

假设函数是指，用数学的方法描述自变量和因变量之间的关系，它们之间可以是一个线性函数或非线性函数。 在本次线性回顾模型中，我们的假设函数为 Y^=aX1+b\hat{Y}= aX_1+b​Y​^​​=aX​1​​+b ，其中，Y^\hat{Y}​Y​^​​表示模型的预测结果（预测房价），用来和真实的Y区分。模型要学习的参数即：a,b。

损失函数：

损失函数是指，用数学的方法衡量假设函数预测结果与真实值之间的误差。这个差距越小预测越准确，而算法的任务就是使这个差距越来越小。

建立模型后，我们需要给模型一个优化目标，使得学到的参数能够让预测值Y^\hat{Y}​Y​^​​尽可能地接近真实值Y。输入任意一个数据样本的目标值yiy_iy​i​​和模型给出的预测值\hat{Y_i，损失函数输出一个非负的实值。这个实值通常用来反映模型误差的大小。

对于线性模型来讲，最常用的损失函数就是均方误差（Mean Squared Error， MSE）。

MSE=1n∑i=1n(Yi^−Yi)2MSE=\frac{1}{n}\sum_{i=1}^{n}(\hat{Y_i}-Y_i)^2MSE=​n​​1​​∑​i=1​n​​(​Y​i​​​^​​−Y​i​​)​2​​

即对于一个大小为n的测试集，MSE是n个数据预测结果误差平方的均值。

优化算法：

在模型训练中优化算法也是至关重要的，它决定了一个模型的精度和运算速度。本章的线性回归实例中主要使用了梯度下降法进行优化。

现在，让我们正式进入实验吧！

 

首先导入必要的包，分别是：

paddle.fluid--->PaddlePaddle深度学习框架

numpy---------->python基本库，用于科学计算

os------------------>python的模块，可使用该模块对操作系统进行操作

matplotlib----->python绘图库，可方便绘制折线图、散点图等图形

 

import paddle.fluid as fluid
import paddle
import numpy as np
import os
import matplotlib.pyplot as plt

 

 

Step1：准备数据。

（1）uci-housing数据集介绍

数据集共506行,每行14列。前13列用来描述房屋的各种信息，最后一列为该类房屋价格中位数。

PaddlePaddle提供了读取uci_housing训练集和测试集的接口，分别为paddle.dataset.uci_housing.train()和paddle.dataset.uci_housing.test()。

(2)train_reader和test_reader

paddle.reader.shuffle()表示每次缓存BUF_SIZE个数据项，并进行打乱

paddle.batch()表示每BATCH_SIZE组成一个batch

BUF_SIZE=500
BATCH_SIZE=20

#用于训练的数据提供器，每次从缓存中随机读取批次大小的数据
train_reader = paddle.batch(
    paddle.reader.shuffle(paddle.dataset.uci_housing.train(), 
                          buf_size=BUF_SIZE),                    
    batch_size=BATCH_SIZE)   
#用于测试的数据提供器，每次从缓存中随机读取批次大小的数据
test_reader = paddle.batch(
    paddle.reader.shuffle(paddle.dataset.uci_housing.test(),
                          buf_size=BUF_SIZE),
    batch_size=BATCH_SIZE)  

 

(3)打印看下数据是什么样的？PaddlePaddle接口提供的数据已经经过归一化等处理

(array([-0.02964322, -0.11363636, 0.39417967, -0.06916996, 0.14260276, -0.10109875, 0.30715859, -0.13176829, -0.24127857, 0.05489093, 0.29196451, -0.2368098 , 0.12850267]), array([15.6])),

#用于打印，查看uci_housing数据
train_data=paddle.dataset.uci_housing.train();
sampledata=next(train_data())
print(sampledata)

 

Step2:网络配置

（1）网络搭建：对于线性回归来讲，它就是一个从输入到输出的简单的全连接层。

对于波士顿房价数据集，假设属性和房价之间的关系可以被属性间的线性组合描述。

 

#定义张量变量x，表示13维的特征值
x = fluid.layers.data(name='x', shape=[13], dtype='float32')
#定义张量y,表示目标值
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
#定义一个简单的线性网络,连接输入和输出的全连接层
#input:输入tensor;
#size:该层输出单元的数目
#act:激活函数
y_predict=fluid.layers.fc(input=x,size=1,act=None)

 

(2)定义损失函数

此处使用均方差损失函数。

square_error_cost(input,lable):接受输入预测值和目标值，并返回方差估计,即为（y-y_predict）的平方

cost = fluid.layers.square_error_cost(input=y_predict, label=y) #求一个batch的损失值
avg_cost = fluid.layers.mean(cost)                              #对损失值求平均值

 

(3)定义优化函数

此处使用的是随机梯度下降。

test_program = fluid.default_main_program().clone(for_test=True)
optimizer = fluid.optimizer.SGDOptimizer(learning_rate=0.001)
opts = optimizer.minimize(avg_cost)

在上述模型配置完毕后，得到两个fluid.Program：fluid.default_startup_program() 与fluid.default_main_program() 配置完毕了。

参数初始化操作会被写入fluid.default_startup_program()

fluid.default_main_program()用于获取默认或全局main program(主程序)。该主程序用于训练和测试模型。fluid.layers 中的所有layer函数可以向 default_main_program 中添加算子和变量。default_main_program 是fluid的许多编程接口（API）的Program参数的缺省值。例如,当用户program没有传入的时候， Executor.run() 会默认执行 default_main_program 。

 

Step3.模型训练 and Step4.模型评估

 

（1）创建Executor

首先定义运算场所 fluid.CPUPlace()和 fluid.CUDAPlace(0)分别表示运算场所为CPU和GPU

Executor:接收传入的program，通过run()方法运行program。

use_cuda = False                         #use_cuda为False,表示运算场所为CPU;use_cuda为True,表示运算场所为GPU           
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)              #创建一个Executor实例exe
exe.run(fluid.default_startup_program()) #Executor的run()方法执行startup_program(),进行参数初始化

（2）定义输入数据维度

DataFeeder负责将数据提供器（train_reader,test_reader）返回的数据转成一种特殊的数据结构，使其可以输入到Executor中。

feed_list设置向模型输入的向变量表或者变量表名

# 定义输入数据维度
feeder = fluid.DataFeeder(place=place, feed_list=[x, y])#feed_list:向模型输入的变量表或变量表名

（3）定义绘制训练过程的损失值变化趋势的方法draw_train_process

 

iter=0;
iters=[]
train_costs=[]

def draw_train_process(iters,train_costs):
    title="training cost"
    plt.title(title, fontsize=24)
    plt.xlabel("iter", fontsize=14)
    plt.ylabel("cost", fontsize=14)
    plt.plot(iters, train_costs,color='red',label='training cost') 
    plt.grid()
    plt.show()

（4）训练并保存模型

Executor接收传入的program,并根据feed map(输入映射表)和fetch_list(结果获取表) 向program中添加feed operators(数据输入算子)和fetch operators（结果获取算子)。 feed map为该program提供输入数据。fetch_list提供program训练结束后用户预期的变量。

使用feed方式送入训练数据，先将reader数据转换为PaddlePaddle可识别的Tensor数据，传入执行器进行训练。

注：enumerate() 函数用于将一个可遍历的数据对象(如列表、元组或字符串)组合为一个索引序列，同时列出数据和数据下标，

EPOCH_NUM=50
model_save_dir = "/home/aistudio/work/fit_a_line.inference.model"

for pass_id in range(EPOCH_NUM):                                  #训练EPOCH_NUM轮
    # 开始训练并输出最后一个batch的损失值
    train_cost = 0
    for batch_id, data in enumerate(train_reader()):              #遍历train_reader迭代器
        train_cost = exe.run(program=fluid.default_main_program(),#运行主程序
                             feed=feeder.feed(data),              #喂入一个batch的训练数据，根据feed_list和data提供的信息，将输入数据转成一种特殊的数据结构
                             fetch_list=[avg_cost])    
        if batch_id % 40 == 0:
            print("Pass:%d, Cost:%0.5f" % (pass_id, train_cost[0][0]))    #打印最后一个batch的损失值
        iter=iter+BATCH_SIZE
        iters.append(iter)
        train_costs.append(train_cost[0][0])
       
   
    # 开始测试并输出最后一个batch的损失值
    test_cost = 0
    for batch_id, data in enumerate(test_reader()):               #遍历test_reader迭代器
        test_cost= exe.run(program=test_program, #运行测试cheng
                            feed=feeder.feed(data),               #喂入一个batch的测试数据
                            fetch_list=[avg_cost])                #fetch均方误差
    print('Test:%d, Cost:%0.5f' % (pass_id, test_cost[0][0]))     #打印最后一个batch的损失值
    
    #保存模型
    # 如果保存路径不存在就创建
if not os.path.exists(model_save_dir):
    os.makedirs(model_save_dir)
print ('save models to %s' % (model_save_dir))
#保存训练参数到指定路径中，构建一个专门用预测的program
fluid.io.save_inference_model(model_save_dir,   #保存推理model的路径
                                  ['x'],            #推理（inference）需要 feed 的数据
                                  [y_predict],      #保存推理（inference）结果的 Variables
                                  exe)              #exe 保存 inference model
draw_train_process(iters,train_costs)

----------------------------------------

Pass:0, Cost:730.52649
Test:0, Cost:129.28802
Pass:1, Cost:935.00702
Test:1, Cost:146.41402
Pass:2, Cost:561.50110
Test:2, Cost:188.96291
Pass:3, Cost:455.02338
Test:3, Cost:296.16476
Pass:4, Cost:347.46710
Test:4, Cost:122.57037
Pass:5, Cost:480.02325
Test:5, Cost:140.77341
Pass:6, Cost:464.05698
Test:6, Cost:127.89626
Pass:7, Cost:276.11606
Test:7, Cost:233.21486
Pass:8, Cost:337.44760
Test:8, Cost:163.82315
Pass:9, Cost:311.88654
Test:9, Cost:13.98091
Pass:10, Cost:308.88275
Test:10, Cost:170.74649
Pass:11, Cost:340.49243
Test:11, Cost:77.21281
Pass:12, Cost:301.12851
Test:12, Cost:31.04134
Pass:13, Cost:150.75267
Test:13, Cost:2.99113
Pass:14, Cost:174.88126
Test:14, Cost:39.84206
Pass:15, Cost:279.36380
Test:15, Cost:102.89651
Pass:16, Cost:184.16774
Test:16, Cost:208.48296
Pass:17, Cost:252.75090
Test:17, Cost:65.50356
Pass:18, Cost:125.28737
Test:18, Cost:5.14324
Pass:19, Cost:241.18799
Test:19, Cost:43.11307
Pass:20, Cost:333.37201
Test:20, Cost:48.84952
Pass:21, Cost:150.72885
Test:21, Cost:58.22155
Pass:22, Cost:73.52397
Test:22, Cost:113.02930
Pass:23, Cost:189.21335
Test:23, Cost:29.70313
Pass:24, Cost:182.14908
Test:24, Cost:16.74845
Pass:25, Cost:128.77292
Test:25, Cost:16.76190
Pass:26, Cost:117.02783
Test:26, Cost:10.72589
Pass:27, Cost:107.32870
Test:27, Cost:4.64500
Pass:28, Cost:138.55495
Test:28, Cost:6.51828
Pass:29, Cost:48.11888
Test:29, Cost:8.40414
Pass:30, Cost:127.07739
Test:30, Cost:123.49804
Pass:31, Cost:169.20230
Test:31, Cost:5.44257
Pass:32, Cost:88.83828
Test:32, Cost:7.61720
Pass:33, Cost:80.49153
Test:33, Cost:22.00040
Pass:34, Cost:59.16454
Test:34, Cost:46.63321
Pass:35, Cost:161.52925
Test:35, Cost:26.65326
Pass:36, Cost:81.94468
Test:36, Cost:28.30224
Pass:37, Cost:35.22042
Test:37, Cost:3.84092
Pass:38, Cost:72.79510
Test:38, Cost:16.40567
Pass:39, Cost:109.47186
Test:39, Cost:4.38933
Pass:40, Cost:59.62152
Test:40, Cost:0.58020
Pass:41, Cost:52.41791
Test:41, Cost:2.84398
Pass:42, Cost:139.88603
Test:42, Cost:11.51844
Pass:43, Cost:31.33353
Test:43, Cost:12.27122
Pass:44, Cost:33.70327
Test:44, Cost:11.24299
Pass:45, Cost:36.93304
Test:45, Cost:3.56746
Pass:46, Cost:69.01217
Test:46, Cost:12.32192
Pass:47, Cost:20.34635
Test:47, Cost:5.79740
Pass:48, Cost:37.24659
Test:48, Cost:9.30209
Pass:49, Cost:104.55357
Test:49, Cost:12.87949
save models to /home/aistudio/work/fit_a_line.inference.model

-------------------------------------

Step5.模型预测

（1）创建预测用的Executor

infer_exe = fluid.Executor(place)    #创建推测用的executor
inference_scope = fluid.core.Scope() #Scope指定作用域

（2）可视化真实值与预测值方法定义

infer_results=[]
groud_truths=[]

#绘制真实值和预测值对比图
def draw_infer_result(groud_truths,infer_results):
    title='Boston'
    plt.title(title, fontsize=24)
    x = np.arange(1,20) 
    y = x
    plt.plot(x, y)
    plt.xlabel('ground truth', fontsize=14)
    plt.ylabel('infer result', fontsize=14)
    plt.scatter(groud_truths, infer_results,color='green',label='training cost') 
    plt.grid()
    plt.show()

 

(3)开始预测

通过fluid.io.load_inference_model，预测器会从params_dirname中读取已经训练好的模型，来对从未遇见过的数据进行预测。

with fluid.scope_guard(inference_scope):#修改全局/默认作用域（scope）, 运行时中的所有变量都将分配给新的scope。
    #从指定目录中加载 推理model(inference model)
    [inference_program,                             #推理的program
     feed_target_names,                             #需要在推理program中提供数据的变量名称
     fetch_targets] = fluid.io.load_inference_model(#fetch_targets: 推断结果
                                    model_save_dir, #model_save_dir:模型训练路径 
                                    infer_exe)      #infer_exe: 预测用executor
    #获取预测数据
    infer_reader = paddle.batch(paddle.dataset.uci_housing.test(),  #获取uci_housing的测试数据
                          batch_size=200)                           #从测试数据中读取一个大小为200的batch数据
    #从test_reader中分割x
    test_data = next(infer_reader())
    test_x = np.array([data[0] for data in test_data]).astype("float32")
    test_y= np.array([data[1] for data in test_data]).astype("float32")
    results = infer_exe.run(inference_program,                              #预测模型
                            feed={feed_target_names[0]: np.array(test_x)},  #喂入要预测的x值
                            fetch_list=fetch_targets)                       #得到推测结果 
                            
    print("infer results: (House Price)")
    for idx, val in enumerate(results[0]):
        print("%d: %.2f" % (idx, val))
        infer_results.append(val)
    print("ground truth:")
    for idx, val in enumerate(test_y):
        print("%d: %.2f" % (idx, val))
        groud_truths.append(val)
    draw_infer_result(groud_truths,infer_results)

 

---------------------------------------------------------------------------------------------------------

infer results: (House Price)
0: 13.23
1: 13.22
2: 13.23
3: 14.47
4: 13.76
5: 14.04
6: 13.25
7: 13.35
8: 11.67
9: 13.38
10: 11.03
11: 12.38
12: 12.95
13: 12.62
14: 12.13
15: 13.39
16: 14.32
17: 14.26
18: 14.61
19: 13.20
20: 13.78
21: 12.54
22: 14.22
23: 13.43
24: 13.58
25: 13.00
26: 14.03
27: 13.89
28: 14.73
29: 13.76
30: 13.53
31: 13.10
32: 13.08
33: 12.20
34: 12.07
35: 13.84
36: 13.83
37: 14.25
38: 14.40
39: 14.27
40: 13.21
41: 12.75
42: 14.15
43: 14.42
44: 14.38
45: 14.07
46: 13.33
47: 14.48
48: 14.61
49: 14.83
50: 13.20
51: 13.52
52: 13.10
53: 13.30
54: 14.40
55: 14.88
56: 14.38
57: 14.92
58: 15.04
59: 15.23
60: 15.58
61: 15.51
62: 13.54
63: 14.46
64: 15.13
65: 15.67
66: 15.32
67: 15.62
68: 15.71
69: 16.01
70: 14.49
71: 14.15
72: 14.95
73: 13.76
74: 14.71
75: 15.18
76: 16.25
77: 16.42
78: 16.55
79: 16.59
80: 16.13
81: 16.34
82: 15.44
83: 16.08
84: 15.77
85: 15.07
86: 14.48
87: 15.83
88: 16.46
89: 20.68
90: 20.86
91: 20.75
92: 19.60
93: 20.27
94: 20.51
95: 20.05
96: 20.16
97: 21.58
98: 21.32
99: 21.59
100: 21.49
101: 21.30
ground truth:
0: 8.50
1: 5.00
2: 11.90
3: 27.90
4: 17.20
5: 27.50
6: 15.00
7: 17.20
8: 17.90
9: 16.30
10: 7.00
11: 7.20
12: 7.50
13: 10.40
14: 8.80
15: 8.40
16: 16.70
17: 14.20
18: 20.80
19: 13.40
20: 11.70
21: 8.30
22: 10.20
23: 10.90
24: 11.00
25: 9.50
26: 14.50
27: 14.10
28: 16.10
29: 14.30
30: 11.70
31: 13.40
32: 9.60
33: 8.70
34: 8.40
35: 12.80
36: 10.50
37: 17.10
38: 18.40
39: 15.40
40: 10.80
41: 11.80
42: 14.90
43: 12.60
44: 14.10
45: 13.00
46: 13.40
47: 15.20
48: 16.10
49: 17.80
50: 14.90
51: 14.10
52: 12.70
53: 13.50
54: 14.90
55: 20.00
56: 16.40
57: 17.70
58: 19.50
59: 20.20
60: 21.40
61: 19.90
62: 19.00
63: 19.10
64: 19.10
65: 20.10
66: 19.90
67: 19.60
68: 23.20
69: 29.80
70: 13.80
71: 13.30
72: 16.70
73: 12.00
74: 14.60
75: 21.40
76: 23.00
77: 23.70
78: 25.00
79: 21.80
80: 20.60
81: 21.20
82: 19.10
83: 20.60
84: 15.20
85: 7.00
86: 8.10
87: 13.60
88: 20.10
89: 21.80
90: 24.50
91: 23.10
92: 19.70
93: 18.30
94: 21.20
95: 17.50
96: 16.80
97: 22.40
98: 20.60
99: 23.90
100: 22.00
101: 11.90

---------------------------------------------------------------------------------------------------------

 

 data.txt

98.87, 599.0
68.74, 450.0
89.24, 440.0
129.19, 780.0
61.64, 450.0
74.0, 315.0
124.07, 998.0
65.0, 435.0
57.52, 435.0
60.42, 225.0
98.78, 685.0
63.3, 320.0
94.8, 568.0
49.95, 365.0
84.76, 530.0
127.82, 720.0
85.19, 709.0
79.91, 510.0
91.56, 600.0
59.0, 300.0
96.22, 580.0
94.69, 380.0
45.0, 210.0
68.0, 320.0
77.37, 630.0
104.0, 415.0
81.0, 798.0
64.33, 450.0
86.43, 935.0
67.29, 310.0
68.69, 469.0
109.01, 480.0
56.78, 450.0
53.0, 550.0
95.26, 625.0
142.0, 850.0
76.19, 450.0
83.42, 450.0
104.88, 580.0
149.13, 750.0
155.51, 888.0
105.15, 570.0
44.98, 215.0
87.48, 680.0
154.9, 1300.0
69.02, 340.0
74.0, 510.0
130.3, 1260.0
160.0, 640.0
88.31, 535.0
80.38, 680.0
94.99, 557.0
87.98, 435.0
88.0, 430.0
65.39, 320.0
89.0, 395.0
97.36, 535.0
53.0, 470.0
120.0, 632.0
129.0, 520.0
138.68, 878.0
81.25, 520.0
81.37, 550.0
101.98, 630.0
99.5, 540.0
59.53, 350.0
60.0, 220.0
50.51, 270.0
120.82, 745.0
107.11, 650.0
57.0, 252.0
145.08, 1350.0
111.8, 625.0
99.51, 518.0
87.09, 705.0
112.82, 510.0
148.93, 1280.0
199.24, 900.0
112.42, 600.0
94.17, 588.0
198.54, 1350.0
77.02, 460.0
153.47, 900.0
69.16, 435.0
84.54, 430.0
100.74, 590.0
110.9, 620.0
109.74, 618.0
107.98, 590.0
108.63, 700.0
67.37, 435.0
138.53, 790.0
44.28, 315.0
107.51, 816.0
59.53, 388.0
118.87, 1155.0
59.38, 260.0
55.0, 360.0
81.02, 410.0
109.68, 680.0
127.0, 630.0
109.23, 545.0
85.03, 699.0
107.27, 620.0
120.3, 480.0
127.08, 680.0
158.63, 850.0
123.25, 895.0
151.68, 1000.0
65.84, 438.0
96.87, 780.0
166.0, 1400.0
59.53, 410.0
137.4, 1150.0
45.0, 209.0
88.54, 722.0
87.22, 720.0
164.0, 2000.0
69.34, 248.0
103.67, 750.0
74.2, 595.0
71.0, 440.0
53.0, 475.0
60.86, 850.0
90.14, 530.0
57.0, 338.0
138.82, 1150.0
89.0, 760.0
98.0, 400.0
143.35, 1150.0
113.81, 575.0
152.2, 1150.0
63.32, 330.0
66.91, 218.0
44.28, 305.0
97.76, 590.0
69.0, 285.0
55.0, 380.0
148.0, 1300.0
154.59, 868.0
131.5, 1020.0
87.1, 780.0
148.68, 749.0
94.22, 590.0
96.79, 670.0
99.19, 578.0
199.0, 1380.0
125.03, 630.0
60.95, 520.0
127.04, 680.0
85.63, 460.0
77.14, 320.0
75.63, 508.0
140.18, 800.0
59.53, 365.0
109.09, 850.0
152.0, 1850.0
122.0, 980.0
111.21, 630.0
56.7, 260.0
84.46, 588.0
83.19, 500.0
132.18, 1260.0
76.14, 480.0
107.29, 585.0
137.71, 780.0
108.22, 610.0
98.81, 570.0
139.0, 1180.0
89.0, 1100.0
89.47, 800.0
75.61, 496.0
84.54, 460.0
75.87, 490.0
61.0, 450.0
83.72, 500.0
53.0, 458.0
86.0, 700.0
98.57, 760.0
84.86, 510.0
82.77, 600.0
102.49, 600.0
139.32, 730.0
145.0, 1290.0
148.0, 1100.0
65.82, 410.0
53.0, 240.0
88.96, 1000.0
86.36, 700.0
65.72, 455.0
88.0, 725.0
65.98, 600.0
99.0, 560.0
131.0, 975.0
59.53, 349.0
86.79, 508.0
110.19, 500.0
42.13, 320.0
89.91, 450.0
44.0, 320.0
107.16, 900.0
98.28, 574.0
109.68, 650.0
65.0, 450.0
103.8, 750.0
71.69, 440.0
94.38, 550.0
107.5, 760.0
85.29, 705.0
152.3, 1000.0
80.66, 665.0
88.0, 600.0
67.0, 350.0
87.0, 700.0
88.15, 430.0
104.14, 600.0
54.0, 250.0
65.44, 435.0
88.93, 525.0
51.0, 338.0
57.0, 500.0
66.94, 470.0
142.99, 800.0
69.16, 500.0
43.0, 202.0
177.0, 1120.0
131.73, 900.0
60.0, 220.0
83.0, 530.0
110.99, 630.0
49.95, 365.0
121.87, 748.0
70.0, 690.0
48.76, 230.0
88.73, 547.0
59.53, 355.0
85.49, 420.0
87.06, 570.0
77.0, 350.0
55.0, 354.0
120.94, 655.0
88.7, 560.0
76.31, 510.0
100.39, 610.0
124.88, 820.0
95.0, 480.0
44.28, 315.0
158.0, 1500.0
55.59, 235.0
87.32, 738.0
64.43, 440.0
77.2, 418.0
89.0, 750.0
130.4, 725.0
98.61, 505.0
55.34, 355.0
132.66, 944.0
88.7, 560.0
67.92, 408.0
88.88, 640.0
57.52, 370.0
71.0, 615.0
86.29, 650.0
51.0, 211.0
53.14, 350.0
63.38, 430.0
90.83, 660.0
95.05, 515.0
96.0, 650.0
135.24, 900.0
80.5, 640.0
132.0, 680.0
69.0, 450.0
56.39, 268.0
59.53, 338.0
74.22, 445.0
88.0, 780.0
112.41, 570.0
140.85, 760.0
108.33, 635.0
104.76, 612.0
86.67, 632.0
169.0, 1550.0
99.19, 570.0
95.0, 780.0
174.0, 1500.0
103.13, 565.0
107.0, 940.0
109.43, 880.0
91.93, 616.0
66.69, 296.0
57.11, 280.0
98.78, 590.0
40.09, 275.0
144.86, 850.0
110.99, 600.0
103.13, 565.0
87.0, 565.0
55.0, 347.0
53.75, 312.0
62.36, 480.0
135.4, 710.0
74.22, 450.0
96.35, 400.0
88.93, 542.0
98.0, 598.0
66.28, 530.0
119.2, 950.0
67.0, 415.0
68.9, 430.0
65.33, 400.0
55.0, 350.0
148.0, 1270.0
60.0, 307.0
88.83, 450.0
85.46, 430.0
137.57, 1130.0
90.59, 440.0
49.51, 220.0
96.87, 780.0
133.24, 820.0
84.0, 650.0
59.53, 420.0
59.16, 418.0
121.0, 670.0
53.0, 489.0
145.0, 1350.0
56.43, 260.0
71.8, 400.0
77.74, 370.0
59.16, 410.0
141.0, 820.0
87.28, 510.0
112.39, 666.0
119.0, 460.0
72.66, 395.0
120.8, 720.0
121.87, 750.0
64.43, 430.0
59.53, 400.0
106.69, 615.0
102.61, 575.0
61.64, 342.0
99.02, 750.0
88.04, 750.0
126.08, 750.0
145.0, 1280.0
84.0, 550.0
109.39, 700.0
199.96, 1850.0
88.0, 410.0
104.65, 750.0
81.0, 760.0
60.0, 230.0
108.0, 760.0
87.0, 550.0
88.15, 415.0
82.86, 490.0
152.0, 750.0
89.0, 565.0
43.5, 260.0
59.62, 356.0
96.6, 800.0
59.53, 450.0
67.48, 270.0
70.0, 455.0
102.84, 600.0
112.0, 423.0
90.23, 720.0
65.0, 380.0
89.67, 730.0
69.53, 480.0
182.36, 870.0
98.56, 569.0
65.0, 430.0
59.53, 325.0
159.83, 880.0
45.0, 260.0
92.64, 628.0
85.63, 413.0
100.43, 400.0
171.68, 1000.0
104.64, 720.0
52.46, 560.0
89.02, 420.0
166.11, 1160.0
67.21, 387.0
71.57, 420.0
68.07, 265.0
170.0, 1395.0
67.0, 455.0
73.5, 480.0
130.53, 760.0
96.04, 570.0
73.57, 265.0
128.6, 750.0
127.09, 870.0
71.0, 450.0
55.43, 230.0
103.0, 560.0
169.0, 1600.0
107.7, 815.0
153.61, 770.0
71.8, 450.0
87.76, 568.0
122.0, 970.0
58.75, 420.0
65.33, 310.0
80.63, 520.0
93.0, 350.0
59.62, 400.0
124.0, 890.0
105.79, 680.0
122.83, 668.0
67.09, 420.0
71.0, 350.0
127.23, 720.0
128.0, 780.0
77.9, 498.0
55.73, 213.0
91.39, 430.0
114.43, 860.0
125.0, 730.0
73.57, 270.0
59.56, 268.0
71.74, 395.0
88.12, 565.0
65.5, 340.0
81.0, 400.0
154.95, 900.0
67.0, 600.0
80.6, 580.0
148.0, 850.0
52.33, 475.0
122.54, 950.0
70.21, 400.0
63.0, 460.0
97.0, 750.0
100.19, 600.0
179.0, 895.0
69.8, 450.0
63.4, 480.0
65.72, 439.0
77.0, 350.0
137.68, 800.0
95.0, 590.0
68.07, 270.0
136.37, 1100.0
57.39, 218.0
83.72, 510.0
125.42, 920.0
99.51, 580.0
73.33, 625.0
53.17, 490.0
53.0, 480.0
51.0, 400.0
131.27, 780.0
95.37, 625.0
59.53, 400.0
88.8, 525.0
67.0, 310.0
129.76, 660.0
98.28, 580.0
101.44, 550.0
89.05, 710.0
157.77, 1310.0
84.73, 640.0
93.96, 540.0
55.24, 365.0
86.0, 740.0
65.8, 395.0
139.0, 1150.0
99.19, 540.0
88.0, 678.0
65.0, 440.0
138.37, 1060.0
65.33, 350.0
140.6, 850.0
90.46, 518.0
53.0, 485.0
73.9, 370.0
71.7, 280.0
80.73, 485.0
113.0, 570.0
97.0, 570.0
65.5, 340.0
77.74, 350.0
145.0, 1280.0
97.46, 800.0
88.8, 530.0
198.04, 1600.0
50.0, 270.0
60.0, 220.0
136.0, 858.0
67.07, 370.0
49.51, 220.0
67.0, 600.0
108.56, 810.0
96.52, 565.0
68.48, 435.0
65.84, 450.0
102.61, 590.0
101.69, 600.0
73.93, 520.0
57.0, 256.0
123.5, 1115.0
154.89, 1260.0
160.34, 1320.0
88.0, 715.0
71.0, 269.0
74.93, 405.0
73.6, 630.0
59.5, 380.0
84.0, 650.0
59.53, 370.0
45.0, 210.0
51.0, 350.0
107.56, 780.0
76.46, 418.0
83.0, 398.0
77.14, 305.0
71.0, 300.0
86.0, 680.0
52.37, 450.0
99.99, 530.0
52.91, 540.0
63.36, 299.0
60.51, 520.0
122.0, 950.0
96.73, 740.0
138.82, 860.0
99.0, 520.0
109.75, 570.0
112.89, 870.0
65.36, 420.0
110.19, 558.0
132.59, 980.0
128.78, 900.0
89.0, 1296.0
182.36, 980.0
146.41, 521.0
90.0, 428.0
157.0, 1280.0
89.0, 528.0
85.19, 695.0
61.0, 686.0
91.56, 720.0
126.25, 670.0
81.0, 775.0
117.31, 745.0
50.6, 350.0
138.53, 740.0
151.66, 1550.0
87.0, 745.0
65.0, 300.0
83.4, 535.0
67.06, 288.0
55.0, 340.0
79.0, 340.0
95.0, 760.0
77.0, 370.0
50.0, 360.0
97.0, 750.0
53.0, 465.0
60.0, 220.0
65.33, 293.0
153.0, 900.0
52.41, 518.0
69.0, 290.0
89.54, 700.0
53.0, 465.0
125.0, 760.0
103.0, 580.0
67.0, 310.0
69.0, 275.0
87.09, 495.0
102.71, 620.0
132.53, 820.0
62.0, 570.0
121.62, 780.0
80.03, 480.0
138.82, 950.0
107.0, 900.0
149.0, 930.0
68.74, 455.0
101.27, 488.0
55.0, 350.0
60.0, 240.0
97.0, 400.0
67.92, 450.0
87.86, 435.0
65.33, 365.0
70.23, 335.0
119.0, 940.0
85.68, 570.0
86.0, 520.0
125.0, 760.0
184.23, 1350.0
77.0, 485.0
57.0, 470.0
108.45, 750.0
107.02, 575.0
65.0, 478.0
97.0, 780.0
107.16, 920.0
53.75, 310.0
122.83, 700.0
63.3, 252.0
80.0, 560.0
123.82, 600.0
87.31, 405.0
126.54, 540.0
132.82, 800.0
152.73, 1280.0
109.0, 650.0
103.0, 845.0
59.62, 420.0
66.28, 525.0
96.33, 400.0
86.29, 350.0
78.78, 308.0
137.36, 1100.0
119.69, 530.0
126.08, 750.0
73.84, 490.0
73.0, 635.0
67.22, 400.0
87.98, 435.0
68.36, 270.0
73.0, 415.0
94.0, 600.0
107.07, 800.0
49.0, 260.0
156.0, 880.0
107.03, 830.0
198.75, 1550.0
60.92, 300.0
83.45, 540.0
57.39, 248.0
68.48, 450.0
140.86, 750.0
146.0, 1150.0
59.53, 430.0
77.14, 350.0
55.0, 360.0
80.15, 615.0
118.6, 660.0
63.3, 330.0
59.53, 350.0
59.65, 325.0
115.59, 470.0
71.0, 460.0
113.0, 500.0
94.7, 650.0
79.15, 460.0
150.04, 975.0
152.73, 1250.0
47.0, 230.0
146.67, 680.0
184.7, 980.0
60.06, 300.0
71.21, 485.0
124.88, 738.0
67.0, 600.0
167.28, 770.0
78.9, 320.0
118.4, 700.0
74.0, 620.0
61.38, 510.0
106.14, 585.0
109.0, 668.0
89.52, 800.0
130.29, 1100.0
136.0, 850.0
99.5, 580.0
84.02, 350.0
118.87, 413.0
88.31, 550.0
88.99, 610.0
65.82, 430.0
59.53, 350.0
120.94, 650.0
67.22, 410.0
184.0, 1380.0
156.45, 1200.0
79.0, 320.0
53.0, 459.0
160.7, 980.0
70.81, 360.0
110.94, 578.0
103.0, 600.0
80.66, 670.0
74.82, 315.0
140.09, 1140.0
89.62, 950.0
97.95, 570.0
88.0, 450.0
50.0, 400.0
112.39, 610.0
148.0, 1350.0
102.85, 565.0
126.71, 700.0
65.0, 350.0
69.0, 439.0
168.0, 770.0
61.88, 399.0
147.26, 1230.0
48.76, 210.0
67.22, 365.0
138.0, 1150.0
71.0, 325.0
115.46, 840.0
96.81, 740.0
90.21, 530.0
120.26, 650.0
53.0, 500.0
136.0, 1030.0
87.0, 660.0
134.6, 900.0
161.61, 850.0
88.88, 635.0
84.66, 485.0
81.79, 500.0
50.0, 259.0
121.0, 760.0
84.94, 595.0
73.32, 630.0
53.0, 500.0
97.86, 580.0
154.0, 1280.0
89.0, 395.0
163.66, 1080.0
101.95, 628.0
55.0, 348.0
68.48, 480.0
154.16, 780.0
157.21, 1350.0
111.0, 600.0
108.73, 580.0
53.0, 390.0
137.69, 810.0
170.83, 1290.0
67.0, 243.0
112.93, 600.0
161.0, 650.0
168.1, 1500.0
86.24, 670.0
63.0, 530.0
128.4, 950.0
98.28, 556.0
107.27, 570.0
84.0, 650.0
79.53, 480.0
110.96, 570.0
107.48, 810.0
56.88, 342.0
106.54, 540.0
59.62, 380.0
60.42, 222.0
88.31, 500.0
58.97, 236.0
88.84, 530.0
44.98, 208.0
167.28, 780.0
65.33, 300.0
55.0, 355.0
77.74, 350.0
78.81, 415.0
135.5, 1100.0
70.3, 380.0
57.0, 248.0
68.47, 330.0
59.42, 360.0
53.0, 202.0
42.13, 326.0
144.5, 1230.0
148.0, 650.0
81.0, 667.0
105.18, 590.0
59.53, 370.0
59.53, 450.0
133.8, 1090.0
59.53, 380.0
55.0, 380.0
53.0, 250.0
57.36, 258.0
137.68, 690.0
96.66, 550.0
49.95, 350.0
67.92, 430.0
81.33, 500.0
98.6, 600.0
76.61, 275.0
87.0, 438.0
84.0, 520.0
111.62, 920.0
153.44, 950.0
61.7, 320.0
90.21, 530.0
68.07, 260.0
98.43, 740.0
135.0, 800.0
65.0, 340.0
139.41, 670.0
72.66, 395.0
56.0, 240.0
84.02, 500.0
161.38, 1300.0
58.64, 420.0
80.42, 400.0
83.45, 560.0
103.0, 545.0
55.0, 380.0
128.0, 750.0
63.2, 440.0
138.01, 700.0
106.99, 625.0
109.01, 480.0
172.71, 1020.0
153.0, 1100.0
112.41, 578.0
138.01, 700.0
139.48, 900.0
103.13, 550.0
71.0, 283.0
96.0, 760.0
133.23, 830.0
79.43, 460.0
87.29, 700.0
65.82, 410.0
70.3, 376.0
96.36, 740.0
155.35, 1150.0
184.0, 1350.0
98.0, 580.0
71.0, 260.0
72.83, 470.0
95.85, 710.0
115.5, 640.0
89.0, 635.0
76.0, 475.0
125.86, 888.0
102.22, 638.0
78.0, 310.0
97.09, 800.0
112.0, 645.0
105.23, 570.0
100.74, 580.0
47.5, 430.0
106.54, 530.0
145.1, 1350.0
108.0, 790.0
59.79, 280.0
107.92, 800.0
124.75, 880.0
126.76, 710.0
91.14, 730.0
67.0, 620.0
137.76, 650.0
99.99, 600.0
150.67, 850.0
107.47, 750.0
138.53, 730.0
65.0, 438.0
53.23, 320.0
89.7, 780.0
134.62, 850.0
89.0, 735.0
59.53, 360.0
97.0, 600.0