Kaggle Learn(2): Intro to Machine Learning

datasets Usage:

https://www.kaggle.com/datasets/dansbecker/melbourne-housing-snapshot

How Models Work？

About how maching learning models work and how they are used.

Your cousin has made millions of dollars speculating on real estate. He's offered to become business partners with you because of your interest in data science. He'll supply the money, and you'll supply models that predict how much various houses are worth.
你的表哥通过房地产投机赚取了数百万美元。由于你对数据科学的兴趣，他提出与你成为商业伙伴。他将提供资金，而你将提供预测各种房屋价值的模型.

You ask your cousin how he's predicted real estate values in the past, and he says it is just intuition. But more questioning reveals that he's identified price patterns from houses he has seen in the past, and he uses those patterns to make predictions for new houses he is considering.
你询问表哥过去是如何预测房地产价值的，他说只是凭直觉。但进一步的询问揭示了他通过过去看到的房屋识别价格模式，并利用这些模式对新考虑的房屋进行预测。

Machine learning works the same way. We'll start with a model called the Decision Tree. There are fancier models that give more accurate predictions. But decision trees are easy to understand, and they are the basic building block for some of the best models in data science.
机器学习的工作原理是相同的。我们将从一个名为决策树的模型开始。有一些更复杂的模型可以提供更准确的预测。但决策树易于理解，它们是数据科学中一些最佳模型的基本构建模块。

Simple Decision Tree:

With only two categories. Under consideration with the historical average price of houses in the same caregory.

The whole process is belike:

We use data to decide how to break the houses into two groups, and then again to determine the predicted price in each group.

This step of capturing patterns from data is called fitting or training the model. The data used to fit the model is called the training data.

After the model has been fit, we can apply it to new data to predict prices of additional homes.

The whole idea of the process is about :train(use the data and result we had) and predict( The new data without result).

Improving the Decision Tree :

Q: The following two Decision Tree Models which one is more likely to result form fitting the real eastate traing data?

Difference: more bedrooms with higher or lower price?

There is no doubt that , the decision tree on the left probably makes more sense ,beacause it captures the reality that houses with more bedrooms tend to sold higher than fewer bedrooms. The biggest shortcoming of this model is that it doesn't capture most factors affecting home price, like number of bathrooms, lot size, location, etc.

Than when can capture more factors using a tree that has more "splits", called "deeper" trees .

A decision tree that also consider the total size of each house;s lot might look like :

By tracing through the decidion tree ,always picking the path corresponding to that house's characteristics.

The predicted price for the house is at the bottom of the tree.

The point at the bottom where we make a prediction is called a** leaf**. The splits and values at the leaves will be determined by the data.

Basic Data Exploration

Using Pandas to get familiar with data：

(熟悉数据是非常重要的，所以进行机器学习的第一步是对数据信息的熟悉)

The first step in any machine learning project is familiarize yourself with the data.

You'll use the Pandas library for this. Pandas is the primary tool data scientists use for exploring and manipulating data.

Usage:

import pandas as pd

The most import part of the Pandas library is the DataFrame(数据帧). A DataFrame holds the type of data you might think of as a table.This is similar to a sheet in Excel, or a table in a SQL database.

(简单地说，就是pandas支持把我们的数据处理成像excel和sql数据库那样的数据表地形式)

Pandas has powerful methods for most things you'll want to do with this type of data.

(功能齐全)

As an example, we'll look at data about home prices in Melbourne, Australia. In the hands-on exercises, you will apply the same processes to a new dataset, which has home prices in Iowa.

[belike with the following ]

Usage:

# save filepath to variable for easier access
melbourne_file_path = '../input/melbourne-housing-snapshot/melb_data.csv'
# read the data and store data in DataFrame titled melbourne_data
melbourne_data = pd.read_csv(melbourne_file_path) 
# print a summary of the data in Melbourne data

melbourne_data.describe()

run belike:

Interpreting Data Description：

The results show 8 numbers for each column in your original dataset. The first number, the count, shows how many rows have non-missing values.

Missing values arise for many reasons. For example, the size of the 2nd bedroom wouldn't be collected when surveying a 1 bedroom house. We'll come back to the topic of missing data.

The second value is the mean, which is the average. Under that, std is the standard deviation, which measures how numerically spread out the values are.

[第二个值是均值，即平均值。在其下方，std 是标准差，它衡量数值的离散程度]

To interpret the min, 25%, 50%, 75% and max values, imagine sorting each column from lowest to highest value. The first (smallest) value is the min. If you go a quarter way through the list, you'll find a number that is bigger than 25% of the values and smaller than 75% of the values. That is the 25% value (pronounced "25th percentile"). The 50th and 75th percentiles are defined analogously, and the max is the largest number.

要解释最小值、25%、50%、75%和最大值，可以想象将每一列从低到高排序。第一个（最小的）值是最小值。如果你在列表中前进四分之一，你会找到一个比 25%的值大、比 75%的值小的数。这就是 25%的值（读作"25 个百分点"）。50%和 75%的百分点类似定义，最大值是最大的数。

Exercise:

To read data file and understand statitics about the data.

Step 1: Loading Data

import pandas as pd

# Path of the file to read
iowa_file_path = '../input/home-data-for-ml-course/train.csv'

# Fill in the line below to read the file into a variable home_data
home_data = pd.read_csv(iowa_file_path)

# Call line below with no argument to check that you've loaded the data correctly
step_1.check()

Step 2: Review The Data

# Print summary statistics in next line

home_data.describe()

# What is the average lot size (rounded to nearest integer)?
avg_lot_size = round(home_data['LotArea'].mean())

# As of today, how old is the newest home (current year - the date in which it was built)
newest_home_age = round(2025 - home_data['YearBuilt'].max())

# Checks your answers
step_2.check()

注意看一下这里关于max和min的使用方法，只能说python fw不太会用。

Your First Machine Learning Model：

Selecting Data for Modeling

Your dataset had too many vars and hard to solve with, so how to deal with the data and understand them has become a key question.

We'll start by picking a few vars bu using our intuition. Then we'll try to solve by using statistical techniques to automatically priotize vars.

To choose ,we should see a list of all columns .

Belike:

import pandas as pd

data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

x=data.columns
print(x)

How to deal with missing data?

# The Melbourne data has some missing values (some houses for which some variables weren't recorded.)
# We'll learn to handle missing values in a later tutorial.  
# Your Iowa data doesn't have missing values in the columns you use. 
# So we will take the simplest option for now, and drop houses from our data. 
# Don't worry about this much for now, though the code is:

# dropna drops missing values (think of na as "not available")
melbourne_data = melbourne_data.dropna(axis=0)

（这里直接使用了dropna函数这个就是会把直接缺失项标记为0，axis默认为0，表示删除行，但是使用参数为1会进行对应列的删除）

There are many ways to select a subset of data. The Pandas covers more ,here just focus on two approcahs:

题外话：

既然这里简单说到了subset of data（数据子集），也就顺便说说这里关于数据子集的一些内容：

数据子集，指的是对其中我们满足特定要求的一些数据信息进行处理操作，这是数据分析的比较核心的步骤，有助于更加高效地观察、清洗和处理数据。

• Dot notation, which we use to select the "prediction target"
• Selecting with a column list, which we use to select the "features"

Selecting The Prediction Target:

• Usage of dot-notation
  • with a single column is stored in a Series , which is broadly like a DataFrame with only a single column of data.
  • We'll use the dot notation to select the column we want to predict, which is called the prediction target. By convention, the prediction target is called y.

[简单说就是：使用点变量法来提取我们特定需要的变量然后直接列出来一个列，命名为y]

y = melbourne_data.Price

Choosing "Features"

The columns that are inputted into our model (and later used to make predictions) are called "features."

For now, we'll build a model with only a few features. Later on you'll see how to iterate and compare models built with different features.

We select multiple features by providing a list of column names inside brackets. Each item in that list should be a string (with quotes).

melbourne_features = ['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_features]

By convention, this data is called X.

we can also check the features by the ways we had learned before:

X.describe()
X.head()

Build Model:

Here we are going to use scikit-learning library to create models . When coding ,this library is written as sklearn. Scikit-learn is easily the most popular library for modeling the types of data typically stored in DataFrames.

The steps to building and using a model are:

• Define: What type of model will it be? A decision tree? Some other type of model? Some other parameters of the model type are specified too.
• Fit : Capture patterns from provided data. This is the heart of modeling.
• Predict: Just what it sounds like
• Evaluate: Determine how accurate the model's predictions are.

定义：选择你的模型；拟合：选择你的数据进行拟合；预测：对新数据进行预测；评估：检查预测结果是否准确。

Example：

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

y=data['price']

features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

model = DecisionTreeRegressor(random_state=1)
#fit model

model.fit(X, y)

这里关于两点进行一个简单的说明：

首先是关于random_state可选参数的说明，虽然关于我们原文的下面也进行了对应的说明，也顺便说说自己的理解吧：

Many machine learning models allow some randomness in model training. Specifying a number for <font style="color:rgb(60, 64, 67);">random_state</font> ensures you get the same results in each run. This is considered a good practice. You use any number, and model quality won't depend meaningfully on exactly what value you choose.

这是我们原文的说明，许多机器学习模型在模型训练过程中允许一定的随机性，这里主要是为了保证每次训练的结果是不是一样的，通过这个数值的设定可以保证原文的随机化程度。

通过下面这个也可以看到对于决策树函数中的这个参数的设置定义：

然后这里也就主要想说明，这个参数有时候对模型影响挺大的，还是要稍微注意一下。

其次是关于fit函数这里，有点像我们从一个比较大的集合中，挑出了最有可能影响我们预测结果的因素，然后y就是我们进行预测学习的关键：

比较像是这样一种关系。

We now have a fitted model that we can use to make predictions：

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

y=data['Price']

features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

model = DecisionTreeRegressor(random_state=1)
#fit model
model.fit(X, y)

print("Making predictions for the following 5 houses:")
print(X.head())

print("The predictions are")
print(model.predict(X.head()))

Run:

Exercise:

https://www.kaggle.com/datasets/dansbecker/home-data-for-ml-course(dataset)

Step 1: Specify Prediction Target

Select the target variable, which corresponds to the sales price.

choose y:

• show cols:

• choose:

y = home_data['SalePrice']

# Check your answer
step_1.check()

choose X:

Now you will create a DataFrame called X holding the predictive features.

Since you want only some columns from the original data, you'll first create a list with the names of the columns you want in X.

You'll use just the following columns in the list (you can copy and paste the whole list to save some typing, though you'll still need to add quotes):

• LotArea
• YearBuilt
• 1stFlrSF
• 2ndFlrSF
• FullBath
• BedroomAbvGr
• TotRmsAbvGrd

After you've created that list of features, use it to create the DataFrame that you'll use to fit the model.

# Create the list of features below
feature_names = ['LotArea','YearBuilt','1stFlrSF','2ndFlrSF','FullBath','BedroomAbvGr','TotRmsAbvGrd']

# Select data corresponding to features in feature_names
X = home_data[feature_names]
# Check your answer
step_2.check()

Review Data：

# Review data
# print description or statistics from X
print(X.describe())

# print the top few lines
print(X.head())

Step 2: Specify and Fit Model

Step 3: Make Predictions

剩下步骤和前面一模一样，就不展开了（）

skip>>

到这里，我们已经简单的进行了一个超级初级的模型的搭建，非常非常简单，但是这里的这个思路还是非常重要的，所以还是进行一个简单的总结，以后为了搭建大型模型也是同样适用的：

首先，我们的目标是为了训练这样一个模型来完成莫一类任务，比如这里的房价预测，或者是最开始说到的有关泰坦尼克的生存预测。

在明确了任务之后，第一步就是要选择目标。这里主要涉及的函数为pandas函数库，进行初步的数据处理，我们需要分析比较有关的X，以及我们最终的预测关系y（因为这里涉及的还是比较单一的一种多对一关系，后面自然也会涉及到其他领域的非单一变量，但是这样的思路是不变的）。

其次，就是模型的选择，比如这里选择的就是sklearn自带的模型，进行了简单的参数调整，后面模型会越来越复杂，甚至我们需要自己来训练我们的模型。

最后就是把训练好的模型进行结果预测。至此，一个超级简单的模型搭建也就完成了。

Model Validation

To use model validation to measure how good the model is .

What is Model Validation:

The main idea: will the model's predictions be close to what actually happens .

Mant people make a huge mistake when mwasyring predictive accuracy. They make predictions with their training data and compare those predictions to the target values in the training data.

（不能使用训练数据进行预测，并把这些预测结果和训练数据的目标值进行比较）

First to summarize the model quality into an understandable way. If you compare predicted and actual home values for 10,000 houses, you'll likely find mix of good and bad predictions. Looking through a list of 10,000 predicted and actual values would be pointless. We need to summarize this into a single metric.

There are many metrics for summarizing model quality, but we'll start with one called Mean Absolute Error (also called MAE 平绝对误差). Let's break down this metric starting with the last word, error.

The prediction error for each house is:

error=actual−predicted
#误差=实际-预测

So, if a house cost $150,000 and you predicted it would cost $100,000 the error is $50,000.

With the MAE metric, we take the absolute value of each error. This converts each error to a positive number. We then take the average of those absolute errors. This is our measure of model quality. In plain English, it can be said as

On average, our predictions are off by about X.

To calculate MAE, we first need a model.

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

data.describe()

y=data['Price']

features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

model = DecisionTreeRegressor(random_state=1)
#fit model
model.fit(X, y)

# print("Making predictions for the following 5 houses:")
# print(X.head())

# print("The predictions are")
# print(model.predict(X.head()))

Once we have a model, here is how we calculate the mean absolute error:

from sklearn.metrics import mean_absolute_error

predicted_home_prices = melbourne_model.predict(X)
mean_absolute_error(y, predicted_home_prices)

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_squared_error, mean_absolute_error

data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

data.describe()

y=data['Price']

features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

model = DecisionTreeRegressor(random_state=1)
#fit model
model.fit(X, y)



print("Making predictions for the following 5 houses:")
print(X.head())

print("The predictions are")
print(model.predict(X.head()))
print("mean absolute error:",mean_absolute_error(y, model.predict(X)))

The Problem with "In-Sample" Scores

（这里会解释一下为什么我们前面说那样的预测是不好的）

The measure we just computed can be called an "in-sample" score. We used a single "sample" of houses for both building the model and evaluating it. Here's why this is bad.

Imagine that, in the large real estate market, door color is unrelated to home price.

However, in the sample of data you used to build the model, all homes with green doors were very expensive. The model's job is to find patterns that predict home prices, so it will see this pattern, and it will always predict high prices for homes with green doors.

Since this pattern was derived from the training data, the model will appear accurate in the training data.

But if this pattern doesn't hold when the model sees new data, the model would be very inaccurate when used in practice.

Since models' practical value come from making predictions on new data, we measure performance on data that wasn't used to build the model. The most straightforward way to do this is to exclude some data from the model-building process, and then use those to test the model's accuracy on data it hasn't seen before. This data is called validation data.

（说白了也很简单，在我训练的时候，存在某一个不那么重要的特征导致某种误导性的训练，这样对我当前的模型训练效果来说就会很好，但是我们把这个模型用到其他预测目标身上就会效果很差。而模型的实际价值来自对新数据的预测。所以我们会用一些未参与构建模型的数据进行性能衡量。最直接的方法是从模型构建过程中排除一些数据，然后使用这些数据来测试模型在它之前未见过数据上的准确性。这些数据就叫做验证数据）

Coding it

To split data：

The scikit-learn library has a function train_test_split to break up the data into two pieces. We'll use some of that data as training data to fit the model, and we'll use the other data as validation data to calculate mean_absolute_error.

from sklearn.model_selection import train_test_split

# split data into training and validation data, for both features and target
# The split is based on a random number generator. Supplying a numeric value to
# the random_state argument guarantees we get the same split every time we
# run this script.
train_X, val_X, train_y, val_y = train_test_split(X, y, random_state = 0)
# Define model
melbourne_model = DecisionTreeRegressor()
# Fit model
melbourne_model.fit(train_X, train_y)

# get predicted prices on validation data
val_predictions = melbourne_model.predict(val_X)
print(mean_absolute_error(val_y, val_predictions))

（偷个懒继续用上一个实验留下来的代码直接进行切割）：

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_squared_error, mean_absolute_error
from sklearn.model_selection import train_test_split

data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

data.describe()

y=data['Price']

features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

train_X, val_X, train_y, val_y = train_test_split(X, y, test_size=0.4, random_state=42)

model = DecisionTreeRegressor(random_state=1)
#fit model
model.fit(train_X, train_y)



print("Making predictions for the following 5 houses:")
print(X.head())

print("The predictions are")
print(model.predict(val_X.head()))
print("mean absolute error:",mean_absolute_error(val_y, model.predict(val_X)))

可以看的出来这个效果就不太好，后面就是怎么进行模型优化的问题了。

Underfitting and Overfitting:

Experimenting with different models:

Now we have learned to measure with differebt models , we can also try to find a better predictioins. But how to choose the models?

In scikit-learn's documentation that the decision tree model has many options .The most important options determine the tree's depth.

In practice, it's not uncommon for a tree to have 10 splits between the top level (all houses) and a leaf. As the tree gets deeper, the dataset gets sliced up into leaves with fewer houses. If a tree only had 1 split, it divides the data into 2 groups. If each group is split again, we would get 4 groups of houses. Splitting each of those again would create 8 groups. If we keep doubling the number of groups by adding more splits at each level, we'll have 2^10 groups of houses by the time we get to the 10th level. That's 1024 leaves.

When we divide the houses amongst many leaves, we also have fewer houses in each leaf. **Leaves with very few houses will make predictions that are quite close to those homes' actual values, but they may make very unreliable predictions for new data **(because each prediction is based on only a few houses).

This is a phenomenon called overfitting , where a model matches the training data almost perfectly, but does poorly in validation and other new data. On the flip side, if we make our tree very shallow, it doesn't divide up the houses into very distinct groups.

At an extreme, if a tree divides houses into only 2 or 4, each group still has a wide variety of houses. Resulting predictions may be far off for most houses, even in the training data (and it will be bad in validation too for the same reason). When a model fails to capture important distinctions and patterns in the data, so it performs poorly even in training data, that is called underfitting.

(简单说就是一个：过拟合：训练过多导致训练结果非常切合当前模型但是不适用其他新的数据；欠拟合：训练不够导致预测非常差)

Since we care about accuracy on new data, which we estimate from our validation data, we want to find the sweet spot between underfitting and overfitting. Visually, we want the low point of the (red) validation curve in the figure below.

Example:

There are a few alternatives for controlling the tree depth, and many allow for some routes through the tree to have greater depth than other routes. But the max_leaf_nodes argument provides a very sensible way to control overfitting vs underfitting. The more leaves we allow the model to make, the more we move from the underfitting area in the above graph to the overfitting area.

We can use a utility function to help compare MAE scores from different values for max_leaf_nodes:

import pandas as pd
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split

data_path='./data/melb_data.csv'
data = pd.read_csv(data_path)

data.describe()

y=data['Price']
features=['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = data[features]

train_X, val_X, train_y, val_y = train_test_split(X, y, test_size=0.4, random_state=42)

#fit model
#get MAE
def get_MAE(max_leaf_nodes,train_X,val_X,train_y,val_y):
    model = DecisionTreeRegressor(max_leaf_nodes=max_leaf_nodes,random_state=0)
    model.fit(train_X, train_y)
    MAE = mean_absolute_error(val_y, model.predict(val_X))
    return MAE

for max_leaf_nodes in [5, 50, 500, 5000]:
    my_mae = get_MAE(max_leaf_nodes, train_X, val_X, train_y, val_y)
    print("Max leaf nodes: %d  \t\t Mean Absolute Error:  %d" %(max_leaf_nodes, my_mae))

(这里依旧用的是之前那个房价预测的那一组测试数据，有点点不一样)

Of the options listed , 500 is the optiomal number of leaves.

Conclusion:

Here's the takeaway: Models can suffer from either:

• Overfitting: capturing spurious patterns that won't recur in the future, leading to less accurate predictions, or
• Underfitting: failing to capture relevant patterns, again leading to less accurate predictions.

We use validation data, which isn't used in model training, to measure a candidate model's accuracy. This lets us try many candidate models and keep the best one.