9、Python 数据分析-2012美国大选献金项目
需求
- 1.加载数据
- 2.查看数据的基本信息
- 3.指定数据截取，将如下字段的数据进行提取，其他数据舍弃
- cand_nm ：候选人姓名
- contbr_nm ： 捐赠人姓名
- contbr_st ：捐赠人所在州
- contbr_employer ： 捐赠人所在公司
- contbr_occupation ： 捐赠人职业
- contb_receipt_amt ：捐赠数额（美元）
- contb_receipt_dt ： 捐款的日期
- 4.对新数据进行总览,查看是否存在缺失数据
- 5.用统计学指标快速描述数值型属性的概要。
- 6.空值处理。可能因为忘记填写或者保密等等原因，相关字段出现了空值，将其填充为NOT PROVIDE
- 7.异常值处理。将捐款金额<=0的数据删除
- 8.新建一列为各个候选人所在党派party
- 9.查看party这一列中有哪些不同的元素
- 10.统计party列中各个元素出现次数
- 11.查看各个党派收到的政治献金总数contb_receipt_amt
- 12.查看具体每天各个党派收到的政治献金总数contb_receipt_amt
- 13.将表中日期格式转换为'yyyy-mm-dd'。
- 14.查看老兵(捐献者职业)DISABLED VETERAN主要支持谁
①
加载数据
import numpy as np
import pandas as pd
df = pd.read_csv('./data/usa_election.txt',low_memory=False)
②
查看数据的基本信息
df.info()
>>>
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 16 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cmte_id 536041 non-null object
1 cand_id 536041 non-null object
2 cand_nm 536041 non-null object
3 contbr_nm 536041 non-null object
4 contbr_city 536026 non-null object
5 contbr_st 536040 non-null object
6 contbr_zip 535973 non-null object
7 contbr_employer 525088 non-null object
8 contbr_occupation 530520 non-null object
9 contb_receipt_amt 536041 non-null float64
10 contb_receipt_dt 536041 non-null object
11 receipt_desc 8479 non-null object
12 memo_cd 49718 non-null object
13 memo_text 52740 non-null object
14 form_tp 536041 non-null object
15 file_num 536041 non-null int64
dtypes: float64(1), int64(1), object(14)
memory usage: 65.4+ MB
③
指定数据截取，将如下字段的数据进行提取，其他数据舍弃
- cand_nm ：候选人姓名
- contbr_nm ： 捐赠人姓名
- contbr_st ：捐赠人所在州
- contbr_employer ： 捐赠人所在公司
- contbr_occupation ： 捐赠人职业
- contb_receipt_amt ：捐赠数额（美元）
- contb_receipt_dt ： 捐款的日期
df = df[['cand_nm','contbr_nm','contbr_st','contbr_employer','contbr_occupation','contb_receipt_amt','contb_receipt_dt']]
④
对新数据进行总览,查看是否存在缺失数据
df.info()
>>>
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cand_nm 536041 non-null object
1 contbr_nm 536041 non-null object
2 contbr_st 536040 non-null object
3 contbr_employer 525088 non-null object
4 contbr_occupation 530520 non-null object
5 contb_receipt_amt 536041 non-null float64
6 contb_receipt_dt 536041 non-null object
dtypes: float64(1), object(6)
memory usage: 28.6+ MB
⑤
用统计学指标快速描述数值型属性的概要
df.describe()
⑥
空值处理,可能因为忘记填写或者保密等等原因，相关字段出现了空值，将其填充为NOT PROVIDE
df.fillna(value='NOT PROVIDE',inplace=True,axis=0)
df.info()
>>>
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cand_nm 536041 non-null object
1 contbr_nm 536041 non-null object
2 contbr_st 536041 non-null object
3 contbr_employer 536041 non-null object
4 contbr_occupation 536041 non-null object
5 contb_receipt_amt 536041 non-null float64
6 contb_receipt_dt 536041 non-null object
dtypes: float64(1), object(6)
memory usage: 28.6+ MB
⑦
异常值处理。将捐款金额<=0的数据删除
df.loc[df['contb_receipt_amt'] <= 0] #异常值对应的行数据
indexs = df.loc[df['contb_receipt_amt'] <= 0].index#提取异常数据的行号
>>>
Int64Index([ 43, 265, 650, 666, 720, 810, 835, 836,
837, 838,
...
535929, 535937, 535942, 535945, 535971, 535975, 536003, 536005,
536006, 536007],
dtype='int64', length=5727)
df.drop(labels=indexs,axis=0,inplace=True)#删除这些异常数据
⑧
新建一列为各个候选人所在党派party
parties = {
'Bachmann, Michelle': 'Republican',
'Romney, Mitt': 'Republican',
'Obama, Barack': 'Democrat',
"Roemer, Charles E. 'Buddy' III": 'Reform',
'Pawlenty, Timothy': 'Republican',
'Johnson, Gary Earl': 'Libertarian',
'Paul, Ron': 'Republican',
'Santorum, Rick': 'Republican',
'Cain, Herman': 'Republican',
'Gingrich, Newt': 'Republican',
'McCotter, Thaddeus G': 'Republican',
'Huntsman, Jon': 'Republican',
'Perry, Rick': 'Republican'
}
df['party'] = df['cand_nm'].map(parties)
df.head()
⑨
查看party这一列中有哪些不同的元素
df['party'].unique()
>>>
array(['Republican', 'Democrat', 'Reform', 'Libertarian'], dtype=object)
⑩
统计party列中各个元素出现次数
df['party'].value_counts()
>>>
Democrat 289999
Republican 234300
Reform 5313
Libertarian 702
Name: party, dtype: int64
⑪
查看各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by='party')['contb_receipt_amt'].sum()
>>>
party
Democrat 8.259441e+07
Libertarian 4.132769e+05
Reform 3.429658e+05
Republican 1.251181e+08
Name: contb_receipt_amt, dtype: float64
⑫
查看具体每天各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by=['contb_receipt_dt','party'])['contb_receipt_amt'].sum()
>>>
contb_receipt_dt party
01-APR-11 Reform 50.00
Republican 12635.00
01-AUG-11 Democrat 182198.00
Libertarian 1000.00
Reform 1847.00
...
31-MAY-11 Republican 313839.80
31-OCT-11 Democrat 216971.87
Libertarian 4250.00
Reform 3205.00
Republican 751542.36
Name: contb_receipt_amt, Length: 1183, dtype: float64
⑬
将表中日期格式转换为'yyyy-mm-dd'
months = {'JAN' : 1, 'FEB' : 2, 'MAR' : 3, 'APR' : 4, 'MAY' : 5, 'JUN' : 6,
'JUL' : 7, 'AUG' : 8, 'SEP' : 9, 'OCT': 10, 'NOV': 11, 'DEC' : 12}
def transform_date(date):
day,month,year = date.split('-')
month = months[month]
return '20'+year+'-'+str(month)+'-'+day
df['contb_receipt_dt'] = df['contb_receipt_dt'].map(transform_date)
df.head()
⑭
查看老兵(捐献者职业)DISABLED VETERAN主要支持谁
df.columns
>>>
Index(['cand_nm', 'contbr_nm', 'contbr_st', 'contbr_employer',
'contbr_occupation', 'contb_receipt_amt', 'contb_receipt_dt', 'party'],
dtype='object')
#1.先把老兵这个职业对应的行数据取出
old_df = df.loc[df['contbr_occupation'] == 'DISABLED VETERAN']
#2.对old_df根据候选人分组，然后对钱数聚合
old_df.groupby(by='cand_nm')['contb_receipt_amt'].sum()
>>>
cand_nm
Cain, Herman 300.00
Obama, Barack 4205.00
Paul, Ron 2425.49
Santorum, Rick 250.00
Name: contb_receipt_amt, dtype: float64
