String Manipulation related with pandas

String Manipulation related with pandas

String object Methods

import pandas as pd

import numpy as np

val='a,b, guido'

val.split(',') # normal python built-in method split

['a', 'b', ' guido']

pieces=[x.strip() for x in val.split(',')];pieces  # strip whitespace

['a', 'b', 'guido']

'::'.join(pieces)

'a:🅱️:guido'

val.count(',')

2

val.count('guido')

1

val.replace(',',':')

'a🅱️ guido'

val.swapcase()

'A,B, GUIDO'

val[::-1]

'odiug ,b,a'

Regular expression

The re module functions fall into 3 categories:pattern matching,substitution,splliting.

import re

text='foo   bar\t baz  \t qux'

re.split('\s+',text)

['foo', 'bar', 'baz', 'qux']

regex=re.compile('\s+')

regex.split(text)

['foo', 'bar', 'baz', 'qux']

regex.findall(text)

[' ', '\t ', ' \t ']

• To avoid unwanted escaping with \ in a regular expression,use raw string literals

text="""Dave dave@google.com
Steve steve@mail.com
Rob rob@mail.com
Ryan ryan@yahoo.com
"""

pattern=r'[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}'

regex=re.compile(pattern,re.I)

Using findall() produces a list of the email address.

regex.findall(text)

['dave@google.com', 'steve@mail.com', 'rob@mail.com', 'ryan@yahoo.com']

regex.findall(r' J.onepy+@w-m.co')

['J.onepy+@w-m.co']

search() returns a specified match object for the first email address in the text.

m=regex.search(text)

m

<re.Match object; span=(5, 20), match='dave@google.com'>

regex.match(text)

text[m.start():m.end()]

'dave@google.com'

regex.match(text) returns None,as it onlyu will match if the pattern occurs at the start of the string.

sub() will return a new string with occurences of the pattern replaced by a new string.

print(regex.sub('READACTED',text))

Dave READACTED
Steve READACTED
Rob READACTED
Ryan READACTED

Vectorized string functions in pandas

data={'Dave':'dave@google.com','Steve':'steve@gmeil.com','Rob':'rob@gmail.com','Wes':np.nan}

data=pd.Series(data);data

Dave dave@google.com
Steve steve@gmeil.com
Rob rob@gmail.com
Wes NaN
dtype: object

data.isnull()

Dave False
Steve False
Rob False
Wes True
dtype: bool

data.str.contains('gmail')

Dave False
Steve False
Rob True
Wes NaN
dtype: object

data

Dave dave@google.com
Steve steve@gmeil.com
Rob rob@gmail.com
Wes NaN
dtype: object

data.map(lambda x:x[:2],na_action='ignore')  # x is the value in data, the returned Series has the same index with caller,data here.

Dave da
Steve st
Rob ro
Wes NaN
dtype: object

help(data.map)

Help on method map in module pandas.core.series:

map(arg, na_action=None) method of pandas.core.series.Series instance
Map values of Series using input correspondence (a dict, Series, or
function).

Parameters
----------
arg : function, dict, or Series
Mapping correspondence.
na_action : {None, 'ignore'}
If 'ignore', propagate NA values, without passing them to the
mapping correspondence.

Returns
-------
y : Series
Same index as caller.

Examples
--------

Map inputs to outputs (both of type Series):

>>> x = pd.Series([1,2,3], index=['one', 'two', 'three'])
>>> x
one 1
two 2
three 3
dtype: int64

>>> y = pd.Series(['foo', 'bar', 'baz'], index=[1,2,3])
>>> y
1 foo
2 bar
3 baz

>>> x.map(y)
one foo
two bar
three baz

If arg is a dictionary, return a new Series with values converted
according to the dictionary's mapping:

>>> z = {1: 'A', 2: 'B', 3: 'C'}

>>> x.map(z)
one A
two B
three C

Use na_action to control whether NA values are affected by the mapping
function.

>>> s = pd.Series([1, 2, 3, np.nan])

>>> s2 = s.map('this is a string {}'.format, na_action=None)
0 this is a string 1.0
1 this is a string 2.0
2 this is a string 3.0
3 this is a string nan
dtype: object

>>> s3 = s.map('this is a string {}'.format, na_action='ignore')
0 this is a string 1.0
1 this is a string 2.0
2 this is a string 3.0
3 NaN
dtype: object

See Also
--------
Series.apply : For applying more complex functions on a Series.
DataFrame.apply : Apply a function row-/column-wise.
DataFrame.applymap : Apply a function elementwise on a whole DataFrame.

Notes
-----
When arg is a dictionary, values in Series that are not in the
dictionary (as keys) are converted to NaN. However, if the
dictionary is a dict subclass that defines __missing__ (i.e.
provides a method for default values), then this default is used
rather than NaN:

>>> from collections import Counter
>>> counter = Counter()
>>> counter['bar'] += 1
>>> y.map(counter)
1 0
2 1
3 0
dtype: int64

pattern

'[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}'

data.str.findall(pattern,flags=re.I)

Dave [dave@google.com]
Steve [steve@gmeil.com]
Rob [rob@gmail.com]
Wes NaN
dtype: object

matches=data.str.match(pattern,flags=re.I);matches

Dave True
Steve True
Rob True
Wes NaN
dtype: object

matches.str.get(1)

Dave NaN
Steve NaN
Rob NaN
Wes NaN
dtype: float64

matches.str[0]

Dave NaN
Steve NaN
Rob NaN
Wes NaN
dtype: float64

data.str[:5]

Dave dave@
Steve steve
Rob rob@g
Wes NaN
dtype: object