notepad++ remove duplicate
step1 to sort and remove space.
Since Notepad++ Version 6 you can use this regex in the search and replace dialogue:
^(.*?)$\s+?^(?=.*^\1$)
and replace with nothing. This leaves from all duplicate rows the last occurrence in the file.