dpkg -P <pkg>

http://www.linuxquestions.org/questions/debian-26/how-do-i-get-rid-of-those-rc-packages-as-seen-in-dpkg-l-output-418956-print/