uniq :txt去掉重复行

sort <txt file>  |uniq