文本聚类分析.pdf

鏂囨湰鑱氱被 飩 鏁版嵁棰勫鐞 飩 鐗瑰緛鎻愬彇LSI銆乂SM 飩 DBSCAN 飩 鏈寸礌KMeans 飩 瀛愮┖闂磋仛绫籆WKMeans 飩 缁撴灉 鎻愬彇鐗瑰緛-LSI 飩 鐗瑰緛闄嶇淮 飩 鎴戜滑灏濊瘯浜哃SI 锛堟綔鍦ㄨ涔夌储寮曪級闄嶇淮鐨勬柟娉曘€ 飩 棣栧厛鐢辨暟鎹泦鏋勯€犱竴涓猼erm-document鐨勭淮搴 涓簃*n(84621*18828)鐨勭煩闃礎 銆 飩 鐒跺悗瀵笰杩涜鐭╅樀濂囧紓鍊?SVD)鍒嗚В锛屽洜涓篈 缁存暟杈冨ぇ锛屽埄鐢ㄤ簡python涓█鐤忕煩闃靛寘 T (scipy.sparse)杩涜銆傚緱鍒 A = U*S*V 銆 飩 鍙栧瑙掔煩闃礢鐨勫墠k*k缁碨 锛孶鐨刴*k缁碪 , VT k k 鐨刱*n缁碫T , 姹傚嚭A 鐨刱绉╄繎浼肩煩闃礎 = U * S k k k k * VT k 鎻愬彇鐗瑰緛-LSI 飩 鐗瑰緛闄嶇淮 - 姹傚嚭鐨勫瑙掗樀S 鎻愬彇鐗瑰緛-LSI 飩 鐗瑰緛闄嶇淮 飩 鏈潵鐨勮鍒掓槸浠嶢 鑾峰緱鏂扮殑鏂囨。鍚戦噺锛岀劧鍚 k 鐢ㄦ柊鐨勫悜閲忚繘琛岃绠椼€備絾鏄悗鏉ュ彂鐜板鐞嗕箣 鍚庣殑A 涓嶅啀鏄竴涓█鐤忕煩闃碉紝杩欐牱涓€鏉ョ畻娉 k 鐨勫鏉傚害灏辫绌哄墠鎻愰珮浜嗐€傛墍浠ュ悗鏉ユ斁寮冧簡 杩欎釜鏂规硶銆 飩 灏辨垜鐨勭悊瑙ｏ紝SVD鐨勪綔鐢ㄧ浉褰撲簬鎵惧埌褰卞搷姣 杈冨ぇ鐨刱缁寸壒寰侊紝鐒跺悗鍦ㄦ暣涓壒寰佺┖闂翠笂鍋氫簡 骞虫粦锛屽洜鑰屾晥鏋滃彲鑳戒細姣旂洿鎺ユ寜鐓f-idf閫夊彇k 涓淮搴﹀ソ锛屼絾鏄绠楁硶澶嶆潅搴﹀疄鍦ㄦ槸鏈夌偣楂樸€ 鎻愬彇鐗瑰緛-VSM+闄嶇淮 飩 TF-IDF 飩 涔嬪悗锛屾牴鎹甦iscriminationscore锛屾垜浠粠 姣忎竴涓被鍒腑鎻愬彇鍑鸿绫诲埆Top 100鐨勫叧 閿瘝锛屼竴鍏?000涓叧閿瘝锛屽叾涓苟涓嶉兘 鏄笉鍚岀殑鍗曡瘝銆備箣鍚庯紝瀵硅繖2000涓叧閿 璇嶈繘琛屽幓閲嶏紝寰楀埌n(n2000)涓叧閿瘝锛 杩欏氨鏄枃鏈殑鐗瑰緛銆傚皢杩檔涓叧閿瘝浣滀负 涓€涓悜閲忥紝瀵规墍鏈夋枃鏈繘琛屾壂鎻忥紝濡傛灉璇 鏂囨湰鍚湁绗琲涓瘝锛岄偅涔堝氨鍦ㄧi缁翠笂鍙?锛 鍚﹀垯鍙? 銆 DBSCAN鈥斺€旂畻娉曞疄鐜 1. 渚濇閬嶅巻鍏ㄩ儴瀵硅薄锛屾彁鍙栧璞[i] 2. 鏌ヨO[i]鐨勯偦鍩焠eighbors锛屽苟鏍囪O[i]涓 visited 3. 鑻[i]鏄潪鏍稿績瀵硅薄锛屾爣璁癘[i]涓簄oise 4. 鑻[i]鏄牳蹇冨璞★紝灏哋[i]鍜宯eighbors鏍 璁颁负C[j] 锛 5. 閫掑綊neighbors 6. 鑻ュ璞″凡缁忓叏閮ㄨ璁块棶锛屽垯缁堟锛涘惁鍒 j++ 锛岃繑鍥? 鍙傛暟璋冩暣鍜岀粨鏋 飩 涓昏鍙傛暟 飩 e锛氶偦灞呭叧绯荤殑鏈€澶у崐寰 飩 minPts锛氭牳蹇冨璞￠偦鍩熷唴鐐圭殑鏈€灏戞暟鐩 飩 缁撴灉 飩 鍚戦噺绌洪棿杩囦簬绋€鐤忥紝澶ч噺鍣偣 飩 澶ч儴鍒嗙偣閮借鍒嗗湪涓€绫讳腑 E minPts 鍣偣鏁 鑱氱被鏁 鐔 150 3