五月天青色头像情侣网名,国产亚洲av片在线观看18女人,黑人巨茎大战俄罗斯美女,扒下她的小内裤打屁股

歡迎光臨散文網(wǎng) 會員登陸 & 注冊

百度LAC分詞(python3實現(xiàn))

2023-02-24 17:32 作者:trisyp  | 我要投稿

對應(yīng)數(shù)據(jù)的鏈接放這里(https://download.csdn.net/download/Trisyp/87486282)了

import pandas as pd
from util.logger import Log
import os
from util.data_dir import root_dir
from LAC import LAC
os_file_name = os.path.split(os.path.realpath(__file__))[-1]


# 加載LAC模型
lac = LAC(mode="lac")
# 載入自定義詞典
lac.load_customization(f'{root_dir}/data/lac_data/lac_define_dict.txt')


def load_stopwords():
? ?"""
? ?加載停用詞
? ?:return: list
? ?"""
? ?stopwords_path = f'{root_dir}/data/lac_data/stop_words.txt'
? ?stopwords_increase_path = f'{root_dir}/data/lac_data/stop_words_increase.txt'
? ?if stopwords_path:
? ? ? ?with open(stopwords_path, 'r', encoding='utf-8-sig') as f:
? ? ? ? ? ?stopwords = [line.strip() for line in f]
? ?else:
? ? ? ?Log().info(os_file_name, '《熱點事件》|沒有停用詞表|')
? ? ? ?stopwords = []
? ?if stopwords_increase_path:
? ? ? ?with open(stopwords_increase_path, 'r', encoding='utf-8-sig') as f:
? ? ? ? ? ?stopwords_increase = [line.strip() for line in f]
? ? ? ?stopwords.extend(stopwords_increase)
? ?else:
? ? ? ?Log().info(os_file_name, '《熱點事件》|沒有停用詞增加表|')
? ?return stopwords


def extract_important_word(content: str):
? ?"""
? ?LAC分詞
? ?:param content: str
? ?:return: str
? ?"""

? ?# 運行l(wèi)ac
? ?lac_result = lac.run(content)

? ?# 將分詞和詞性轉(zhuǎn)為dataframe
? ?lac_result_df = pd.DataFrame(data=zip(lac_result[1], lac_result[0]), columns=['char', 'word'])

? ?# 選取重要詞性的dataframe
? ?seg_data = list(lac_result_df[(lac_result_df['char'] == 'n') | (lac_result_df['char'] == 'vn')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'an') | (lac_result_df['char'] == 'nz')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'ORG') | (lac_result_df['char'] == 'v')
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| (lac_result_df['char'] == 'LOC')].word)

? ?# 獲取重要詞語,不在停用詞內(nèi)
? ?docu_process = ','.join([word for word in seg_data if len(word) >= 2 and word not in load_stopwords()])
? ?return docu_process


if __name__ == "__main__":
? ?sentence = "小區(qū)物業(yè)沒人管,垃圾也沒人處理"
? ?sentence = extract_important_word(sentence)
? ?print(sentence)

百度LAC分詞(python3實現(xiàn))的評論 (共 條)

分享到微博請遵守國家法律
新郑市| 邓州市| 当阳市| 邛崃市| 富源县| 合作市| 曲靖市| 习水县| 三江| 淮北市| 松潘县| 博客| 阳朔县| 襄汾县| 平乡县| 伊金霍洛旗| 清苑县| 仁化县| 县级市| 嘉义县| 游戏| 剑川县| 九江市| 五河县| 枞阳县| 扎囊县| 太康县| 九寨沟县| 方正县| 麟游县| 巨野县| 雷波县| 资溪县| 焉耆| 安泽县| 麻城市| 通海县| 嘉禾县| 文昌市| 安义县| 安丘市|