ćē§ćÆćć³ćęć£ć¦ćć¾ććāāćć£ćäøęć§ćććććććčŖåć§ćē§ļ¼ćÆļ¼ćć³ļ¼ćļ¼ęć£ļ¼ć¦ļ¼ćļ¼ć¾ćććØåčŖć«åćåŗććåč©ćę“»ēØå½¢ć¾ć§å¤å®ćć¦ćććä»ēµćæćććć¾ććććć å½¢ę ē“ č§£ęćę¤ē“¢ćØć³ćøć³ććć£ććććććęåę°å¶éć®ććē»é¢ć®ę¹č”å¶å¾”ćććć°ć®ćć¼ćÆć¼ćę½åŗćŖć©ćę„ę¬čŖćę±ććććććµć¼ćć¹ć®åå°ć«ćŖć£ć¦ććęč”ć§ćāØ
PythonćØMeCabćēµćæåćććć°ććć£ćę°č”ć®ć³ć¼ćć§ę„ę¬čŖććµćÆććØč§£ęć§ććććć«ćŖćć¾ććććć¼ćæåęć£ć¦é£ćććććØęćć¦ććäŗŗć»ć©ćęåć®ęåć大ćććÆćć§ćš
å½¢ę ē“ ćØćÆćęå³ćęć¤ęå°åä½ćć®ććØćå½¢ę ē“ č§£ęć§ćÆćęē« ććć®ęå°åä½ć«åå²ććććććć®åč©ć»ę“»ēØå½¢ć»čŖćæć»ēŗé³ćŖć©ćęØå®ćć¾ććććć«ćåčŖć®éć«åč§ć¹ćć¼ć¹ćęćć ćåćć”ęøćććåŗåććććØćåÆč½ć§ćć
ę„ę¬čŖć®å½¢ę ē“ č§£ęćć¼ć«ć«ćÆMeCabć»Janomeć»SudachićŖć©ćććć¾ććä»åćÆć¤ć³ć¹ćć¼ć«å¾ć®å¦ēéåŗ¦ć«å®č©ć®ććMeCabć軸ć«é²ćć¾ćć
MeCabćPythonććę±ćć«ćÆćPythonć©ććć¼ć®mecabććć±ć¼ćøćØMeCabę¬ä½ćććć¦č¾ęøćåæ č¦ć§ććęåć®ęŗåćå°ćå¤ćć§ćććäøåŗ¦ēµććć°å¾ćÆć³ć¼ććęøćć ćć
pip install -U pipć§ę“ę°brew install mecabć§MeCabę¬ä½ćå°å
„pip install unidic-liteć§č»½éč¾ęøćčæ½å pip install mecab-python3ć§ć¤ć³ć¹ćć¼ć«ęŗåćę“ćć°ćPythonć³ć¼ććÆćØć¦ćć·ć³ćć«ćimport MeCabćć¦TaggerćŖććøć§ćÆććä½ććparse()ć«ęē« ćęø”ćć ćć§č§£ęēµęćčæć£ć¦ćć¾ććTagger("-Owakati")ćØęå®ććć°ćåčŖć®éć«åč§ć¹ćć¼ć¹ćå
„ććåćć”ęøććäøēŗć§ćć
parseToNode()ć使ććØćåčŖććØć«ć蔨層形ļ¼surfaceļ¼ććØćē“ ę§ę
å ±ļ¼featureļ¼ććęć£ććć¼ććé ć«ćć©ćć¾ććfeaturećÆć«ć³ćåŗåćć®ęååć§ćå
é ć«åč©ę
å ±ćå
„ć£ć¦ććć®ć§ćfeature.split(",")[0]ć§åćåŗćć¦ćåč©ćć ććć£ć«ćæćŖć³ć°ććØćć£ćå¦ēćē°”åć«ęøćć¾ćććæć¤ćć«ććåŗęåč©ćęćåŗććććććć°čØäŗć®ćć¼ćÆć¼ćåč£ćä½ć£ććććć®ć«ęé©ć§ćš
ęØęŗč¾ęøć ćć§ćÆćę°čŖćē¬čŖć®åŗęåč©ććć¾ć1åčŖćØćć¦čŖčćććŖćććØćććć¾ććććØćć°ćć“ćććć®ćććŖę°ććčØčćÆćć°ćć°ćć«åå²ććć¦ćć¾ććć”ćććć§ćCSVćć”ć¤ć«ć«č”Øå±¤å½¢ć»åč©ć»čŖćæć»ēŗé³ćŖć©ćęøććmecab-dict-indexć³ćć³ćć§ćć¤ććŖč¾ęøćēęććTagger("-u čŖä½č¾ęøć®ćć¹")ć§čŖćæč¾¼ć¾ććć°ćčŖä½ć®čŖå½ćäøę°ć«å¼·åć§ćć¾ćććµć¼ćć¹ē¹ęć®ēØčŖć«å¼·ćč¾ęøćč²ć¦ćä½ę„ćÆćå®åć§ćä¾”å¤ć®é«ćć¹ćć«ć«ćŖćć¾ćšŖ
ćåćććć ćć§ēµććććć使ćććŖććć¬ćć«ć¾ć§ęć£ć¦ććććć®å¦ēæć¢ć¤ćć ćå³éøćć¾ććš
å½¢ę ē“ č§£ęćę“»ććć«ćÆćęååć»ćŖć¹ćć»č¾ęøć»é¢ę°ćØćć£ćPythonć®åŗē¤ć大åęć§ććå³č§£ćØę¼ēæćč±åÆćŖå „éęøćŖććē¬å¦ć§ćę«ęććć«čµ°ćåćć¾ćć
å½¢ę ē“ č§£ęćÆčŖē¶čØčŖå¦ēļ¼NLPļ¼ć®å „ćå£ććć¼ćÆć³åć»TF-IDFć»Word2VecćŖć©ć®åŗē¤ćä½ē³»ēć«å¦ć¹ćęøē±ć1åęć£ć¦ćććØćć³ć¼ćć®č£å“ć«ććčćę¹ć¾ć§ēč§£ć§ććććć«ćŖćć¾ćć
åćć”ęøćććććć¹ććpandasćmatplotlibć§éčØć»åÆč¦åć§ććć°ćć¬ćć¼ććę„ååęć®å¹ ććć£ćØåŗććć¾ććå®ååÆćć®ććć¹ććć¤ćć³ć°ę¬ćēµćæåććććØćå¦ć³ćē¾å “ć§ä½æććć¹ćć«ć«å¤ććć¾ćć
ē°å¢ę§ēÆć§ćæć¼ććć«ćå©ćęéćé·ćåéć ććććććć¼ćć¼ćć®ęéµęćÆå¦ēæå¹ēć«ē“ēµćć¾ććéé³ćæć¤ććŖććå¤ć®éäøćæć¤ć ć§ćå®¶ęćčæęćę°ć«ćććæć¤ćć³ć°ć§ćć¾ćš
å·¦ć«ćØćć£ćæć¼ćå³ć«UniDicćMeCabć®ććć„ć”ć³ććäøć«ćæć¼ććć«āāēę³ć®ä½ę„ē°å¢ćÆć¢ćć¤ć«ć¢ććæć¼1ęć§å®ē¾åÆč½ććć¼ćPCć«USB-Cć§ē¹ćć ćć§ćčŖå® ć§ćć«ćć§ć§ćåæ«é©ćŖNLPē°å¢ćęć«å „ćć¾ćć
MeCabćÆC++製ć§é«éćJanomećÆPure Python製ć§ć¤ć³ć¹ćć¼ć«ćē°”åćØććē¹å¾“ćććć¾ććę軽ć«č©¦ććććŖćJanomećę¬ę ¼ēć«å¤§éć®ęē« ćå¦ēććććŖćMeCabćå®ēŖć§ćć
ć·ć³ćć«ć«å§ćććŖćä»å瓹ä»ććUniDic-litećę軽ć§ććę°čŖć»åŗęåč©ć«å¼·ćććććŖćNEologdćå¦č”ēćŖč”ØčØć®ēµ±äøćéč¦ćććŖćę¬å®¶UniDicćåćć¦ćć¾ćć
Pythonć®ćć¼ćøć§ć³ļ¼3.6仄äøļ¼ćpipć®ę“ę°ćVisual C++ć©ć³ćæć¤ć ć®å°å „ćMeCabę¬ä½ć®64bitēć¤ć³ć¹ćć¼ć«ćē°å¢å¤ę°ć®PATHčØå®ććć§ććÆćć¾ććććęé ć1ć¤ćć¤ē¢ŗčŖććć®ćčæéć§ćć
mecab-dict-indexć³ćć³ćć®å®č”ćć£ć¬ćÆććŖļ¼ć·ć¹ćć č¾ęøćć£ć¬ćÆććŖļ¼ćØćCSVćć”ć¤ć«ć®ęåć³ć¼ćļ¼UTF-8ļ¼ćåć£ć¦ćććć確čŖćć¾ććććTaggerēęęć®-ućŖćć·ć§ć³ć®ćć¹ęå®ćć¹ćććććåå ć§ćć
ę½åŗććåč©ć§WordCloudćä½ććTF-IDFć§ē¹å¾“čŖćę½åŗćććć³ćµć¤ć³é”似度ć§é”ä¼¼ęęøę¤ē“¢ćä½ććWord2VecćBERTć§ęå³ććÆćć«åććććØćć£ćēŗå±ć«ć¼ććč±åÆć«ććć¾ććå½¢ę ē“ č§£ęćÆćNLPćØććåŗå¤§ćŖę£®ćøć®å „å£ć§ćš³
ē°å¢ę§ēÆć®ćć¼ćć«ććč¶ććć°ćPythonćØMeCabćÆććŖćć®å¼·åćŖēøę£ć«ćŖćć¾ććęē« ćåčŖć«åč§£ććåč©ććć£ć«ćæććčŖä½č¾ęøć§åŗęåč©ć«å¼·ćććāāćć®3ć¹ććććę¼ćććć ćć§ćę¤ē“¢ę¹åć»ććć¹ćåęć»ćć£ćććććéēŗćŖć©ćę“»ēØå ćÆäøę°ć«åŗććć¾ćć
ęøē±ć§åå°ćåŗććåæ«é©ćŖććć¤ć¹ē°å¢ć§ęćåććć°ćę„ę¬čŖćć¼ćæć®äøēćÆćć£ćØčŗ«čæć«ćä»ę„ć¤ć³ć¹ćć¼ć«ććMeCabććę„ęć®ććŖćć®ä»äŗćå¦ć³ćå¤ćććÆćć§ćšāØ
ā”ļø ćAIć使ćććććAI…
šØ AIćęŖēØćććµć¤ćć¼ę»ę…
ā ļø ćčŖåćÆå¤§äøå¤«ććéēØć…
š¦ ćåćć¦ććć¤ććććäøēŖ…
š AIę代ć®ę³¢ć«ä¹ććč³ē£ć…