elasticsearch v6.5.3 v7.17.4 mysql 热加载 扩展词 停止词 同义词
安装
官网
https://www.elastic.co/cn/downloads/past-releases/elasticsearch-6-5-3
kibana
https://www.elastic.co/cn/downloads/past-releases/kibana-6-5-3
kibana汉化
https://github.com/JGMa-java/kibana_zh-cn
es 内容
es 和 数据库 的联系
分词案例
原生的
http://localhost:9200/test
{
"settings": {
"analysis": {
"analyzer": {
"ik_max_word": {
"type": "custom",
"tokenizer": "ik_max_word"
}
}
}
},
"mappings": {
"doc": {
"properties": {
"content": {
"type": "text",
"analyzer": "ik_max_word"
}
}
}
}
}
http://localhost:9200/test/doc
{
"content": "北京我来了！"
}
http://localhost:9200/test/_search
{
"query": {
"match": {
"content": "我"
}
}
}
http://localhost:9200/test/_analyze
{
"text":"我是中国人"
}
ik 相关内容博客
https://zhuanlan.zhihu.com/p/381913918
https://www.cnblogs.com/bubu99/p/13599687.html
安装 分词插件
https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v6.5.3
可以先下载zip
到相应目录
elasticsearch-plugin install file:///D:/opt/es/elasticsearch-6.5.3/elasticsearch-analysis-ik-6.5.3.zip
下面命令访问github网络不稳定容易不成功
./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.3/elasticsearch-analysis-ik-6.5.3.zip
然后重启es
http://localhost:9200/test/_analyze
{
"analyzer": "ik_smart",
"text": "我是中国人"
}
http://localhost:9200/test/_analyze
{
"analyzer": "ik_max_word",
"text": "我是中国人"
}
自定义词库
D:\opt\es\elasticsearch-6.5.3\config\analysis-ik
然后重启es
http://localhost:9200/test/_analyze
{
"analyzer": "ik_max_word",
"text": "我是中国人"
}
定义远程词库
安装nginx
在html目录下
修改文件内容
远程文件是uft8编码
es也会重新加载
mysql 词库
下载相应版本源码
https://github.com/medcl/elasticsearch-analysis-ik/tree/v6.5.3
idea 打开
maven
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.26</version>
</dependency>
jdbc.yml
jdbc:
url: jdbc:mysql://localhost:3306/esciku?useUnicode=true&autoReconnect=true&failOverReadOnly=false&characterEncoding=utf8&useSSL=false&serverTimezone=UTC
user: root
password : root
sql: SELECT keyword FROM hot_words WHERE flag=0
plugin.xml
<dependencySet>
<outputDirectory/>
<useProjectArtifact>true</useProjectArtifact>
<useTransitiveFiltering>true</useTransitiveFiltering>
<excludes>
<exclude>mysql:mysql-connector-java</exclude>
</excludes>
</dependencySet>
代码
static {
try {
//利用反射把mysql驱动加载到内存
Class.forName("com.mysql.cj.jdbc.Driver").newInstance();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 从mysql加载热更新词典
*/
private void loadExtDictByMysql() {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
Properties prop;
InputStream inputStream = null;
try {
prop = new Properties();
inputStream = Files.newInputStream(PathUtils.get(getDictRoot(), "jdbc.yml").toFile().toPath());
prop.load(inputStream);
conn = DriverManager.getConnection(
prop.getProperty("url"),
prop.getProperty("user"),
prop.getProperty("password"));
stmt = conn.createStatement();
rs = stmt.executeQuery(prop.getProperty("sql"));
while (rs.next()) {
String theWord = rs.getString("keyword");
_MainDict.fillSegment(theWord.trim().toCharArray());
}
logger.info("从mysql加载热更新词典成功!");
} catch (Exception e) {
logger.error("error", e);
} finally {
try {
if (inputStream != null) {
inputStream.close();
}
} catch (IOException e) {
e.printStackTrace();
}
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
logger.error("error", e);
}
}
if (stmt != null) {
try {
stmt.close();
} catch (SQLException e) {
logger.error("error", e);
}
}
if (conn != null) {
try {
conn.close();
} catch (SQLException e) {
logger.error("error", e);
}
}
}
}
scheduleAtFixedRate 方法的参数如下：
- 第一个参数是一个Lambda表达式或匿名函数，用于执行具体的任务逻辑。
- 第二个参数是初始延迟时间，表示任务在启动后多久开始执行第一次。
- 第三个参数是任务执行的周期，表示任务之间的时间间隔。
- 第四个参数是时间单位，用于指定第二个和第三个参数的时间单位。
// mysql 资产热加载
pool.scheduleAtFixedRate(() -> Dictionary.getSingleton().loadExtDictByMysql(), 10, 120, TimeUnit.SECONDS);
打包项目： mvn package,执行完之后会在elasticsearch-analysis-ik\target\releases文件夹下生成一个新的 elasticsearch-analysis-ik-6.5.3.zip 压缩包，解压之后将elasticsearch-analysis-ik-6.5.3.jar 和 mysql-connector-java-6.0.6.jar 都拷贝到es文件中plugins\ik文件夹下即可
问题1
这个是JRE的类的创建设值权限不对
在jre/lib/security文件夹中有一个java.policy文件,在其grant{}中加入授权即可
permission java.lang.RuntimePermission "createClassLoader";
permission java.lang.RuntimePermission "getClassLoader";
permission java.lang.RuntimePermission "accessDeclaredMembers";
permission java.lang.RuntimePermission "setContextClassLoader";
问题2
这个是通信链接等权限不对
也是,在jre/lib/security文件夹中有一个java.policy文件,在其grant{}中加入授权即可
permission java.net.SocketPermission "127.0.0.1:3306","accept";
permission java.net.SocketPermission "127.0.0.1:3306","listen";
permission java.net.SocketPermission "127.0.0.1:3306","resolve";
permission java.net.SocketPermission "127.0.0.1:3306","connect";
最后
加了个我是中国
测试
{
"analyzer": "ik_max_word",
"text": "我是中国人"
}
插件代码
https://gitee.com/linzm1007/elasticsearch-analysis-ik-6.5.4-mysql8
https://gitee.com/linzm1007/elasticsearch-analysis-ik-7.x-mysql8
同义词
创建index
http://localhost:9200/synonyms_index
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"index": {
"analysis": {
"filter": {
"doc_synonym": {
"type": "synonym",
"synonyms": ["苹果,iphone,ipad","理想,理想汽车"]
}
},
"analyzer": {
"my_doc_syno": {
"type": "custom",
"tokenizer": "ik_smart",
"filter": [
"doc_synonym"
]
}
}
}
}
},
"mappings": {
"properties": {
"name": {
"type": "text",
"analyzer": "my_doc_syno"
}
}
}
}
http://localhost:9200/synonyms_index/_analyze
{
"field": "name",
"text": "苹果,理想"
}
https://zhuanlan.zhihu.com/p/381936025
代码
https://gitee.com/linzm1007/elasticsearch-analysis-dynamic-synonym-7.x-mysql8
