Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
南宫伊儿
elasticsearch-analysis-ik
提交
6480991b
E
elasticsearch-analysis-ik
项目概览
南宫伊儿
/
elasticsearch-analysis-ik
与 Fork 源项目一致
从无法访问的项目Fork
通知
3
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
E
elasticsearch-analysis-ik
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
6480991b
编写于
11月 10, 2013
作者:
A
arkxu
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
make it work with the latest elasticsearch
上级
ce6e7fd0
变更
10
隐藏空白更改
内联
并排
Showing
10 changed file
with
40 addition
and
49 deletion
+40
-49
pom.xml
pom.xml
+2
-1
src/main/java/org/elasticsearch/index/analysis/IkAnalyzerProvider.java
.../org/elasticsearch/index/analysis/IkAnalyzerProvider.java
+3
-17
src/main/java/org/elasticsearch/index/analysis/IkTokenizerFactory.java
.../org/elasticsearch/index/analysis/IkTokenizerFactory.java
+8
-5
src/main/java/org/wltea/analyzer/cfg/Configuration.java
src/main/java/org/wltea/analyzer/cfg/Configuration.java
+6
-7
src/main/java/org/wltea/analyzer/core/AnalyzeContext.java
src/main/java/org/wltea/analyzer/core/AnalyzeContext.java
+0
-1
src/main/java/org/wltea/analyzer/core/IKSegmenter.java
src/main/java/org/wltea/analyzer/core/IKSegmenter.java
+7
-2
src/main/java/org/wltea/analyzer/dic/Dictionary.java
src/main/java/org/wltea/analyzer/dic/Dictionary.java
+4
-6
src/main/java/org/wltea/analyzer/lucene/IKAnalyzer.java
src/main/java/org/wltea/analyzer/lucene/IKAnalyzer.java
+5
-3
src/main/java/org/wltea/analyzer/lucene/IKTokenizer.java
src/main/java/org/wltea/analyzer/lucene/IKTokenizer.java
+3
-4
src/main/java/org/wltea/analyzer/query/SWMCQueryBuilder.java
src/main/java/org/wltea/analyzer/query/SWMCQueryBuilder.java
+2
-3
未找到文件。
pom.xml
浏览文件 @
6480991b
...
@@ -31,7 +31,7 @@
...
@@ -31,7 +31,7 @@
</parent>
</parent>
<properties>
<properties>
<elasticsearch.version>
0.90.
2
</elasticsearch.version>
<elasticsearch.version>
0.90.
6
</elasticsearch.version>
</properties>
</properties>
<repositories>
<repositories>
...
@@ -76,6 +76,7 @@
...
@@ -76,6 +76,7 @@
<groupId>
junit
</groupId>
<groupId>
junit
</groupId>
<artifactId>
junit
</artifactId>
<artifactId>
junit
</artifactId>
<version>
4.10
</version>
<version>
4.10
</version>
<scope>
test
</scope>
</dependency>
</dependency>
</dependencies>
</dependencies>
...
...
src/main/java/org/elasticsearch/index/analysis/IkAnalyzerProvider.java
浏览文件 @
6480991b
...
@@ -12,28 +12,14 @@ import org.wltea.analyzer.lucene.IKAnalyzer;
...
@@ -12,28 +12,14 @@ import org.wltea.analyzer.lucene.IKAnalyzer;
public
class
IkAnalyzerProvider
extends
AbstractIndexAnalyzerProvider
<
IKAnalyzer
>
{
public
class
IkAnalyzerProvider
extends
AbstractIndexAnalyzerProvider
<
IKAnalyzer
>
{
private
final
IKAnalyzer
analyzer
;
private
final
IKAnalyzer
analyzer
;
@Inject
@Inject
public
IkAnalyzerProvider
(
Index
index
,
@IndexSettings
Settings
indexSettings
,
Environment
env
,
@Assisted
String
name
,
@Assisted
Settings
settings
)
{
public
IkAnalyzerProvider
(
Index
index
,
@IndexSettings
Settings
indexSettings
,
Environment
env
,
@Assisted
String
name
,
@Assisted
Settings
settings
)
{
super
(
index
,
indexSettings
,
name
,
settings
);
super
(
index
,
indexSettings
,
name
,
settings
);
Dictionary
.
initial
(
new
Configuration
(
settings
));
Dictionary
.
initial
(
new
Configuration
(
env
));
analyzer
=
new
IKAnalyzer
(
indexSettings
,
settings
);
analyzer
=
new
IKAnalyzer
(
indexSettings
,
settings
,
env
);
}
}
public
IkAnalyzerProvider
(
Index
index
,
Settings
indexSettings
,
String
name
,
Settings
settings
)
{
super
(
index
,
indexSettings
,
name
,
settings
);
Dictionary
.
initial
(
new
Configuration
(
settings
));
analyzer
=
new
IKAnalyzer
(
indexSettings
,
settings
);
}
public
IkAnalyzerProvider
(
Index
index
,
Settings
indexSettings
,
String
prefixSettings
,
String
name
,
Settings
settings
)
{
super
(
index
,
indexSettings
,
prefixSettings
,
name
,
settings
);
Dictionary
.
initial
(
new
Configuration
(
settings
));
analyzer
=
new
IKAnalyzer
(
indexSettings
,
settings
);
}
@Override
public
IKAnalyzer
get
()
{
@Override
public
IKAnalyzer
get
()
{
return
this
.
analyzer
;
return
this
.
analyzer
;
}
}
...
...
src/main/java/org/elasticsearch/index/analysis/IkTokenizerFactory.java
浏览文件 @
6480991b
...
@@ -4,6 +4,7 @@ import org.apache.lucene.analysis.Tokenizer;
...
@@ -4,6 +4,7 @@ import org.apache.lucene.analysis.Tokenizer;
import
org.elasticsearch.common.inject.Inject
;
import
org.elasticsearch.common.inject.Inject
;
import
org.elasticsearch.common.inject.assistedinject.Assisted
;
import
org.elasticsearch.common.inject.assistedinject.Assisted
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.env.Environment
;
import
org.elasticsearch.index.Index
;
import
org.elasticsearch.index.Index
;
import
org.elasticsearch.index.settings.IndexSettings
;
import
org.elasticsearch.index.settings.IndexSettings
;
import
org.wltea.analyzer.cfg.Configuration
;
import
org.wltea.analyzer.cfg.Configuration
;
...
@@ -13,18 +14,20 @@ import org.wltea.analyzer.lucene.IKTokenizer;
...
@@ -13,18 +14,20 @@ import org.wltea.analyzer.lucene.IKTokenizer;
import
java.io.Reader
;
import
java.io.Reader
;
public
class
IkTokenizerFactory
extends
AbstractTokenizerFactory
{
public
class
IkTokenizerFactory
extends
AbstractTokenizerFactory
{
private
Environment
environment
;
private
Settings
settings
;
private
Settings
settings
;
@Inject
@Inject
public
IkTokenizerFactory
(
Index
index
,
@IndexSettings
Settings
indexSettings
,
@Assisted
String
name
,
@Assisted
Settings
settings
)
{
public
IkTokenizerFactory
(
Index
index
,
@IndexSettings
Settings
indexSettings
,
Environment
env
,
@Assisted
String
name
,
@Assisted
Settings
settings
)
{
super
(
index
,
indexSettings
,
name
,
settings
);
super
(
index
,
indexSettings
,
name
,
settings
);
this
.
settings
=
settings
;
this
.
environment
=
env
;
Dictionary
.
initial
(
new
Configuration
(
settings
));
this
.
settings
=
settings
;
Dictionary
.
initial
(
new
Configuration
(
env
));
}
}
@Override
@Override
public
Tokenizer
create
(
Reader
reader
)
{
public
Tokenizer
create
(
Reader
reader
)
{
return
new
IKTokenizer
(
reader
,
settings
);
return
new
IKTokenizer
(
reader
,
settings
,
environment
);
}
}
}
}
src/main/java/org/wltea/analyzer/cfg/Configuration.java
浏览文件 @
6480991b
...
@@ -3,17 +3,16 @@
...
@@ -3,17 +3,16 @@
*/
*/
package
org.wltea.analyzer.cfg
;
package
org.wltea.analyzer.cfg
;
import
org.elasticsearch.common.logging.ESLogger
;
import
org.elasticsearch.common.logging.Loggers
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.env.Environment
;
import
java.io.*
;
import
java.io.*
;
import
java.util.ArrayList
;
import
java.util.ArrayList
;
import
java.util.InvalidPropertiesFormatException
;
import
java.util.InvalidPropertiesFormatException
;
import
java.util.List
;
import
java.util.List
;
import
java.util.Properties
;
import
java.util.Properties
;
import
org.elasticsearch.common.logging.ESLogger
;
import
org.elasticsearch.common.logging.Loggers
;
import
org.elasticsearch.env.Environment
;
public
class
Configuration
{
public
class
Configuration
{
private
static
String
FILE_NAME
=
"ik/IKAnalyzer.cfg.xml"
;
private
static
String
FILE_NAME
=
"ik/IKAnalyzer.cfg.xml"
;
...
@@ -23,10 +22,10 @@ public class Configuration {
...
@@ -23,10 +22,10 @@ public class Configuration {
private
Properties
props
;
private
Properties
props
;
private
Environment
environment
;
private
Environment
environment
;
public
Configuration
(
Settings
settings
){
public
Configuration
(
Environment
env
){
logger
=
Loggers
.
getLogger
(
"ik-analyzer"
);
logger
=
Loggers
.
getLogger
(
"ik-analyzer"
);
props
=
new
Properties
();
props
=
new
Properties
();
environment
=
new
Environment
(
settings
)
;
environment
=
env
;
File
fileConfig
=
new
File
(
environment
.
configFile
(),
FILE_NAME
);
File
fileConfig
=
new
File
(
environment
.
configFile
(),
FILE_NAME
);
...
...
src/main/java/org/wltea/analyzer/core/AnalyzeContext.java
浏览文件 @
6480991b
...
@@ -32,7 +32,6 @@ import java.util.LinkedList;
...
@@ -32,7 +32,6 @@ import java.util.LinkedList;
import
java.util.Map
;
import
java.util.Map
;
import
java.util.Set
;
import
java.util.Set
;
import
org.wltea.analyzer.cfg.Configuration
;
import
org.wltea.analyzer.dic.Dictionary
;
import
org.wltea.analyzer.dic.Dictionary
;
/**
/**
...
...
src/main/java/org/wltea/analyzer/core/IKSegmenter.java
浏览文件 @
6480991b
...
@@ -29,6 +29,7 @@ import java.util.ArrayList;
...
@@ -29,6 +29,7 @@ import java.util.ArrayList;
import
java.util.List
;
import
java.util.List
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.env.Environment
;
import
org.wltea.analyzer.cfg.Configuration
;
import
org.wltea.analyzer.cfg.Configuration
;
import
org.wltea.analyzer.dic.Dictionary
;
import
org.wltea.analyzer.dic.Dictionary
;
...
@@ -55,13 +56,17 @@ public final class IKSegmenter {
...
@@ -55,13 +56,17 @@ public final class IKSegmenter {
* IK分词器构造函数
* IK分词器构造函数
* @param input
* @param input
*/
*/
public
IKSegmenter
(
Reader
input
,
Settings
settings
){
public
IKSegmenter
(
Reader
input
,
Settings
settings
,
Environment
environment
){
this
.
input
=
input
;
this
.
input
=
input
;
this
.
cfg
=
new
Configuration
(
settings
);
this
.
cfg
=
new
Configuration
(
environment
);
this
.
useSmart
=
settings
.
get
(
"use_smart"
,
"true"
).
equals
(
"true"
);
this
.
useSmart
=
settings
.
get
(
"use_smart"
,
"true"
).
equals
(
"true"
);
this
.
init
();
this
.
init
();
}
}
public
IKSegmenter
(
Reader
input
){
new
IKSegmenter
(
input
,
null
,
null
);
}
// /**
// /**
// * IK分词器构造函数
// * IK分词器构造函数
// * @param input
// * @param input
...
...
src/main/java/org/wltea/analyzer/dic/Dictionary.java
浏览文件 @
6480991b
...
@@ -25,16 +25,14 @@
...
@@ -25,16 +25,14 @@
*/
*/
package
org.wltea.analyzer.dic
;
package
org.wltea.analyzer.dic
;
import
org.elasticsearch.common.logging.ESLogger
;
import
org.elasticsearch.common.logging.Loggers
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.env.Environment
;
import
org.wltea.analyzer.cfg.Configuration
;
import
java.io.*
;
import
java.io.*
;
import
java.util.Collection
;
import
java.util.Collection
;
import
java.util.List
;
import
java.util.List
;
import
org.elasticsearch.common.logging.ESLogger
;
import
org.elasticsearch.common.logging.Loggers
;
import
org.wltea.analyzer.cfg.Configuration
;
/**
/**
* 词典管理类,单子模式
* 词典管理类,单子模式
*/
*/
...
...
src/main/java/org/wltea/analyzer/lucene/IKAnalyzer.java
浏览文件 @
6480991b
...
@@ -29,7 +29,7 @@ import java.io.Reader;
...
@@ -29,7 +29,7 @@ import java.io.Reader;
import
org.apache.lucene.analysis.Analyzer
;
import
org.apache.lucene.analysis.Analyzer
;
import
org.apache.lucene.analysis.Tokenizer
;
import
org.apache.lucene.analysis.Tokenizer
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.common.settings.Settings
;
import
org.
wltea.analyzer.dic.Dictionary
;
import
org.
elasticsearch.env.Environment
;
/**
/**
* IK分词器,Lucene Analyzer接口实现
* IK分词器,Lucene Analyzer接口实现
...
@@ -67,10 +67,12 @@ public final class IKAnalyzer extends Analyzer{
...
@@ -67,10 +67,12 @@ public final class IKAnalyzer extends Analyzer{
}
}
Settings
settings
;
Settings
settings
;
Environment
environment
;
public
IKAnalyzer
(
Settings
indexSetting
,
Settings
settings
)
{
public
IKAnalyzer
(
Settings
indexSetting
,
Settings
settings
,
Environment
environment
)
{
super
();
super
();
this
.
settings
=
settings
;
this
.
settings
=
settings
;
this
.
environment
=
environment
;
}
}
/**
/**
...
@@ -78,7 +80,7 @@ public final class IKAnalyzer extends Analyzer{
...
@@ -78,7 +80,7 @@ public final class IKAnalyzer extends Analyzer{
*/
*/
@Override
@Override
protected
TokenStreamComponents
createComponents
(
String
fieldName
,
final
Reader
in
)
{
protected
TokenStreamComponents
createComponents
(
String
fieldName
,
final
Reader
in
)
{
Tokenizer
_IKTokenizer
=
new
IKTokenizer
(
in
,
settings
);
Tokenizer
_IKTokenizer
=
new
IKTokenizer
(
in
,
settings
,
environment
);
return
new
TokenStreamComponents
(
_IKTokenizer
);
return
new
TokenStreamComponents
(
_IKTokenizer
);
}
}
...
...
src/main/java/org/wltea/analyzer/lucene/IKTokenizer.java
浏览文件 @
6480991b
...
@@ -33,9 +33,8 @@ import org.apache.lucene.analysis.Tokenizer;
...
@@ -33,9 +33,8 @@ import org.apache.lucene.analysis.Tokenizer;
import
org.apache.lucene.analysis.tokenattributes.CharTermAttribute
;
import
org.apache.lucene.analysis.tokenattributes.CharTermAttribute
;
import
org.apache.lucene.analysis.tokenattributes.OffsetAttribute
;
import
org.apache.lucene.analysis.tokenattributes.OffsetAttribute
;
import
org.apache.lucene.analysis.tokenattributes.TypeAttribute
;
import
org.apache.lucene.analysis.tokenattributes.TypeAttribute
;
import
org.elasticsearch.common.settings.ImmutableSettings
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.common.settings.Settings
;
import
org.elasticsearch.env.Environment
;
import
org.wltea.analyzer.core.IKSegmenter
;
import
org.wltea.analyzer.core.IKSegmenter
;
import
org.wltea.analyzer.core.Lexeme
;
import
org.wltea.analyzer.core.Lexeme
;
...
@@ -61,13 +60,13 @@ public final class IKTokenizer extends Tokenizer {
...
@@ -61,13 +60,13 @@ public final class IKTokenizer extends Tokenizer {
* Lucene 4.0 Tokenizer适配器类构造函数
* Lucene 4.0 Tokenizer适配器类构造函数
* @param in
* @param in
*/
*/
public
IKTokenizer
(
Reader
in
,
Settings
settings
){
public
IKTokenizer
(
Reader
in
,
Settings
settings
,
Environment
environment
){
super
(
in
);
super
(
in
);
offsetAtt
=
addAttribute
(
OffsetAttribute
.
class
);
offsetAtt
=
addAttribute
(
OffsetAttribute
.
class
);
termAtt
=
addAttribute
(
CharTermAttribute
.
class
);
termAtt
=
addAttribute
(
CharTermAttribute
.
class
);
typeAtt
=
addAttribute
(
TypeAttribute
.
class
);
typeAtt
=
addAttribute
(
TypeAttribute
.
class
);
_IKImplement
=
new
IKSegmenter
(
input
,
settings
);
_IKImplement
=
new
IKSegmenter
(
input
,
settings
,
environment
);
}
}
/* (non-Javadoc)
/* (non-Javadoc)
...
...
src/main/java/org/wltea/analyzer/query/SWMCQueryBuilder.java
浏览文件 @
6480991b
...
@@ -34,7 +34,6 @@ import org.apache.lucene.queryparser.classic.ParseException;
...
@@ -34,7 +34,6 @@ import org.apache.lucene.queryparser.classic.ParseException;
import
org.apache.lucene.queryparser.classic.QueryParser
;
import
org.apache.lucene.queryparser.classic.QueryParser
;
import
org.apache.lucene.search.Query
;
import
org.apache.lucene.search.Query
;
import
org.apache.lucene.util.Version
;
import
org.apache.lucene.util.Version
;
import
org.elasticsearch.common.settings.Settings
;
import
org.wltea.analyzer.core.IKSegmenter
;
import
org.wltea.analyzer.core.IKSegmenter
;
import
org.wltea.analyzer.core.Lexeme
;
import
org.wltea.analyzer.core.Lexeme
;
...
@@ -71,8 +70,8 @@ public class SWMCQueryBuilder {
...
@@ -71,8 +70,8 @@ public class SWMCQueryBuilder {
*/
*/
private
static
List
<
Lexeme
>
doAnalyze
(
String
keywords
){
private
static
List
<
Lexeme
>
doAnalyze
(
String
keywords
){
List
<
Lexeme
>
lexemes
=
new
ArrayList
<
Lexeme
>();
List
<
Lexeme
>
lexemes
=
new
ArrayList
<
Lexeme
>();
Settings
settings
=
null
;
IKSegmenter
ikSeg
=
new
IKSegmenter
(
new
StringReader
(
keywords
)
,
settings
);
IKSegmenter
ikSeg
=
new
IKSegmenter
(
new
StringReader
(
keywords
));
try
{
try
{
Lexeme
l
=
null
;
Lexeme
l
=
null
;
while
(
(
l
=
ikSeg
.
next
())
!=
null
){
while
(
(
l
=
ikSeg
.
next
())
!=
null
){
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录