Lucene分词报错:”TokenStream contract violation: close() call missing”
报错 missing call CLOSE 分词 violation Lucene TokenStream
2023-06-13 09:12:43 时间
大家好,又见面了,我是你们的朋友全栈君。
Lucene使用IKAnalyzer分词时报错:”TokenStream contract violation: close() call missing” 解决办法是每次完成后必须调用关闭方法。
如果报错:java.lang.illegalstateexception: tokenstream contract violation: reset()/close() call missing,则要在tokenStream.incrementToken(),原因是lucene从4.6.0开始tokenstream使用方法更改的问题,在使用incrementtoken方法前必须调用reset方法,详见api http://lucene.apache.org/core/4_6_0/core/index.html 。
以下正确示例代码(第10行和22行调用reset()和close()方法):
public Set<String> slicing(String text){
Set<String> result = new HashSet<>();
StringReader reader = null;
TokenStream tokenStream = null;
try {
reader = new StringReader(text);
tokenStream = analyzer.tokenStream("", reader);
CharTermAttribute charTermAttribute = tokenStream.getAttribute(CharTermAttribute.class);
OffsetAttribute offsetAttribute = tokenStream.addAttribute(OffsetAttribute.class);
tokenStream.reset();
while (tokenStream.incrementToken()) {
int startOffset = offsetAttribute.startOffset();
int endOffset = offsetAttribute.endOffset();
if((endOffset - startOffset) > 1){
String term = charTermAttribute.toString();
result.add(term);
}
}
} catch (IOException e) {
e.printStackTrace();
} finally{
IOs.close(tokenStream, reader);
}
return result;
}
http://www.lizi.pw/archives/56
org.wltea.analyzer.lucene.IKAnalyzer
Exception in thread "main" java.lang.IllegalStateException: 词典尚未初始化,请先调用initial方法
at org.wltea.analyzer.dic.Dictionary.getSingleton(Dictionary.java:137)
at org.wltea.analyzer.core.CJKSegmenter.analyze(CJKSegmenter.java:80)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:116)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:88)
转载于:https://www.cnblogs.com/softidea/p/6084043.html
发布者:全栈程序员栈长,转载请注明出处:https://javaforall.cn/163117.html原文链接:https://javaforall.cn
相关文章
- Kibana报错 Privileges are missing and can't be removed, currently
- 【错误记录】约束布局报错 ( Missing Constraints in ConstraintLayout. This view is not constrained. It only has )
- ORA-00953: missing or invalid index name ORACLE 报错 故障修复 远程处理
- ORA-01166: file number string is larger than string (string) ORACLE 报错 故障修复 远程处理
- ORA-01977: Missing thread number ORACLE 报错 故障修复 远程处理
- ORA-02051: another session or branch in same transaction failed or finalized ORACLE 报错 故障修复 远程处理
- ORA-24780: cannot recover a transaction while in an existing transaction ORACLE 报错 故障修复 远程处理
- ORA-28555: pass-through SQL: required parameter missing or NULL ORACLE 报错 故障修复 远程处理
- ORA-29985: Query Registration is only supported for automatic undo management ORACLE 报错 故障修复 远程处理
- ORA-30400: identical JOIN KEY clauses ORACLE 报错 故障修复 远程处理
- ORA-32584: missing LOG keyword ORACLE 报错 故障修复 远程处理
- ORA-32606: missing NAV keyword in MODEL clause ORACLE 报错 故障修复 远程处理
- ORA-40119: nested columns incompatible with missing value treatment ORACLE 报错 故障修复 远程处理
- ORA-48228: Missing Define Call [string] ORACLE 报错 故障修复 远程处理
- ORA-56723: I/O request limit exceeded – session terminated ORACLE 报错 故障修复 远程处理
- MySQL Error number: MY-010153; Symbol: ER_RPL_REWRITEDB_MISSING_ARROW; SQLSTATE: HY000 报错 故障修复 远程处理
- MySQL Error number: MY-010546; Symbol: ER_RPL_SERVER_ID_MISSING; SQLSTATE: HY000 报错 故障修复 远程处理
- ORA-01978: Missing sequence number ORACLE 报错 故障修复 远程处理
- ORA-02438: Column check constraint cannot reference other columns ORACLE 报错 故障修复 远程处理
- MySQL Error number: MY-011967; Symbol: ER_IB_MSG_142; SQLSTATE: HY000 报错 故障修复 远程处理
- ORA-13368: simple polygon type has more than one exterior ring ORACLE 报错 故障修复 远程处理
- ORA-13983: Missing tag from HTML ORACLE 报错 故障修复 远程处理
- ORA-14155: missing PARTITION or SUBPARTITION keyword ORACLE 报错 故障修复 远程处理
- ORA-15513: cannot access the input directory ORACLE 报错 故障修复 远程处理
- ORA-18010: command missing mandatory CATEGORY keyword ORACLE 报错 故障修复 远程处理
- MySQL Error number: MY-012621; Symbol: ER_IB_MSG_796; SQLSTATE: HY000 报错 故障修复 远程处理
- MySQL Error number: MY-012671; Symbol: ER_IB_MSG_846; SQLSTATE: HY000 报错 故障修复 远程处理
- 关于plsqldeveloper打开报错解决方法MSVCR71.dll is missing from your compute详解数据库
- Oracle数据库报错:无效数据类型,如何解决?(oracle无效数据类型)