ES搜索排序,文档相关度评分介绍——Field-length norm
Field-length norm
How long is the field? The shorter the field, the higher the weight. If a term appears in a short field, such as a title
field, it is more likely that the content of that field is about the term than if the same term appears in a much bigger body
field. The field length norm is calculated as follows:
norm(d) = 1 / √numTerms
![](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/callouts/1.png)
While the field-length norm is important for full-text search, many other fields don’t need norms. Norms consume approximately 1 byte per string
field per document in the index, whether or not a document contains the field. Exact-value not_analyzed
string fields have norms disabled by default, but you can use the field mapping to disable norms on analyzed
fields as well:
PUT /my_index
{
"mappings": {
"doc": {
"properties": {
"text": {
"type": "string",
"norms": { "enabled": false }
}
}
}
}
}
This field will not take the field-length norm into account. A long field and a short field will be scored as if they were the same length. |
For use cases such as logging, norms are not useful. All you care about is whether a field contains a particular error code or a particular browser identifier. The length of the field does not affect the outcome. Disabling norms can save a significant amount of memory.
These three factors—term frequency, inverse document frequency, and field-length norm—are calculated and stored at index time. Together, they are used to calculate the weight of a single term in a particular document.
![Tip](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/tip.png)
When we refer to documents in the preceding formulae, we are actually talking about a field within a document. Each field has its own inverted index and thus, for TF/IDF purposes, the value of the field is the value of the document.
When we run a simple term
query with explain
set to true
(see Understanding the Score), you will see that the only factors involved in calculating the score are the ones explained in the preceding sections:
PUT /my_index/doc/1
{ "text" : "quick brown fox" }
GET /my_index/doc/_search?explain
{
"query": {
"term": {
"text": "fox"
}
}
}
The (abbreviated) explanation
from the preceding request is as follows:
weight(text:fox in 0) [PerFieldSimilarity]: 0.15342641
![](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/callouts/1.png)
result of: fieldWeight in 0 0.15342641 product of: tf(freq=1.0), with freq of 1: 1.0
![](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/callouts/2.png)
idf(docFreq=1, maxDocs=1): 0.30685282
![](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/callouts/3.png)
fieldNorm(doc=0): 0.5
![](https://www.elastic.co/guide/en/elasticsearch/guide/current/images/icons/callouts/4.png)
The final |
|
The term |
|
The inverse document frequency of |
|
The field-length normalization factor for this field. |
Of course, queries usually consist of more than one term, so we need a way of combining the weights of multiple terms. For this, we turn to the vector space model.
相关文章
- ES-通过Sense插件方便的操作ES
- MySQL如何实时同步数据到ES?试试这款阿里开源的神器!
- TextureView+SurfaceTexture+OpenGL ES来播放视频(三)
- OpenGL ES着色器语言之变量和数据类型(二)(官方文档第四章)
- OpenGL ES着色器语言之着色概览(官方文档)
- 如何使用Android中的OpenGL ES媒体效果
- Linux服务集群部署实战--MySQL、Redis、ES、RocketMQ、Zookeeper
- 使用Adobe Lifecycle ES将若干个word合并成一个PDF
- 音视频开发(十二):OpenGL ES之纹理
- ElasticSearch---es用should表示or的逻辑
- ES profile 性能优化用——返回各个shard的耗时
- ES 断路器——本质上保护OOM提前抛出异常而已
- ES shrink ——一般是结合rollover一起使用的,一开始没有看懂官方shrink文档,当看了这个之后就明白了
- ES transport client底层是netty实现,netty本质上是异步方式,但是netty自身可以使用sync或者await(future超时机制)来实现类似同步调用!因此,ES transport client可以同步调用也可以异步(不过底层的socket必然是异步实现)
- ES API 备忘
- ES搜索排序,文档相关度评分介绍——TF-IDF—term frequency, inverse document frequency, and field-length norm—are calculated and stored at index time.
- ES _source字段介绍——json文档,去掉的话无法更新部分文档,最重要的是无法reindex
- How to reduce Index size on disk?减少ES索引大小的一些小手段
- ES里设置索引中倒排列表仅仅存文档ID——采用docs存储后可以降低pos文件和cfs文件大小
- ES mlockall作用——preventing that memory from being paged to the swap area
- ElasticSearch_12_ES的高性能设计