Elasticsearch高手进阶篇(11)
深度探秘搜索技术_案例实战基于dis_max实现best fields策略进行多字段搜索
1、为帖子数据增加content字段
POST /waws/article/_bulk
{ "update": { "_id": "1"} }
{ "doc" : {"content" : "i like to write best elasticsearch article"}}
{ "update": { "_id": "2"} }
{ "doc" : {"content" : "i think java is the best programming language"}}
{ "update": { "_id": "3"} }
{ "doc" : {"content" : "i am only an elasticsearch beginner"}}
{ "update": { "_id": "4"} }
{ "doc" : {"content" : "elasticsearch and hadoop are all very good solution, i am a beginner"}}
{ "update": { "_id": "5"} }
{ "doc" : {"content" : "spark is best big data solution based on scala ,an programming language similar to java"}}
2、搜索title或content中包含java或solution的帖子
下面这个就是multi-field搜索，多字段搜索
GET /waws/article/_search
{
"query": {
"bool": {
"should": [
{ "match": { "title": "java solution" }},
{ "match": { "content": "java solution" }}
]
}
}
}
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 0.8849759,
"hits": [
{
"_index": "waws",
"_type": "article",
"_id": "2",
"_score": 0.8849759,
"_source": {
"articleID": "KDKE-B-9947-#kL5",
"userID": 1,
"hidden": false,
"postDate": "2017-01-02",
"tag": [
"java"
],
"tag_cnt": 1,
"view_cnt": 50,
"title": "this is java blog",
"content": "i think java is the best programming language"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "4",
"_score": 0.7120095,
"_source": {
"articleID": "QQPX-R-3956-#aD8",
"userID": 2,
"hidden": true,
"postDate": "2017-01-02",
"tag": [
"java",
"elasticsearch"
],
"tag_cnt": 2,
"view_cnt": 80,
"title": "this is java, elasticsearch, hadoop blog",
"content": "elasticsearch and hadoop are all very good solution, i am a beginner"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "5",
"_score": 0.56008905,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2017-03-01",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "1",
"_score": 0.26742277,
"_source": {
"articleID": "XHDK-A-1293-#fJ3",
"userID": 1,
"hidden": false,
"postDate": "2017-01-01",
"tag": [
"java",
"hadoop"
],
"tag_cnt": 2,
"view_cnt": 30,
"title": "this is java and elasticsearch blog",
"content": "i like to write best elasticsearch article"
}
}
]
}
}
3、结果分析
-
期望的是doc5，结果是doc2,doc4排在了前面
- 计算每个document的relevance score：每个query的分数，乘以matched query数量，除以总query数量
-
算一下doc4的分数
- { "match": { "title": "java solution" }}，针对doc4，是有一个分数的
- { "match": { "content": "java solution" }}，针对doc4，也是有一个分数的
- 所以是两个分数加起来，比如说，1.1 + 1.2 = 2.3
- matched query数量 = 2
- 总query数量 = 2
- 2.3 * 2 / 2 = 2.3
-
算一下doc5的分数
- { "match": { "title": "java solution" }}，针对doc5，是没有分数的
- { "match": { "content": "java solution" }}，针对doc5，是有一个分数的
- 所以说，只有一个query是有分数的，比如2.3
- matched query数量 = 1
- 总query数量 = 2
- 2.3 * 1 / 2 = 1.15
doc5的分数 = 1.15 < doc4的分数 = 2.3
4、best fields策略，dis_max
-
best fields策略，就是说，搜索到的结果，应该是某一个field中匹配到了尽可能多的关键词，被排在前面；而不是尽可能多的field匹配到了少数的关键词，排在了前面
-
dis_max语法，直接取多个query中，分数最高的那一个query的分数即可
{ "match": { "title": "java solution" }}，针对doc4，是有一个分数的，1.1 { "match": { "content": "java solution" }}，针对doc4，也是有一个分数的，1.2 取最大分数，1.2
{ "match": { "title": "java solution" }}，针对doc5，是没有分数的 { "match": { "content": "java solution" }}，针对doc5，是有一个分数的，2.3 取最大分数，2.3
然后doc4的分数 = 1.2 < doc5的分数 = 2.3，所以doc5就可以排在更前面的地方，符合我们的需要
GET /forum/article/_search
{
"query": {
"dis_max": {
"queries": [
{ "match": { "title": "java solution" }},
{ "match": { "content": "java solution" }}
]
}
}
}
{
"took": 24,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 0.68640786,
"hits": [
{
"_index": "waws",
"_type": "article",
"_id": "2",
"_score": 0.68640786,
"_source": {
"articleID": "KDKE-B-9947-#kL5",
"userID": 1,
"hidden": false,
"postDate": "2017-01-02",
"tag": [
"java"
],
"tag_cnt": 1,
"view_cnt": 50,
"title": "this is java blog",
"content": "i think java is the best programming language"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "5",
"_score": 0.56008905,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2017-03-01",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "4",
"_score": 0.5565415,
"_source": {
"articleID": "QQPX-R-3956-#aD8",
"userID": 2,
"hidden": true,
"postDate": "2017-01-02",
"tag": [
"java",
"elasticsearch"
],
"tag_cnt": 2,
"view_cnt": 80,
"title": "this is java, elasticsearch, hadoop blog",
"content": "elasticsearch and hadoop are all very good solution, i am a beginner"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "1",
"_score": 0.26742277,
"_source": {
"articleID": "XHDK-A-1293-#fJ3",
"userID": 1,
"hidden": false,
"postDate": "2017-01-01",
"tag": [
"java",
"hadoop"
],
"tag_cnt": 2,
"view_cnt": 30,
"title": "this is java and elasticsearch blog",
"content": "i like to write best elasticsearch article"
}
}
]
}
}
Elasticsearch高手进阶篇(12)
深度探秘搜索技术_案例实战基于tie_breaker参数优化dis_max搜索效果
1、搜索title或content中包含java beginner的帖子
GET /waws/article/_search
{
"query": {
"dis_max": {
"queries": [
{ "match": { "title": "java beginner" }},
{ "match": { "body": "java beginner" }}
]
}
}
}
有些场景不是太好复现的，因为是这样，你需要尝试去构造不同的文本，然后去构造一些搜索出来，去达到你要的一个效果
可能在实际场景中出现的一个情况是这样的：
（1）某个帖子，doc1，title中包含java，content不包含java beginner任何一个关键词
（2）某个帖子，doc2，content中包含beginner，title中不包含任何一个关键词
（3）某个帖子，doc3，title中包含java，content中包含beginner
（4）最终搜索，可能出来的结果是，doc1和doc2排在doc3的前面，而不是我们期望的doc3排在最前面
dis_max，只是取分数最高的那个query的分数而已。
2、dis_max只取某一个query最大的分数，完全不考虑其他query的分数
3、使用tie_breaker将其他query的分数也考虑进去
tie_breaker参数的意义，在于说，将其他query的分数，乘以tie_breaker，然后综合与最高分数的那个query的分数，综合在一起进行计算 除了取最高分以外，还会考虑其他的query的分数
- tie_breaker的值，在0~1之间，是个小数，就ok
GET /waws/article/_search
{
"query": {
"dis_max": {
"queries": [
{ "match": { "title": "java beginner" }},
{ "match": { "body": "java beginner" }}
],
"tie_breaker": 0.3
}
}
}
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 3,
"max_score": 0.26742277,
"hits": [
{
"_index": "waws",
"_type": "article",
"_id": "1",
"_score": 0.26742277,
"_source": {
"articleID": "XHDK-A-1293-#fJ3",
"userID": 1,
"hidden": false,
"postDate": "2017-01-01",
"tag": [
"java",
"hadoop"
],
"tag_cnt": 2,
"view_cnt": 30,
"title": "this is java and elasticsearch blog",
"content": "i like to write best elasticsearch article"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "2",
"_score": 0.19856805,
"_source": {
"articleID": "KDKE-B-9947-#kL5",
"userID": 1,
"hidden": false,
"postDate": "2017-01-02",
"tag": [
"java"
],
"tag_cnt": 1,
"view_cnt": 50,
"title": "this is java blog",
"content": "i think java is the best programming language"
}
},
{
"_index": "waws",
"_type": "article",
"_id": "4",
"_score": 0.155468,
"_source": {
"articleID": "QQPX-R-3956-#aD8",
"userID": 2,
"hidden": true,
"postDate": "2017-01-02",
"tag": [
"java",
"elasticsearch"
],
"tag_cnt": 2,
"view_cnt": 80,
"title": "this is java, elasticsearch, hadoop blog",
"content": "elasticsearch and hadoop are all very good solution, i am a beginner"
}
}
]
}
}
总结：
- dis_max:只会考虑相关度最大的那个TF/IDF的数值作为score,选中的那个query排在最前面，最匹配
- tie_breaker:相关度最大的那个TF/IDF的数值的那个query所代表的数据匹配 + 其他的匹配的TF/IDF的数值 * tie_breaker比例，最终作为score的数值，更加综合
