优化Elasticsearch性能

在使用Elasticsearch查询新闻文档时，发现查询结果返回时间较长，大约需要850毫秒。目标是减少查询时间，提高查询效率。

查询代码如下：

def search_news(term, limit, page, flagged_articles):
    query = {
        "query": {
            "from": page*limit,
            "size": limit,
            "multi_match" : {
                "query" : term,
                "fields" : [ "title^3" , "category^5" , "entities.name^5", "art_text^1", "summary^1"]
            }
        },
        "filter" : {
            "not" : {
                "filter" : {
                    "ids" : {
                        "values" : flagged_articles
                    }
                },
                "_cache" : True
            }
        }
    }
    es_query = json_util.dumps(query)
    uri = 'http://localhost:9200/newsidx/_search'
    r = requests.get(uri, data=es_query)
    results = json.loads( r.text )
    data = []
    for res in results['hits']['hits']:
        data.append(res['_source'])

    return data

索引映射如下：

{
  "news": {
    "properties": {
      "actual_rank": {
        "type": "long"
      },
      "added": {
        "type": "date",
        "format": "dateOptionalTime"
      },
      "api_id": {
        "type": "long"
      },
      "art_text": {
        "type": "string"
      },
      "category": {
        "type": "string"
      },
      "downvotes": {
        "type": "long"
      },
      "entities": {
        "properties": {
          "etype": {
            "type": "string"
          },
          "name": {
            "type": "string"
          }
        }
      },
      "flags": {
        "properties": {
          "a": {
            "type": "long"
          },
          "b": {
            "type": "long"
          },
          "bad_image": {
            "type": "long"
          },
          "c": {
            "type": "long"
          },
          "d": {
            "type": "long"
          },
          "innapropiate": {
            "type": "long"
          },
          "irrelevant_info": {
            "type": "long"
          },
          "miscategorized": {
            "type": "long"
          }
        }
      },
      "media": {
        "type": "string"
      },
      "published": {
        "type": "string"
      },
      "published_date": {
        "type": "date",
        "format": "dateOptionalTime"
      },
      "show": {
        "type": "boolean"
      },
      "source": {
        "type": "string"
      },
      "source_rank": {
        "type": "double"
      },
      "summary": {
        "type": "string"
      },
      "times_showed": {
        "type": "long"
      },
      "title": {
        "type": "string"
      },
      "top_entities": {
        "properties": {
          "einfo_test": {
            "type": "string"
          },
          "etype": {
            "type": "string"
          },
          "name": {
            "type": "string"
          }
        }
      },
      "tweet_article_poster": {
        "type": "string"
      },
      "tweet_favourites": {
        "type": "long"
      },
      "tweet_retweets": {
        "type": "long"
      },
      "tweet_user_rank": {
        "type": "double"
      },
      "upvotes": {
        "type": "long"
      },
      "url": {
        "type": "string"
      }
    }
  }
}

2. 解决方案

• 调整查询语句

将查询语句中的 not 过滤器替换为 bool/must_not 过滤器，并将其设置为缓存。

{
    "query": {
        "filtered": {
            "query": {
                "multi_match" : {
                    "query" : term,
                    "fields" : [ "title^3" , "category^5" , "entities.name^5", "art_text^1", "summary^1"]
                }
            },
            "filter" : {
                "bool" : {
                    "must_not" : {
                        "ids" : {"values" : flagged_articles}
                    }
                }
            }
        }
    }
    "from": page * limit,
    "size": limit,
}

• 优化索引映射

为索引字段添加权重值，并使用 bool 查询而不是 multi_match 查询。

{
  "news": {
    "properties": {
      "title": {
        "boost": 3.0,
        "type": "string"
      },
      "category": {
        "boost": 5.0,
        "type": "string"
      },
      "art_text": {
        "type": "string"
      },
      "summary": {
        "type": "string"
      },
      // ...
    }
  }
}

"query": {
  "bool" : {
    "should" : [ {
      "field" : {
        "title" : term
      }
    }, {
      "field" : {
        "category" : term
      }
    } ],
    "must_not" : {
      "ids" : {"values" : flagged_articles}
    }
  }
} 
"from": page * limit,
"size": limit

• 启用缓存

在 bool/must_not 过滤器上启用缓存。