es - elasticsearch 自定义分析器 - 内建字符过滤器世界上并没有完美的程序，但是我们并不因此而沮丧，因

世界上并没有完美的程序，但是我们并不因此而沮丧，因为写程序就是一个不断追求完美的过程。

官网地址：www.elastic.co/guide/en/el…

自定义分析器主要由以下三部分组成 :

Character filters :
1. 作用 : 字符的增、删、改转换
2. 数量限制 : 可以有0个或多个
3. 内建字符过滤器 :
1. HTML Strip Character filter : 去除html标签
2. Mapping Character filter : 映射替换
3. Pattern Replace Character filter : 正则替换
Tokenizer :
1. 作用 : 分词
2. 数量限制 : 有且只能有一个
Token filters :
1. 作用 : 分词的增、删、改转换
2. 数量限制 : 可以有0个或多个

今天重点演示character filter字符过滤器，以下为使用实例 :

# HTML Strip Character filter
# 作用 : 去除html标签
# 配置项 :
#   1. escaped_tags : 要跳过的html元素
GET /_analyze
{
  "tokenizer": "keyword",
  "char_filter": [{
    "type" : "html_strip",
    "escaped_tags" : ["p"]
  }],
  "text": ["<html><p>this is my name - 这是我的名字</p></html>",
  "<p>I&apos;m so <b>happy</b>!</p>"]
}

# 结果
{
  "tokens" : [
    {
      "token" : """
<p>this is my name - 这是我的名字</p>
""",
      "start_offset" : 0,
      "end_offset" : 44,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "<p>I'm so happy!</p>",
      "start_offset" : 45,
      "end_offset" : 77,
      "type" : "word",
      "position" : 101
    }
  ]
}

# Mappging Charater filter
# 作用   : 替换映射的字符
# 配置项 :
#   1. mappings
#   2. mappings_path
GET /_analyze
{
  "tokenizer": "keyword",
  "char_filter": [{
    "type" : "mapping",
    "mappings" : [
        "٠ => 0",
        "١ => 1",
        "٢ => 2",
        "٣ => 3",
        "٤ => 4",
        "٥ => 5",
        "٦ => 6",
        "٧ => 7",
        "٨ => 8",
        "٩ => 9",
        "a => 97",
        "y => 是"
      ]
  }],
  "text": "My license plate is ٢٥٠١٥"
}

# 结果
{
  "tokens" : [
    {
      "token" : "M是 license pl97te is 25015",
      "start_offset" : 0,
      "end_offset" : 25,
      "type" : "word",
      "position" : 0
    }
  ]
}

# Pattern Replace Character filter
# 作用   : 正则替换
# 配置项 :
#   1. pattern     : 正则
#   2. replacement : 替换的词
#   3. flags       : java正则标记
GET /_analyze
{
  "tokenizer": "keyword",
  "char_filter": [{
    "type" : "pattern_replace",
    "pattern" : "\\d",
    "replacement" : "2"
  }, {
    "type" : "pattern_replace",
    "pattern" : "(\\d+)-(?=\\d)",
    "replacement" : "$1_"
  }],
  "text": ["12dsfds", "My credit card is 123-456-789"]
}

# 结果
{
  "tokens" : [
    {
      "token" : "22dsfds",
      "start_offset" : 0,
      "end_offset" : 7,
      "type" : "word",
      "position" : 0
    },
    {
      "token" : "My credit card is 222_222_222",
      "start_offset" : 8,
      "end_offset" : 37,
      "type" : "word",
      "position" : 101
    }
  ]
}

# character filter 组合自定义分析器
PUT /test_char_filter
{
  "settings": {"number_of_shards": 1,
    "analysis": {
      "analyzer": {
        "my_analyzer" : {
          "tokenizer" : "keyword",
          "char_filter" : [
            "my_char_filter_html",
            "my_char_filter_mapping",
            "my_char_filter_pattern"
            ]
        }
      },
      "char_filter": {
        "my_char_filter_html" : {
          "type" : "html_strip",
          "escaped_tags" : "p"
        },
        "my_char_filter_mapping" : {
          "type" : "mapping",
          "mappings" : [
              "a => 97",
              "b => 98"
            ]
        },
        "my_char_filter_pattern" : {
          "type" : "pattern_replace",
          "pattern" : "-",
          "replacement" : "_"
        }
      }
    }
  }
}

GET /test_char_filter/_analyze
{
  "analyzer": "my_analyzer",
  "text": ["this is a b c d 2-3-5-7 <p>hello</p> <span>good</span>"]
}

# 结果
{
  "tokens" : [
    {
      "token" : "this is 97 98 c d 2_3_5_7 <p>hello</p> good",
      "start_offset" : 0,
      "end_offset" : 54,
      "type" : "word",
      "position" : 0
    }
  ]
}