ElasticSearch 的版本选择

提要：我是m1的芯片有些版本是不支持的，又考虑springboot支持的elasticsearch版本是7的版本，所以选择了7.1.3版本，这个版本基本上可以直接启动也不用过多的配置，对于新手，或者复习来说搭建难度还是比较小的，我算是踩了一堆坑吧，反正不要选择最新的版本就好了，虽然可以直接启动但是对于只想了解api的使用来说的话没必要。

对了mac用户最好别用brew下载docker 初级选手容易找不到开启的位置，可以直接去官网下载可视版本，虽然最后还是需要命令行来进行操作，但是会给一个很壮观的感受

下载地址：Past Releases of Elastic Stack Software | Elastic

ElasticSearch 和kibana 的搭建

方式1:直接载下来，然后运行这里直接是启动不来的，需要添加一个启动参数，我贴出来-其实主要就是 discovery.type=single-node 设置一下节点信息，不然的话会有启动的问题

方式2:使用docker pull下来命令docker pull elasticsearch:7.1.3

docker命令:docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -e "discovery.type=single-node" -d -p 9200:9200 -p 9300:9300 --name ES01 e082d8ac7e5e

ElasticSearch的Maven版本选择

没有标明版本号均采用springboot版本号进行管理

1: spring-boot-starter-parent 2.4.9

2:spring-boot-starter-data-elasticsearch

3:lombok

4:spring-boot-starter-test

5:fastjson 1.2.80

Kibana

基础知识介绍：索引：可以理解为一个库，就是叫索引库

类型：可以理解为为一个库里面的一张表（elastic7 被删除了全文只有一个_doc）

文档：可以理解为一行一行的数据

字段：可以理解为一列

1:如何创建一个索引

请求方式: 创建一个索引并且同时做一些映射～对于字段

PUT /dpyuyutest
{
  "mappings": {
    "properties": {
      "title":{
        "type": "keyword"
      },
      "name":{
        "type": "text"
      }
    }
  },
  "settings": {
    "number_of_replicas": 5, 
    "number_of_shards": 1
  }
}

返回参数：

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "dpyuyutest"
}

number_of_replicas：副本数量 number_of_shards：分片数量

2:查询一个索引

请求方式：

GET /dpyuyutest

响应参数：

{
  "dpyuyutest" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "name" : {
          "type" : "text"
        },
        "title" : {
          "type" : "keyword"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "dpyuyutest",
        "creation_date" : "1654445455453",
        "number_of_replicas" : "5",
        "uuid" : "UzkJB6SFQYW1pCLjfUWDvQ",
        "version" : {
          "created" : "7160299"
        }
      }
    }
  }
}

这里也是正常显示了一些mappings和settings 的设置

3:删除索引

请求方式：

DELETE /dpyuyutest

返回参数：

{
  "acknowledged" : true
}

4:添加文档

请求方式：

POST /dpyuyutest/_doc
{
  "title":"我是二号标题",
  "name":"bmw"
}

返回参数：

{
  "_index" : "dpyuyutest",
  "_type" : "_doc",
  "_id" : "vuquNIEBJMhLMVgDlISa",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 6,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

注意：这里需要注意的是，_doc是elasticsearch7之后全文唯一类型，所以添加的时候需要写上不写上的会报错。

5:查询指定索引全部文档

请求方式：

GET dpyuyutest/_search
{
  "query": {
    "match_all": {}
  }
}

返回参数：

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "veqtNIEBJMhLMVgDooRm",
        "_score" : 1.0,
        "_source" : {
          "title" : "我是一号标题",
          "name" : "bmw"
        }
      },
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "vuquNIEBJMhLMVgDlISa",
        "_score" : 1.0,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      }
    ]
  }
}

hits：里面是我添加进去的两个文档。

6:文档删除

请求方式：

DELETE /dpyuyutest/_doc/veqtNIEBJMhLMVgDooRm

返回参数：

{
  "_index" : "dpyuyutest",
  "_type" : "_doc",
  "_id" : "veqtNIEBJMhLMVgDooRm",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 6,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 1
}

DELETE /索引/唯一类型/id

7:match查询（分词查询）

请求方式：

GET /dpyuyutest/_search
{
  "query": {
    "match": {
      "name": "bmw"
    }
  }
}

//还可以这样写，制定查询精度
GET /dpyuyutest/_search
{
  "query": {
    "match": {
      "type":{
        "query": "b",
        "minimum_should_match": "75%" //是指最少匹配百分比
      }
    }
  }
}

返回参数：

{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "vuquNIEBJMhLMVgDlISa",
        "_score" : 0.2876821,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      }
    ]
  }
}

8:term查询（完全匹配，不分词）

请求方式：

GET /dpyuyutest/_search
{
  "query": {
    "term": {
      "name": "bmw"
    }
  }
}

返回参数：

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "vuquNIEBJMhLMVgDlISa",
        "_score" : 0.2876821,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      }
    ]
  }
}

9:boolean 查询（与或非）

请求方式：

GET /dpyuyutest/_search
{
  "query": {
   "bool": {
     "must": [
       {"term": {"name": {"value": "bmw"}}}
       ],
      "should": [
        {"match": {
          "title": "我们是二号标题"
        }}
      ],
      "must_not": [
        {"match": {"title": "54"}}
      ]
   }
  }
}

返回参数：

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.276251,
    "hits" : [
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "v-rBNIEBJMhLMVgD4oTh",
        "_score" : 1.276251,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      },
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "wOrCNIEBJMhLMVgD64Rt",
        "_score" : 1.276251,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      }
    ]
  }
}

must（与）子句必须出现在匹配的文档中，并将对分数起作用

must_not（非）子句不得出现在匹配的文档中。

shuold （或）子句应出现在匹配文档中。

10:简单的聚合查询

请求参数

GET /dpyuyutest/_search
{
  "aggs": {
    "agg_title": {
      "terms": {
        "field": "title.keyword",
        "size": 10
      }
    }
  }
}

返回参数：

{
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "v-rBNIEBJMhLMVgD4oTh",
        "_score" : 1.0,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      },
      {
        "_index" : "dpyuyutest",
        "_type" : "_doc",
        "_id" : "wOrCNIEBJMhLMVgD64Rt",
        "_score" : 1.0,
        "_source" : {
          "title" : "我是二号标题",
          "name" : "bmw"
        }
      }
    ]
  },
  "aggregations" : {
    "agg_title" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "我是二号标题",
          "doc_count" : 2
        }
      ]
    }
  }
}

这个参数： "field": "title.keyword", 在6版本的时候是不用加keyword参数的； agg_title：这个参数是自定义参数

ElasticSearch 的增删改查

1:配置类

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration;

/**
 * ElasticSearch 客户端配置
 */
@Configuration
public class RestClientConfig extends AbstractElasticsearchConfiguration {
    @Override
    @Bean
    public RestHighLevelClient elasticsearchClient() {
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "********"));  //es账号密码（默认用户名为elastic）
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                                new HttpHost("localhost", 9200, "http"))
                        .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                                httpClientBuilder.disableAuthCaching();
                                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                            }
                        }));
        return client;
    }
}

2:crud测试类

@SpringBootTest
public class ElasticTest {

    @Autowired
    public RestHighLevelClient client;

    // 测试索引的创建， Request PUT liuyou_index
    @Test
    public void testCreateIndex() throws IOException {
        CreateIndexRequest request = new CreateIndexRequest("dpyuyutest");
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        //   System.out.println(response);
        System.out.println(response.isAcknowledged());// 查看是否创建成功
        System.out.println(response);// 查看返回对象
        client.close();
    }

    // 测试获取索引，并判断其是否存在
    @Test
    public void testIndexIsExists() throws IOException {
        GetIndexRequest request = new GetIndexRequest("dpyuyutest");
        boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);
        System.out.println(exists);// 索引是否存在
        client.close();
    }

    // 索引删除
    @Test
    public void testDeleteIndex() throws IOException {
        DeleteIndexRequest request = new DeleteIndexRequest("dpyuyutest");
        AcknowledgedResponse response = client.indices().delete(request, RequestOptions.DEFAULT);
        System.out.println(response.isAcknowledged());// 是否删除成功
        client.close();
    }

    // 测试添加文档(先创建一个User实体类，添加fastjson依赖)
    @Test
    public void testAddDocument() throws IOException {
        // 创建一个User对象
        Book book = new Book("2","elasticsearch","1",2.0d,new Date(),new Date());
        // 创建请求
        IndexRequest request = new IndexRequest("dpyuyutest");
        // 制定规则 PUT /dpyuyutest/_doc/1
        request.id("1");// 设置文档ID
        request.timeout(TimeValue.timeValueMillis(1000));// request.timeout("1s")
        // 将我们的数据放入请求中
        request.source(JSON.toJSONString(book), XContentType.JSON);
        // 客户端发送请求，获取响应的结果
        IndexResponse response = client.index(request, RequestOptions.DEFAULT);
        System.out.println(response.status());// 获取建立索引的状态信息 CREATED
        System.out.println(response);// 查看返回内容 IndexResponse[index=liuyou_index,type=_doc,id=1,version=1,result=created,seqNo=0,primaryTerm=1,shards={"total":2,"successful":1,"failed":0}]
    }

    // 获取文档，判断是否存在 get /index/_doc/1
    @Test
    void testIsExists() throws IOException {
        GetRequest request = new GetRequest("dpyuyutest", "1");
        // 不获取返回的 _source 的上下文了
        request.fetchSourceContext(new FetchSourceContext(false));
        request.storedFields("_none_");
        boolean exists = client.exists(request, RequestOptions.DEFAULT);
        System.out.println(exists);
    }

    // 测试获得文档信息
    @Test
    public void testGetDocument() throws IOException {
        GetRequest request = new GetRequest("dpyuyutest","1");
        request.storedFields("");
        GetResponse response = client.get(request, RequestOptions.DEFAULT);
        System.out.println(response.getSourceAsString());// 打印文档内容
        System.out.println(request);// 返回的全部内容和命令是一样的
        client.close();
    }

    // 测试更新文档内容
    @Test
    public void testUpdateDocument() throws IOException {
        UpdateRequest request = new UpdateRequest("dpyuyutest", "1");
        Book book = new Book("2","elasticsearch","1",2.0d,new Date(),new Date());
        request.doc(JSON.toJSONString(book),XContentType.JSON);
        UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
        System.out.println(response.status()); // OK
        client.close();
    }

    // 删除文档
    @Test
    public void testDeleteDocument() throws IOException {
        DeleteRequest request = new DeleteRequest("dpyuyutest", "1");
        request.timeout("1s");
        DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
        System.out.println(response.status());// OK
    }

    // 批量插入数据
    @Test
    public void testBulk() throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
        bulkRequest.timeout("10s");
        ArrayList<Book> users = new ArrayList<>();
        users.add(new Book("2","elasticsearch","1",2.0d,new Date(),new Date()));
        users.add(new Book("3","elasticsearch","1",2.0d,new Date(),new Date()));
        users.add(new Book("4","elasticsearch","1",2.0d,new Date(),new Date()));
        users.add(new Book("5","elasticsearch","1",2.0d,new Date(),new Date()));
        users.add(new Book("6","elasticsearch","1",2.0d,new Date(),new Date()));

        // 批量请求处理
        for (int i = 0; i < users.size(); i++) {
            bulkRequest.add(
                    // 这里是数据信息
                    new IndexRequest("dpyuyutest")
                            .id(""+(i + 1)) // 没有设置id 会自定生成一个随机id
                            .source(JSON.toJSONString(users.get(i)),XContentType.JSON)
            );
        }
        BulkResponse bulk = client.bulk(bulkRequest, RequestOptions.DEFAULT);
        System.out.println(bulk.status());// ok
    }

    // 查询
// SearchRequest 搜索请求
// SearchSourceBuilder 条件构造
// HighlightBuilder 高亮
// TermQueryBuilder 精确查询
// MatchAllQueryBuilder
// xxxQueryBuilder ...
    @Test
    public void testSearch() throws IOException {
        // 1.创建查询请求对象
        SearchRequest searchRequest = new SearchRequest("dpyuyutest");
        // 2.构建搜索条件
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        // (1)查询条件 使用QueryBuilders工具类创建
        // 精确查询
        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("title", "elasticsearch");
        // 匹配查询
        //  MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
        AvgAggregationBuilder title = AggregationBuilders.avg("title");
        // (2)其他<可有可无>：（可以参考 SearchSourceBuilder 的字段部分）
        // 设置高亮
        searchSourceBuilder.highlighter(new HighlightBuilder());
        //        // 分页
        //        searchSourceBuilder.from();
        //        searchSourceBuilder.size();
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        // (3)条件投入
        searchSourceBuilder.query(termQueryBuilder);
        // 3.添加条件到请求
        searchRequest.source(searchSourceBuilder);
        // 4.客户端查询请求
        SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
        // 5.查看返回结果
        SearchHits hits = search.getHits();
        System.out.println(JSON.toJSONString(hits));
        System.out.println("=======================");
        for (SearchHit documentFields : hits.getHits()) {
            System.out.println(documentFields.getSourceAsMap());
        }
    }
//聚合查询，是可以查询到buckets 可以拿kibana 进行一个聚合查询 
    @Test
    public void Agg() throws IOException {
        //es聚合测试
        //构建bool查询
        SearchRequest searchRequest = new SearchRequest("dpyuyutest");
        TermsAggregationBuilder agg = AggregationBuilders.terms("agg_title").field("title");
       searchRequest.source().aggregation(agg);
        //构建聚合查询
        /*   .subAggregation(AggregationBuilders.stats("res_score").field("score"))
                .subAggregation(AggregationBuilders.histogram("avg_price").field("price").interval(500))*/
        //解析返回值

        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        //取出第一次聚合的terms
        Terms terms = searchResponse.getAggregations().get("agg_title");
        List<? extends Terms.Bucket> buckets = terms.getBuckets();
        Map<String,Object> resultMap = new HashMap<>();
        //在terms的buckets下，取出stats结果
        terms.getBuckets().forEach(item -> {
            System.out.println(item.getKey());
            System.out.println(item.getDocCount());
        });
        System.out.println(resultMap.toString());


    }

ElasticSearch 和kibana 搭建和api介绍

ElasticSearch 的版本选择

ElasticSearch 和kibana 的搭建

ElasticSearch的Maven版本选择

Kibana

1:如何创建一个索引

2:查询一个索引

3:删除索引

4:添加文档

5:查询指定索引全部文档

6:文档删除

7:match查询（分词查询）

8:term查询（完全匹配，不分词）

9:boolean 查询 （与或非）

10:简单的聚合查询

ElasticSearch 的增删改查

1:配置类

2:crud测试类

9:boolean 查询（与或非）