前言
目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话,给个关注❤️ ~
本节给大家讲一下文档高级查询部分的内容~
本文偏实战一些,好了, 废话不多说直接开整吧~
文档操作
本节继续使用上节的项目,在api下新建DocSearchApi类,为了方便演示,我们使用之前建好的索引req_log
查询更新
上节给大家讲的是通过doc_id来进行更新,在真实场景中显然不好使,因为我们不可能记得每一个doc_id,往往是通过条件来更新,在java中如何操作呢?下面一起来看下
- 首先构建
UpdateByQueryRequest,可以使用QueryBuilder来构建查询条件,然后将其添加到UpdateByQueryRequest中,下面的示例中,我们定义了查询path字段进行条件匹配
/**
* 通过查询进行更新
* @throws IOException
*/
@Test
public void updateByQuery() throws IOException {
UpdateByQueryRequest request = new UpdateByQueryRequest(index);
request.setQuery(QueryBuilders.matchQuery("path", "/api/post/4/update"));
}
- 执行脚本更新,将
times字段进行更新操作,+10因为之前我建的times是text类型,所以这里加完之后为12,010
request.setScript(new Script(ScriptType.INLINE, "painless", "ctx._source.times += params.count", Collections.singletonMap("count", 10)));
- 执行请求操作
BulkByScrollResponse response = client.updateByQuery(request, RequestOptions.DEFAULT);
if(!response.isTimedOut()) {
log.info("status >>>> {}", String.valueOf(response.getStatus()));
// BulkIndexByScrollResponse[sliceId=null,updated=1,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s]
}
我们可以看到响应的结果,updated=1表示我们更新成功了,也可以 返回kibana查下数据有没有变化, 完整示例:
@Slf4j
@RunWith(SpringRunner.class)
@SpringBootTest(classes = { EsStudyApplication.class })
public class DocSearchApi {
/**
* es 索引
*/
public static final String index = "req_log";
@Autowired
private RestHighLevelClient client;
/**
* 通过查询进行更新
* @throws IOException
*/
@Test
public void updateByQuery() throws IOException {
// 构建UpdateByQueryRequest。可以使用QueryBuilder来构建查询条件,然后将其添加到UpdateByQueryRequest中
UpdateByQueryRequest request = new UpdateByQueryRequest(index);
request.setQuery(QueryBuilders.matchQuery("path", "/api/post/4/update"));
// 更新times字段 +10 因为之前我建的times是text类型,所以这里加完之后为 '12,010'
request.setScript(new Script(ScriptType.INLINE, "painless", "ctx._source.times += params.count", Collections.singletonMap("count", 10)));
BulkByScrollResponse response = client.updateByQuery(request, RequestOptions.DEFAULT);
if(!response.isTimedOut()) {
log.info("status >>>> {}", String.valueOf(response.getStatus()));
// BulkIndexByScrollResponse[sliceId=null,updated=1,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s]
}
}
}
通过上边的例子,我们了解到了QueryBuilders, 下面我们就一起看下今天的主角Search API
Search API & 查询所有
接下来看一个例子,我们要将所有文档查询出来,并输出文档数据, api很简单,一起来看下吧~
/**
* 查询全部
* @throws IOException
*/
@Test
public void searchAll() throws IOException {
SearchRequest searchRequest = new SearchRequest(index);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchRequest.source(searchSourceBuilder);
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
}
下面看下如何获取查询的数据
if(!response.isTimedOut()) {
log.info("status >>>> {}", response.status());
SearchHits hits = response.getHits();
TotalHits totalHits = hits.getTotalHits();
long numHits = totalHits.value;
log.info("numHits >>>> {}", numHits);
TotalHits.Relation relation = totalHits.relation;
float maxScore = hits.getMaxScore();
log.info("maxScore >>>> {}", maxScore);
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
String index = hit.getIndex();
log.info("index >>>> {}", index);
String id = hit.getId();
log.info("id >>>> {}", id);
float score = hit.getScore();
log.info("score >>>> {}", score);
String sourceAsString = hit.getSourceAsString();
log.info("sourceAsString >>>> {}", sourceAsString);
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String path = (String) sourceAsMap.get("path");
log.info("path >>>> {}", path);
}
}
通过代码来看,字段是不是很熟悉,都是之前给大家讲的
返回结果:
2023-03-01 10:21:04.977 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : status >>>> OK
2023-03-01 10:21:04.978 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : numHits >>>> 29
2023-03-01 10:21:04.978 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : maxScore >>>> 1.0
2023-03-01 10:21:04.978 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.978 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> GUK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.978 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.981 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":80,"method":"GET","path":"/api/post/1","created":"2023-02-09"}
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/1
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> GkK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":30,"method":"GET","path":"/api/post/2","created":"2023-02-07"}
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/2
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> G0K3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":20,"method":"GET","path":"/api/post/3","created":"2023-02-08"}
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/3
2023-03-01 10:21:04.987 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> HEK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":120,"method":"GET","path":"/api/post/20","created":"2023-02-06"}
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/20
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> HUK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":150,"method":"GET","path":"/api/post/1","created":"2023-02-05"}
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/1
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> HkK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":80,"method":"GET","path":"/api/post/3","created":"2023-02-04"}
2023-03-01 10:21:04.988 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/3
2023-03-01 10:21:04.989 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.989 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> H0K3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.989 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.989 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":960,"method":"GET","path":"/api/post/6","created":"2023-02-03"}
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/6
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> IEK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":9000,"method":"GET","path":"/api/post/8","created":"2023-02-02"}
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/8
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> IUK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":1300,"method":"GET","path":"/api/post/6","created":"2023-02-01"}
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/6
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : id >>>> IkK3NIYBdXrpvlCF01bz
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.0
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"times":400,"method":"GET","path":"/api/post/4","created":"2023-02-10"}
2023-03-01 10:21:04.990 INFO 15872 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/4
同样的,search支持异步查询:
client.searchAsync(searchRequest, RequestOptions.DEFAULT, listener);
ActionListener<SearchResponse> listener = new ActionListener<SearchResponse>() {
@Override
public void onResponse(SearchResponse searchResponse) {
}
@Override
public void onFailure(Exception e) {
}
};
Search API & 分页
下面一起看下如何进行分页查询
/**
* 分页查询
* @throws IOException
*/
@Test
public void searchByPage() throws IOException {
SearchRequest searchRequest = new SearchRequest(index);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
// 分页
searchSourceBuilder.from(0);
searchSourceBuilder.size(5);
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
searchRequest.source(searchSourceBuilder);
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
// ....
}
跟之前讲的from和size一样
Search API & QueryBuilders
接下来,我们就看下这个QueryBuilders
QueryBuilders是Elasticsearch提供的一个Java API,用于构建各种类型的查询条件。它提供了丰富的方法和类,用于构建常见的查询条件,例如term、match、range等, 跟我们之前讲的一样,只不过封了一道。
以下是QueryBuilders常用的几个方法:
-
termQuery(String name, Object value):创建一个精确匹配的查询条件,用于查找指定字段中包含指定值的文档。 -
matchQuery(String field, Object text):创建一个全文搜索的查询条件,用于查找指定字段中包含指定文本的文档。 -
boolQuery():创建一个bool查询条件,用于组合多个查询条件,例如must、should、must_not等。 -
rangeQuery(String field):创建一个范围查询条件,用于查找指定字段中的值在指定范围内的文档。 -
existsQuery(String field):创建一个存在查询条件,用于查找指定字段存在的文档。 -
idsQuery(String... ids):创建一个ID查询条件,用于查找指定ID的文档。 -
wildcardQuery(String name, String value):创建一个通配符查询条件,用于查找指定字段中符合通配符模式的文档。 -
fuzzyQuery(String name, Object value):创建一个模糊查询条件,用于查找指定字段中与指定值相似的文档。
这些方法和类都可以用于构建查询条件,并通过QueryBuilder类的build()方法生成查询条件的实例,用于在搜索请求中使用。
接下来,通过一个小案例给大家演示一下:
查询method为POST,times范围为0到100的数据
/**
* 分页查询
* @throws IOException
*/
@Test
public void searchByQuery() throws IOException {
SearchRequest searchRequest = new SearchRequest(index);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
// 精确匹配
QueryBuilder termQueryBuilder = QueryBuilders.termQuery("method", "POST");
// 范围查询
QueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("times").from(0).to(100);
// 组合条件查询
QueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
.must(termQueryBuilder)
.filter(rangeQueryBuilder);
searchSourceBuilder.query(boolQueryBuilder);
searchRequest.source(searchSourceBuilder);
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
if(!response.isTimedOut()) {
log.info("status >>>> {}", response.status());
SearchHits hits = response.getHits();
TotalHits totalHits = hits.getTotalHits();
long numHits = totalHits.value;
log.info("numHits >>>> {}", numHits);
TotalHits.Relation relation = totalHits.relation;
float maxScore = hits.getMaxScore();
log.info("maxScore >>>> {}", maxScore);
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
String index = hit.getIndex();
log.info("index >>>> {}", index);
String id = hit.getId();
log.info("id >>>> {}", id);
float score = hit.getScore();
log.info("score >>>> {}", score);
String sourceAsString = hit.getSourceAsString();
log.info("sourceAsString >>>> {}", sourceAsString);
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String path = (String) sourceAsMap.get("path");
log.info("path >>>> {}", path);
}
}
}
返回结果:
// 2023-03-01 10:54:02.706 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : status >>>> OK
// 2023-03-01 10:54:02.707 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : numHits >>>> 5
// 2023-03-01 10:54:02.708 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : maxScore >>>> 1.2144442
// 2023-03-01 10:54:02.708 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
// 2023-03-01 10:54:02.708 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : id >>>> 2
// 2023-03-01 10:54:02.708 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.2144442
// 2023-03-01 10:54:02.713 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"path":"/api/post/2/update","times":"40","method":"POST","created":"2023-02-28"}
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/2/update
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : id >>>> 3
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.2144442
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"method":"POST","times":"90","path":"/api/post/3/update","created":"2023-02-28"}
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/3/update
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : id >>>> 1
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.2144442
// 2023-03-01 10:54:02.723 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"path":"/api/post/1/update1","times":"60","method":"POST","created":"2023-02-28"}
// 2023-03-01 10:54:02.724 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/1/update1
// 2023-03-01 10:54:02.724 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
// 2023-03-01 10:54:02.724 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : id >>>> GaXWloYBBkiEpNgm4EUT
// 2023-03-01 10:54:02.724 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.2144442
// 2023-03-01 10:54:02.724 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"method":"POST","times":"60","path":"/api/post/5/update","created":"2023-02-28"}
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/5/update
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : index >>>> req_log
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : id >>>> GqXWloYBBkiEpNgm4EUT
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : score >>>> 1.2144442
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : sourceAsString >>>> {"method":"POST","times":"60","path":"/api/post/6/update","created":"2023-02-28"}
// 2023-03-01 10:54:02.725 INFO 17484 --- [ main] com.es.all.api.DocSearchApi : path >>>> /api/post/6/update
其它api大家可以自己试试,就不一一给大家演示了,大家可以试着写一个复杂点的查询,方法都是一样的
结束语
到此,es专题系列教程就结束了,为什么不接着讲api?,其实RestHighLevelClien的Java Api非常好用,用起来很顺手,大家可以查阅官方文档。根据自己的需求寻找合适的api,这个不用全部记住,用到查文档就可以了,知道是怎么一回事就可以了。
官方文档地址也给大家贴出来, [https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html),文档写的还是比较清楚的
下节我们进入Kafka专题的学习~
本着把自己知道的都告诉大家,如果本文对您有所帮助,点赞+关注鼓励一下呗~
相关文章
项目源码(源码已更新 欢迎star⭐️)
往期并发编程内容推荐
- Java多线程专题之线程与进程概述
- Java多线程专题之线程类和接口入门
- Java多线程专题之进阶学习Thread(含源码分析)
- Java多线程专题之Callable、Future与FutureTask(含源码分析)
- 面试官: 有了解过线程组和线程优先级吗
- 面试官: 说一下线程的生命周期过程
- 面试官: 说一下线程间的通信
- 面试官: 说一下Java的共享内存模型
- 面试官: 有了解过指令重排吗,什么是happens-before
- 面试官: 有了解过volatile关键字吗 说说看
- 面试官: 有了解过Synchronized吗 说说看
- Java多线程专题之Lock锁的使用
- 面试官: 有了解过ReentrantLock的底层实现吗?说说看
- 面试官: 有了解过CAS和原子操作吗?说说看
- Java多线程专题之线程池的基本使用
- 面试官: 有了解过线程池的工作原理吗?说说看
- 面试官: 线程池是如何做到线程复用的?有了解过吗,说说看
- 面试官: 阻塞队列有了解过吗?说说看
- 面试官: 阻塞队列的底层实现有了解过吗? 说说看
- 面试官: 同步容器和并发容器有用过吗? 说说看
- 面试官: CopyOnWrite容器有了解过吗? 说说看
- 面试官: Semaphore在项目中有使用过吗?说说看(源码剖析)
- 面试官: Exchanger在项目中有使用过吗?说说看(源码剖析)
- 面试官: CountDownLatch有了解过吗?说说看(源码剖析)
- 面试官: CyclicBarrier有了解过吗?说说看(源码剖析)
- 面试官: Phaser有了解过吗?说说看
- 面试官: Fork/Join 有了解过吗?说说看(含源码分析)
- 面试官: Stream并行流有了解过吗?说说看