windows搭建elasticsearch集群+kibana并整合java

994 阅读4分钟
  • 环境准备
  • 环境搭建
  • 整合java代码

1.环境准备

版本说明

名称版本
elasticsearch7.8.0
kibana7.8.0
spring-boot-starter-data-elasticsearch2.3.1.RELEASE
jdk1.8

注意:elasticsearch 7.8 自带JDK11,也可以设置jdk

环境说明

window环境下的伪集群,三台实例。

名称http端口tcp端口
第一台92019300
第二台92029400
第三台92039500

kibana 使用默认

软件下载

ElasticSearch 下载地址

Kibana 下载地址

最好下载zip包

2.搭建Elasticsearch集群和Kibana

  • 解压elasticsearch-7.8.0-windows-x86_64

1.根目录创建data 文件夹

2.配置文件elasticsearch.yml

#集群名称
cluster.name: elasticsearch-cluster
#节点名称
node.name: node-1
#是不是有资格主节点
node.master: true
#是否存储数据
node.data: true
#最大集群节点数
node.max_local_storage_nodes: 3
#数据
path.data: C:/software/elasticsearch/elasticsearch-7.8.0-1/data
#日志
path.logs: C:/software/elasticsearch/elasticsearch-7.8.0-1/logs
#配置ip (如果需要外网访问,配置外网ip)
network.host: 127.0.0.1
http.port: 9201
transport.tcp.port: 9300
#其他两个节点地址(如果需要外网访问,配置外网ip)
discovery.seed_hosts: ["localhost:9300", "localhost:9400"]
#其他两个节点
cluster.initial_master_nodes: ["node-1", "node-2","node-3"]

3.完整配置文件

#集群名称
cluster.name: elasticsearch-cluster
#节点名称
node.name: node-1
#是不是有资格主节点
node.master: true
#是否存储数据
node.data: true
#最大集群节点数
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: C:/software/elasticsearch/elasticsearch-7.8.0-3/data
#
# Path to log files:
#
path.logs: C:/software/elasticsearch/elasticsearch-7.8.0-3/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 127.0.0.1
#
# Set a custom port for HTTP:
#
http.port: 9201
#
# For more information, consult the network module documentation.
transport.tcp.port: 9300
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["localhost:9300", "localhost:9400","localhost:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#配置跨域
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true
#开启x-pack验证
#xpack.security.enabled: true
#是否开启服务端 SSL 证书
#xpack.license.self_generated.type: basic
#xpack.security.transport.ssl.enabled: true
#xpack.security.transport.ssl.verification_mode: certificate
#xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
#xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

注意: 配置这里有坑,配置为yml文件,["node-1", "node-2", "node-3"] 注意前面有空格,有空格,有空格!!!, xpack.security.enabled: true 该配置如需要密码验证才打开

如果开启x-pack认证 遇到报DecoderException: javax.net.ssl.SSLHandshakeException: No available authentic异常

4.启动elasticsearch集群

  • 创建其他两个节点

复制 三个文件夹 elasticsearch-7.8.0-1 ,elasticsearch-7.8.0-2 ,elasticsearch-7.8.0-3

修改对对应elasticsearch.yml配置文件 分别有端口,tcp端口

分别运行三个elasticsearch.bat

集群验证 http://localhost:9201,http://localhost:9202

查看集群 http://localhost:9201/_cat/health?v 状态为green

  • 如果需要设置密码

    打开三台配置, xpack.security.enabled: true

    修改配置后重启集群,再进入http://localhost:9202 则需要密码了。

    设置密码 cmd 进入第一台的bin目录

    运行 elasticsearch-setup-passwords.bat interactive 自定义密码 或者 运行 elasticsearch-setup-passwords.bat auto 自动生成密码

5.kibana安装

解压zip,修改配置文件

server.port: 5601
server.host: "127.0.0.1"
server.name: "your-hostname"
elasticsearch.hosts: ["http://127.0.0.1:9202"]
elasticsearch.username: "elastic"
elasticsearch.password: "123456"

配置elasticsearch的master节点和密码

启动kibana ./bin/kibana.bat

启动稍微有点慢,耐心等待

http://127.0.0.1:5601 访问kibana界面

另外附上一个启动脚本

    @echo off
    title elasticsearch-cluster
    set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181
    set PATH=%JAVA_HOME%\bin;
    cd /d C:\software\elasticsearch\elasticsearch-7.8.0-1\bin
    start "elasticsearch-cluster:node-1" elasticsearch.bat
    cd /d C:\software\elasticsearch\elasticsearch-7.8.0-2\bin
    start "elasticsearch-cluster:node-2" elasticsearch.bat
    cd /d C:\software\elasticsearch\kibana-7.8.0-windows-x86_64\bin
    start "kibana" kibana.bat

创建后缀为 start.bat文件, 修改路径,点击启动start.bat 就不用一个一个去启动了

elasticsearch支持SQL了,kibana中的devtools界面查询

POST /_sql?format=txt
{
  "query": "SELECT id,district,province,street,streetNumber FROM address_components LIMIT 10"
}

用起来很方便的,谁用谁知道

3.整合JAVA

  • maven配置
       <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
            <version>2.3.1.RELEASE</version>
        </dependency>
  • application.yml文件
 spring:
  elasticsearch:
    rest:
      uris:
        - 127.0.0.1:9201
        - 127.0.0.1:9202
        - 127.0.0.1:9203
      username: elastic
      password: 123456
  • Java实体类对应ES的文档索引
@Document(indexName = "address_components")
@Data
public class AddressComponents {
    private String id;
    /**
     * 城市
     */
    private String city;
    /**
     * 区县
     */
    private String district;
    /**
     * 省份
     */
    private String province;
    /**
     * 路
     */
    private String street;
    /**
     * 号
     */
    private String streetNumber;
   }
  • Java对应操作ES JPA
public interface AddressComponentsResponsitory extends ElasticsearchRepository<AddressComponents, String> {
}
  • Java对应操作ES 调用
增加/修改
responsitory.save(addressComponents);
查询单个
responsitory.findById(id);
  • Java对应操作ES 调用 (高级查询)

注入操作类

	 @Autowired
    private ElasticsearchRestTemplate restTemplate;
    NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
            //条件查询
            .withQuery(QueryBuilders.termQuery("userId", userId))
            //排序
            .withSort(SortBuilders.fieldSort("updateTime").order(SortOrder.DESC))
            .build();
        SearchHits<TicketEs> search = this.restTemplate.search(searchQuery, TicketEs.class);
        List<TicketEs> list = (List<TicketEs>) SearchHitSupport.unwrapSearchHits(search);