Es snapshot hdfs存储插件使用（翻译）

环境

Es5.6.X
CDH 6.0.x
Hadoop 3.0.0

下载插件

sudo bin/elasticsearch-plugin install repository-hdfs

插件必须安装在集群的每一个节点,安装后需要重启.

移除插件

移除插件之前必须先停止节点

sudo bin/elasticsearch-plugin remove repository-hdfs

在hdfs上创建es用户

adduser elasticsearch
cd /$hadoop_home/bin
./hdfs dfs -mkdir -p /user/elasticsearch
./hdfs dfs -chown elasticsearch:hdfs /user/elasticsearch

调用rest api添加配置

PUT _snapshot/my_hdfs_repository
{
  "type": "hdfs",
  "settings": {
    "uri": "hdfs://namenode:9000/",
    "path": "elasticsearch/respositories/my_hdfs_repository",
    "conf.dfs.client.read.shortcircuit": "true"
  }
}

配置	说明	是否必须
uri	hdfs访问地址	Required
path	文件存储、恢复路径	Required
load_defaults	是否加载hadoop默认配置	Enabled by default
conf.<key>	添加Hadoop配置	Optional
compress	是否压缩metadata（index mapping and settings）	default true
chunk_size	对数据文件分块，i.e. 1g, 10m, 5k	默认NULL（无限制块大小）
security.principal	连接受保护hadoop使用	simple（default），kerberos

对索引进行快照

对所有开启的索引进行备份
#wait_for_completion=true快照完成后返回，false快照初始化完成后返回
PUT /_snapshot/my_backup/snapshot_1?wait_for_completion=true

自定义索引进行备份
PUT /_snapshot/my_backup/snapshot_1
{
  "indices": "index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": false
}

获取快照信息
GET /_snapshot/my_backup/snapshot_1

获取所有存储的快照
GET /_snapshot/my_backup/_all

删除快照
DELETE /_snapshot/my_backup/snapshot_1

删除快照存储
DELETE /_snapshot/my_backup

恢复

恢复snapshot_1下所有的索引
POST /_snapshot/my_backup/snapshot_1/_restore

恢复指定索引
POST /_snapshot/my_backup/snapshot_1/_restore
{
  "indices": "index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": true, //
  "rename_pattern": "index_(.+)", //对恢复索引重命名
  "rename_replacement": "restored_index_$1", //对恢复索引重命名
  "include_aliases": false  //默认false，不恢复别名
}

还原并覆盖属性
POST /_snapshot/my_backup/snapshot_1/_restore
{
  "indices": "index_1",
  "index_settings": {
    "index.number_of_replicas": 0
  },
  "ignore_index_settings": [
    "index.refresh_interval"
  ]
}

快照状态

GET _snapshot/_status
GET _snapshot/my_backup/_status
GET _snapshot/my_backup/snapshot_1/_status
GET _snapshot/my_backup/snapshot_1,snapshot_2/_status

删除进行中的快照/恢复操作
DELETE _snapshot/my_backup/snapshot_1