docker安装Hadoop+HDFS

1.下载 Docker Compose 的当前稳定版本

curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

2.对二进制文件应用可执行权限

chmod +x /usr/local/bin/docker-compose

3.查看是否安装成功

docker-compose --version

编写docker-compose.yml

version: "2"
​
services:
  namenode:
    image: singularities/hadoop
    command: start-hadoop namenode
    hostname: namenode
    environment:
      HDFS_USER: hdfsuser
    ports:
      - "8020:8020"
      - "8088:8088"
      - "9870:9870"
      - "14000:14000"
      - "50070:50070"
      - "50075:50075"
      - "10020:10020"
      - "13562:13562"
      - "19888:19888"
  datanode:
    image: singularities/hadoop
    command: start-hadoop datanode namenode
    environment:
      HDFS_USER: hdfsuser
    links:
      - namenode

启动

docker-compose up -d

扩容节点

设置3个datanode 一个 namenode

docker-compose scale datanode=2

常见端口

• 50070：namenode的页面
• 50075：datanode的页面
• 50090：status的页面
• 9000：是客户端使用hdfs连接端口
• 8088：yarn http端口号