关于Crawlab在docker下的配置

810 阅读1分钟

问题描述

我最近在弄一个爬虫的项目,需要使用Crawlab这个框架。于是乎便开始学习。但是在配置框架的过程中,遇到了一个问题:就是使用非docker-compose方式配置时,虽然可以进入到Crawlab的登录界面,但是输入默认用户名和密码后,点击登录,一直在转圈;如果使用docker-compose的方式配置,在输入命令docker-compose后,会报错。(docker配置方式

解决办法

在CentOS的环境下

1.逐个输入如下命令,其中'${USER}'替换成你的CentOS的用户名,如我的就是root

mkdir -p /opt/docker
chown ${USER} -R /opt/docker
mkdir -p /opt/docker/crawlab/data/mongodb
mkdir -p /opt/docker/crawlab/data/redis
mkdir -p /opt/docker/crawlab/logs/crawlab
mkdir -p /opt/docker/crawlab/spiders
mkdir -p /opt/docker/crawlab/tmp
touch /opt/docker/crawlab/docker-compose.yml
vim /opt/docker/crawlab/docker-compose.yml

2.编写docversion: '3.3'(docker-compose内容)。这里需要注意,CRAWLAB_API_ADDRESS: "192.168.99.100:8000"指的是你的CentOS的IP地址。

version: '3.3'
services:
  master: 
    image: tikazyq/crawlab:latest
    container_name: crawlab-master
    environment:
      CRAWLAB_API_ADDRESS: "192.168.99.100:8000"
      CRAWLAB_SERVER_MASTER: "Y"
      CRAWLAB_MONGO_HOST: "mongo"
      CRAWLAB_REDIS_ADDRESS: "redis"
    ports:    
      - "8080:8080" # frontend
      - "8000:8000" # backend
    depends_on:
      - mongo
      - redis
  worker:
    image: tikazyq/crawlab:latest
    container_name: crawlab-worker
    environment:
      CRAWLAB_SERVER_MASTER: "N"
      CRAWLAB_MONGO_HOST: "mongo"
      CRAWLAB_REDIS_ADDRESS: "redis"
    depends_on:
      - mongo
      - redis
  mongo:
    image: mongo:latest
    restart: always
    ports:
      - "27017:27017"
  redis:
    image: redis:latest
    restart: always
    ports:
      - "6379:6379"

3.进入目录/opt/docker/crawlab/,在终端输入nohup docker-compose up > /tmp/crawlabs.log 2>&1 &

至此完成