问题描述
我最近在弄一个爬虫的项目,需要使用Crawlab这个框架。于是乎便开始学习。但是在配置框架的过程中,遇到了一个问题:就是使用非docker-compose方式配置时,虽然可以进入到Crawlab的登录界面,但是输入默认用户名和密码后,点击登录,一直在转圈;如果使用docker-compose的方式配置,在输入命令docker-compose后,会报错。(docker配置方式)
解决办法
在CentOS的环境下
1.逐个输入如下命令,其中'${USER}'替换成你的CentOS的用户名,如我的就是root
mkdir -p /opt/docker
chown ${USER} -R /opt/docker
mkdir -p /opt/docker/crawlab/data/mongodb
mkdir -p /opt/docker/crawlab/data/redis
mkdir -p /opt/docker/crawlab/logs/crawlab
mkdir -p /opt/docker/crawlab/spiders
mkdir -p /opt/docker/crawlab/tmp
touch /opt/docker/crawlab/docker-compose.yml
vim /opt/docker/crawlab/docker-compose.yml
2.编写docversion: '3.3'(docker-compose内容)。这里需要注意,CRAWLAB_API_ADDRESS: "192.168.99.100:8000"指的是你的CentOS的IP地址。
version: '3.3'
services:
master:
image: tikazyq/crawlab:latest
container_name: crawlab-master
environment:
CRAWLAB_API_ADDRESS: "192.168.99.100:8000"
CRAWLAB_SERVER_MASTER: "Y"
CRAWLAB_MONGO_HOST: "mongo"
CRAWLAB_REDIS_ADDRESS: "redis"
ports:
- "8080:8080" # frontend
- "8000:8000" # backend
depends_on:
- mongo
- redis
worker:
image: tikazyq/crawlab:latest
container_name: crawlab-worker
environment:
CRAWLAB_SERVER_MASTER: "N"
CRAWLAB_MONGO_HOST: "mongo"
CRAWLAB_REDIS_ADDRESS: "redis"
depends_on:
- mongo
- redis
mongo:
image: mongo:latest
restart: always
ports:
- "27017:27017"
redis:
image: redis:latest
restart: always
ports:
- "6379:6379"
3.进入目录/opt/docker/crawlab/,在终端输入nohup docker-compose up > /tmp/crawlabs.log 2>&1 &
至此完成