在python中执行命令行指令
from scrapy.cmdline import execute
if __name__ == '__main__':
execute(['scrapy','crawl','quotes'])
创建项目
python -m scrapy startproject tutorial
创建爬虫
python -m scrapy genspider quotes_spider quotes.toscrape.com
拼接url路径
python原生
from urllib.parse import urljoin
url = urljoin(prefix, href)
scrapy风格
url = resp.urljoin(href)
