1 Star 1 Fork 3

yuzhishui/biliob-spider

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
start_spider_task.sh 757 Bytes
一键复制 编辑 原始数据 按行查看 历史
见齐 提交于 2020-02-28 21:31 . FEATURE: Get reply data and jannchie inde
#!/bin/bash
simpyders=(author video)
for spider in ${simpyders[@]}
do
ps -ef | grep $spider | grep -v grep | cut -c 9-15 | xargs kill -9
nohup python ./spiders/$spider.py -L INFO 1>log-$spider.out 2>&1 &
done
spiders=(bangumiAndDonghua authorAutoAdd videoAutoAdd site online)
for var in ${spiders[@]}
do
ps -ef | grep $var | grep -v grep | cut -c 9-15 | xargs kill -9
nohup scrapy crawl $var -L INFO 1>$var.log 2>&1 &
done
ps -ef | grep tag_adder | grep -v grep | cut -c 9-15 | xargs kill -9
nohup python ./biliob_requests/tag_adder.py 1>tag_adder.log 2>&1 &
ps -ef | grep DanmakuAggregate | grep -v grep | cut -c 9-15 | xargs kill -9
cd danmaku_spider/ && nohup scrapy crawl DanmakuAggregate 1>log.log 2>&1 &
ps -ef |grep py
Loading...
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/yuzhishui/biliob-spider.git
[email protected]:yuzhishui/biliob-spider.git
yuzhishui
biliob-spider
biliob-spider
master

搜索帮助