zl程序教程

您现在的位置是:首页 >  后端

当前栏目

python编程(rq调度系统)

2023-09-27 14:27:11 时间

【 声明:版权所有,欢迎转载,请勿用于商业用途。 联系信箱:feixiaoxing @163.com】


    昨天看到一个基于python的队列调度系统,觉得挺有意思。项目的地址在。基本操作步骤如下,


1、安装redis、rq的python库

sudo pip install redis
sudo pip install rq

2、编写work.py

import requests

def count_words_at_url(url):
    resp = requests.get(url)
    return len(resp.text.split())

3、编写process.py

from redis import Redis
from rq import Queue

q = Queue(connection=Redis())
from work import count_words_at_url
result = q.enqueue(count_words_at_url, 'https://www.baidu.com')

4、运行process.py,这个时候只是将工作压进堆栈

python process.py

5、开启工作进程

rq worker

6、此时,你就可以看到计算机结果了

feixiaoxing@hp330:~$ rq worker
08:35:08 RQ worker u'rq:worker:hp330.27580' started, version 0.10.0
08:35:08 *** Listening on default...
08:35:08 Cleaning registries for queue: default
08:35:08 default: work.count_words_at_url('https://www.baidu.com') (6deae914-53be-45a1-acc9-9c5c5fc65c6c)
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
08:35:08 default: Job OK (6deae914-53be-45a1-acc9-9c5c5fc65c6c)
08:35:08 Result is kept for 500 seconds