python3 分布式进程(跨机器)BaseManager(multiprocessing.managers)
2023-09-14 09:12:28 时间
A机器负责发送任务和接受结果:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
#task_master.py import random,time,queue from multiprocessing.managers import BaseManager task_queue = queue.Queue() result_queue = queue.Queue() class QueueManager(BaseManager): pass if __name__ = = '__main__' : print ( "master start." ) QueueManager.register( 'get_task_queue' , callable = lambda :task_queue) QueueManager.register( 'get_result_queue' , callable = lambda :result_queue) manager = QueueManager(address = ( '10.10.100.11' , 9833 ),authkey = b 'abc' ) manager.start() task = manager.get_task_queue() result = manager.get_result_queue() for i in range ( 10 ): n = random.randint( 0 , 1000 ) print ( 'put task %d ...' % n) task.put(n) print ( 'try get results...' ) for i in range ( 10 ): r = result.get(timeout = 100 ) print ( 'Result:%s' % r) manager.shutdown() print ( 'master exit.' ) |
B机器负责处理任务和发送结果:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
#task_worker.py import sys,time,queue from multiprocessing.managers import BaseManager class QueueManager(BaseManager): pass QueueManager.register( 'get_task_queue' ) QueueManager.register( 'get_result_queue' ) server_addr = '10.10.100.11' print ( 'connect to server %s...' % server_addr) m = QueueManager(address = (server_addr, 9833 ),authkey = b 'abc' ) m.connect() task = m.get_task_queue() result = m.get_result_queue() for i in range ( 10 ): try : n = task.get(timeout = 10 ) print ( 'run task %d * %d' % (n,n)) r = '%d * %d = %d' % (n,n,n * n) time.sleep( 1 ) result.put(r) except Queue.Empty: print ( 'task queue is empty' ) print ( 'worker exit' ) |
相关文章
- python2 和 python3 常见差异及兼容方式梳理
- dockerfile制作:制作python3下的robotframework运行环境
- Linux系统安装Python3环境(超详细)
- Python3 面向对象编程基础
- Python3 函数式编程
- Python3 Socket
- python3 gil锁_python gil 多线程
- python3·7创建虚拟环境_python激活虚拟环境
- python3爆破服务器_dirsearch「建议收藏」
- python3 牛客网:OJ在线编程常见输入输出练习(ACM模式)
- Python3.X 爬虫实战缓存与持久化详解程序员
- python3 写的一个通过ftp远程批量处理的工具__界面处理代码详解编程语言
- Python3.x:访问带参数链接并且获取返回json串详解编程语言
- Python3.x:代理ip刷点赞详解编程语言
- Python3.x:获取代理ip以及使用详解编程语言
- python3使用urllib示例取googletranslate(谷歌翻译)