CDH 版本Hadoop Yarn配置容量调度器(默认是公平调度器)
2023-09-14 09:02:04 时间
配置
中进行如下调整:
其实就是在在yarn-site.xm中添加yarn.resourcemanager.scheduler.class
,修改成容量调度器:
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>
配置中搜索yarn.scheduler.capacity.root.queues
,以XML格式查看
,导入以下配置后重启yarn:
修改后的配置:
<configuration>
<property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
</property>
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.1</value>
</property>
<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default,dmgeo</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.capacity</name>
<value>60</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.user-limit-factor</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.maximum-capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_application_max_priority</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.acl_application_max_priority</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-application-lifetime </name>
<value>-1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.maximum-application-lifetime </name>
<value>-1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.default-application-lifetime </name>
<value>-1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.dmgeo.default-application-lifetime </name>
<value>-1</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.rack-locality-additional-delay</name>
<value>-1</value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings</name>
<value></value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments</name>
<value>1</value>
</property>
</configuration>
默认的配置:
<configuration><property><name>yarn.scheduler.capacity.root.queues</name><value>default</value></property><property><name>yarn.scheduler.capacity.root.capacity</name><value>100</value></property><property><name>yarn.scheduler.capacity.root.default.capacity</name><value>100</value></property></configuration>
参考链接:https://www.cnblogs.com/hongfeng2019/p/11477583.html
相关文章
- 前端系列——React开发必不可少的eslint配置
- hadoop集群配置SSH免登陆
- Python的Django框架中的URL配置与松耦合
- 配置防火墙we登录
- nginx做负载均衡器以及proxy缓存配置
- Win配置Apache+mod_wsgi+django环境+域名
- Hadoop单机模式安装-(3)安装和配置Hadoop
- activemq持久化配置,设置为主从模式(带复制的主从模式,应用mysql数据库)
- Spring Boot jackson配置使用详解
- Hadoop集群最迅速的配置免密码登陆方法
- 一脸懵逼学习基于CentOs的Hadoop集群安装与配置(三台机器跑集群)
- 一步一步跟我学hadoop(1)----hadoop概述和安装配置
- AP3010DN-V2 安装 配置
- hadoop入门(九):日志聚集功能配置
- 【taro react】---- 配置目录别名
- 静态Vxlan多活网关实验配置(集中式网关)