java.util.concurrent.TimeoutException: Heartbeat of TaskManager with id container_1566481621886_4397244_01_000004 timed out.
JAVA of with ID 01 out container util
2023-09-11 14:18:40 时间
根据Heartbeat of TaskManager with id和The heartbeat of ResourceManager with id在源码中找出这样的代码
private class TaskManagerHeartbeatListener implements HeartbeatListener<AccumulatorReport, Void> { private final JobMasterGateway jobMasterGateway; private TaskManagerHeartbeatListener(JobMasterGateway jobMasterGateway) { this.jobMasterGateway = Preconditions.checkNotNull(jobMasterGateway); } @Override public void notifyHeartbeatTimeout(ResourceID resourceID) { jobMasterGateway.disconnectTaskManager( resourceID, new TimeoutException("Heartbeat of TaskManager with id " + resourceID + " timed out.")); } @Override public void reportPayload(ResourceID resourceID, AccumulatorReport payload) { for (AccumulatorSnapshot snapshot : payload.getAccumulatorSnapshots()) { schedulerNG.updateAccumulators(snapshot); } } @Override public CompletableFuture<Void> retrievePayload(ResourceID resourceID) { return CompletableFuture.completedFuture(null); } } private class ResourceManagerHeartbeatListener implements HeartbeatListener<Void, Void> { @Override public void notifyHeartbeatTimeout(final ResourceID resourceId) { runAsync(() -> { log.info("The heartbeat of ResourceManager with id {} timed out.", resourceId); if (establishedResourceManagerConnection != null && establishedResourceManagerConnection.getResourceManagerResourceID().equals(resourceId)) { reconnectToResourceManager( new JobMasterException( String.format("The heartbeat of ResourceManager with id %s timed out.", resourceId))); } }); } @Override public void reportPayload(ResourceID resourceID, Void payload) { // nothing to do since the payload is of type Void } @Override public CompletableFuture<Void> retrievePayload(ResourceID resourceID) { return CompletableFuture.completedFuture(null); } }
然后在这实例化
this.taskManagerHeartbeatManager = heartbeatServices.createHeartbeatManagerSender(resourceId,new TaskManagerHeartbeatListener(selfGateway),rpcService.getScheduledExecutor(),log);
顺着去heartbeatServices瞅瞅了
/** * HeartbeatServices gives access to all services needed for heartbeating. This includes the * creation of heartbeat receivers and heartbeat senders. */ public class HeartbeatServices { /** Heartbeat interval for the created services. */ protected final long heartbeatInterval; /** Heartbeat timeout for the created services. */ protected final long heartbeatTimeout; public HeartbeatServices(long heartbeatInterval, long heartbeatTimeout) { Preconditions.checkArgument(0L < heartbeatInterval, "The heartbeat interval must be larger than 0."); Preconditions.checkArgument(heartbeatInterval <= heartbeatTimeout, "The heartbeat timeout should be larger or equal than the heartbeat interval."); this.heartbeatInterval = heartbeatInterval; this.heartbeatTimeout = heartbeatTimeout; } /** * Creates a heartbeat manager which does not actively send heartbeats. * * @param resourceId Resource Id which identifies the owner of the heartbeat manager * @param heartbeatListener Listener which will be notified upon heartbeat timeouts for registered * targets * @param scheduledExecutor Scheduled executor to be used for scheduling heartbeat timeouts * @param log Logger to be used for the logging * @param <I> Type of the incoming payload * @param <O> Type of the outgoing payload * @return A new HeartbeatManager instance */ public <I, O> HeartbeatManager<I, O> createHeartbeatManager( ResourceID resourceId, HeartbeatListener<I, O> heartbeatListener, ScheduledExecutor scheduledExecutor, Logger log) { return new HeartbeatManagerImpl<>( heartbeatTimeout, resourceId, heartbeatListener, scheduledExecutor, scheduledExecutor, log); } /** * Creates a heartbeat manager which actively sends heartbeats to monitoring targets. * * @param resourceId Resource Id which identifies the owner of the heartbeat manager * @param heartbeatListener Listener which will be notified upon heartbeat timeouts for registered * targets * @param scheduledExecutor Scheduled executor to be used for scheduling heartbeat timeouts * @param log Logger to be used for the logging * @param <I> Type of the incoming payload * @param <O> Type of the outgoing payload * @return A new HeartbeatManager instance which actively sends heartbeats */ public <I, O> HeartbeatManager<I, O> createHeartbeatManagerSender( ResourceID resourceId, HeartbeatListener<I, O> heartbeatListener, ScheduledExecutor scheduledExecutor, Logger log) { return new HeartbeatManagerSenderImpl<>( heartbeatInterval, heartbeatTimeout, resourceId, heartbeatListener, scheduledExecutor, scheduledExecutor, log); } /** * Creates an HeartbeatServices instance from a {@link Configuration}. * * @param configuration Configuration to be used for the HeartbeatServices creation * @return An HeartbeatServices instance created from the given configuration */ public static HeartbeatServices fromConfiguration(Configuration configuration) { long heartbeatInterval = configuration.getLong(HeartbeatManagerOptions.HEARTBEAT_INTERVAL); long heartbeatTimeout = configuration.getLong(HeartbeatManagerOptions.HEARTBEAT_TIMEOUT); return new HeartbeatServices(heartbeatInterval, heartbeatTimeout); } }
没错超时时间就在HeartbeatManagerOptions.HEARTBEAT_TIMEOUT
/** Timeout for requesting and receiving heartbeat for both sender and receiver sides. */ public static final ConfigOption<Long> HEARTBEAT_TIMEOUT = key("heartbeat.timeout") .defaultValue(50000L) .withDescription("Timeout for requesting and receiving heartbeat for both sender and receiver sides.");
引起心跳超时有可能是yarn压力比较大引起的,先暂时在conf/flink-conf.yaml将这个值调大一点,再观察。
#Timeout for requesting and receiving heartbeat for both sender and receiver sides. heartbeat.timeout: 180000
相关文章
- java.sql.SQLException Parameter index out of range (0 1 ).
- Java 中Timer和TimerTask 定时器和定时任务使用的例子
- Java实现 蓝桥杯 算法提高 八数码(BFS)
- Java实现 LeetCode 478 在圆内随机生成点
- Java实现 LeetCode 268 缺失数字
- Java实现 蓝桥杯 素因子去重
- Java实现LeetCode 111. Minimum Depth of Binary Tree
- java实现输入日期
- java实现连续数的公倍数
- java实现第二届蓝桥杯异或加密法
- Java实现 蓝桥杯 历届试题 邮局
- Java实现第七届蓝桥杯国赛 赢球票
- java核心知识点学习----多线程间的数据共享和对象独立,ThreadLocal详解
- 【JAVA】 03-Java中的异常和包的使用
- 【JAVA】java编译错误:编码UTF8/GBK的不可映射字符
- 【Java】java使用反射访问对象方法和成员变量
- Java数组对象的内存布局
- Java AOP学习之method invoke of class JdkDynamicAopProxy
- java语言学习003_开发环境基础配置和安装(jdk,jre,eclipse&myeclipse)
- 【java】Java生成微信小程序二维码
- Java中String字符串截取几种方法(substring,split)
- 通过宠物商店理解java面向对象
- Java中String类的concat方法___java的String字符串的concat()方法连接字符串和“+“连接字符串解释
- java死锁(Java-level deadlock)
- mysql-connector-java与Mysql、Java的对应版本
- java 不同意同一账户不同IP 同一时候登录系统解决的方法 兼容IE Firefox
- Java如何比较两个数组是否相等?
- Java中读入读出文件
- Android Gradle plugin requires Java 11 to run. You are currently using Java 1.8. You can try some of
- 【java】Java 重写(Override)与重载(Overload)
- 【java】Java并发编程--Java实现多线程的4种方式