【大数据系列】broadcastTimeout报错
数据 报错 系列
2023-09-27 14:29:28 时间
💝💝💝欢迎来到我的博客,很高兴能够在这里和您见面!希望您在这里可以感受到一份轻松愉快的氛围,不仅可以获得有趣的内容和知识,也可以畅所欲言、分享您的想法和见解。
- 推荐:kuan 的首页,持续学习,不断总结,共同进步,活到老学到老
- 导航
非常期待和您一起在这个小小的网络世界里共同探索、学习和成长。💝💝💝 ✨✨ 欢迎订阅本专栏 ✨✨
博客目录
broadcastTimeout 报错
设置 set spark.sql.broadcastTimeout=3000;
22/05/18 14:39:26 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 9.0 (TID 20589) in 3911 ms on bjm8-bdc-cdh-prd-10-251-35-132.belle.lan (executor 9) (32/35)
22/05/18 14:39:37 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 9.0 (TID 20607) in 14961 ms on bjm8-bdc-cdh-prd-10-251-35-51.belle.lan (executor 2) (33/35)
22/05/18 14:39:37 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 9.0 (TID 20576) in 16008 ms on bjm8-bdc-cdh-prd-10-251-35-132.belle.lan (executor 9) (34/35)
22/05/18 14:39:38 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 9.0 (TID 20577) in 16116 ms on bjm8-bdc-cdh-prd-10-251-35-132.belle.lan (executor 7) (35/35)
22/05/18 14:39:38 INFO cluster.YarnClusterScheduler: Removed TaskSet 9.0, whose tasks have all completed, from pool
22/05/18 14:39:38 INFO scheduler.DAGScheduler: ResultStage 9 (run at ThreadPoolExecutor.java:1149) finished in 16.118 s
22/05/18 14:39:38 INFO scheduler.DAGScheduler: Job 9 finished: run at ThreadPoolExecutor.java:1149, took 16.594440 s
22/05/18 14:39:38 INFO memory.MemoryStore: Block broadcast_18 stored as values in memory (estimated size 5.0 MB, free 3.0 GB)
22/05/18 14:39:38 INFO memory.MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 801.9 KB, free 3.0 GB)
22/05/18 14:39:38 INFO storage.BlockManagerInfo: Added broadcast_18_piece0 in memory on 10.251.35.177:12696 (size: 801.9 KB, free: 3.0 GB)
22/05/18 14:39:38 INFO spark.SparkContext: Created broadcast 18 from run at ThreadPoolExecutor.java:1149
22/05/18 14:39:38 INFO codegen.CodeGenerator: Code generated in 21.867176 ms
22/05/18 14:39:38 INFO memory.MemoryStore: Block broadcast_19 stored as values in memory (estimated size 437.5 KB, free 3.0 GB)
22/05/18 14:39:38 INFO memory.MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 47.8 KB, free 3.0 GB)
22/05/18 14:39:38 INFO storage.BlockManagerInfo: Added broadcast_19_piece0 in memory on 10.251.35.177:12696 (size: 47.8 KB, free: 3.0 GB)
22/05/18 14:39:38 INFO spark.SparkContext: Created broadcast 19 from run at ThreadPoolExecutor.java:1149
22/05/18 14:39:38 INFO execution.FileSourceScanExec: Planning scan with bin packing, max size: 134217728 bytes, open cost is considered as scanning 4194304 bytes.
22/05/18 14:39:38 ERROR datasources.FileFormatWriter: Aborting job null.
觉得有用的话点个赞 👍🏻
呗。
❤️❤️❤️本人水平有限,如有纰漏,欢迎各位大佬评论批评指正!😄😄😄
💘💘💘如果觉得这篇文对你有帮助的话,也请给个点赞、收藏下吧,非常感谢!👍 👍 👍
🔥🔥🔥Stay Hungry Stay Foolish 道阻且长,行则将至,让我们一起加油吧!🌙🌙🌙
相关文章
- 大数据导出报错
- el-table显示树形数据报错
- Google Earth Engine(GEE)——全球8,502个污水处理厂(WWTPs)数据集
- 找不到完美数据科学家?你还可以组建一支数据科学梦之队
- 《驾驭大数据》一3.1 汽车保险业:车载信息服务数据的价值
- 挑战传统数据建模技术 大数据工具成趋势
- Python数据可视化 Pyecharts 制作 Grid 组合组件
- Flink CDC 专题首发|每天 10 分钟,解锁新一代数据集成框架
- mysql导出数据报错The MySQL server is running with the --secure-file-priv option so it cannot execute this statement
- mysql数据库中不能插入0000-00-00 00:00:00日期数据(报错Invalid datetime format: 1292 Incorrect datetime value: '0000-00-00 00:00:00')
- 数据驱动业务决策的5个步骤
- 大数据全球战略布局全面升级 进入关键转折期
- 获取数据库元数据
- 王兴山:互联网与大数据环境下的管理会计
- 龙岩:“大数据”助力传统产业升级