Sqoop-MySQL导入hive时id为文本解决
2023-09-11 14:14:34 时间
错误如下
19/05/27 15:32:51 ERROR tool.ImportTool: Import failed: java.io.IOException: Generating splits for a textual index column allowed only in case of "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:195) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:327) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588) at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:199) at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:172) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:269) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:124) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:526) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:634) at org.apache.sqoop.Sqoop.run(Sqoop.java:145) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:232) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:241) at org.apache.sqoop.Sqoop.main(Sqoop.java:250) Caused by: Generating splits for a textual index column allowed only in case of "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:65) at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:192) ... 23 more
解决
修改命令如下
sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=true"
相关文章
- MySQL存储过程详解 mysql 存储过程
- 【MySQL】Mysql 日志
- 【MySQL高级】MySql中常用工具及Mysql 日志
- ubuntu mysql 导入外部sql文件
- 【转】mysql 大数据字段longblob 默认长度参数max_allowed_packet 过小导致导入失败
- MySQL root修改普通用户密码及Mysql 修改root密码
- 【MySQL】解决导入sql脚本报错:Got a packet bigger than ‘max_allowed_packet‘ bytes
- SQL Server 数据导入Mysql详细教程
- linux 安装mysql数据库——yum安装法
- MySQL 优化之 Linux系统层面调优
- 《PHP、MySQL和Apache入门经典(第5版)》一一2.3 在Linux/UNIX上安装MySQL
- 基于Java+Mysql实现酒店预订系统【100010045】
- 基于Java+Mysql实现(web)大型企业管理系统【100010019】
- 使用navicat连接mysql连接错误:Lost connection to Mysql server at 'waiting for initial communication packet'
- MySQL案列之主从复制出错问题以及pt-slave-restart工具的使用
- python操作mysql数据库系列-操作MySql数据库(一)
- mysql视频链接
- Mysql 根据时间戳按年月日分组统计
- Mysql学习---使用Python执行存储过程
- ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var mysql (转)
- mysql 导入数据时一定要改这个值 max_allowed_packet = 1000M
- 记录MySQL运行的SQL
- mysql导入数据报错:Incorrect datetime value
- Mac - Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
- Eclipse中使用MySql遇到:Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading o