Hadoop Failed to set permissions of path
在Eclipse里面写了个測试程序:把HDFS中的数据批量导入到HBase中
写好后,在本地測试遇到了例如以下问题:
14/04/21 16:49:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/04/21 16:49:53 ERROR security.UserGroupInformation: PriviledgedActionException as:admin cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1518427785\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1518427785\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:690)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:918)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at com.plocc.hadoop.hbase.BatchImport.main(BatchImport.java:104)
此时我把程序打包到Linux上执行是okey的,那说明是Windows下文件权限问题,
然后依据at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:690)这个错误信息找到了源代码,
然后一步步调试到Win32FileSystem.class中的
public native boolean setPermission(File f, int access, boolean enable, boolean owneronly);方法,
这个native格式的方法可能在c或者c++里面调用了,没法继续跟踪了,所以我这里的解决方法是把
private static void checkReturnValue(boolean rv, File p,
FsPermission permission
) throws IOException {
if (!rv) {
throw new IOException("Failed to set permissions of path: " + p +
" to " +
String.format("%04o", permission.toShort()));
}
}
标记经色部分的代码先凝视掉,保证在windows下測试通过。
相关文章
- asp.net core signalr Error: Failed to start the transport 'WebSockets': null
- 【RAC】failed to online diskgroup resource ora.GI.dg
- 解决问题 The Tomcat connector configured to listen on port 8080 failed to start.
- Tomcat7+Spring3异常Failed to start component
- Tomcat 启动或者发布项目时提示Publishing failed:Resource /xxxx does not exist
- 如何分析SAP Fiori运行时报出的HTTP Request Failed错误消息
- 【已解决】linux下mysql报Failed to restart mysqld.service: Unit not found
- Java如何获取IP属地 ip2region failed to create searcher with x:java.io.FileNotFoundException:( 系统找不到指定的路径)
- Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container..
- Android ADT中增大AVD内存后无法启动:emulator failed to allocate memory 8 (转)
- Execution failed for task ‘:app:processDebugResources‘. > com.android.ide.common.process.ProcessExce
- /storage/emulated/0/Pictures/eWorld/1644986847.jpeg: open failed: ENOENT (No such file or directory)
- 【问题】2.Bind for 0.0.0.0:8887 failed: port is already allocated错误