Error message looks like this:
Couldnot load history file hdfs://namenodeha:8020/mr-history/tmp/hdfs/job_1392049860497_0005-1392129567754-hdfs-word+count-1392129599308-1-1-SUCCEEDED-default.jhist
Actually, I know the answer to the problem. The defaul settings of /mr-history files is:
hadoop fs -chown -R $MAPRED_USER:$HDFS_USER /mr-history
But when running a job (under $HDFS_USER), job file is saved to /mr-history/tmp/hdfs under $HDFS_USER:$HDFS_USER and then not accessible to $MAPRED_USER (where JobHistory server is running). After changing the permissions back again the job file can be load.
But it is happening again with every new job. So can someone help me, what is the pernament solution to this, thank you.
I ran into the same problem. As a workaround I added the $MAPRED_USER user to the $HDFS_USER group, it helped.
我qeekey
的解决办法办,把mapred这个用户加入到 supergroup组,这样它就有权限读其它人生成的日志了。
groupadd supergroup
usermod -a -G supergroup mapred
相关推荐
官网荡下的源码,自己亲手编译的hadoop2.x 64位,给大家分享下 文件太大放的是百度网盘链接,目前有hadoop2.3.0,后面的版本我会陆续更新。
hadoop 2.3.0 SRC包
oozie 4.0.1 hadoop2.3.0 安装
Hadoop 2.3.0分布式集群搭建图文
本文详细描述了Hadoop2.3.0安装步骤。其中有转载于其他部分的内容。
hadoop2.3.0-eclipse-plugin htrace-core-3.0.4.jar有这个jar包。
资源如题,自己想办法编译的,亲测可用,因为等级低,所以分了三部分上传。(我不会分卷压缩tar.gz,所以压缩成了.zip,见谅哈)
资源如题,自己编译的,亲测可用,分成了三部分,下载完成自行解压就好
从apache官网下载的hadoop-2.3.0.tar.gz的native库是32位的,但是我们的机器一般是64位的操作系统,所以需要重新编译为64位的,附件就是我编译好的64位的native
hadoop-eclipse-plugin-2.3.0插件 eclipse版本4.4.0,经验证win7下可以正常运行。
解压后改名并修改spark-env.sh和slaves,提前安装scala11.x和hadoop2.6。
赠送Maven依赖信息文件:hadoop-yarn-server-resourcemanager-2.6.0.pom; 包含翻译后的API文档:hadoop-yarn-server-resourcemanager-2.6.0-javadoc-API文档-中文(简体)版.zip; Maven坐标:org.apache.hadoop:...
spark-2.3.0-bin-hadoop2.7版本.zip
详细介绍Spark2.3.0和Hadoop2.7.4集群在RedHat服务器部署,内涵hadoop 基于NFS 的HA高可用模式, yarn HA高可用, zookeeper安装,spark集群部署,NFS目录创建。对相关参数有详细介绍,以及提供了涉及到的Linux命令...
赠送Maven依赖信息文件:hadoop-yarn-server-common-2.6.5.pom; 包含翻译后的API文档:hadoop-yarn-server-common-2.6.5-javadoc-API文档-中文(简体)版.zip; Maven坐标:org.apache.hadoop:hadoop-yarn-server-...
hadoop的dll文件 hadoop.zip
hadoop-eclipse-plugin-2.3.0.jar x86 x64 eclipse kepler release2
赠送Maven依赖信息文件:hadoop-yarn-server-web-proxy-2.6.0.pom; 包含翻译后的API文档:hadoop-yarn-server-web-proxy-2.6.0-javadoc-API文档-中文(简体)版.zip; Maven坐标:org.apache.hadoop:hadoop-yarn-...
Hadoop HA 集群搭建所需要的配置文件:core-site,hdfs-site,mapred-site,yarn-site四个xml文件和一个slaves文件
Hadoop分布式文件系统的模型分析,Hadoop 分布式文件系统是遵循Google 文件系统原理进行开发和实现的,受到了业界极大关注,并 已被广泛应用。 鉴于当前缺乏从系统设计理论的角度对其开展的相关研究,本文从 Hadoop ...