Linux 编译 hadoop-cdh4.2.1

配置

安装Maven, Ant, Findbugs, protobuf, cmake
参考:http://cn.soulmachine.me/blog/20140214/ 注意CDH4.2.1需要用protobuf2.4.1编译，否则hadoop-common那里报错 在apache-maven-3.0.5/conf/settings.xml 更改maven镜像

<mirror>
  <id>nexus-osc</id>
  <mirrorOf>central</mirrorOf>
  <name>Nexus osc</name>
  <url>http://maven.oschina.net/content/groups/public/</url>
  </mirror>

  <mirror>
  <id>nexus-spring</id>
  <mirrorOf>cdh.repo</mirrorOf>
  <name>spring</name>
  <url>http://repo.spring.io/repo/</url>
  </mirror>
  
  <mirror>
  <id>nexus-spring2</id>
  <mirrorOf>cdh.releases.repo</mirrorOf>
  <name>spring2</name>
  <url>http://repo.spring.io/repo/</url>
  </mirror>

编译

mvn package -DskipTests -Pdist,native -Dtar

替换jar包

从src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/target里面拿出mr-app.jar重命名为hadoop-mapreduce-client-app-2.0.0-cdh4.2.1-20140801.jar

cd /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
# 上传jar包
chmod 755 hadoop-mapreduce-client-app-2.0.0-cdh4.2.1-20140801.jar
ln -snf hadoop-mapreduce-client-app-2.0.0-cdh4.2.1-20140801.jar hadoop-mapreduce-client-app.jar
mv hadoop-mapreduce-client-app-2.0.0-cdh4.2.1.jar /tmp

错误处理

Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/home/xx/hadoop-fix/hadoop/build/hadoop-2.0.0-cdh4.2.1/src/hadoop-common-project/hadoop-common/target/native"): java.io.IOException: error=2, No such file or directory -> [Help 1]

没有安装cmake，yum install cmake