Hadoop-HBase编译安装snappy压缩工具（矿坑）

准备所需组件Hadoop2.7.3:https://github.com/apache/had...Snappy1.1.3:https://github.com/google/sna...protobuf-2.5.0:https:///github.com/google/pro...安装依赖包ant,gcc,gcc-c++,make,autoconf,automake,cmake,gzip,libssl-devel,openssl-devel,libtool,Java7+,maven3+大部分都可以直接yuminstall安装snappy解压unzipsnappy-1.1.3.zip进入目录cdsnappy-1.1.3编译安装./configure,make&&makeinstall；编译成功后，/usr/local/lib/下会有如下#ls/usr/local/lib/libsnappy.alibsnappy.lalibsnappy.solibsnappy.so.1libsnappy.so.1.3.0installprotobuf解压:解压protobuf-2.5.0.zip进入目录:cdprotobuf-2.5.0/编译安装:./configure,make&&makeinstall配置环境变量:exportPATH=/usr/local/protoc/bin:$PATH立即生效：source/etc/profile查看版本：protoc–version[root@SparkWorker3data]#protoc--versionlibprotoc2.5.0编译hadoop源码安装jdk，maven，安装依赖库：yum-yinstallsvnncurses-develgcc*yum-yinstalllzo-develzlib-develautoconfautomakelibtoolcmakeopenssl-develunzipunziphadoop-release-2.7.3-RC2.zip进入目录cdhadoop-release-2.7.3-RC2/设置maven内存，exportMAVEN_OPTS="-Xms256m-Xmx512m"compilemvnpackage-DskipTests-Pdist,native-Dtar-Dsnappy.lib=/usr/local/lib-Dbundle.snappy会花很长时间time，一般的错误是缺少依??赖。结果如下#lshadoop-release-2.7.3-RC2/hadoop-dist/target/hadoop-2.7.3/lib/nativelibhadoop.alibhadooputils.alibsnappy.alibsnappy.so.1.3。0libhadooppipes.alibhdfs.alibsnappy.lalibhadoop.solibhdfs.solibsnappy.solibhadoop.so.1.0.0libhdfs.so.0.0.0libsnappy.so.1hadoop和hbase添加snappy拷贝库文件到hadoop#cp-r/data/hadoop-compile/hadoop-release-2.7.3-RC2/hadoop-dist/target/hadoop-2.7.3/lib/native/*$HADOOP_HOME/lib/native/在hbase中创建目录#mkdir-p$HBASE_HOME/lib/native/Linux-amd64-64拷贝库文件到Hbase#cp-r/data/hadoop-compile/hadoop-release-2.7.3-RC2/hadoop-dist/target/hadoop-2.7.3/lib/native/*$HBASE_HOME/lib/native/Linux-amd64-64/$HADOOP_/HOME/etc/hadoop/hadoop-env.sh添加如下环境变量：exportLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/hadoop/hadoop-2.7.3/lib/native/:/usr/local/lib/$HBASE_HOME/conf/hbase-env.sh添加如下环境变量：exportLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/hadoop/hadoop-2.7.3/lib/native/:/usr/local/lib/exportHBASE_LIBRARY_PATH=$HBASE_LIBRARY_PATH:$HBASE_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/修改配置文件添加$HADOOP_HOOME/etc/hadoop/core-site.xml<属性>io.compression.codecsorg.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec添加$HADOOP_HOOME/etc/hadoop/mapred-site.xmlmapred.output.compresstruemapred.output.compression.codecorg.apache.hadoop.io.compress.SnappyCodecmapred.compress.map.outputtruemapred.map.output。compression.codecorg.apache.hadoop.io.compress.SnappyCodec添加$HBASE_HOME/conf/hbase-site.xmlhbase.regionserver.codecssnappy重启动hadoop、hbase，并测试hadoop测试：$HADOOP_HOME/bin/hadoopchecknative-a#$HADOOP_HOME/bin/hadoopchecknative-a17/09/2110:31:50INFOzlib.ZlibFactory：成功加载并初始化native-zliblibraryNativelibrarychecking：hadoop：true/usr/local/hadoop/hadoop-2.7.3/lib/native/libhadoop.sozlib：true/lib64/libz。so.1snappy:true/usr/local/hadoop/hadoop-2.7.3/lib/native/libsnappy.so.1lz4:true修订版：99bzip2:falseopenssl:true/lib64/libcrypto.so17/09/2110:31:50INFOutil.ExitUtil:Exitingwithstatus1hbase创建表：create'snappytest',{NAME=>'info',COMPRESSION=>'活泼'}