本文已参与「新人创作礼」活动,一起开启掘金创作之路。
ambari2.7.3 安装 impala
接着上面第四步开始
5.hdfs增加配置
修改core-site.xml
#HDFS config we need add below config to /etc/hadoop/conf/core-site.xml
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.client.read.shortcircuit.skip.checksum</name>
<value>false</value>
</property>
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
修改hdfs-site.xml
we need add below config to /etc/hadoop/conf/hdfs-site.xml
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.block.local-path-access.user</name>
<value>impala</value>
</property>
<property>
<name>dfs.client.file-block-storage-locations.timeout.millis</name>
<value>60000</value>
</property>
6.重启hdfs
直接在ambari界面重启就行
7.启动impala
启动失败看日志
cat impala-server.log (每台impala机器都要做)
报错:
Unable to find Java. JAVA_HOME should be set in /etc/default/bigtop-utils
解决
vim /etc/default/bigtop-utils
JAVA_HOME=/opt/jdk1.8.0_201 (与/etc/profile保持一样)
启动失败看日志 cat /var/log/impala/impalad.ERROR
报错
hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:\
ClassNotFoundException: com.ctc.wstx.io.InputBootstrapperjava.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper\
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper\
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)\
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)\
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)\
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)\
Wrote minidump to /var/log/impala/minidumps/impalad/0a803f89-a804-4bb4-242e02b9-94eba9b1.dmp
解决
把hbase的jar包拷贝到 /usr/lib/impala/lib 并创建软连接
这里注意根据安装的impala的版本对应的cdh的版本选择对应的hbase的版本,否则可能出现版本不兼容的问题
ln -s hbase-annotations-1.2.0-cdh5.11.0.jar hbase-annotations.jar
ln -s hbase-client-1.2.0-cdh5.11.0.jar hbase-client.jar
ln -s hbase-common-1.2.0-cdh5.11.0.jar hbase-common.jar
ln -s hbase-examples-1.2.0-cdh5.11.0.jar hbase-examples.jar
ln -s hbase-external-blockcache-1.2.0-cdh5.11.0.jar hbase-external-blockcache.jar
ln -s hbase-hadoop2-compat-1.2.0-cdh5.11.0.jar hbase-hadoop2-compat.jar
ln -s hbase-hadoop-compat-1.2.0-cdh5.11.0.jar hbase-hadoop-compat.jar
ln -s hbase-it-1.2.0-cdh5.11.0.jar hbase-it.jar
ln -s hbase-prefix-tree-1.2.0-cdh5.11.0.jar hbase-prefix-tree.jar
ln -s hbase-procedure-1.2.0-cdh5.11.0.jar hbase-procedure.jar
ln -s hbase-protocol-1.2.0-cdh5.11.0.jar hbase-protocol.jar
ln -s hbase-resource-bundle-1.2.0-cdh5.11.0.jar hbase-resource-bundle.jar
ln -s hbase-rest-1.2.0-cdh5.11.0.jar hbase-rest.jar
ln -s hbase-rsgroup-1.2.0-cdh5.11.0.jar hbase-rsgroup.jar
ln -s hbase-server-1.2.0-cdh5.11.0.jar hbase-server.jar
ln -s hbase-shell-1.2.0-cdh5.11.0.jar hbase-shell.jar
ln -s hbase-spark-1.2.0-cdh5.11.0.jar hbase-spark.jar
ln -s hbase-thrift-1.2.0-cdh5.11.0.jar hbase-thrift.jar
报错
E0524 17:18:04.260188 1741 impala-server.cc:279] Invalid short-circuit reads co\
- Impala cannot read or execute the parent directory of dfs.domain.socket.path
解决
在配置文件中查看dfs.domain.socket.path 为$path
chmod 775 ../$path
参考:
github.com/cas-bigdata… github官网有详细步骤持续更新)