Kylin下载

根据需要的软件版本下载Kylin

下载地址：https://archive.apache.org/dist/kylin/

环境准备

由于我要在CDH上部署，根据官方文档还需要三个jar包

官方文档：https://cwiki.apache.org/confluence/display/KYLIN/Deploy+Kylin+4+on+CDH+6

需要的jar包：commons-configuration、hive-exec、stax2-api在官方文档中也有下载连接

把jar包放到指定目录

1
2
3
4
5
6
$ cd $KYLIN_HOME

$ mkdir -p bin/hadoop3_jars/cdh6
$ cp commons-configuration-1.10.jar bin/hadoop3_jars/cdh6
$ cp hive-exec-1.21.2.3.1.0.0-78.jar bin/hadoop3_jars/cdh6
$ cp stax2-api-3.1.4.jar bin/hadoop3_jars/cdh6

准备mysql连接

1
2
$ mkdir ext
$ cp mysql-connector-java-5.1.40.jar ext/

修改配置

1
2
3
4
5
6
7
8
9
10
11
12
13
$ vi conf/kylin.properties
 
#spark配置
kylin.query.spark-conf.spark.sql.hive.metastore.version=2.1.1
kylin.query.spark-conf.spark.sql.hive.metastore.jars=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive/lib/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce/*
kylin.engine.spark-conf.spark.sql.hive.metastore.version=2.1.1
kylin.engine.spark-conf.spark.sql.hive.metastore.jars=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive/lib/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn/*:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce/*

#mysql配置
kylin.metadata.url=kylin_metadata@jdbc,driverClassName=com.mysql.jdbc.Driver,url=jdbc:mysql://localhost:3306/kylin_test,username=,password=
#zk配置
kylin.env.zookeeper-connect-string=ip

启动

1
$ sh bin/kylin.sh start

停止

1
$ sh bin/kylin.sh stop

遇到的问题

1.Permission denied: user=root, access=WRITE, inode=“/“:hdfs:supergroup:drwxr-xr-x

权限不足，需要在hdfs中配置dfs.permissions.enabled，值为false

然后重启

2.spark not found, set SPARK_HOME, or run bin/download-spark.sh

在/etc/profile中加入环境变量

1
2
3
export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
export KYLIN_HOME=/opt/apache-kylin-4.0.0-bin-spark2
export PATH=$KYLIN_HOME/bin:$JAVA_HOME/bin:$PATH

3. Caused by: java.lang.NoSuchFieldError: METASTORE_CLIENT_FILTER_ENABLED

原因：hive连接jar包与hive版本不一致

解决方案：到hive中把连接jar包copy到kylin中