kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [kylin] hit-lacus edited a comment on issue #1172: KYLIN-4385 Fix HiveProducer can not write to Hive Table(AWS S3)
Date Wed, 01 Apr 2020 07:46:38 GMT
hit-lacus edited a comment on issue #1172: KYLIN-4385 Fix HiveProducer can not write to Hive
Table(AWS S3)
URL: https://github.com/apache/kylin/pull/1172#issuecomment-607089486
 
 
   # Test on AWS EMR 5.28
   
   - tomcat/webapps/kylin/WEB-INF/classes/kylinMetrics.xml
   
   ```xml
   <bean id="initMetricsManager" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
       <property name="targetClass" value="org.apache.kylin.metrics.MetricsManager"/>
       <property name="targetMethod" value="initMetricsManager"/>
       <property name="arguments">
           <list>
               <ref bean="hiveSink"/>
               <map key-type="org.apache.kylin.metrics.lib.ActiveReservoir" value-type="java.util.List">
                   <entry key-ref="blockingReservoir">
                       <list>
                           <bean class="org.apache.kylin.common.util.Pair">
                               <property name="first"
                                         value="org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter"/>
                               <property name="second">
                                   <props>
                                       <prop key="fs.defaultFS">s3://YOUR_BUCKET</prop>
                                   </props>
                               </property>
                           </bean>
                       </list>
                   </entry>
               </map>
           </list>
       </property>
   </bean>
   ``` 
   
   - conf/kylin.properties
   
   ```properties
   kylin.metadata.url=kylin_metadata_failover@hbase
   kylin.env.hdfs-working-dir=s3://xiaoxiang-yu/kylin/
   kylin.storage.hbase.cluster-fs=s3://xiaoxiang-yu
   kylin.source.hive.redistribute-flat-table=false
   kylin.source.hive.metadata-type=gluecatalog
   
   kylin.engine.spark-conf.spark.eventLog.dir=s3\://xiaoxiang-yu/kylin/spark-history
   kylin.engine.spark-conf.spark.history.fs.logDirectory=s3\://xiaoxiang-yu/kylin/spark-history
   
   kylin.query.max-return-rows=10000000
   kylin.engine.spark-conf.spark.yarn.archive=s3://xiaoxiang-yu/package/kylin_binary_AWS_GLUE/spark-libs.jar
   
   
   
   kylin.metrics.monitor-enabled=true
   kylin.metrics.reporter-job-enabled=true
   kylin.metrics.reporter-query-enabled=true
   kylin.web.dashboard-enabled=true
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message