kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nixon Rodrigues <nixon.rodrig...@freestoneinfotech.com>
Subject java.lang.OutOfMemoryError: Requested array size exceeds VM limit
Date Wed, 10 Oct 2018 13:36:52 GMT
Kafka community,

Getting *java.lang.OutOfMemoryError: Requested array size exceeds VM *
*limit,*  when Atlas hook in Hive tries to send Kafka message.
Let me know if any workaround possible on this issue or this is because of
JDK issue https://bugs.openjdk.java.net/browse/JDK-8154035.


2018-09-10 15:12:57,961 ERROR HiveHook.java:160 - HiveHook.run():
failed to process operation QUERY *java.lang.OutOfMemoryError:
Requested array size exceeds VM limit
*at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
at java.lang.StringCoding.encode(StringCoding.java:344)
at java.lang.String.getBytes(String.java:918)
at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:47)
at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:28)
at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:65)
at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:55)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:783)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:760)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:648)
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:197)
at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:182)
at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:117)
at org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:143)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:128)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:181)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:157)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:52)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1619)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message