samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukáš Havrlant <lu...@havrlant.cz>
Subject Re: How to set up RocksDB
Date Mon, 09 Nov 2015 17:22:03 GMT
Yes I think I saw something like "permission denied" in the logs. But I
cannot find the config that tells the Samza the destination folder. We
tried to set different -Djava.io.tmpdir but it didn't help. Can you please
give me a hint what config should it be?

2015-11-09 18:05 GMT+01:00 Tommy Becker <tobecker@tivo.com>:

> The native RocksDB libs are included in the jar file, but they are
> extracted at runtime before they can be loaded. Make sure the user running
> the job has permission to write to...wherever it is that it tries to put it
> (sorry don't know offhand). But you shouldn't have to set any environment
> variables or anything for it to work.
>
>
> On 11/09/2015 08:50 AM, Lukáš Havrlant wrote:
>
> Hello,
> we have a problem with configuring RocksDB as a key-value store with our
> Samza 0.9.0 project. We added this line to our config:
>
>
> stores.my-store.factory=org.apache.samza.storage.kv.RocksDbKeyValueStorageEngineFactory
>
> But then our Samza job couldn't locate the
> RocksDbKeyValueStorageEngineFactory class. So we added compile
> "org.apache.samza:samza-kv-rocksdb_2.10:$samzaVersion" to our gradle build
> script. It helped but now Samza failed to start because it couldn't
> find librocksdbjni-linux64.so library. We added export
> ROCKSDB_SHAREDLIB_DIR=/opt/samza/lib to our init script and it didn't help.
> Then we manually copied the librocksdbjni-linux64.so from
> the rocksdbjni-3.5.1.jar to the /opt/samza/lib folder and Samza still
> couldn't start, now because of
>
> 2015-11-09T14:35:06,209 INFO [main]
> org.apache.samza.container.SamzaContainer$ - Got store consumers:
> Map(tracking-info ->
> org.apache.samza.system.kafka.KafkaSystemConsumer@78461bc4)
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x0000003c4de092ab, pid=4410, tid=139728971949824
>
> And we're out of ideas at the moment. Can you please help me set up RocksDB
> correctly? I suppose we are doing something completely wrong.
>
> Is it possibble to use another key-value store at least?
>
> Lukáš Havrlant
>
>
>
> --
> Tommy Becker
> Senior Software Engineer
>
> Digitalsmiths
> A TiVo Company
>
> www.digitalsmiths.com<http://www.digitalsmiths.com>
> tobecker@tivo.com<mailto:tobecker@tivo.com>
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo Inc. is authorized to conclude any binding
> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> Inc. may only be made by a signed written agreement.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message