pulsar-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Pulsar Slack" <apache.pulsar.sl...@gmail.com>
Subject Slack digest for #general - 2019-04-11
Date Thu, 11 Apr 2019 09:11:02 GMT
2019-04-10 09:11:54 UTC - Olivier Chicha: @Ali Ahmed Thank you for you answer.
I am not familiar with Erlang / Elixir, but another entity of our company is using Elixir,
and we would like to make sure that they will be able to use pulsar.
If I understands correctly what you say is that from Elixir they could access through a wrapper
(they would need to develop, I guess) to the C++ implementation of the client.
Is that correct?
2019-04-10 13:21:09 UTC - bhagesharora: @Matteo Merli
I am implementing message deduplication through java, I have one produce class which is reading
a text file, text file hold(1, 2, 3, 4, 5) String type Data, and we have consume class which
is reading same Data.
When I am running/pushing messages from producer class I am getting error..
[main] WARN org.apache.pulsar.shade.com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed
to load Circe JNI library. Falling back to Java based CRC32c provider
Exception in thread "main" java.util.NoSuchElementException
        at java.util.Scanner.throwFor(Scanner.java:862)
        at java.util.Scanner.next(Scanner.java:1371)
        at pulsar_producer_consumer.pulsar_producer_consumer.MessageDeduplicationProducer1.main(MessageDeduplicationProducer1.java:83)
[pulsar-timer-4-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [<persistent://public/default/topic-1>]
[standalone-7-3] Pending messages: 0 --- Publish throughput: 0.03 msg/s --- 0.00 Mbit/s ---
Latency: med: 207.656 ms - 95pct: 207.656 ms - 99pct: 207.656 ms - 99.9pct: 207.656 ms - max:
207.656 ms --- Ack received rate: 0.03 ack/s --- Failed messages: 0
When I am running Consumer class or trying to consume same messages, so I can see in output,
it's consuming only two messages, see output below
Created Pulsar consumer
[pulsar-client-io-1-1] WARN org.apache.pulsar.shade.com.scurrilous.circe.checksum.Crc32cIntChecksum
- Failed to load Circe JNI library. Falling back to Java based CRC32c provider
inside loop 2
inside loop 4
[pulsar-timer-4-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - [<persistent://public/default/topic-1>]
[tutorial-subscription] [05a87] Prefetched messages: 0 --- Consume throughput: 0.03 msgs/s
--- Throughput received: 0.00 msg/s --- 0.03 Mbit/s --- Ack sent rate: 0 ack/s --- Failed
messages: 0 --- Failed acks: {}

Why Its not able to consume all messages, Why It's consuming even no, or producing odd no.
2019-04-10 13:23:43 UTC - Matteo Merli: Created <https://github.com/apache/pulsar/pull/4015>
as a pre-req
2019-04-10 14:21:46 UTC - Olivier Chicha: Hello I would like, for the purpose of unit test
to embed multiple pulsar broker (+1zookeeper +1bookeeper) inside a single JVM
I assume this is feasible by copying and modifying the code of PulsarStandalone, but I was
wondering if there is already a project / class that would provide that.
2019-04-10 17:08:30 UTC - Bicou: @Bicou has joined the channel
2019-04-10 18:04:11 UTC - Ali Ahmed: currently yes that wrapper has not been implemented yet
since no one requested it.
2019-04-10 18:37:56 UTC - Steve Kim: Thanks for your help. I looked at `BlobStoreBackedInputStreamImpl`
and `BlobStoreBackedReadHandleImpl` to understand how performance could be improved. I think
that reads from blob store would be faster with speculative read-ahead, so that `refillBufferIfNeeded`
can swap an already-populated buffer instead of blocking on a call to `blobStore.getBlob`.
2019-04-10 19:35:58 UTC - Douglas Mutart: @Douglas Mutart has joined the channel
2019-04-10 22:34:49 UTC - Ali Ahmed: @Olivier Chicha you can use this
2019-04-11 02:27:59 UTC - hao zhang: @hao zhang has joined the channel
2019-04-11 04:01:09 UTC - jia zhai: :+1:
2019-04-11 04:05:48 UTC - jia zhai: Yes, we should do that.
Are you willing to contribute to this feature?:slightly_smiling_face:
2019-04-11 04:08:02 UTC - jia zhai: Opened Issue <https://github.com/apache/pulsar/issues/4025>
to track it.
2019-04-11 07:52:30 UTC - Kev Jackson: just reading the deployment instructions and we would
like to use a pre-deployed/defined zookeeper cluster - is this comment in the documentation
valid " You'll need to install the Pulsar binary package on each machine in the cluster, including
machines running ZooKeeper and BookKeeper."
2019-04-11 07:53:35 UTC - Ali Ahmed: you can just connect to an existing zookeeper instance
2019-04-11 07:55:18 UTC - Kev Jackson: ok so I'm going to deploy a standard zookeeper cluster
and then deploy pulsar as a separate cluster - I will still need to download the pulsar installation
tarball so I can run this step as it requires a local connection <https://pulsar.apache.org/docs/en/deploy-bare-metal/#initializing-cluster-metadata>
2019-04-11 08:14:27 UTC - Kev Jackson: what happens if I execute: `bin/initialize-cluster-metadata`
with the same configuration twice? is it idempotent (ie. safe, the second initialize is a
2019-04-11 08:37:54 UTC - Sijie Guo: it should be idemponent. the second one is a no-op
View raw message