pulsar-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Pulsar Slack" <apache.pulsar.sl...@gmail.com>
Subject Slack digest for #general - 2019-01-30
Date Wed, 30 Jan 2019 09:11:03 GMT
2019-01-29 09:32:32 UTC - jia zhai: right, if also need the config in bk and broker, it need
a re-config and reboot
----
2019-01-29 11:01:20 UTC - Yuvaraj Loganathan: What is the right order to upgrade the cluster
? Rolling update Bookkeeper nodes  and rolling update broker and then zookeeper nodes? is
there any doc available ?
----
2019-01-29 13:37:25 UTC - jia zhai: Seems there is no doc for upgrade currently.
The order of bk - &gt; broker -&gt; zk seems good
----
2019-01-29 17:34:55 UTC - Emma Pollum: I'm consistently seeing this message in my pulsar logs
INFO  org.apache.pulsar.broker.admin.v2.NonPersistentTopics - [null] Namespace bundle is not
owned by any broker prod/prod/0x1a800000_0x1b000000
What is this referring to?
----
2019-01-29 21:18:59 UTC - Grant Wu: I’m getting some weird problems with Pulsar, seems related
to my use of Pulsar functions:
----
2019-01-29 21:20:01 UTC - Grant Wu: 
----
2019-01-29 21:35:27 UTC - Grant Wu: Actually, scratch that, we had it happen once without
Pulsar functions being involved
----
2019-01-29 21:38:35 UTC - Ali Ahmed: @Emma Pollum do you have more details
----
2019-01-29 21:42:20 UTC - Shalin: Is there a max redelivery configuration that can be set
for python consumer client?
----
2019-01-29 21:45:14 UTC - Grant Wu: @Shalin does `acknowledge_cumulative` work?
----
2019-01-29 21:46:36 UTC - Shalin: Not quite. I am thinking in terms of dead letter topic,
<https://github.com/apache/pulsar/pull/2400>
----
2019-01-29 22:59:49 UTC - Dilara Çakır: @Dilara Çakır has joined the channel
----
2019-01-29 23:01:24 UTC - Patrick Johnmeyer: @Patrick Johnmeyer has joined the channel
----
2019-01-29 23:01:28 UTC - Dilara Çakır: Hi everyone I'm Dilara
wave : Matteo Merli, Ali Ahmed, Karthik Ramasamy, Joe Francis, bossbaby, Nazrvl
----
2019-01-29 23:05:34 UTC - Grant Wu: :wave:
----
2019-01-29 23:55:59 UTC - Vincent Ngan: Does a message id carry information about its ordering?
Can I compare two message ids to determine their ordering? For example, if (messageId1 &lt;
messageId2) means message1 comes before message2.
----
2019-01-29 23:56:48 UTC - Matteo Merli: Yes, `MessageId` interface extends `Comparable&lt;MessageId&gt;`
----
2019-01-29 23:57:45 UTC - Matteo Merli: Even though it’s “opaque”, it can be compared.
The “comes before” here would mean “was written earlier in the topic”
----
2019-01-30 01:10:59 UTC - Vincent Ngan: Thanks for your explanation. I have another question
about the ordering of messages regarding the use of async call. If I send two messages in
parallel using async. The ordering of these two messages is determined by (A) the ordering
of making two async calls; (B) the ordering of the actual completion sequence of the async
calls; or (C) the ordering information of the message ids returned when the async calls complete?
----
2019-01-30 01:22:51 UTC - Matteo Merli: All three A, B and C are guaranteed to be the same
----
2019-01-30 01:32:57 UTC - Vincent Ngan: Great! Thanks a lot.
----
2019-01-30 01:41:05 UTC - Vincent Ngan: So, that means the Pulsar client serialises certain
part of its logic to make sure that, even for async calls, the ordering of the messages written
to the topic is same as the calling sequence of the async calls.
----
2019-01-30 01:46:16 UTC - Joe Francis: @Matteo Merli how so?
----
2019-01-30 01:49:57 UTC - Matteo Merli: * Messages are enqueued based on the sequence of `sendAsync()`
calls
 * Producer maintains items in queue until an ack is received
 * If there’s a failure, producer will re-send all the pending items in the queue (in same
order)
 * Broker will send acks back in the same order. There’s validation in producer that a successful
ack from broker will have to match the 1st item in the queue
 * Completable futures from send operations are therefore triggered in order
 * Message Id for these published messages will be in order too
----
2019-01-30 01:50:31 UTC - Matteo Merli: &gt; So, that means the Pulsar client serialises
certain part of its logic to make sure that, even for async calls, the ordering of the messages
written to the topic is same as the calling sequence of the async calls.

Yes, a portion of `sendAsync()` is protected by mutex (only for the duration of the enqueuing)
----
2019-01-30 01:51:50 UTC - Joe Francis: But parallel async calls - is there a guarantee between
them?  I thought before entering the queue, there is no ordering among threads.
----
2019-01-30 01:55:05 UTC - Matteo Merli: If you have 2 threads calling sendAsync(), there’s
an inherent race. The 2 calls will be serialized (in some way) though that’s non-deterministic.

What is deterministic is if you have one single thread doing:

```java
producer.sendAsync("Hello-1");
producer.sendAsync("Hello-2");
producer.flush();
```
----
2019-01-30 01:55:58 UTC - Matteo Merli: In this case, it’s guaranteed that “Hello-1”
will always come before “Hello-2", even if there are other threads writing on the same producer
instance
----
2019-01-30 01:57:15 UTC - Joe Francis: I think @Vincent Ngan qn was about parallel calls -
and I dont know whether we return comparables when the async call returns to see who entered
the queue first - Do we?
----
2019-01-30 02:01:35 UTC - Matteo Merli: &gt; If I send two messages in parallel using
async. The ordering of these two messages is determined by (A) the ordering of making two
async calls;

More that “in parallel” it would be “pipelining”.

&gt; and I dont know whether we return comparables when the async call returns to see
who entered the queue first - Do we?

`sendAsync()` just returns the future, to get the actual message id one has to wait until
the future is completed
----
2019-01-30 02:02:09 UTC - Matteo Merli: so, you cannot compare the futures directly, though
they will be triggered in the same order
----
2019-01-30 06:56:33 UTC - bossbaby: Once zookeeper stops or dies, how to recover data from
the bookie when connecting to another zookeeper?
----
2019-01-30 08:34:44 UTC - jia zhai: @bossbaby zookeeper contains the metadata of bookkeeper
cluster. It would be troublesome if the data in zookeeper lost.
----
Mime
View raw message