pulsar-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Pulsar Slack" <apache.pulsar.sl...@gmail.com>
Subject Slack digest for #general - 2019-01-11
Date Fri, 11 Jan 2019 09:11:04 GMT
2019-01-10 12:49:31 UTC - Chris Bartholomew: If you are using AWS EKS for Kubernetes, then
the LoadBalancer service type creates an NLB for you, no need to add an ingress controller.
This proxy service config works for me in EKS: ```apiVersion: v1
kind: Service
metadata:
  name: broker-proxy
  labels:
    app: pulsar
    component: proxy
spec:
  type: LoadBalancer
  ports:
    - name: pulsar
      port: 6650
      protocol: TCP
    - name: http
      port: 8080
      protocol: TCP
  selector:
    app: pulsar
    component: proxy
```
----
2019-01-10 12:50:00 UTC - Chris Bartholomew: More info about loadbalancer in EKS: <https://docs.aws.amazon.com/eks/latest/userguide/load-balancing.html>
----
2019-01-10 14:29:38 UTC - Enrico Olivelli: @Enrico Olivelli has joined the channel
----
2019-01-10 14:33:33 UTC - Enrico Olivelli: Hi, I am going to share bookies of an existing
cluster with a Pulsar cluster. Is there any "official" way to understand which ledger is from
Pulsar ? I have procedures which check ledgers and for each ledger I have to know which application
is the owner
----
2019-01-10 14:42:54 UTC - Shalin: Thanks @Chris Bartholomew. Unfortunately I am running my
own k8s cluster. Fortunately, I am only running my  own k8s cluster to get more hands on with
k8s before moving to a more managed EKS.
----
2019-01-10 14:43:09 UTC - Shalin: Also realized these questions are not relevant to pulsar
anymore.
----
2019-01-10 15:06:48 UTC - Ivan Kelly: @Enrico Olivelli I don't think so, we should start adding
something to the ledger custom metadata to make this easier to figure out
----
2019-01-10 15:42:01 UTC - Jack: In pulsar how to connect between 2 bookie in 2 vms to replicate
data
----
2019-01-10 16:06:37 UTC - Romain Castagnet: I just find my error... I used tlsEnabled=true
but I didn't configured TLS between brokers. Thank you for your help.
----
2019-01-10 16:41:50 UTC - Matteo Merli: @Enrico Olivelli the main reason we haven’t added
that metadata yet is to avoid the ledger metadata compatibility issues with the text based
protobuf. After bk 4.9 we’ll be able to move forward and tag all ledgers
----
2019-01-10 17:13:53 UTC - Matteo Merli: :+1:
----
2019-01-10 17:27:41 UTC - Emma Pollum: I'm trying to understand the bookkeeper-ack-quorum
for the persistence policies. Is the ack quorum saying it should wait for the message to be
acknowledged by two consumers before it is removed?
----
2019-01-10 17:28:20 UTC - Matteo Merli: No, ack quorum is the number of guaranteed replicas
of your data
----
2019-01-10 17:28:45 UTC - Matteo Merli: eg: write to 3 bookies (write-quorum) and wait for
at least 2 acks (ack-quorum)
----
2019-01-10 17:28:53 UTC - Emma Pollum: :thumbsup:
----
2019-01-10 17:35:29 UTC - Emma Pollum: What is "individually deleted messages" representative
of? Are these messages that have been deleted by a client, not expired by pulsar
----
2019-01-10 17:35:31 UTC - Emma Pollum: ?
----
2019-01-10 17:37:54 UTC - Matteo Merli: Consumers can acknowledge messages out-of-order. So
we keep track of messages acknoweledged (“deleted”) on an individual basis (with few tricks
to make it efficient). Once all messages prior to a certain positions are acked, then the
cursor will move forward and everything before that can be deleted from disk (again, that
deletion will not happen instantly, rather in background for efficiency reasons).
----
2019-01-10 17:39:06 UTC - Grant Wu: The deletion bit can be changed by tweaking retention
settings, I believe merlimat is describing the defaults.
----
2019-01-10 17:42:32 UTC - Emma Pollum: Do functions automatically ack messages?
----
2019-01-10 17:42:44 UTC - Matteo Merli: correct, there are many more tunables aroudn when
data gets phisically deleted
----
2019-01-10 17:43:19 UTC - Matteo Merli: by default, yes, you return a value, the value gets
published on downstream topic and the original message gets acked
----
2019-01-10 17:43:28 UTC - Grant Wu: When you say “functions”, are you referring to Pulsar
Functions?
----
2019-01-10 17:43:36 UTC - Matteo Merli: you can disable the auto ack and control it manually
----
2019-01-10 17:44:09 UTC - Emma Pollum: How do I disable that?
----
2019-01-10 17:58:10 UTC - Emma Pollum: And if I set a retention policy to 28 days does that
mean no messages will be deleted for 28 days?
----
2019-01-10 18:43:58 UTC - Matteo Merli: That’s correct, retention will keep these messages
around even if they were acknowledged
----
2019-01-10 18:44:26 UTC - Matteo Merli: then you can either use a Reader, or just reset the
subscription to an earlier timestamp
----
2019-01-10 20:54:49 UTC - Karthik Palanivelu: Cool Thanks @Matteo Merli it worked. I will
do a small write up. Thanks @Sijie Guo as well for your time.
----
2019-01-10 20:55:11 UTC - Matteo Merli: Good to hear!
----
2019-01-10 21:08:28 UTC - Victor Li: 
----
2019-01-10 21:09:23 UTC - Victor Li: @Matteo Merli @Sijie Guo Do you know what might be the
cause?
----
2019-01-11 01:50:26 UTC - Shalin: I am getting `Policies not found for public/default namespace`
when trying to create a producer for `my-topic`. Isn't `public/default` namespace created
by default in pulsar
----
2019-01-11 01:51:17 UTC - Shalin: This is when deploying pulsar on k8s cluster using helm
----
2019-01-11 07:00:43 UTC - Samuel Sun: ? actually what I meant is how to create a topic (no
partitions/ or only 1 partition) by command “pulsar-admin topics”
----
2019-01-11 09:04:23 UTC - bossbaby: Hi all, i was Deploy a cluster on bare metal successful,
then i test by stop 1 in 2 bookeeper and client be disconnect while server still have 1 bookeeper
running.
Please explain help me.
Thanks you all
----
Mime
View raw message