pulsar-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Pulsar Slack" <apache.pulsar.sl...@gmail.com>
Subject Slack digest for #general - 2019-04-07
Date Sun, 07 Apr 2019 09:11:01 GMT
2019-04-06 13:34:08 UTC - Ryan Samo: Hey guys,
I’m checking out Sinks in Pulsar IO. It looks like a Sink it just a wrapped Consumer, is
that a correct assessment? Just asking so I can look into adjusting performance a bit with
params such as receiverQueueSize, etc. This is in regards to building custom sinks or maybe
even adjusting sinks that are currently available.
----
2019-04-06 13:40:54 UTC - Samuel Sun: Hi , one question for buffer of MESSAGE(aaaa, 61 61
61 61):
```
00 00 00 48 00 00 00 17 08 09 4A 13 08 00 12 0F 
08 41 10 19 18 FF FF FF FF FF FF FF FF FF 01 0E 
01 D5 CA 13 EE 00 00 00 1F 0A 14 70 72 6F 64 75 
63 65 72 5F 6E 61 6D 65 5F 73 69 6E 67 6C 65 10 
00 18 9B CF BD 96 9F 2D 61 61 61 61 00 00 00 48 
00 00 00 17 08 09 4A 13 08 00 12 0F 08 41 10 1A 
18 FF FF FF FF FF FF FF FF FF 01 0E 01 0A 1B 10 
DB 00 00 00 1F 0A 14 70 72 6F 64 75 63 65 72 5F 
6E 61 6D 65 5F 73 69 6E 67 6C 65 10 00 18 8B E7 
BD 96 9F 2D 61 61 61 61 
```
can I know which 4 bytes are checksum in this buffer ? according to function here <https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/api/Commands.java#L339>
----
2019-04-06 16:47:09 UTC - Jannis: @Jannis has joined the channel
----
2019-04-06 18:46:23 UTC - Ali Ahmed: @Ryan Samo sinks are wrapper over pulsar functions
----
2019-04-06 19:36:15 UTC - Ryan Samo: @Ali Ahmed yup I see that now, just wondering if functions
are wrapping consumers and producers so that I can potentially downcast to get additional
functionality that is not immediately exposed to the IO or Function interface?
----
2019-04-06 19:37:21 UTC - Ali Ahmed: No I don’t think you can downcast to get access to
the producer consumer instance directly
----
2019-04-06 19:38:34 UTC - Ryan Samo: Ok thanks, my problem was with having a sink that couldn’t
keep up with a message volume but my consumers can. Just looking at options to address the
sink backlog 
----
2019-04-06 19:39:04 UTC - Ryan Samo: Using partitions and parallelism does help, just not
enough 
----
2019-04-06 19:39:49 UTC - Ali Ahmed: I don’t there is bottleneck in sink framework the implementation
might have something to do with it
----
2019-04-06 19:43:19 UTC - Ryan Samo: Yeah I think is on the receiving end, I’ll keep tweaking,
thanks!
----
2019-04-06 21:51:51 UTC - Emma Pollum: I'm running into an issue with 2.3.0 proxy when tokens
are enabled. I'm getting the following stack trace
----
2019-04-06 21:52:30 UTC - Emma Pollum: 
----
2019-04-06 21:53:02 UTC - Emma Pollum: Some research showed me you are aware of similiar problem
with the broker and that I should manually specify the numHttpServerThreads, however that
configuration does not seem to exist in the proxy.conf
----
2019-04-06 21:54:04 UTC - Matteo Merli: It should be `httpNumThreads=8`
----
2019-04-06 21:56:09 UTC - Emma Pollum: Thanks!
----
2019-04-07 00:06:08 UTC - Emma Pollum: I'm trying to use tokens to authenticate through the
proxy, but now when I issue commands like bin/pulsar-client produce
I recieve this error
org.apache.pulsar.proxy.server.ProxyConnection - [/127.0.0.1:47434] Unable to authenticate:
Unsupported authentication mode: none
----
2019-04-07 00:08:22 UTC - Emma Pollum: I have tried submitting the command with --auth-params
and --auth-plugin
----
2019-04-07 00:52:47 UTC - Anqing Xu: Pulsar seems to have a superior architecture compared
to either Kafka or RabbitMQ. How come not many companies are using it? I know Yahoo has been
using it internally. A Chinese company named Zhaopin is also using it
----
2019-04-07 01:54:14 UTC - Sanjeev Kulkarni: @Ryan Samo you can set the recieveQueueSize as
part of the input spec in Sinkconfig. Look inside the ConsumerConfig class of SinkConfig
----
2019-04-07 01:55:46 UTC - Sanjeev Kulkarni: @Anqing Xu Pulsar came out of Yahoo into open
source just in 2016.
----
2019-04-07 01:56:38 UTC - Ryan Samo: Awesome @Sanjeev Kulkarni, that’s exactly what I was
looking for. I appreciate it!
----
2019-04-07 08:31:38 UTC - young: @young has joined the channel
----
Mime
View raw message