samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com.INVALID>
Subject Re: How to get number of partitions for a downstream topic?
Date Thu, 18 Dec 2014 19:11:56 GMT
Hey Roger,

You shouldn't need to use Scala, though you can if you want. The
SystemFactory and SystemAdmin APIs are both Java-based.

Set<String> streamNames = new HashSet<String>();
streamNames.add("my-topic");
SystemAdmin kafkaSystemAdmin = new KafkaSystemFactory().getAdmin("kafka",
config);
Map<String, SystemStreamMetadata> metadata =
SyskafkaSystemAdmin.getSystemStreamMetadata(streamNames);

You can get the config object that you need from InitableTask.config().

Cheers,
Chris

On 12/18/14 10:37 AM, "Roger Hoover" <roger.hoover@gmail.com> wrote:

>Looking around the code base, I think it's only available in Scala
>(samza-core).  That's ok.  I can write the job in Scala.  Need a little
>help on how to instantiate the object though...
>
>On Thu, Dec 18, 2014 at 10:18 AM, Roger Hoover <roger.hoover@gmail.com>
>wrote:
>>
>> To do this, I think I need a SystemFactory object so that I can
>> call getAdmin() on it to get the SystemAdmin object you mentioned.  Can
>>you
>> give me a pointer on how to do that in the task init()?
>>
>> Thanks!
>>
>> On Wed, Dec 17, 2014 at 5:20 PM, Roger Hoover <roger.hoover@gmail.com>
>> wrote:
>>>
>>> Thanks, Chris!
>>>
>>> Sent from my iPhone
>>>
>>> > On Dec 17, 2014, at 5:10 PM, Chris Riccomini
>>> <criccomini@linkedin.com.INVALID> wrote:
>>> >
>>> > Hey Roger,
>>> >
>>> > You could instantiate a SystemAdmin for the topic in question. This
>>> > interface has a getSystemStreamMetadata call that will give you
>>> partition
>>> > information for arbitrary topics and systems.
>>> >
>>> > Cheers,
>>> > Chris
>>> >
>>> >> On 12/17/14 4:51 PM, "Roger Hoover" <roger.hoover@gmail.com> wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> Is there an easy way to get the number of partitions for a
>>>downstream
>>> >> topic?  Or would I have to instantiate a Kafka producer myself in
>>>the
>>> >> Task.init().
>>> >>
>>> >> I appreciate any suggestions.  Thanks,
>>> >>
>>> >> Roger
>>> >
>>>
>>


Mime
View raw message