samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roger Hoover <roger.hoo...@gmail.com>
Subject Re: How to get number of partitions for a downstream topic?
Date Thu, 18 Dec 2014 19:14:52 GMT
Fantastic.  Much appreciated, Chris.

On Thu, Dec 18, 2014 at 11:11 AM, Chris Riccomini <
criccomini@linkedin.com.invalid> wrote:
>
> Hey Roger,
>
> You shouldn't need to use Scala, though you can if you want. The
> SystemFactory and SystemAdmin APIs are both Java-based.
>
> Set<String> streamNames = new HashSet<String>();
> streamNames.add("my-topic");
> SystemAdmin kafkaSystemAdmin = new KafkaSystemFactory().getAdmin("kafka",
> config);
> Map<String, SystemStreamMetadata> metadata =
> SyskafkaSystemAdmin.getSystemStreamMetadata(streamNames);
>
> You can get the config object that you need from InitableTask.config().
>
> Cheers,
> Chris
>
> On 12/18/14 10:37 AM, "Roger Hoover" <roger.hoover@gmail.com> wrote:
>
> >Looking around the code base, I think it's only available in Scala
> >(samza-core).  That's ok.  I can write the job in Scala.  Need a little
> >help on how to instantiate the object though...
> >
> >On Thu, Dec 18, 2014 at 10:18 AM, Roger Hoover <roger.hoover@gmail.com>
> >wrote:
> >>
> >> To do this, I think I need a SystemFactory object so that I can
> >> call getAdmin() on it to get the SystemAdmin object you mentioned.  Can
> >>you
> >> give me a pointer on how to do that in the task init()?
> >>
> >> Thanks!
> >>
> >> On Wed, Dec 17, 2014 at 5:20 PM, Roger Hoover <roger.hoover@gmail.com>
> >> wrote:
> >>>
> >>> Thanks, Chris!
> >>>
> >>> Sent from my iPhone
> >>>
> >>> > On Dec 17, 2014, at 5:10 PM, Chris Riccomini
> >>> <criccomini@linkedin.com.INVALID> wrote:
> >>> >
> >>> > Hey Roger,
> >>> >
> >>> > You could instantiate a SystemAdmin for the topic in question. This
> >>> > interface has a getSystemStreamMetadata call that will give you
> >>> partition
> >>> > information for arbitrary topics and systems.
> >>> >
> >>> > Cheers,
> >>> > Chris
> >>> >
> >>> >> On 12/17/14 4:51 PM, "Roger Hoover" <roger.hoover@gmail.com>
wrote:
> >>> >>
> >>> >> Hi all,
> >>> >>
> >>> >> Is there an easy way to get the number of partitions for a
> >>>downstream
> >>> >> topic?  Or would I have to instantiate a Kafka producer myself
in
> >>>the
> >>> >> Task.init().
> >>> >>
> >>> >> I appreciate any suggestions.  Thanks,
> >>> >>
> >>> >> Roger
> >>> >
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message