From user-return-69252-apmail-spark-user-archive=spark.apache.org@spark.apache.org Fri Apr 21 07:22:49 2017 Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 76F9619641 for ; Fri, 21 Apr 2017 07:22:49 +0000 (UTC) Received: (qmail 55661 invoked by uid 500); 21 Apr 2017 07:22:44 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 55540 invoked by uid 500); 21 Apr 2017 07:22:44 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 55530 invoked by uid 99); 21 Apr 2017 07:22:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Apr 2017 07:22:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id E7D63C0375 for ; Fri, 21 Apr 2017 07:22:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.39 X-Spam-Level: ** X-Spam-Status: No, score=2.39 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id X4bx_cOp1exi for ; Fri, 21 Apr 2017 07:22:41 +0000 (UTC) Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com [209.85.218.53]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id F0D6C5F39D for ; Fri, 21 Apr 2017 07:22:35 +0000 (UTC) Received: by mail-oi0-f53.google.com with SMTP id x184so82903301oia.1 for ; Fri, 21 Apr 2017 00:22:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Ya4gJ0jcFdb1L96FkfmQYVkcc29VnJLn7cCkE6aFwQ8=; b=KI0UJsNLfxJYVGAOrZCdlT31l7mQLH3OgXMeBWFAQOA8OTWlVA7wAT4nDi2dn9ZMGl 21LuyfwYF9zVoyJIPeKFQf7bIOIW6Thxr4Bux7gsjbS6sh4ncS+EtIU5PIwRXmb+hfnH +OsGWf5HzE5449CX7sOTB+CyeJ/PNBsszUwcyPw1V8dIMEesU1DUokXtkb+GO2zmwgc3 vAc2igjikxURk5DYPtkNScTWhPNqmRs6bJOsGGy4IcQHup7lipVY08SUMPum3lKq55k2 Q/Jgo+dqs+DSLEV9ZCvAYMNhBdNWI2Ys7gUeiYlcsaw+U5aWw6AqcvhWE0BgwwA36AK3 /I+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Ya4gJ0jcFdb1L96FkfmQYVkcc29VnJLn7cCkE6aFwQ8=; b=shqX9SSkcPV+3m17/SqkEpwRotLCcWTZ05b0h85VJVJtbmVdJaqYF5fyPDYC8z4D7o j+FfgOMNivGzebvq12h2yPEMl1M3X7VavkkyWco84JXuZCquM886mFOVwB619tgh1nbQ EIrkluttxQlus7L6Q+b3QrDh+RrozBLy3AhP7ctoH7ad60GLi9ai50Z5/aGPZ1OUmzdX mZ7kzyihQvDjWBt3wt572Q85jKHEfKt5rWaiaytZWdQruQeDoVVydHM+2hCZC2hGjRme Ydry4Q4XRliT9nWSP9qgbLB1I3dCxJwiRjDm0a3lPO7P1ebM9ZMAgM9zIF93z1QNnBmE EqUw== X-Gm-Message-State: AN3rC/752IQzhVWFzIOgcxgokizLjxstLEotRvQxgk8Echb77KTdl5uz 39rwZB31ybA1PZf9oSKwWlce+p4/Dw== X-Received: by 10.98.133.145 with SMTP id m17mr11179260pfk.51.1492759349305; Fri, 21 Apr 2017 00:22:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.144.151 with HTTP; Fri, 21 Apr 2017 00:22:28 -0700 (PDT) In-Reply-To: References: <1492746633772.1921785828@boxbe> From: ayan guha Date: Fri, 21 Apr 2017 17:22:28 +1000 Message-ID: Subject: Re: Azure Event Hub with Pyspark To: Denny Lee Cc: Nan Zhu , Ashish Singh , user Content-Type: multipart/alternative; boundary=94eb2c0bba74c02666054da81e0b --94eb2c0bba74c02666054da81e0b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi I need either all scala or all python(preferable). I will check the one suggested by Denny, thanks. On Fri, Apr 21, 2017 at 3:13 PM, Denny Lee wrote: > As well, perhaps another option could be to use the Spark Connector to > DocumentDB (https://github.com/Azure/azure-documentdb-spark) if sticking > with Scala? > On Thu, Apr 20, 2017 at 21:46 Nan Zhu wrote: > >> DocDB does have a java client? Anything prevent you using that? >> >> Get Outlook for iOS >> ------------------------------ >> *From:* ayan guha >> *Sent:* Thursday, April 20, 2017 9:24:03 PM >> *To:* Ashish Singh >> *Cc:* user >> *Subject:* Re: Azure Event Hub with Pyspark >> >> Hi >> >> yes, its only scala. I am looking for a pyspark version, as i want to >> write to documentDB which has good python integration. >> >> Thanks in advance >> >> best >> Ayan >> >> On Fri, Apr 21, 2017 at 2:02 PM, Ashish Singh >> wrote: >> >>> Hi , >>> >>> You can try https://github.com/hdinsight/spark-eventhubs : which is >>> eventhub receiver for spark streaming >>> We are using it but you have scala version only i guess >>> >>> >>> Thanks, >>> Ashish Singh >>> >>> On Fri, Apr 21, 2017 at 9:19 AM, ayan guha wrote: >>> >>>> [image: Boxbe] This message is >>>> eligible for Automatic Cleanup! (guha.ayan@gmail.com) Add cleanup rule >>>> >>>> | More info >>>> >>>> >>>> Hi >>>> >>>> I am not able to find any conector to be used to connect spark >>>> streaming with Azure Event Hub, using pyspark. >>>> >>>> Does anyone know if there is such library/package exists>? >>>> >>>> -- >>>> Best Regards, >>>> Ayan Guha >>>> >>>> >>> >> >> >> -- >> Best Regards, >> Ayan Guha >> > --=20 Best Regards, Ayan Guha --94eb2c0bba74c02666054da81e0b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi

I need either all scala or all pytho= n(preferable). I will check the one suggested by Denny, thanks.=C2=A0
=

On Fri, Apr= 21, 2017 at 3:13 PM, Denny Lee <denny.g.lee@gmail.com> = wrote:
As well, perhaps another option co= uld be to use the Spark Connector to DocumentDB (https://github.com/Azur= e/azure-documentdb-spark) if sticking with Scala?
O= n Thu, Apr 20, 2017 at 21:46 Nan Zhu <zhunanmcgill@gmail.com> wrote:
DocDB does have a java client? Anything preven= t you using that?

Get Outlook for iOS
From: ayan guha <guha.ayan@gmail.com>
Sent: Thursday, April 20, 2017 9:24:03 PM
To: Ashish Singh
Cc: user
Subject: Re: Azure Event Hub with Pyspark
=C2=A0
Hi

yes, its only scala. I am looking for a pyspark version, as i want to = write to documentDB which has good python integration.=C2=A0

Thanks in advance

best
Ayan

On Fri, Apr 21, 2017 at 2:02 PM, Ashish Singh <ashishleo25@= gmail.com> wrote:
Hi ,

You can try=C2=A0https://github.com/hdinsight/spark-eventhubs= : which is eventhub receiver for spark streaming
We are using it but you have scala version only i guess


Thanks,
Ashish Singh

On Fri, Apr 21, 2017 at 9:19 AM, ayan guha <guha.ayan@gmai= l.com> wrote:
3D"Boxbe" This message is eligible for Au= tomatic Cleanup! (= guha.ayan@gmail.com) Add cleanup rule | More info

Hi

I am not able to find any conector to be used to connect spark streami= ng with Azure Event Hub, using pyspark.=C2=A0

Does anyone know if there is such library/package exists>?

--
Best Regards,
Ayan Guha





--
B= est Regards,
Ayan Guha



--
=
Best Rega= rds,
Ayan Guha
--94eb2c0bba74c02666054da81e0b--