nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie Wang <jam...@opentext.com>
Subject RE: [EXTERNAL] - Re: GetTwitter to stream tweets instead pull
Date Thu, 04 May 2017 23:18:53 GMT
Hi Joey,

Thanks for the information. The name actually plays only a small part as you indicated. But
I sort of got convinced it is pulling after reading the help documentation for GetTwitter.
You can see it here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.twitter.GetTwitter/.
The line "Pulls status changes from Twitter's streaming API" sort of got me to believe it
is a pull instead streaming. Also it'd be a good idea if possible to add a line to explicitly
document it is actually streaming. Thanks again for your note and my apology for the late
response.

Jamie

From: Joey Frazee [mailto:joey.frazee@icloud.com]
Sent: Wednesday, May 03, 2017 12:51 PM
To: users@nifi.apache.org
Subject: [EXTERNAL] - Re: GetTwitter to stream tweets instead pull

Jamie, can you explain a little bit more about what you’re looking for?

The GetTwitter processor is accessing the spritzer/decahose/firehouse, what have you, via
Twitter’s Hosebird library. This library is indeed streaming the Tweets from their sample
and filter APIs in the usual way with a persistent, chunk-encoded HTTP connection to https://stream.twitter.com/1.1/statuses/sample.json<https://urldefense.proofpoint.com/v2/url?u=https-3A__stream.twitter.com_1.1_statuses_sample.json&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=a8suuN1dyAV6Q9X5Z0tEns7WODrBD8VcmH0ctYLhJto&e=>
and https://stream.twitter.com/1.1/statuses/filter.json<https://urldefense.proofpoint.com/v2/url?u=https-3A__stream.twitter.com_1.1_statuses_filter.json&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=0TueQgZE0p1YdPjGiDCQ5IVHtDBL5h_C0DZsOFKpEsA&e=>.

I’ll admit the name might be a little confusing since the Get might suggest it’s hitting
one of the REST https://api.twitter.com/1.1/statuses/<https://urldefense.proofpoint.com/v2/url?u=https-3A__api.twitter.com_1.1_statuses_&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=-8N0_UEmgW8Yb56o43fS87OzYVjtoFlZmSBZzVTfa1Y&e=>
resources periodically instead of using a long-term HTTP connection.

-joey

On May 3, 2017, at 2:00 PM, Jamie Wang <jamiew@opentext.com<mailto:jamiew@opentext.com>>
wrote:

Hi,

I understand the built-in processor GetTwitter is a pull. Are there streaming based processor
for getting Tweets available? If no, any suggestions on how would l go by to build one?

Thanks
Jamie

Mime
View raw message