flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Perez (Jira)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-22436) twitter datastream connector hangs
Date Sat, 01 May 2021 13:45:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-22436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Perez updated FLINK-22436:
--------------------------------
    Summary: twitter datastream connector hangs  (was: twitter datastream connector gets Invalid
'expires' attribute)

> twitter datastream connector hangs
> ----------------------------------
>
>                 Key: FLINK-22436
>                 URL: https://issues.apache.org/jira/browse/FLINK-22436
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataStream, Connectors / Common
>         Environment: My environment is just simple local flink env, using the twitter
connector example.
>            Reporter: Jason Perez
>            Priority: Major
>
> Sorry for selecting Connectors/Common, Twitter didn't show up in the Connectors / "X"
list, I'm not sure why that is.
>  
> It appears the twitter datastream connector is getting an error previously mentioned
in a few other areas:
> https://issues.apache.org/jira/browse/HTTPCLIENT-1763
> [https://jira.spring.io/browse/SOCIAL-486]
>  
> basically I am following the example here:
> https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/twitter/TwitterExample.java
>  
> which just seems to run indefinitely but not have any data throughput/results or exceptions.
>  
> task manager log looks something like this:
>  
> {code:java}
> 2021-04-23 10:51:00,102 INFO  org.apache.flink.streaming.connectors.twitter.TwitterSource
 [] - Initializing Twitter Streaming API connection2021-04-23 10:51:00,234 INFO  com.twitter.hbc.httpclient.BasicClient
                      [] - New connection executed: flink-twitter-source, endpoint: /1.1/statuses/sample.json2021-04-23
10:51:00,234 INFO  org.apache.flink.streaming.connectors.twitter.TwitterSource  [] - Twitter
Streaming API connection established successfully2021-04-23 10:51:00,254 INFO  com.twitter.hbc.httpclient.ClientBase
                       [] - flink-twitter-source Establishing a connection2021-04-23 10:51:00,861
WARN  org.apache.http.client.protocol.ResponseProcessCookies       [] - Invalid cookie header:
"set-cookie: personalization_id="v1_eokcdF92+g1ZaDhF6ztjng=="; Max-Age=63072000; Expires=Sun,
23 Apr 2023 14:51:00 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None". Invalid 'expires'
attribute: Sun, 23 Apr 2023 14:51:00 GMT2021-04-23 10:51:00,861 WARN  org.apache.http.client.protocol.ResponseProcessCookies
      [] - Invalid cookie header: "set-cookie: guest_id=v1%3A161918946086118384; Max-Age=63072000;
Expires=Sun, 23 Apr 2023 14:51:00 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None".
Invalid 'expires' attribute: Sun, 23 Apr 2023 14:51:00 GMT2021-04-23 10:51:00,865 INFO  com.twitter.hbc.httpclient.ClientBase
                       [] - flink-twitter-source Processing connection data2021-04-23 10:51:40,448
INFO  org.apache.flink.runtime.taskmanager.Task                    [] - Attempting to cancel
task Source: Custom Source -> Flat Map -> Sink: Unnamed (1/1)#0 (360c4d19bf8443d3535c256f89f38215).2021-04-23
10:51:40,448 INFO  org.apache.flink.runtime.taskmanager.Task                    [] - Source:
Custom Source -> Flat Map -> Sink: Unnamed (1/1)#0 (360c4d19bf8443d3535c256f89f38215)
switched from RUNNING to CANCELING.2021-04-23 10:51:40,449 INFO  org.apache.flink.runtime.taskmanager.Task
                   [] - Triggering cancellation of task code Source: Custom Source -> Flat
Map -> Sink: Unnamed (1/1)#0 (360c4d19bf8443d3535c256f89f38215).2021-04-23 10:51:40,451
INFO  org.apache.flink.streaming.connectors.twitter.TwitterSource  [] - Cancelling Twitter
source2021-04-23 10:51:40,452 INFO  org.apache.flink.streaming.connectors.twitter.TwitterSource
 [] - Closing source2021-04-23 10:51:40,452 INFO  com.twitter.hbc.httpclient.BasicClient 
                     [] - Stopping the client: flink-twitter-source, endpoint: /1.1/statuses/sample.json2021-04-23
10:51:40,452 INFO  com.twitter.hbc.httpclient.ClientBase                        [] - flink-twitter-source
exit event - Stopped by user: waiting for 5000 ms2021-04-23 10:51:40,547 INFO  org.apache.flink.streaming.connectors.twitter.TwitterSource
 [] - Closing source2021-04-23 10:51:40,548 INFO  com.twitter.hbc.httpclient.BasicClient 
                     [] - Stopping the client: flink-twitter-source, endpoint: /1.1/statuses/sample.json2021-04-23
10:51:40,585 INFO  com.twitter.hbc.httpclient.ClientBase                        [] - flink-twitter-source
Done processing, preparing to close connection2021-04-23 10:51:40,595 INFO  com.twitter.hbc.httpclient.ClientBase
                       [] - flink-twitter-source Shutting down httpclient connection manager2021-04-23
10:51:40,596 INFO  com.twitter.hbc.httpclient.BasicClient                       [] - Successfully
stopped the client: flink-twitter-source, endpoint: /1.1/statuses/sample.json2021-04-23 10:51:40,596
INFO  com.twitter.hbc.httpclient.BasicClient                       [] - Successfully stopped
the client: flink-twitter-source, endpoint: /1.1/statuses/sample.json2021-04-23 10:51:40,596
INFO  org.apache.flink.runtime.taskmanager.Task                    [] - Source: Custom Source
-> Flat Map -> Sink: Unnamed (1/1)#0 (360c4d19bf8443d3535c256f89f38215) switched from
CANCELING to CANCELED.
> {code}
> and this just seems to repeat in the log.
> I can provide more info if needed but my setup is really basic/simple.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message