asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raman Grover <ramangrove...@gmail.com>
Subject Re: Socket feed questions
Date Mon, 23 Nov 2015 21:24:47 GMT
I should have an update on this in next couple of days.
Sorry for the delay.

Regards,
Raman

On Wed, Nov 18, 2015 at 3:27 PM, Mike Carey <dtabass@gmail.com> wrote:

> Any update on this?
>
>
> On 11/2/15 5:54 AM, Raman Grover wrote:
>
>> Hi,
>>
>> I should be able to look at this one in a few days when I return and
>> resume
>> work (I am currently on a vacation till 5th Nov).
>>
>> However, the exception suggests that building the secondary feed pipeline
>> encountered an exception. Can you share the logs so that I can get a
>> better
>> understanding of the sequence of steps that happened as your statement
>> executed.
>>
>> I have intermittent access to net, but should be able to revert with some
>> delays.
>> On Nov 2, 2015 2:32 AM, "Jianfeng Jia" <jianfeng.jia@gmail.com> wrote:
>>
>> I’ve tried the “apply AQLfunction” idea, but got an NullPointException:
>>> java.lang.NullPointerException
>>>          at
>>>
>>> org.apache.asterix.aql.translator.AqlTranslator.getFeedJointKey(AqlTranslator.java:2268)
>>>          at
>>>
>>> org.apache.asterix.aql.translator.AqlTranslator.getFeedConnectionRequest(AqlTranslator.java:2214)
>>>          at
>>>
>>> org.apache.asterix.aql.translator.AqlTranslator.handleConnectFeedStatement(AqlTranslator.java:2130)
>>>          at
>>>
>>> org.apache.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:362)
>>>          at
>>>
>>> org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:114)
>>>          at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
>>>          at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>>>          at
>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
>>>          at
>>>
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
>>>          at
>>>
>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>>>          at
>>>
>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
>>>          at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
>>>          at
>>>
>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
>>>          at
>>>
>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
>>>          at
>>>
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>          at
>>>
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>          at org.eclipse.jetty.server.Server.handle(Server.java:347)
>>>          at
>>>
>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
>>>          at
>>>
>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
>>>          at
>>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
>>>          at
>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
>>>          at
>>>
>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
>>>          at
>>>
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
>>>          at
>>>
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
>>>          at
>>>
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>          at java.lang.Thread.run(Thread.java:745)
>>>
>>> The ddl that I was using is here:
>>>
>>> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
>>> <
>>>
>>> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
>>> Anyone has any idea? Thank you!
>>>
>>> On Oct 30, 2015, at 12:30 AM, Heri Ramampiaro <heriram@gmail.com> wrote:
>>>>
>>>> Although I haven’t tested this (I have mostly used & created java-based
>>>>
>>> UDFs),
>>>
>>>> one can execute AQL calls from within a UDF. Feeds allows functions to
>>>> execute arbitrary AQL statements (DDL, DMLs etc). I.e., I believe what
>>>> you are trying to do is possible.
>>>>
>>>> For example you could do:
>>>> (Given that you have a rec. type called Tweet, and a dataset
>>>>
>>> ProcessedTweets
>>>
>>>> create feed CoordTwitterFeed if not exists
>>>> using “push_twitter" (("type-name"="Tweet”))
>>>>        apply function find-intersection;
>>>>
>>>> (Here "find-intersection” is an AQL function that does the step
>>>>
>>> specified under 2)).
>>>
>>>> To do nr. 3 the easiest way is to have a modified version of the
>>>>
>>> “hashTag”
>>>
>>>> (let’s call this “tweetlocator”).  Java-based UDF (see my previous
>>>>
>>> message with the
>>>
>>>> TweetLib example). You can then connect this  as a secondary feed,
>>>>
>>> connected to
>>>
>>>> “CoordTwitterFeed”.
>>>>
>>>> E.g.:
>>>> create secondary feed ProcessedTwitterFeed from feed CoordTwitterFeed
>>>> apply function “tweelib#tweetlocator";
>>>>
>>>> Thereafter you can connect the feeds to appropriate datasets.
>>>>
>>>> (Since the find-intersection is an AQL function, you can also call
>>>>
>>> “tweelib#tweetlocator”
>>>
>>>> inside this function).
>>>>
>>>> Best,
>>>> -heri
>>>>
>>>> On Oct 30, 2015, at 12:18 AM, Young-Seok Kim <kisskys@gmail.com> wrote:
>>>>>
>>>>> I'm not sure whether the following UDF is possible or not, but
>>>>>
>>>> hopefully it
>>>
>>>> is.
>>>>> What we're trying to do is to have the following UDF.
>>>>>
>>>>> The UDF
>>>>> 1) accepts an incoming tweet record through a feed job as an input and
>>>>>
>>>> then
>>>
>>>> 2) takes a field, more specifically, coordinate field value from the
>>>>>
>>>> tweet
>>>
>>>> record and sends a spatial-intersect query using the coordinate in
>>>>>
>>>> order to
>>>
>>>> find out the corresponding county of the coordinates (we have created
>>>>> AsterixDB instance which stores US county shapes records into a dataset
>>>>>
>>>> and
>>>
>>>> created R-tree index on the polygon field of it, so the query will
>>>>>
>>>> return
>>>
>>>> the county effectively using the R-tree index)
>>>>> 3) creates a new tweet record consisting of the original record's
>>>>>
>>>> fields +
>>>
>>>> the returned county value
>>>>> 4) ingests to a tweet dataset in the AsterixDB instance.
>>>>>
>>>>> Can we have such an UDF?
>>>>>
>>>>> Best,
>>>>> Young-Seok
>>>>>
>>>>>
>>>>> On Thu, Oct 29, 2015 at 3:53 PM, Heri Ramampiaro <heriram@gmail.com>
>>>>>
>>>> wrote:
>>>
>>>> 2. Can we use AQL function in those kind of feed UDFs?
>>>>>>>
>>>>>> Can you give an example of what you are trying to do?
>>>>>> I.e do you want to run an AQL inside a UDF or use an AQL function
as a
>>>>>> UDF connected to a running feed?
>>>>>>
>>>>>> -heri
>>>>>>
>>>>>
>>>
>>> Best,
>>>
>>> Jianfeng Jia
>>> PhD Candidate of Computer Science
>>> University of California, Irvine
>>>
>>>
>>>
>


-- 
Raman

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message