drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Bates <jba...@maprtech.com>
Subject How do I make json files les painful
Date Thu, 19 Mar 2015 16:22:24 GMT
I constantly, constantly, constantly hit this.

I have json files that are just a huge collection of an array of json


My issue is in exploring the data, I hit this.

Query failed: Query stopped., Record was too large to copy into vector. [
39186288-2e01-408c-b886-dcee0a2c25c5 on maprdemo:31010 ]

I can explore csv, tab, maprdb, hive at fairly large data sets and limit
the response to what fits in my system limitations but not json in this

The two options I have come up with to move forward are..

   1. I strip out 90% of the array values in a file and explore that to get
   to my view. then go to a larger system and see if I have enough to get the
   job done.
   2. Move to the larger system and explore there taking resources that
   don't need to be spent on a science project.

Hoping the smart people have a different option for me,


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message