spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Takeshi Yamamuro <linguin....@gmail.com>
Subject Re: [Spark 2.0] Problem with Spark Thrift Server show NULL instead of showing BIGINT value
Date Thu, 04 Aug 2016 10:59:29 GMT
Hi,

When changing the long type into int one, does the issue also happen?
And also, could you show more simple query to reproduce the issue?

// maropu

On Thu, Aug 4, 2016 at 7:35 PM, Chanh Le <giaosudau@gmail.com> wrote:

>
> Hi everyone,
>
> I have a parquet file and it has data but when I use Spark Thrift Server
> to query it shows NULL for *userid.*
> As you can see I can get data by Spark Scala but STS is not.
>
>
> The file schema
> root
> |-- time: string (nullable = true)
> |-- topic_id: integer (nullable = true)
> |-- interest_id: integer (nullable = true)
> |-- inmarket_id: integer (nullable = true)
> |-- os_id: integer (nullable = true)
> |-- browser_id: integer (nullable = true)
> |-- device_type: integer (nullable = true)
> |-- device_id: integer (nullable = true)
> |-- location_id: integer (nullable = true)
> |-- age_id: integer (nullable = true)
> |-- gender_id: integer (nullable = true)
> |-- website_id: integer (nullable = true)
> |-- channel_id: integer (nullable = true)
> |-- section_id: integer (nullable = true)
> |-- zone_id: integer (nullable = true)
> |-- placement_id: integer (nullable = true)
> |-- advertiser_id: integer (nullable = true)
> |-- campaign_id: integer (nullable = true)
> |-- payment_id: integer (nullable = true)
> |-- creative_id: integer (nullable = true)
> |-- audience_id: integer (nullable = true)
> |-- merchant_cate: integer (nullable = true)
> |-- ad_default: integer (nullable = true)
> |-- userId: long (nullable = true)
> |-- impression: integer (nullable = true)
> |-- viewable: integer (nullable = true)
> |-- click: integer (nullable = true)
> |-- click_fraud: integer (nullable = true)
> |-- revenue: double (nullable = true)
> |-- proceeds: double (nullable = true)
> |-- spent: double (nullable = true)
> |-- network_id: integer (nullable = true)
>
>
> I create a table in Spark Thrift Server by.
>
> CREATE EXTERNAL TABLE *ad_cookie_report* (time String, advertiser_id int,
> campaign_id int, payment_id int, creative_id int, website_id int,
> channel_id int, section_id int, zone_id int, ad_default int, placment_id
> int, topic_id int, interest_id int, inmarket_id int, audience_id int, os_id
> int, browser_id int, device_type int, device_id int, location_id int,
> age_id int, gender_id int, merchant_cate int,* userid bigint*, impression
> int, viewable int, click int, click_fraud int, revenue double, proceeds
> double, spent double, network_id integer)
> STORED AS PARQUET LOCATION 'alluxio://master2:19998/AD_COOKIE_REPORT';
>
> But when I query it got all in  NULL values.
>
> *0: jdbc:hive2://master1:10000> select userid from ad_cookie_report limit
> 10;*
> *+---------+--+*
> *| userid  |*
> *+---------+--+*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *| NULL    |*
> *+---------+--+*
> *10 rows selected (3.507 seconds)*
>
> How to solve the problem? Is that related to field with Uppercase?
> How to change the field name in this situation.
>
>
> Regards,
> Chanh
>
>


-- 
---
Takeshi Yamamuro

Mime
View raw message