spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin Keunen <robin.keu...@lampiris.be>
Subject Re: Type problem in Java when using flatMapValues
Date Fri, 03 Oct 2014 07:33:38 GMT
Damn, you're right, I wasn't looking at it properly. I was confused by 
intelliJ I guess.

Many thanks!


On 2014-10-02 19:02, Sean Owen wrote:
> Eh, is it not that you are mapping the values of an RDD whose keys are
> StringStrings, but expecting the keys are Strings? That's also about
> what the compiler is saying too.
>
> On Thu, Oct 2, 2014 at 4:15 PM, Robin Keunen <robin.keunen@lampiris.be> wrote:
>> Hi all,
>>
>> I successfully implemented my algorithm in Scala but my team wants it in
>> Java. I have a problem with Generics, can anyone help me?
>>
>> I have a first JavaPairRDD with a structure like ((ean, key), [from, to,
>> value])
>>
>> ean and key are string
>> from and to are DateTime
>> value is a Double
>>
>> JavaPairRDD<StringString, List<Serializable>> eanKeyTsParameters =
>> javaRDD.mapToPair( ... );
>>
>> Then I try to do flatMapValues to apply the GenerateTimeSeries Function, it
>> takes the from, to and values to generate a List<LongDouble>. Here is the
>> error I get when compiling:
>>
>> error: incompatible types: no instance(s) of type variable(s) U exist so
>> that JavaPairRDD<StringString,U> conforms to JavaPairRDD<String,LongDouble>
>>
>> Here is what IntelliJ tells me:
>> flatMapValues( Function<List<Serializable>, Iterable<U>> ) in JavaPairRDD
>> cannot be applied to Transformations.GenerateTimeSeries
>>
>> Here is the problematic transformation:
>>
>> JavaPairRDD<String, LongDouble> keyLongDouble =
>>          eanKeyTsParameters.flatMapValues(new
>> Transformations.GenerateTimeSeries());
>>
>> And here is the Function:
>>
>> import org.apache.spark.api.java.function.Function; [...]
>>
>> public class Transformations {
>>      public static class GenerateTimeSeries
>>              implements Function<List<Serializable>, Iterable<LongDouble>>
{
>>
>>          @Override
>>          public Iterable<LongDouble> call(List<Serializable> args) {
>>              DateTime start = (DateTime) args.get(0);
>>              DateTime end = (DateTime) args.get(1);
>>              Double value  = (Double) args.get(2);
>>              int granularity = 24*60*60*1000; // 1 day
>>
>>              return AggregationUtils.createTimeSeries(start, end, value,
>> granularity);
>>          }
>>      }
>> }
>>
>> Any idea?
>>
>> Thanks
>>
>> --
>>
>> Robin Keunen
>> Software Engineer
>> robin.keunen@lampiris.be
>> www.lampiris.be

-- 

Robin Keunen
Software Engineer
robin.keunen@lampiris.be
www.lampiris.be


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message