crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Natkins (JIRA)" <>
Subject [jira] [Created] (CRUNCH-129) AvroGroupedTableType is not compatible with by()
Date Tue, 11 Dec 2012 21:27:21 GMT
Jonathan Natkins created CRUNCH-129:

             Summary: AvroGroupedTableType is not compatible with by()
                 Key: CRUNCH-129
             Project: Crunch
          Issue Type: Bug
            Reporter: Jonathan Natkins

Given a simple Avro pipeline like this:

    PGroupedTable<String, MyAvroObject> processedData = data.parallelDo(new DoFn<String,
Pair<String, MyAvroObject>>() {
      public void process(String line, Emitter<Pair<String, MyAvroObject>> emitter)
        String key = getKey(line);
        MyAvroObject value = convertToAvroObject(line);
        emitter.emit(Pair.of(key, value));
    }, Avros.tableOf(Avros.strings(), Avros.specifics(MyAvroObject.class)))

    PTable<MyAvroGroup, Pair<String, Iterable<MyAvroObject>>> groupedData
= MapFn<Pair<String, Iterable<MyAvroObject>>, MyAvroGroup>()
            public MyAvroGroup map(Pair<String, Iterable<MyAvroObject>> input)
              MyAvroGroup group = new MyAvroGroup();
              group.objects = Lists.<MyAvroObject>newArrayList();
              for (MyAvroObject obj : input.second()) {
              return group;

An exception is thrown when the by() code is run:

12/12/10 14:11:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.ClassCastException: org.apache.crunch.types.avro.AvroGroupedTableType
cannot be cast to org.apache.crunch.types.avro.AvroType
    at org.apache.crunch.types.avro.Avros.tableOf(
    at org.apache.crunch.types.avro.AvroTypeFamily.tableOf(

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message