Wednesday, April 6, 2011

Hadoop, java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text

If you work on a Mapper which expects the file format of keyvalue, and forget to specify the inputformat to jobconf, you will get the following errors.

java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at MR.map(MR.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)



here is why,

for the jobconf, the default inputformat is TextInputFormat,
image

you will get Key/value type of longwritable/text in your Map function.
if you put something like Text as the key, you will get the classcast exception

public void map(Text key, Text value,
OutputCollector<Text, Text> collector,
Reporter paramReporter) throws IOException {

So always remember to specify the InputFormat.
image

No comments:

 
Locations of visitors to this page