Wednesday, April 6, 2011

Hadoop, java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text

If you work on a Mapper which expects the file format of keyvalue, and forget to specify the inputformat to jobconf, you will get the following errors.

java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at MR.map(MR.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)



here is why,

for the jobconf, the default inputformat is TextInputFormat,
image

you will get Key/value type of longwritable/text in your Map function.
if you put something like Text as the key, you will get the classcast exception

public void map(Text key, Text value,
OutputCollector<Text, Text> collector,
Reporter paramReporter) throws IOException {

So always remember to specify the InputFormat.
image

1 comment:

Home Services said...

The Google Ads Performance Planner is a tool that allows you to plan your advertising spend. You can see how changes to campaigns can affect critical metrics and overall performance.
https://ppcexpo.com/blog/what-can-the-performance-planner-recommend

 
Locations of visitors to this page