写MR Job的时候遇到一个坑爹的异常:
LongWritable cannot be cast to org.apache.hadoop.io.IntWritable
当写Map的时候,key的默认输入就是LongWritable。
因为LongWritable指代Block中的数据偏移量。
所以把它强行转换成Text当然就Error了。。
public static class TempMapper extends Mapper<LongWritable, Text, IntWritable, FloatWritable>{
@Override
protected void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
//code for getting date and temperature
String temp = columns.get(3);
context.write(new IntWritable(year), new FloatWritable(Float.valueOf(temp)));
}
}
坑啊。。
相关推荐
与hadoop有关的io文件在hadoop-commom-x.x.x.jar中如:org.apache.hadoop.io.LongWritable, org.apache.hadoop.io.IntWritable org.apache.hadoop.io.Text等,相关的conf配置文件也在该jar文件中
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapred.*; public class BookCount { public static Logger logger = Logger.getLogger(BookCount.class...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends Mapper<LongWritable, Text, Text, ...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import java.io.IOException; public class WordcountMapper extends Mapper...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....
import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends ...
4. **org.apache.hadoop.io**: 此模块包含了各种基本的数据类型和序列化机制,如`IntWritable`, `Text`, `LongWritable`等,它们是Hadoop数据处理中的基本元素。`Writable`接口定义了对象如何能够被序列化和反序列化...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends Mapper<LongWritable, Text, Text, ...
- 实现`org.apache.hadoop.mapreduce.Mapper`接口。 - 定义输入类型为`LongWritable`和`Text`,输出类型为`Text`和`IntWritable`。 - 在`map`方法中处理输入数据,提取单词并输出单词及其出现次数。 3. **编写...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import java.io.IOException; import ...
import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class ReplaceDelimiterMapper ...
import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; public class UDFNowDate extends UDF { private SimpleDateFormat formatter; ...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class MultiOutPutMapper extends Mapper<LongWritable, Text, IntWritable, ...
在实际开发中,需要导入的其他包可能包括Hadoop的API,例如`org.apache.hadoop.conf.Configuration`、`org.apache.hadoop.fs.Path`、`org.apache.hadoop.io.IntWritable`、`org.apache.hadoop.io.LongWritable`和`...
import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....