`
dacoolbaby
  • 浏览: 1267390 次
  • 性别: Icon_minigender_1
  • 来自: 广州
社区版块
存档分类
最新评论

Hadoop: LongWritable cannot be cast to org.apache.hadoop.io.IntWritable

阅读更多

写MR Job的时候遇到一个坑爹的异常:

LongWritable cannot be cast to org.apache.hadoop.io.IntWritable

 

当写Map的时候,key的默认输入就是LongWritable。

因为LongWritable指代Block中的数据偏移量。

 

所以把它强行转换成Text当然就Error了。。

public static class TempMapper extends Mapper<LongWritable, Text, IntWritable, FloatWritable>{

  @Override
  protected void map(LongWritable key, Text value, Context context)
                throws IOException, InterruptedException {
       //code for getting date and temperature
       String temp = columns.get(3);
       context.write(new IntWritable(year), new FloatWritable(Float.valueOf(temp)));
  }
}

 

 

坑啊。。

分享到:
评论

相关推荐

    hadoop-commom-3.0.0.jar

    与hadoop有关的io文件在hadoop-commom-x.x.x.jar中如:org.apache.hadoop.io.LongWritable, org.apache.hadoop.io.IntWritable org.apache.hadoop.io.Text等,相关的conf配置文件也在该jar文件中

    使用命令行编译打包运行自己的MapReduce程序 Hadoop2.6.0

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    教你如何查看API及使用hadoop新api编程

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapred.*; public class BookCount { public static Logger logger = Logger.getLogger(BookCount.class...

    linux下maven在eclipse安装测试Hadoop.pdf

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    java操作hadoop之mapreduce计算整数的最大值和最小值实战源码

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    Hadoop CountWord 例子

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends Mapper&lt;LongWritable, Text, Text, ...

    Hadoop&Spark安装、环境配置、使用教程.pdf

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    mapretuce的WordCount例子

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    编写Java程序,实现统计单词个数功能

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import java.io.IOException; public class WordcountMapper extends Mapper...

    MapReduce之wordcount范例代码

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

    基于Windows eclipse maven Hadoop 的WordCount源码

    import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends ...

    Hadoop 0.20.2 API

    4. **org.apache.hadoop.io**: 此模块包含了各种基本的数据类型和序列化机制,如`IntWritable`, `Text`, `LongWritable`等,它们是Hadoop数据处理中的基本元素。`Writable`接口定义了对象如何能够被序列化和反序列化...

    大数据入门资源.doc

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends Mapper&lt;LongWritable, Text, Text, ...

    hadoop进行数据分析教程.docx

    - 实现`org.apache.hadoop.mapreduce.Mapper`接口。 - 定义输入类型为`LongWritable`和`Text`,输出类型为`Text`和`IntWritable`。 - 在`map`方法中处理输入数据,提取单词并输出单词及其出现次数。 3. **编写...

    大数据资源宝典

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import java.io.IOException; import ...

    第5章:Hive函数重要应用案例1

    import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class ReplaceDelimiterMapper ...

    hive的自定义函数

    import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; public class UDFNowDate extends UDF { private SimpleDateFormat formatter; ...

    Hadoop MultipleOutputs输出到多个文件中的实现方法

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class MultiOutPutMapper extends Mapper&lt;LongWritable, Text, IntWritable, ...

    mapreduce框架学习之天气统计

    在实际开发中,需要导入的其他包可能包括Hadoop的API,例如`org.apache.hadoop.conf.Configuration`、`org.apache.hadoop.fs.Path`、`org.apache.hadoop.io.IntWritable`、`org.apache.hadoop.io.LongWritable`和`...

    WordCount:使用 Hadoop MapReduce 框架的字数统计

    import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce....

Global site tag (gtag.js) - Google Analytics