`
tenght
  • 浏览: 50453 次
社区版块
存档分类
最新评论

Mapper for the maximum temperature example

 
阅读更多

Format of a National Climate Data Center record:

( The line has been split into multiple lines to show each field; in the real file,
fields are packed into one line with no delimiters.)

0057
332130   # USAF weather station identifier
99999    # WBAN weather station identifier
19500101 # observation date
0300     # observation time
4
+51317   # latitude (degrees x 1000)
+028783  # longitude (degrees x 1000)
FM-12
+0171    # elevation (meters)
99999
V020
320      # wind direction (degrees)
1        # quality code
N
0072
1
00450    # sky ceiling height (meters)
1        # quality code
C
N
010000   # visibility distance (meters)
1        # quality code
N
9
-0128    # air temperature (degrees Celsius x 10)
1        # quality code
-0139    # dew point temperature (degrees Celsius x 10)
1        # quality code
10268    # atmospheric pressure (hectopascals x 10)
1        # quality code


Mapper:
import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
public class MaxTemperatureMapper
  extends Mapper<LongWritable, Text, Text, IntWritable> {
  private static final int MISSING = 9999;
  
  @Override
  public void map(LongWritable key, Text value, Context context)
      throws IOException, InterruptedException {
    
    String line = value.toString();
    String year = line.substring(15, 19);
    int airTemperature;
    if (line.charAt(87) == '+') { // parseInt doesn't like leading plus signs
      airTemperature = Integer.parseInt(line.substring(88, 92));
    } else {
      airTemperature = Integer.parseInt(line.substring(87, 92));
    }
    String quality = line.substring(92, 93);
    if (airTemperature != MISSING && quality.matches("[01459]")) {
      context.write(new Text(year), new IntWritable(airTemperature));
    }
  }
}

Mapper接口的四个形参:

the input key is a long integer offset,

the input value is a line of text

the output key is a year,

the output value is an air temperature (an integer).

该方法中用到的Hadoop基本类型:

LongWritable( 类似于java的long类型)

Text (like Java String)

IntWritable (like Java Integer)

The map() method is passed a key and a value. We convert the Text value containing
the line of input into a Java String, then use its substring() method to extract the
columns we are interested in.

map()方法的输入是一个键和一个值。我们将包含有一行输入的Text值转换为Java的String类型,

然后使用substring()方法提取我们感兴趣的列。

The map() method also provides an instance of Context to write the output to. In this
case, we write the year as a Text object (since we are just using it as a key), and the
temperature is wrapped in an IntWritable. We write an output record only if the tem-
perature is present and the quality code indicates the temperature reading is OK.

map()方法还提供了Context实例用于输出内容的写入。在这种情况下,我们将年份数据按Text对象进行

读写(因为我们把年份当做键),将气温值封装在IntWritable类型中。我们只在气温数据不缺失并且所

对应质量代码显示为正确的气温读数时,才将其写入输出记录中。


分享到:
评论

相关推荐

    汽车编程-Using the TASKING Pin Mapper for AURIX.pdf

    汽车编程-Using the TASKING Pin Mapper for AURIX 任务编程在汽车电子领域中扮演着不可或缺的角色,TASKING Pin Mapper for AURIX正是其中的一款强大工具。本文档旨在详细介绍TASKING Pin Mapper的使用方法,并对...

    Mapper_mapper_

    #!/usr/bin/env python&quot;... # what we output here will be the input for the # Reduce step i.e. the input for reducer.py # # tab-delimited; the trivial word count is 1 print '%s\t%s' % (word 1)

    Tidy Tile Mapper v1.4.4

    Tidy Tile Mapper is an easy-to-use, intuitive and powerful tile-based game creator for Unity, perfect for creating both 2D and 3D tile-based games. See the website, watch the videos. API ...

    Tidy Tile Mapper V1.4.6

    Tidy Tile Mapper is an easy-to-use, intuitive and powerful tile-based game creator for Unity, perfect for creating both 2D and 3D tile-based games. See the website, watch the videos. API documentation...

    FC转mapper教程

    FC的ROM独有的mapper使得制作卡带比较麻烦, 通常mapper4时最常用的mapper, 此教程讲解如何将mapper0, mapper1, mapper2, mapper3,mapper23(VRC2)转换为mapper4

    Elasticsearch for Hadoop

    Downloading the example code Downloading the color images of this book Errata Piracy Questions 1. Setting Up Environment Setting up Hadoop for Elasticsearch Setting up Java Setting up a ...

    GlobalMapper14安装包

    《全球地图神器:GlobalMapper14安装指南及详解》 GlobalMapper是一款强大的地理信息系统(GIS)软件,由Blue Marble Geographics公司开发。该软件以其易用性和全面的功能集深受全球用户的喜爱,尤其在地理信息处理...

    GlobalMapper17补丁

    《全球地图神器:GlobalMapper17补丁详解与汉化探讨》 GlobalMapper是一款功能强大的全球地理信息系统(GIS)软件,被广泛应用于地理信息数据处理、地图制作、数据分析等领域。这款软件以其丰富的功能和易用性深受...

    MyBatis SQL mapper framework for Java.zip

    MyBatis SQL mapper framework for Java

    mapper_src_VirtualNES_mapper_源码

    【Mapper与VirtualNES模拟器】 Mapper,又称为“映射器”,是FC(Family Computer,即任天堂红白机)游戏开发中的一个重要概念。在FC游戏卡带的设计中,为了突破硬件限制,开发者引入了Mapper来扩展游戏的功能和...

    mybatis-3-mapper.rar

    在处理`mybatis-3-mapper.dtd`这个问题时,我们需要深入理解MyBatis的Mapper体系以及DTD的作用。 首先,`mybatis-3-mapper.dtd`是MyBatis中用于验证XML映射文件的文档类型定义(Document Type Definition)。在XML...

    通用Mapper映射视频

    ### 通用Mapper映射视频知识点解析 #### 一、映射文件的概念与作用 映射文件是MyBatis框架中的核心组成部分之一,主要用于定义SQL语句,并将SQL执行结果映射到Java对象上。通过映射文件,我们可以更加灵活地处理...

    fc-mapper4_MAPPER4_fc-mapper4_

    标题中的"fc-mapper4_MAPPER4_fc-mapper4_"似乎是一个标识符,它可能代表一个与FC(Family Computer,即任天堂的红白机)游戏卡带映射器相关的项目或程序。在FC游戏开发中,映射器(Mapper)是关键组件,它的作用是...

    Mybatis中配置Mapper的方法

    &lt;property name="basePackage" value="com.example.mapper"/&gt; ``` 以上内容详细介绍了MyBatis中配置Mapper的几种方式,包括XML配置、注解配置、动态SQL、结果映射以及在Spring环境下的集成。通过灵活运用这些...

    idea工具中直接从mapper.Java文件中跳转到mapper.xml文件的插件,挺不错的

    IDEA的这款插件通过解析Java代码中的注解和类结构,自动关联Mapper接口与Mapper XML文件,使得开发者只需在Mapper Java文件中右键点击方法,就能快速打开相应的Mapper XML。这种方式极大地简化了开发者的操作步骤,...

    Linkage Mapper用户手册,实验教程

    《Linkage Mapper用户手册与实验教程详解》 在IT领域,数据分析和基因组研究日益重要,其中Linkage Mapper是一款被广泛使用的工具,尤其在遗传学和生物信息学中扮演着关键角色。本教程旨在帮助用户深入理解并熟练...

    springboot mybatis mapper.xml 配置

    在Spring Boot集成MyBatis的过程中,`mapper.xml`配置文件起着至关重要的作用。它用于定义SQL语句,实现数据库的CRUD操作。本示例将深入探讨如何配置`mapper.xml`,并提供新增、修改、删除、查询及分页查询的实践...

    globalmapper操作手册

    globalmapper操作手册(上帝之眼Global Mapper系列教程开课啦,本系列教程由风侠老师主讲,坐沙发蹲地板的同Global Mapper 是一款地图绘制软件,不仅能够将数据(例如:SRTM数据)显示为光栅地图、高程地图、矢量...

    mapper-4.1.5.jar

    &lt;tk-mapper.version&gt;4.1.5&lt;/tk-mapper.version&gt; &lt;pagehelper.version&gt;1.2.10 &lt;!-- Mybatis通用Mapper --&gt; &lt;groupId&gt;tk.mybatis &lt;artifactId&gt;mapper ${tk-mapper.version} ...

Global site tag (gtag.js) - Google Analytics