`
sunwinner
  • 浏览: 203763 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

Failed to Run Pig Script with Macro

 
阅读更多

Pig version:

[root@n8 examples]# pig -version
Apache Pig version 0.11.0-cdh4.3.0 (rexported) 
compiled May 27 2013, 20:48:21

 Hadoop version:

[root@n8 examples]# hadoop version
Hadoop 2.0.0-cdh4.3.0
Subversion file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.3.0/src/hadoop-common-project/hadoop-common -r 48a9315b342ca16de92fcc5be95ae3650629155a
Compiled by jenkins on Mon May 27 19:45:25 PDT 2013
From source with checksum a4218d77f9b12df4e3e49ef96f9d357d
This command was run using /opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/hadoop/hadoop-common-2.0.0-cdh4.3.0.jar

 

 

Try to run below Pig script 

define dividend_analysis (daily, year, daily_symbol, daily_open, daily_close) returns analyzed {
        divs          = load 'NYSE_dividends' as (exchange:chararray,
                             symbol:chararray, date:chararray, dividends:float);
        divsthisyear  = filter divs by date matches '$year-.*';
        dailythisyear = filter $daily by date matches '$year-.*';
        jnd           = join divsthisyear by symbol, dailythisyear 
                               by $daily_symbol;
        $analyzed     = foreach jnd generate dailythisyear::$daily_symbol, 
                               $daily_close - $daily_open;
};

daily   = load 'NYSE_daily' as (exchange:chararray, symbol:chararray,
          date:chararray, open:float, high:float, low:float, close:float,
                        volume:int, adj_close:float);
results = dividend_analysis(daily, '2009', 'symbol', 'open', 'close');
dump results;

 

Error info as follows

grunt> exec ch6/macro.pig
2013-08-16 19:39:17,936 [main] WARN  org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2013-08-16 19:39:17,937 [main] WARN  org.apache.hadoop.conf.Configuration - dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
2013-08-16 19:39:17,937 [main] WARN  org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
<line 21, column 53>  Unexpected character '$'
2013-08-16 19:39:17,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <line 21, column 53>  Unexpected character '$'
Details at logfile: /root/git/programmingpig/examples/pig_1376647314921.log

 

The error happen in this line

 $analyzed     = foreach jnd generate dailythisyear::$daily_symbol, $daily_close - $daily_open;

 

Now sure why this error happen for now, pig version?? Apparently we have to use the prefix 'dailythisyear::' here because both datasets have one field names symbol. If I remove the prefix and run again, the error will be:

2013-08-16 19:33:10,570 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1025: 
at expanding macro 'dividend_analysis' (null:24)
<line 19, column 43> Invalid field projection. Projected field [symbol] does not exist in schema: macro_dividend_analysis_divsthisyear_0::exchange:chararray,macro_dividend_analysis_divsthisyear_0::symbol:chararray,macro_dividend_analysis_divsthisyear_0::date:chararray,macro_dividend_analysis_divsthisyear_0::dividends:float,macro_dividend_analysis_dailythisyear_0::exchange:chararray,macro_dividend_analysis_dailythisyear_0::symbol:chararray,macro_dividend_analysis_dailythisyear_0::date:chararray,macro_dividend_analysis_dailythisyear_0::open:float,macro_dividend_analysis_dailythisyear_0::high:float,macro_dividend_analysis_dailythisyear_0::low:float,macro_dividend_analysis_dailythisyear_0::close:float,macro_dividend_analysis_dailythisyear_0::volume:int,macro_dividend_analysis_dailythisyear_0::adj_close:float.
Details at logfile: /root/git/programmingpig/examples/pig_1376647314921.log

 

 

分享到:
评论

相关推荐

    CATIA V5 Macro Programming with Visual Basic Script

    CATIA V5 Macro Programming with Visual Basic Script

    Hypermill 5 Axis- Automatic Creat to Multi Blade toolpaths with Macro.mkv

    Hypermill 5 Axis- Automatic Creat to Multi Blade toolpaths with Macro.mkv

    Excel Macro to Generate Database Insert Script.zip

    Excel Macro to Generate Database Insert Script Excel宏生成数据库插入脚本 对于开发人员来说,一次在数据库中添加/编辑主数据或静态数据一直很麻烦,因为它需要为每个小的文本更改集编写脚本。 对于不了解SQL...

    Carpenter's Complete Guide to the SAS Macro Language

    Art Carpenter所著的《Carpenter's Complete Guide to the SAS Macro Language》第三版,是一本深入探讨SAS宏语言的权威指南,旨在帮助读者全面掌握这一强大工具。 该书作为SAS Institute Inc.出版的作品,拥有多种...

    macro to update windows 7 to last updates

    标题中的“macro to update windows 7 to last updates”和描述中的“macro to update iso windows 7 to last update”都在强调一个自动化脚本(macro)用于更新Windows 7系统至最新的更新版本。这涉及到Windows操作...

    source insight常用macro语言脚本

    标题提到的"source insight常用macro语言脚本"指的是利用Source Insight内置的宏语言来扩展其功能,以提高编程效率。宏语言允许用户自定义快捷操作,例如批量注释或取消注解代码,这些都是编程过程中常见的需求。 ...

    [confluence插件] confluence-markdown-macro-1.6.22.jar

    [confluence插件] confluence-markdown-macro-1.6.22.jar [confluence插件] confluence-markdown-macro-1.6.22.jar [confluence插件] confluence-markdown-macro-1.6.22.jar [confluence插件] confluence-markdown-...

    新代数控系统 OpenCNC MACRO手册

    根据给定文件的信息,我们可以总结出以下关于新代数控系统OpenCNC MACRO手册的关键知识点: ### 一、概述 新代数控系统OpenCNC MACRO手册是针对SYNTEC OpenCNC MACRO开发工具的操作手册。该手册旨在帮助用户理解和...

    excel macro

    好用 的 excel macro,可以build好用 的 excel macro,可以build好用 的 excel macro,可以build好用 的 excel macro,可以build

    cortexm3_macro

    标题 "cortexm3_macro" 指向的是与Cortex-M3处理器相关的宏定义,这通常涉及到微控制器编程,特别是使用ARM公司的Cortex-M3内核的微控制器。Cortex-M3是一款高效的32位RISC(精简指令集计算机)处理器核心,广泛应用...

    Finding Feasible Routes with ReinforcementLearning Using Macro

    “Finding Feasible Routes with Reinforcement Learning Using Macro-Level Traffic Measurements” 是一篇探讨如何利用强化学习和宏观交通测量数据来寻找可行路径的研究论文。以下是这篇论文的概述: 背景与动机...

    EE.rar_On The Run

    i. You open the DPHU_Format_26Dec.xls, then apply DPHU_Match, after the program starts to run, you select the after-meeting test report from which you want to generate a dphu report. ii. Use “Save ...

    SAS/MACRO

    SAS/MACRO是SAS系统中的一个强大工具,它允许用户通过自定义宏语言来创建可重用的代码块,提高编程效率和代码的可维护性。MACRO语言不仅用于简单的数据处理,还能实现复杂的统计分析流程的自动化,使得在大数据环境...

    FreeMarker使用Macro实例

    FreeMarker使用Macro实例,很好的学习资料。

    SAS Macro Make it easy

    SAS宏设施是SAS系统中的一个功能,它允许程序员使用宏变量(Macro Variables)、宏程序(Macro Programs)和宏语言(Macro Language)来编写可重用的代码,这些代码可以自动执行复杂的SAS程序任务。SAS宏设施的主要...

    Token Macro Plugin 1.10 hpi

    Run mvn clean package to create the plugin .hpi file. To install: 1. copy the resulting ./target/credentials.hpi file to the $JENKINS_HOME/plugins directory. Don't forget to restart Jenkins ...

    Macro Recorder 2024 自动化程序

    Macro Recorder 2024 是一个高效可靠的应用程序,可以让您自动化计算机上的各种重复性任务,从而节省宝贵的时间。这是一个多功能应用程序,不仅可以在特定应用程序内自动化任务,还可以跨多个程序自动化操作。它可以...

    QuickMacro 8

    QuickMacro 8是一款高效实用的自动化工具,被誉为“按键精灵”的免费版本。它专为用户提供了便捷的表单操作和宏录制功能,极大地提升了工作效率,尤其适用于那些需要频繁重复相同键盘和鼠标操作的工作场景。 首先,...

    Macro1_ExcelVBA_Cool_

    标题 "Macro1_ExcelVBA_Cool_" 暗示我们关注的是一个使用Excel VBA(Visual Basic for Applications)创建的宏,这个宏被称为“Macro1”,并且它具有某些独特的功能,被描述为“Cool”。描述中提到,该宏能够进行...

Global site tag (gtag.js) - Google Analytics