`
bupt04406
  • 浏览: 348353 次
  • 性别: Icon_minigender_1
  • 来自: 杭州
社区版块
存档分类
最新评论
文章列表

Hive UT

    博客分类:
  • Hive
ant package eclipse-files -Doffline=true -Divy.skip=true ant clean package eclipse-files 记录下(可以去各个目录下的build.xml中查找到): ant test -Doffline=true ant testreport ant test -Dtestcase=TestCliDriver -Dqfile=create_view.q -Dtest.silent=false ant test -Dtestcase=TestPaser -Dqfile=input_part1.q -Dtest.silent ...
srcTables:         "src", "src1", "srcbucket", "srcbucket2", "src_json", "src_thrift",         "src_sequencefile", "srcpart" src、src1两表都是两个columns: src : create table src(key string, value string); load data local inp ...

hive - bug

    博客分类:
  • Hive
CREATE TABLE part_test (key STRING, value STRING) PARTITIONED BY (ds STRING) STORED AS SEQUENCEFILE; desc extended part_test; ALTER TABLE part_test ADD PARTITION(ds='1'); ALTER TABLE part_test ADD PARTITION(ds='2'); show partitions part_test; desc extended part_test; desc extended part_test partit ...
http://sigizmund.com/debugging-hadoop-applications-using-your-eclipse/ Well, it can be annoying – it can be awfully annoying, in fact, to debug Hadoop applications. But sometimes you need it, because logging doesn’t show anything, and you’ve tried anything but still cannot get under the Hadoop’s cov ...

hive 脚本

    博客分类:
  • Hive
hive bin=`dirname "$0"` bin=`cd "$bin"; pwd` //$0 命令行的第一个参数,代表命令本身(当前Shell程序的文件名) //dirname $0,获取当前Shell程序的路径(返回命令的路径) // . "$bin"/hive-config.sh  //运行hive-config.sh if [ "$SERVICE" = "" ] ; then   if [ "$HELP" = "_help" ] ; th ...

hive 的 test case

    博客分类:
  • Hive
src/data 目录下面是输入数据 src/ql/src/test/queries下面是测试用例,clientpositive是运行成功的用例,clientnegative是运行失败,返回非0的用例。 src/ql/src/test/results 下面是测试用例对应的输出结果。如src/ql/src/test/queries/case_sensitivity.q对应的输出结果是src/ql/src/test/results/case_sensitivity.q.out 测试会运行case_sensitivity.q产生的输出放入build/ql/test/logs/clientpositiv ...

eclipse使用

eclipse常用快捷键 http://superuser.com/questions/310197/how-do-i-fix-a-cannot-open-display-error-when-opening-an-x-program-after-sshi Ctrl+Shift+T可以按名称查找一个类 Ctrl+O   ( 查找方法 ) 〔Ctrl+L〕  定位到当前编辑器的某一行。 Ctrl+\ 取消注释 Ctrl+/ 注释 Tab /〔Shift + Tab〕   Eclipse出现不编译(clean和勾选Build Automatically都不能自动编译)的情况 ...
hiveserver ./hive --service start-hive ./hive --service stop-hive hive的metastore使用了datanucleus 下载datanucleus-samples-jdo-tutorial-2.0-src.zip 解压datanucleus-samples-jdo-tutorial-2.0,把里面的lib目录,src目录,datanucleus.properties  log4j.properties orm.xml都拷到另外一个目录下面 创建 target/classes文件。 修改datanucleus.prope ...
ubuntu下面: (1)安装好java 设置 JAVA_HOME 在文件/etc/profile中设置 JAVA_HOME=/usr/lib/jvm/java-6-sun export JAVA_HOME tianzhao@tianzhao-VirtualBox:~/apache/trunk$ echo $JAVA_HOME /usr/ (2)ssh tianzhao@tianzhao-VirtualBox:~$ ssh-keygen -t rsa -P "" tianzhao@tianzhao-VirtualBox:~$ cat $HOME/.ssh/id_rsa. ...
search.yahoo.com ubuntu下java home目录是 tianzhao@tianzhao-VirtualBox:~/hadoop-0.20.2-cdh3u0/conf$ echo $JAVA_HOME /usr/ 相关java-options: -server -Xmx40g -Xms40g -Xmn10g -XX:MaxDirectMemorySize=1024m -XX:PermSize=128m -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC 参数的含义 -vmargs -Xms128M -Xmx512M -XX:Perm ...
sudo apt-get install ant which javac sudo update-java-alternatives -l sudo update-java-alternatives -s java-6-sun -v install python3 http://blog.csdn.net/u011538446/article/details/45172949 rpm2cpio xxx.rpm | cpio -t find service | grep "\.svn" |xargs -i rm {} -rf $ find -name .svn | xar ...
svn 相关使用 下载: svn co http://svn.apache.org/repos/asf/hive/trunk trunk 下载trunk版本到当前目录下面的trunk目录下 svn checkout https://(projectname).(domain)/svn/(projectname)/(DIR) (projectname) --username [type-user-name-here] 打patch包 当修改了某个文件后,如 ObjectStore.java tianzhao@tianzhao-VirtualBox:~/apache/trunk$ svn ...
修改 conf/hadoop-env.sh 的相关设置如: export HADOOP_HEAPSIZE=64 export HADOOP_CLIENT_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/tianzhao/oom.hprof" hive启动的时候会应用上面的设置,当OOM的时候,会dump映像到oom.hprof文件,可用java的VisualVM来查看内存相关的信息 partition相关: hive在设置的内存相对比较小(64或128等)的时候会,对partition数有限制,写了 ...
hadoop in action
mahout in action
Global site tag (gtag.js) - Google Analytics