ylzhj02
- 浏览: 245983 次
- 性别:
- 来自: 成都
-
最新评论
-
oldrat：
https://github.com/oldratlee/tr ...
Kafka: High Qulity Posts
文章列表
mysql table user
hive metastore table
sqoop import scripts
import --verbose --connect jdbc:mysql://192.168.122.1:3306/sample
--username zhaohj --password 123456
--table user
--split-by user_id
-m 2
--hive-import
--hive-database user
--hive-partition-key first_letter
--cre ...
Pig: Using Hcat Example
- 博客分类：
- Pig
HCatLoader
//use default database
A = LOAD 'tablename' USING org.apache.hcatalog.pig.HCatLoader();
//specify database 'exampledb' and you could create the database and table using
//Hsql statements in Pig script after 0.10.
A = LOAD 'exampledb.tablename' USING org.apache.hcatalog.pig ...
hue UI components is conposed of the following ones:
Reference
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_installing_manually_book/content/rpm-chap-hue.html
ENV: hive-0.12.0 hadoop2.3.0
1.install protobuf
a.download protobuf-2.5.0.tar.gz from https://code.google.com/p/protobuf/downloads/detail?name=protobuf-2.5.0.tar.gz
b. tar -xzf protobuf-2.5.0.tar.gz
c. #cd protobuf-2.5.0
d. #./configure
e. #make
f. #make check
g. #make install
2. inst ...
1. When I alter a talbe in a database, I can get the altered table information by hive cli and hwi, but error comes when click the table in Hue Metastore Manager .
A: Find no reasons,but fixed it by restart hiveserver2
2.When I run a pig script in Hue using HCatLoader, error emits
ERROR ...
usage: sqoop import [GENERIC-ARGS] [TOOL-ARGS]Common arguments: --connect <jdbc-uri> Specify JDBC connect string --connection-manager <class-name> Specify connection manager class name --connection-param-f ...
I create a HCat table using Hue metastore manager, and submit a sqoop job with hcat through hue, the command show as blow:
import --connect jdbc:mysql://192.168.122.1:3306/sample --username zhj
--password 123456 --table sample_user --split-by user_id -m 2
--hcatalog-table sample_raw.sample_use ...
Import
import --connect jdbc:mysql://192.168.122.1:3306/sample
--username zhj --password 123456
--table actor
--columns "id, name, birthday"
--where "id>40"
--m 3
--target-dir /user/zhj/sqoop/actor // or --warehouse-dir /user/zhj/s ...
sqoop2: Architecture
- 博客分类：
- Sqoop
The architecture of Sqoop2 is :
The Sqoop server is :
References:
https://cwiki.apache.org/confluence/display/SQOOP/Home
ubuntu 12.04安装有道词典
＃sudo gedit /etc/apt/sources.list
add the following two lines to the end of sources.list
deb http://ppa.launchpad.net/justzx2011/openyoudao/ubuntu precise main
deb-src http://ppa.launchpad.net/justzx2011/openyoudao/ubuntu precise main
＃sudo apt-get update
# sudo ...
setup
set hcatalog/bin to path and source ~/.bashrc
#hcat -h
Note:Many hcat commands can be issued as hive commands, including all HCatalog DDL commands. The Hive CLI includes some commands that are not available in the HCatalog CLI.
HCatalog DDL
HCatalog supports all Hive Data Definit ...
ENV : oozie4.0.1 hadoop2.3.0 sqoop1.4.4
When run sqoop example with command
oozie job --oozie http://localhost:11000/oozie --config examples/apps/sqoop/job.properties -run
some errors produced due to the default sqoop sharelib is not targeted to hadoop2.x when I compile oozie
with comm ...
Auto login
- 博客分类：
- operatingsystem
Fedora20 minimal server
change /usr/lib/systemd/system/getty@.service likes this and reboot
ExecStart=-/sbin/agetty --noclear %I
==>
ExecStart=-/sbin/agetty --autologin zhj --noclear %I
Centos6.5 minimal server
change /etc/init/tty.conf
exec /sbin/mingetty $TTY
===>
ex ...
Hue: Job Designer
- 博客分类：
- Hue
1. hue [jobsub] saving a design shows an empty page after.
when I check the system log whic likes followings:
The main error is :
WebHdfsException: SecurityException: Failed to obtain user group information:
org.apache.hadoop.security.authorize.AuthorizationException: User:
hue is not all ...
The Hive Web Interface, abbreviated as HWI, is a simple graphical user interface (GUI). HWI is an alternative to using the Hive command line interface.
Another web interface that can be used for Hive commands is WebHCat, a REST API (not GUI). With WebHCat, applications can make HTTP requests to ...