EclipseEye
- 浏览: 149587 次
- 性别:
- 来自: 北京
-
最新评论
-
EclipseEye：
fair_jm 写道不错 蛮详细的 谢谢分享
SWT/JFace专题 --- SWT中Display和多线程 -
fair_jm：
不错 蛮详细的 谢谢分享
SWT/JFace专题 --- SWT中Display和多线程
文章列表
@Public
@Stable
A Writable which is also Comparable.
WritableComparables can be compared to each other, typically via Comparators. Any type which is to be used as a key in the Hadoop Map-Reduce framework should implement this interface.
Note that hashCode() is frequently used in Hadoop to partiti ...
package sort;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.T ...
package cmd;
/**
* MapReduce 读取hdfs上的文件，
* 以HTable.put(put)的方式在map中完成数据写入，无reduce过程
*/
import java.io.IOException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.C ...
package cmd;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Counter;
import org.apache.hadoop.mapreduce ...
一、SequenceFileInputFormat及SequenceFileRecordReader
/** An {@link InputFormat} for {@link SequenceFile}s. */
@InterfaceAudience.Public
@InterfaceStability.Stable
public class SequenceFileInputFormat<K, V> extends FileInputFormat<K, V> {
@Override
public RecordReader<K, V> ...
/** An {@link InputFormat} for plain text files. Files are broken into lines.
* Either linefeed or carriage-return are used to signal end of line. Keys are
* the position in the file, and values are the line of text.. */
@InterfaceAudience.Public
@InterfaceStability.Stable
public class TextI ...
@Public
@Evolving
A RecordReader that reads records from a SQL table. Emits LongWritables containing the record number as key and DBWritables as value.
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class DBRecordReader<T extends DBWritable> extends
RecordReader< ...
@Public
@Stable
A base class for file-based InputFormats.
FileInputFormat is the base class for all file-based InputFormats. This provides a generic implementation of getSplits(JobContext). Subclasses of FileInputFormat can also override the isSplitable(JobContext, Path) method to ensure input-fil ...
@Public
@Stable
A InputFormat that reads input data from an SQL table.
DBInputFormat emits LongWritables containing the record number as key and DBWritables as value. The SQL query, and input class can be using one of the two setInput methods.
InputFormat核心方法有两个：getSplits和createRecordReader
核心方法一 ...
@Public
@Stable
The record reader breaks the data into key/value pairs for input to the Mapper.
Parameters:
<KEYIN>
<VALUEIN>
----------------
@InterfaceAudience.Public
@InterfaceStability.Stable
public abstract class RecordReader<KEYIN, VALUEIN> implements Closeable {
...
@Public
@Stable
InputFormat describes the input-specification for a Map-Reduce job.
The Map-Reduce framework relies on the InputFormat of the job to:
Validate the input-specification of the job.
Split-up the input file(s) into logical InputSplits, each of which is then assigned to an individua ...
@Public
@Stable
InputSplit represents the data to be processed by an individual Mapper.
Typically, it presents a byte-oriented view on the input and is the responsibility of RecordReader of the job to process this and present a record-oriented view.
---------------------
@InterfaceAudience.Public ...
@Public
@Stable
Objects that are read from/written to a database should implement DBWritable. DBWritable, is similar to Writable except that the write(PreparedStatement) method takes a PreparedStatement, and readFields(ResultSet) takes a ResultSet.
Implementations are responsible for writing the f ...
1. sed,主要用于处理文件输入流的文本替换
1.1
sed 's/[A-Z]/\l&/g' < mysql.sql
mv mysql.sql mysql.sql.old
--------------a2A.sed----------------
#! /bin/sed
#n
s/[a-z]/\U&/g
-------------------------
sed -f a2A.sed mysql.sql.old > mysql.sql
1.2
----------------merge-file.sh------------------------
#! / ...
http://ant.apache.org/manual/Tasks/fixcrlf.html
<fixcrlf srcdir="${src}" includes="**/*.sh"
eol="lf" eof="remove" />
Replaces EOLs with LF characters and removes eof characters from the shell scripts. Tabs and spaces are left as is.