Pig: Data Model

• Data types

 

• Nulls

In Pig a null data element means the value is unknown.which is completely different from the concept of null in C, Java, Python, etc.

• Schemas

dividends = load 'NYSE_dividends' as (exchange:chararray, symbol:chararray, date:chararray, dividend:float);

dividends = load 'NYSE_dividends' as (exchange, symbol, date, dividend);  //bytearray

mdata = load 'mydata' using HCatLoader();     //load schema from HCatalog

 

• Type Casts

player= load 'baseball' as (name:chararray, team:chararray,pos:bag{t:(p:chararray)}, bat:map[]);
unintended = foreach player generate (int)bat#'base_on_balls' - (int)bat#'ibbs';

Casts to bytearrays are never allowed because Pig does not know how to represent the various data types in binary format. Casts from bytearrays to any type are allowed. Casts to and from complex types currently are not allowed, except from bytearray, although conceptually in some cases they could be.