`
han_zw
  • 浏览: 174184 次
  • 性别: Icon_minigender_1
  • 来自: 天津
社区版块
存档分类
最新评论

hadoop 2.7.2 yarn中文文档——The YARN Timeline Server

 
阅读更多
概述
介绍
在YARN中通过Timeline Server用一种通用的形式解决对application的当前和历史信息的存储和检索。它有两个职责：
持久化Application特定信息
Application或者framework的完整特定信息的收集和检索。例如，Hadoop Mapreduce framework会包含诸如map task数目，reduce task数目，counter数目等等。Application的开发者在Application Master 或者application的container中，可以通过TimelineClient发布这些信息到Timeline server。
之后，这些信息可以通过REST APIs查询，用于生成application/framework的特定UI.
 
持久化已经完成的application的通用信息
之前该功能只是通过application history server支持MapReduce作业。引入timeline server之后，Application History Server变成了Timeline Server的一个应用。
通用信息包含application级别的数据，如*queue-name，*用户信息以及在ApplicationSubmissionContext中的其他类似设置，* 运行一个application的application-attempts列表，*每个application-attempt的信息，*每个application-attempt运行的容器列表，*每个容器的信息。
已经完成的application的通用数据通过YARN Resource Manager发布至timeline存储，采用它的web-UI展现这些信息。
 
当前状态以及未来计划
当前状态
  1. timeline server的核心功能已经完成。
  2. 可以工作在安全和非安全集群。
  3. 通用的history服务基于timeline store建立。
  4. history可以存储在内存中，或者采用leveldb数据库存储；后者可以保证history在timeline server 重启后仍会保留。
  5. 不支持在YARN中安装framework特定的UI。
  6. 获取Application特性信息只支持采用JSON type content的RESTful APIs。
  7. 在“Timeline Server v1” 中已经声明为兼容的的REST APIs在未来的发布中依然会进行维护。
  8. Timeline Server的single-server的实现在服务的扩展能力方面会有限制；它也妨碍了该服务称为YARN架构中的高可用组件。
未来的计划
  1. 未来的发布会引入易扩展和更可靠的下一代timeline service——“Timeline Server v2”。
  2. 对采用Timeline Server v1 REST API的application来说，扩展的服务特性可能会不可用。它包括扩展的的数据结构以及client在Timeline Server 实例间的failover能力。
 
Timeline 结构
 
 Timeline Domain
"Domain"首先存储属主信息，读写的ACL信息，创建和修改时间戳。每个Domain通过ID标识，ID必须是在YARN 集群全局唯一的。
Timeline Domain为Timeline server提供了一个命名空间，允许用户管理多个Entities，将它们与其他用户和application隔离开来。Timeline server 安全机制定义在这一层级。
Timeline Entity
Timeline Entity 包含了概念实体和它的相关event的元信息。Entity可以是一个application，application attempt，容器或者任意用户自定义的对象。它包含Primary filters，用于索引timeline Store中的entities。因此，用户/application 应该谨慎选择他们想要存储为primary filters的信息。其他的数据可以存储为非索引信息。每个Entity通过EntityId和EntityType作为唯一标识。
Timeline Events
Timeline Event描述了一个application相关的特定Timeline Entity的事件。用户可以自由定义一个event的含义——比如 启动一个application，获得一个容器分配，操作失败或者其他用户和集群操作相关的信息。
 
部署
配置
基本配置
Configuration Property Description
yarn.timeline-service.enabled 向client标示Timeline Services是否是启用状态。如果启用，application使用的TimelineClient library会发送entitiy和event到Timeline server。默认为false。
yarn.resourcemanager.system-metrics-publisher.enabled Defaults to false.该设置控制RM是否发布YARN系统度量值到timeline server。默认为false。
yarn.timeline-service.generic-application-history.enabled 标示client是否通过timeline history-service查询通用的application数据。如果未启用，则application数据只能从Resource Manager查询。默认为false。
Timeline store and state store configuration
Configuration Property Description
yarn.timeline-service.store-class Timeline store的store class name。 默认为org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore.
yarn.timeline-service.leveldb-timeline-store.path lever timeline store的存储文件名。默认为 ${hadoop.tmp.dir}/yarn/timeline.
yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms leveldb timeline store删除周期间的事件长度。默认为300000。
yarn.timeline-service.leveldb-timeline-store.read-cache-size Defaults to 104857600. 从leveldb timeline store中读取的cache大小，单位bytes。默认为 104857600。
yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size 最近读取的entity启动时间记录的缓存大小，单位为entity数量。默认为10000.
yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size 最近写入的entity启动时间记录的缓存大小，单位为entity数量。默认为10000.
yarn.timeline-service.recovery.enabled 默认为false。
yarn.timeline-service.state-store-class timeline state store的store class名。默认为org.apache.hadoop.yarn.server.timeline. recovery.LeveldbTimelineStateStore.
yarn.timeline-service.leveldb-state-store.path leveldb timeline state store的存储文件名。
Web 和 RPC 配置
Configuration Property Description
yarn.timeline-service.hostname Timeline service web application的hostname。默认为0.0.0.0.
yarn.timeline-service.address Timeline server启动RPC server的地址。默认为 ${yarn.timeline-service.hostname}:10200。
yarn.timeline-service.webapp.address Timeline service 的web application的http地址。默认为${yarn.timeline-service.hostname}:8188。
yarn.timeline-service.webapp.https.address Timeline service的web https地址。默认为 ${yarn.timeline-service.hostname}:8190.
yarn.timeline-service.bind-host Server将要绑定的真实地址。如果设置了 这个可选的地址，RPC和webapp server将会绑定这个地址，端口在 yarn.timeline-service.address 和 yarn.timeline-service.webapp.address中分别指定。这个通常在设置为0.0.0.0，让service监听所有interface时有用。
yarn.timeline-service.http-cross-origin.enabled 启用web service的跨域（CORS）支持。例如，javascript发起到timeline server的webservice请求。默认为false。
yarn.timeline-service.http-cross-origin.allowed-origins 允许访问跨域 (CORS)   支持的web service的 origin列表（逗号分隔）。支持通配符和正则。默认为*。
yarn.timeline-service.http-cross-origin.allowed-methods 需要跨域 (CORS) 支持的web service的允许的方法列表（逗号分隔）。默认为 GET,POST,HEAD.
yarn.timeline-service.http-cross-origin.allowed-headers 需要跨域（CORS）支持的web service允许的headers列表（逗号分隔）。默认为 X-Requested-With,Content-Type,Accept,Origin。
yarn.timeline-service.http-cross-origin.max-age 对于需要跨域（CORS）支持的web service，一次请求能够被缓存的秒数。默认为1800.
TimelineClient中HTTP和HTTPS两者的选择是基于YARN的配置选项 yarn.http.policy的值；如果policy 是HTTPS_ONLY 或者HTTP_AND_HTTPS，那么HTTPS会被选择。
Server端高级配置
Configuration Property Description
yarn.timeline-service.ttl-enable 启用删除timeline store中过期数据。默认为true。
yarn.timeline-service.ttl-ms timeline store数据存活的毫秒数。默认为604800000 (7天)
yarn.timeline-service.handler-thread-count 处理来自client的RPC请求的handler的线程数。默认为10.
yarn.timeline-service.client.max-retries 尝试发布数据到timeline service的最大尝试次数。默认为30.
yarn.timeline-service.client.retry-interval-ms timeline service client重试时间间隔（毫秒）。默认为1000。
yarn.timeline-service.generic-application-history.max-applications 能够通过REST API或者application history协议获取的并显示在timeline server web ui的最大application数量。默认为10000.
UI Hosting Configuration
timeline service能够支持多个UI。service能够支持目录结构的静态web site，或者war包。
Configuration Property Description
yarn.timeline-service.ui-names 支持的UI列表（逗号分隔）。默认为none。
yarn.timeline-service.ui-on-disk-path.$name 对于每一个UI-name，需要指定一个静态内容的目录在磁盘的路径，或者一个war包文件的位置。
yarn.timeline-service.ui-web-path.$name 每一个ui-name，需要指定相对于timeline service的相对路径。path需要以"/"开头。
 
安全设置
通过设置yarn.timeline-service.http-authentication.type为kerberos可以启用Security，之后以下配置项是可用的：
Configuration Property Description
yarn.timeline-service.http-authentication.type 用于timeline server HTTP终端的认证的 定义。支持的值为： simple / kerberos / #AUTHENTICATION_HANDLER_CLASSNAME#。
yarn.timeline-service.http-authentication.simple.anonymous.allowed 标示在使用'simple'认证时，timeline server是否允许匿名的请求。默认为true。
yarn.timeline-service.principal timeline server的 Kerberos 策略。
yarn.timeline-service.keytab timeline server的 Kerberos  keytab。在unix上的默认为 /etc/krb5.keytab.
yarn.timeline-service.delegation.key.update-interval 默认为 86400000 （1天）
yarn.timeline-service.delegation.token.renew-interval 默认为 86400000 （1天）
yarn.timeline-service.delegation.token.max-lifetime 默认为604800000 (7天).
yarn.timeline-service.best-effort 如果未能获得令牌是认为application失败（ option = false  ），还是client丢弃它试图继续发布信息 (option=true)。默认为false。
启用timeline service和通用history service
以下是启用timeline server的基本配置
<property>
  <description>Indicate to clients whether Timeline service is enabled or not.
  If enabled, the TimelineClient library used by end-users will post entities
  and events to the Timeline server.</description>
  <name>yarn.timeline-service.enabled</name>
  <value>true</value>
</property>

<property>
  <description>The setting that controls whether yarn system metrics is
  published on the timeline server or not by RM.</description>
  <name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
  <value>true</value>
</property>

<property>
  <description>Indicate to clients whether to query generic application
  data from timeline history-service or not. If not enabled then application
  data is queried only from Resource Manager.</description>
  <name>yarn.timeline-service.generic-application-history.enabled</name>
  <value>true</value>
</property>
 运行timeline server
确定上述所有配置项都已经正确设置，管理员可以通过以下命令启动Timeline server/history service：
yarn timelineserver
以守护进程方式启动Timeline server/history service的命令是：
$HADOOP_YARN_HOME/sbin/yarn-daemon.sh start timelineserver
 通过命令行访问通用数据
用户能够通过以下命令访问application的通用历史数据：
$ yarn application -status <Application ID>
$ yarn applicationattempt -list <Application ID>
$ yarn applicationattempt -status <Application Attempt ID>
$ yarn container -list <Application Attempt ID>
$ yarn container -status <Container ID>
 注意相同命令可以用于获取运行中的application的相应信息。
发布application特定数据
通过构件TimelineEntity 和 TimelineEvent ，开发者可以定义他们想要为applicatoin记录的任何信息，之后通过TimelineClient API发布这些entity和event到timeline server。
举例：
// Create and start the Timeline client
TimelineClient client = TimelineClient.createTimelineClient();
client.init(conf);
client.start();

try {
  TimelineDomain myDomain = new TimelineDomain();
  myDomain.setID("MyDomain");
  // Compose other Domain info ....

  client.putDomain(myDomain);

  TimelineEntity myEntity = new TimelineEntity();
  myEntity.setDomainId(myDomain.getId());
  myEntity.setEntityType("APPLICATION");
  myEntity.setEntityID("MyApp1")
  // Compose other entity info

  TimelinePutResponse response = client.putEntities(entity);

  TimelineEvent event = new TimelineEvent();
  event.setEventType("APP_FINISHED");
  event.setTimestamp(System.currentTimeMillis());
  event.addEventInfo("Exit Status", "SUCCESS");
  // Compose other Event info ....

  myEntity.addEvent(event);
  TimelinePutResponse response = timelineClient.putEntities(entity);

} catch (IOException e) {
  // Handle the exception
} catch (RuntimeException e) {
  // In Hadoop 2.6, if attempts submit information to the Timeline Server fail more than the retry limit,
  // a RuntimeException will be raised. This may change in future releases, being
  // replaced with a IOException that is (or wraps) that which triggered retry failures.
} catch (YarnException e) {
  // Handle the exception
} finally {
  // Stop the Timeline client
  client.stop();
}
  1. 发布数据到Timeline Server是一个同步操作；调用直到成功后才返回。
  2. TimelineClient实现类是YARN Service API的子类；它可以放置在CompositeService下轻松完成生命周期管理。
  3. putEntities() 调用的结果是一个TimelinePutResponse 对象。它包含一个（可能为空）被timeline server拒绝的timeline entity列表，以及一个错误码标示每一个失败的原因。
在Hadoop2.6 和2.7，错误码如下：
Error Code Description
1 No start time
2 IOException
3 System Filter conflict (reserved filter key used)
4 Access Denied
5 No domain
6 Forbidden relation
以后会定义更多的错误码。
注意: 在更新entity时需要注意以下几点：
  • 对于已经存在的entity不能修改其Domain ID。
  • 修改Primary filter的值之后，新值会追加到旧值，原值不会被替换。
  • 建议对entity的所有更新采用相同的primary filter。任何primary filter的修改在更新之前，不会从查询中获取到相应信息。
 Generic Data Web UI
用户能通过web UI 访问application的通用历史信息,url为：http(s)://<timeline server http(s) address:port>/applicationhistory
Timeline Server REST API V1
查询timeline server目前只支持REST API调用，在YARN library中没有API client的实现。采用java，Jersey client在查询server时是比较高效的，即使是在安全模式下（调用者有相应的Kerberos 令牌 或者 keytab）。
V1版本的REST API是在timeline server 的/ws/v1/timeline/ 路径下。这里是一个非规范的API描述.
Root path
GET /ws/v1/timeline/
返回一个描述server实例的JSON对象。
{"About":"Timeline API"}
Domains /ws/v1/timeline/domain
Domain 汇总信息 /ws/v1/timeline/domain
GET /ws/v1/timeline/domain?owner=$OWNER
返回属于指定用户的domain列表，使用JSON序列化的TimelineDomain数据结构。
在未认证的GET请求时，owner必须设置。
在已经认证的请求中，owner默认为调用者。
PUT /ws/v1/timeline/domain
PUT一个序列化的TimelineDomain结构到这个path，会新增该domain到指定的user或者当前用户的domain列表中。操作成功返回状态码200并且TimelinePutResponse不包含错误。
关于domain的特性信息 /ws/v1/timeline/domain/{domainId}
返回描述domain的JSON序列化TimelineDomain结构。如果domain没有找到，则返回HTTP 404.
POST new domain /ws/v1/timeline/domain
创建一个新的timeline domain，或者覆盖一个已经存在的。
当试图创建一个新的domain，提交信息中的ID必须是全局唯一。
当试图更新已经存在的Domain，domain的ID必须设置。提交者必须具有更新该domain的相应权限。
submission: TimelineDomain
response: TimelinePutResponse
List domains of a user: GET /ws/v1/timeline/domain
获取用户的domain列表。
如果指定了owner，会用owner覆盖当前调用者信息。
Query Parameter Description
owner domain 列表的owner。
GET http://localhost:8188/ws/v1/timeline/domain?owner=alice
{
"domains":
  [
    {
    "id":"DS_DOMAIN_2",
    "owner":"alice",
    "readers":"peter",
    "writers":"john",
    "createdtime":1430425000337,
    "modifiedtime":1430425000337
    },
    {
    "id":"DS_DOMAIN_1",
    "owner":"alice",
    "readers":"bar",
    "writers":"foo",
    "createdtime":1430424955963,
    "modifiedtime":1430424955963
    }
    ,
    {"id":"DEFAULT",
    "description":"System Default Domain",
    "owner":"alice",
    "readers":"*",
    "writers":"*",
    "createdtime":1430424022699,
    "modifiedtime":1430424022699
    }
  ]
}
 response: TimelineDomains
如果用户缺少获取指定用户domain列表的权限，则返回没有domain列表的TimelineDomains 。
获取指定domain的详情:
GET /ws/v1/timeline/domain/{domainId}
获取单个domain的详情
GET http://localhost:8188/ws/v1/timeline/domain/DS_DOMAIN_1
Response: TimelineDomain
{
  "id":"DS_DOMAIN_1",
  "owner":"zshen",
  "readers":"bar",
  "writers":"foo",
  "createdtime":1430424955963,
  "modifiedtime":1430424955963
}
 如果用户缺少查询domain详情的权限，将会返回404 not found的异常——如果不存在指定的ID的实体也会返回相同的结果。
Posting Timeline Entities
利用 Posting Entities API，你可以将想要记录的包含每个框架的信息的entity和event post至timeline server。
URI:
http(s)://<timeline server http(s) address:port>/ws/v1/timeline
HTTP Operations Supported:
POST
Query Parameters Supported:
None
response: TimelinePutResponse
Request 样例：
JSON request
HTTP Request:
POST http://<timeline server http address:port>/ws/v1/timeline
Request Header:
POST /ws/v1/timeline HTTP/1.1 Accept: application/json Content-Type: application/json Transfer-Encoding: chunked
Request Body:
{
  "entities" : [ {
    "entity" : "entity id 0",
    "entitytype" : "entity type 0",
    "relatedentities" : {
      "test ref type 2" : [ "test ref id 2" ],
      "test ref type 1" : [ "test ref id 1" ]
    },
    "events" : [ {
      "timestamp" : 1395818851590,
      "eventtype" : "event type 0",
      "eventinfo" : {
        "key2" : "val2",
        "key1" : "val1"
      }
    }, {
      "timestamp" : 1395818851590,
      "eventtype" : "event type 1",
      "eventinfo" : {
        "key2" : "val2",
        "key1" : "val1"
      }
    } ],
    "primaryfilters" : {
      "pkey2" : [ "pval2" ],
      "pkey1" : [ "pval1" ]
    },
    "otherinfo" : {
      "okey2" : "oval2",
      "okey1" : "oval1"
    },
    "starttime" : 1395818851588
  }, {
    "entity" : "entity id 1",
    "entitytype" : "entity type 0",
    "relatedentities" : {
      "test ref type 2" : [ "test ref id 2" ],
      "test ref type 1" : [ "test ref id 1" ]
    },
    "events" : [ {
      "timestamp" : 1395818851590,
      "eventtype" : "event type 0",
      "eventinfo" : {
        "key2" : "val2",
        "key1" : "val1"
      }
    }, {
      "timestamp" : 1395818851590,
      "eventtype" : "event type 1",
      "eventinfo" : {
        "key2" : "val2",
        "key1" : "val1"
      }
    } ],
    "primaryfilters" : {
      "pkey2" : [ "pval2" ],
      "pkey1" : [ "pval1" ]
    },
    "otherinfo" : {
      "okey2" : "oval2",
      "okey1" : "oval1"
    },
    "starttime" : 1395818851590
  } ]
}
 必选字段
Entity: type 和id. starttime在entity包含一个或者多个event时必选。Event: type 和 timestamp.
Timeline Entity List
利用Timeline Entity List API,你可以获取按照entity start timestamp 戳降序排列的entity list。entity的start timestamp可以是你的application指定的。如果没有明确指定，store则会选择之前接受到的该entity的event的最早的时间戳。
URI:
使用以下URI获得给定eventType的所有entity。
http(s)://<timeline server http(s) address:port>/ws/v1/timeline/{entityType}
HTTP Operations Supported:
GET http://localhost:8188/ws/v1/timeline/DS_APP_ATTEMPT
支持的查询参数：
  1. limit - 返回的entity数量的限制。如果为null，则默认为100.
  2. windowStart - 获取的最早的start timestamp（不包含）。如果为null，默认为获取所有entity直到到达数量上限。
  3. windowEnd - 获取的最晚的start timestamp（包含）。如果为null，默认为Long型的最大值。
  4. fromId - 如果fromId不为null，获取entity早于或者等于指定的ID。如果没有找到指定ID的start time，将会返回空的entity列表。如果entity的start time晚于windowEnd将优先采用windowEnd 参数。
  5. fromTs - 如果fromTs不为null，忽略那些插入store在指定时间戳之后的entity。entity用于比较的的插入时间戳是以第一次插入store的系统时间为准，而不是entity的start time。
  6. primaryFilter - 只获取匹配指定primary filter的entity。如果为null，获取所有的entity。这是索引方式的获取，并不是全局扫描。
  7. secondaryFilters - 获取在已经匹配primary filters的结果中匹配指定的filter的entity。这个不是索引方式的的获取，所以entity会被全部扫描，只有匹配filter的会被返回。
  8. fields - 指定获取entity对象的哪些字段：EVENTS, RELATED_ENTITIES, PRIMARY_FILTERS, OTHER_INFO, LAST_EVENT_ONLY。如果字段集包含LAST_EVENT_ONLY且不包含EVENTS，则返回每个entity的最近event。如果为null，获取所有的字段。
有一点需要注意，primaryFilter 和 secondaryFilters 参数的键值对可以是不同的数据类型，并且匹配是数据类型敏感的。用户需要正确格式化该值。例如，123和“123”表示一个数值和字符串。如果entity的primaryFilter有一个字符串“123”，但是参数设置为数值123，则该entity不会被匹配到。类似的，true意思是boolean值，然而“true”意思是字符串。通常情况下，会用jackson library解析JSON将值转换为java类型。
entitys（Timeline Entity List）对象中的元素
当你请求Timeline entity列表时，会返回一个包含对象的集合信息。
Item Data Type Description
entities array of timeline entity objects(JSON) The collection of timeline entity objects
响应样例：
JSON 响应
HTTP Request:
GET http://<timeline server http address:port>/ws/v1/timeline/{entity-type}
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "entities":[
  {
  "entitytype":"DS_APP_ATTEMPT",
  "entity":"appattempt_1430424020775_0004_000001",
  "events":[
    {
    "timestamp":1430425008796,
    "eventtype":"DS_APP_ATTEMPT_END",
    "eventinfo": { }
    }
    {
    "timestamp":1430425004161,
    "eventtype":"DS_APP_ATTEMPT_START",
    "eventinfo": { }
    }
  ]
  "starttime":1430425004161,
  "domain":"DS_DOMAIN_2",
  "relatedentities": { },
  "primaryfilters":
    {
    "user":["zshen"]
    },
  "otherinfo": { }
  }
  {
  "entitytype":"DS_APP_ATTEMPT",
  "entity":"appattempt_1430424020775_0003_000001",
  "starttime":1430424959169,
  "domain":"DS_DOMAIN_1",
  "events":[
    {
    "timestamp":1430424963836,
    "eventinfo": { }
     }
    {
    "timestamp":1430424959169,
    "eventinfo": { }
    }
   ]
  "relatedentities": { },
  "primaryfilters": {
    "user":["zshen"]
   },
  "otherinfo": { }
   }
  ]
}
 Timeline Entity
利用Timeline Entity API，你可以获取指定ID的Entity的信息。
URI:
使用以下URI获得entity对象，用entityType和EntityId作为标识.
http(s)://<timeline server http(s) address:port>/ws/v1/timeline/{entityType}/{entityId}
HTTP Operations Supported:
GET
Query Parameters Supported:
  1. fields - 指定获取entity对象哪些字段：EVENTS, RELATED_ENTITIES, PRIMARY_FILTERS,  OTHER_INFO, LAST_EVENT_ONLY.如果字段集合包含LAST_EVENT_ONLY 而不包含EVENTS，将会返回每个entity的最近event。如果为null，则获取所有字段。
entity（Timeline Entity）对象中的元素：
注意一点，primaryfilter和otherinfo的值由对象，而不是string。
Item Data Type Description
entity string entity id
entitytype string entity type
relatedentities map 相关entity的标识，以entityType为key的map结构组织，值为:[ entity1, entity2, …  ]
events list entity的event列表
primaryfilters map entity的primary filter，以key组织的map结构组织，值为：  [value1, value2, …]
otherinfo map entity的其他信息，以map结构组织。
starttime long entity的start time。
响应样例：
JSON response
HTTP Request:
GET http://<timeline server http address:port>/ws/v1/timeline/{entity-type}/{entity-id}
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
http://localhost:8188/ws/v1/timeline/DS_APP_ATTEMPT/appattempt_1430424020775_0003_000001
{
  "events":[
    {
    "timestamp":1430424959169,
    "eventtype":"DS_APP_ATTEMPT_START",
    "eventinfo":  {}}],
    "entitytype":"DS_APP_ATTEMPT",
    "entity":"appattempt_1430424020775_0003_000001",
    "starttime":1430424959169,
    "domain":"DS_DOMAIN_1",
    "relatedentities":  {},
    "primaryfilters":  {
        "user":["zshen"]
        },
    "otherinfo":  {}
    }
  ]
}
 Timeline Event 列表
利用Timeline Event API，你可以获取相同entity type的entity列表的event对象。每一个entity的event按照他们的timestamp降序排列。
URI:
使用以下URI获取给定eventType的event对象。
http(s)://<timeline server http(s) address:port>/ws/v1/timeline/{entityType}/events
HTTP Operations Supported:
GET
Query Parameters Supported:
  1. entityIds - 要获取event的entity id。
  2. limit - 每一个entity的返回的event的数量限制。如果为null，默认为每个entity 100 event。
  3. windowStart - 如果不为null，只获取晚于(不含)给定时间的event。
  4. windowEnd - 如果不为null，只获取早于（包含）给定时间的event。
  5. eventTypes - 限制返回指定类型的event。如果为null，返回所有类型的event。
 
event对象中的元素
当发起timeline event的请求时，将会返回一个event对象的集合信息。
Item Data Type Description
events array of timeline event objects(JSON) The collection of timeline event objects
以下是单个event对象的元素。eventinfo和otherinfo的值为Object。
Item Data Type Description
eventtype string event type
eventinfo map 以键值对的map组织的event的信息
timestamp long event的timestamp
响应样例
JSON response
HTTP Request:
GET http://<timeline server http address:port>/ws/v1/timeline/entity%20type%200/events
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
GET http://localhost:8188/ws/v1/timeline/DS_APP_ATTEMPT/events?entityId=appattempt_1430424020775_0003_000001
{
"events": [
  {
  "entity":"appattempt_1430424020775_0003_000001",
  "entitytype":"DS_APP_ATTEMPT"}
  "events":[
    {
    "timestamp":1430424963836,
    "eventtype":"DS_APP_ATTEMPT_END",
    "eventinfo":{}},
    {
    "timestamp":1430424959169,
    "eventtype":"DS_APP_ATTEMPT_START",
    "eventinfo":{}}
    ],
   }
  ]
}
 Generic Data REST APIs
用户通过REST API可以访问application的通用历史信息。
Application List
利用Application List API，你可以获取一个resource集合，其中每一个代表一个application。当你在这个resource上运行GET操作时，你会获取到application对象集合。
URI:
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps
HTTP Operations Supported:
GET
Query Parameters Supported:
  1. states - application需要匹配的application states（以逗号分隔的列表）
  2. finalStatus - application的final status——由application自身上报。
  3. user - 用户名
  4. queue - 队列名
  5. limit - 返回的app对象的总数量限制。
  6. startedTimeBegin - 这个time作为application的start time的起始点。指定值为epoch以来的毫秒数。
  7. startedTimeEnd - 这个time作为application的start time的截止点。指定值为epoch以来的毫秒数。
  8. finishedTimeBegin - 指定epoch以来的毫秒数，作为application finish time的起始点。
  9. finishedTimeEnd - 指定epoch以来的毫秒数，作为application finish time的截止点。
  10. applicationTypes - application需要匹配的application type，用逗号分隔的列表指定。
Elements of the apps (Application List) Object
当你发起请求application list，将会返回一个application对象集合的信息。
Item Data Type Description
app array of app objects(JSON)/zero or more application objects(XML) The collection of application objects
响应样例：
JSON response
HTTP Request:
GET http://<timeline server http address:port>/ws/v1/applicationhistory/apps
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "app":
  [
      {
      "appId":"application_1430424020775_0004",
      "currentAppAttemptId":"appattempt_1430424020775_0004_000001",
      "user":"zshen",
      "name":"DistributedShell",
      "queue":"default",
      "type":"YARN",
      "host":"d-69-91-129-173.dhcp4.washington.edu/69.91.129.173",
      "rpcPort":-1,
      "appState":"FINISHED",
      "progress":100.0,
      "diagnosticsInfo":"",
      "originalTrackingUrl":"N/A",
      "trackingUrl":"http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0004/",
      "finalAppStatus":"SUCCEEDED",
      "submittedTime":1430425001004,
      "startedTime":1430425001004,
      "finishedTime":1430425008861,
      "elapsedTime":7857},
      {
      "appId":"application_1430424020775_0003",
      "currentAppAttemptId":"appattempt_1430424020775_0003_000001",
      "user":"zshen",
      "name":"DistributedShell",
      "queue":"default",
      "type":"YARN",
      "host":"d-69-91-129-173.dhcp4.washington.edu/69.91.129.173",
      "rpcPort":-1,
      "appState":"FINISHED",
      "progress":100.0,
      "diagnosticsInfo":"",
      "originalTrackingUrl":"N/A",
      "trackingUrl":"http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0003/",
      "finalAppStatus":"SUCCEEDED",
      "submittedTime":1430424956650,
      "startedTime":1430424956650,
      "finishedTime":1430424963907,
      "elapsedTime":7257},
      {
      "appId":"application_1430424020775_0002",
      "currentAppAttemptId":"appattempt_1430424020775_0002_000001",
      "user":"zshen",
      "name":"DistributedShell",
      "queue":"default",
      "type":"YARN",
      "host":"d-69-91-129-173.dhcp4.washington.edu/69.91.129.173",
      "rpcPort":-1,
      "appState":"FINISHED",
      "progress":100.0,
      "diagnosticsInfo":"",
      "originalTrackingUrl":"N/A",
      "trackingUrl":"http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0002/",
      "finalAppStatus":"SUCCEEDED",
      "submittedTime":1430424769395,
      "startedTime":1430424769395,
      "finishedTime":1430424776594,
      "elapsedTime":7199
      }
  ]
}
 XML response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 1710
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<apps>
  <app>
    <appId>application_1430424020775_0004</appId>
    <currentAppAttemptId>appattempt_1430424020775_0004_000001</currentAppAttemptId>
    <user>zshen</user>
    <name>DistributedShell</name>
    <queue>default</queue>
    <type>YARN</type>
    <host>d-69-91-129-173.dhcp4.washington.edu/69.91.129.173</host>
    <rpcPort>-1</rpcPort>
    <appState>FINISHED</appState>
    <progress>100.0</progress>
    <diagnosticsInfo></diagnosticsInfo>
    <originalTrackingUrl>N/A</originalTrackingUrl>
    <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0004/</trackingUrl>
    <finalAppStatus>SUCCEEDED</finalAppStatus>
    <submittedTime>1430425001004</submittedTime>
    <startedTime>1430425001004</startedTime>
    <finishedTime>1430425008861</finishedTime>
    <elapsedTime>7857</elapsedTime>
  </app>
  <app>
    <appId>application_1430424020775_0003</appId>
    <currentAppAttemptId>appattempt_1430424020775_0003_000001</currentAppAttemptId>
    <user>zshen</user>
    <name>DistributedShell</name>
    <queue>default</queue>
    <type>YARN</type>
    <host>d-69-91-129-173.dhcp4.washington.edu/69.91.129.173</host>
    <rpcPort>-1</rpcPort>
    <appState>FINISHED</appState>
    <progress>100.0</progress>
    <diagnosticsInfo></diagnosticsInfo>
    <originalTrackingUrl>N/A</originalTrackingUrl>
    <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0003/</trackingUrl>
    <finalAppStatus>SUCCEEDED</finalAppStatus>
    <submittedTime>1430424956650</submittedTime>
    <startedTime>1430424956650</startedTime>
    <finishedTime>1430424963907</finishedTime>
    <elapsedTime>7257</elapsedTime>
  </app>
  <app>
    <appId>application_1430424020775_0002</appId>
    <currentAppAttemptId>appattempt_1430424020775_0002_000001</currentAppAttemptId>
    <user>zshen</user>
    <name>DistributedShell</name>
    <queue>default</queue>
    <type>YARN</type>
    <host>d-69-91-129-173.dhcp4.washington.edu/69.91.129.173</host>
    <rpcPort>-1</rpcPort>
    <appState>FINISHED</appState>
    <progress>100.0</progress>
    <diagnosticsInfo></diagnosticsInfo>
    <originalTrackingUrl>N/A</originalTrackingUrl>
    <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0002/</trackingUrl>
    <finalAppStatus>SUCCEEDED</finalAppStatus>
    <submittedTime>1430424769395</submittedTime>
    <startedTime>1430424769395</startedTime>
    <finishedTime>1430424776594</finishedTime>
    <elapsedTime>7199</elapsedTime>
  </app>
  <app>
    <appId>application_1430424020775_0001</appId>
    <currentAppAttemptId>appattempt_1430424020775_0001_000001</currentAppAttemptId>
    <user>zshen</user>
    <name>QuasiMonteCarlo</name>
    <queue>default</queue>
    <type>MAPREDUCE</type>
    <host>localhost</host>
    <rpcPort>56264</rpcPort>
    <appState>FINISHED</appState>
    <progress>100.0</progress>
    <diagnosticsInfo></diagnosticsInfo>
    <originalTrackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001</originalTrackingUrl>
    <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/</trackingUrl>
    <finalAppStatus>SUCCEEDED</finalAppStatus>
    <submittedTime>1430424053809</submittedTime>
    <startedTime>1430424053809</startedTime>
    <finishedTime>1430424072153</finishedTime>
    <elapsedTime>18344</elapsedTime>
  </app>
</apps>
 Application
利用Application API,你可以获得关于运行在YARN集群中的特性application的application resource信息。
It is essentially a JSON-serialized form of the YARN ApplicationReport structure.
URI:
使用以下URI获取一个通过appId值标示的application对象
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps/{appid}
HTTP Operations Supported:
GET
Query Parameters Supported:
None
Elements of the app (Application) Object:
Item Data Type Description
appId string application ID
user string 启动该application的用户
name string application name
type string application type
queue string application提交到的队列
appState string 根据ResourceManager的application state-有效的值是 YarnApplicationState enum ： FINISHED, FAILED, KILLED
finalStatus string application完成后的final status-application自身上报的-有效的值为：UNDEFINED, SUCCEEDED, FAILED, KILLED
progress float 上报的application进度的百分比。Long-Live YARN 服务可能不会提供一个有意义的值，或者使用它作为实际的与所需的容器计数的度量。
trackingUrl string application的Web URL（通过RM proxy）
originalTrackingUrl string 实际的applicationWeb URL
diagnosticsInfo string application 完成后的诊断信息
startedTime long application的started时间（epoch以来的毫秒数）
finishedTime long application的完成时间(epoch以来的毫秒数)
elapsedTime long 从application启动后的耗时(毫秒数)
allocatedMB int 分配给application的运行容器的内存总和（以M为单位）
allocatedVCores int 分配给application的运行容器的虚拟计算核总和.
currentAppAttemptId string 当前application的attempt id。
host string applicationMaster的host。
rpcPort int Application的RPC port;zero表示没有IPC service。
响应样例：
JSON response
HTTP Request:
http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "appId": "application_1430424020775_0001",
  "currentAppAttemptId": "appattempt_1430424020775_0001_000001",
  "user": "zshen",
  "name": "QuasiMonteCarlo",
  "queue": "default",
  "type": "MAPREDUCE",
  "host": "localhost",
  "rpcPort": 56264,
  "appState": "FINISHED",
  "progress": 100.0,
  "diagnosticsInfo": "",
  "originalTrackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001",
  "trackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/",
  "finalAppStatus": "SUCCEEDED",
  "submittedTime": 1430424053809,
  "startedTime": 1430424053809,
  "finishedTime": 1430424072153,
  "elapsedTime": 18344
}
 
XML response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001 Accept: application/xml
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 873
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <app>
   <appId>application_1430424020775_0001</appId>
   <currentAppAttemptId>appattempt_1430424020775_0001_000001</currentAppAttemptId>
   <user>zshen</user>
   <name>QuasiMonteCarlo</name>
   <queue>default</queue>
   <type>MAPREDUCE</type>
   <host>localhost</host>
   <rpcPort>56264</rpcPort>
   <appState>FINISHED</appState>
   <progress>100.0</progress>
   <diagnosticsInfo></diagnosticsInfo>
   <originalTrackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001</originalTrackingUrl>
   <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/</trackingUrl>
   <finalAppStatus>SUCCEEDED</finalAppStatus>
   <submittedTime>1430424053809</submittedTime>
   <startedTime>1430424053809</startedTime>
   <finishedTime>1430424072153</finishedTime>
   <elapsedTime>18344</elapsedTime>
 </app>
 Application Attempt List
利用Application Attempt List API，你可以获取一个resource集合，其中每个资源为application attempt。当你运行在这个resouce上指定GET请求，你会获取到一个application attempt对象的集合。
URI:
利用以下URL获取appId标识的application的所有attempt对象。
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps/{appid}/appattempts
HTTP Operations Supported:
GET
Query Parameters Supported:
None
Elements of the appattempts (Application Attempt List) Object
当你发起对application attempt列表的请求时，会收到application attempt对象的集合。
Item Data Type Description
appattempt array of appattempt objects(JSON)/zero or more application attempt objects(XML) application attempt对象的集合
响应样例：
JSON response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "appAttempt": [
    {
      "appAttemptId": "appattempt_1430424020775_0001_000001",
      "host": "localhost",
      "rpcPort": 56264,
      "trackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/",
      "originalTrackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001",
      "diagnosticsInfo": "",
      "appAttemptState": "FINISHED",
      "amContainerId": "container_1430424020775_0001_01_000001"
    }
  ]
}
 XML response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts Accept: application/xml
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<appAttempts>
  <appAttempt>
    <appAttemptId>appattempt_1430424020775_0001_000001</appAttemptId>
    <host>localhost</host>
    <rpcPort>56264</rpcPort>
    <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/</trackingUrl>
    <originalTrackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001</originalTrackingUrl>
    <diagnosticsInfo></diagnosticsInfo>
    <appAttemptState>FINISHED</appAttemptState>
    <amContainerId>container_1430424020775_0001_01_000001</amContainerId>
  </appAttempt>
</appAttempts>
 Application Attempt
利用Application Attempt API，你可以获取到运行在YARN集群中的application的特定application attempt的信息。
URI:
利用以下URI获取application attempt对象，用过appid和appattemptid标识.
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps/{appid}/appattempts/{appattemptid}
HTTP Operations Supported:
GET
Query Parameters Supported:
None
Elements of the appattempt (Application Attempt) Object（这部分不翻译了）:
Item Data Type Description
appAttemptId string The application attempt Id
amContainerId string The ApplicationMaster container Id
appAttemptState string The application attempt state according to the ResourceManager - valid values are members of the YarnApplicationAttemptState enum: FINISHED, FAILED, KILLED
trackingUrl string The web URL that can be used to track the application
originalTrackingUrl string The actual web URL of the application
diagnosticsInfo string Detailed diagnostics information
host string The host of the ApplicationMaster
rpcPort int The rpc port of the ApplicationMaster
响应样例
JSON response
HTTP Request:
http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts/appattempt_1430424020775_0001_000001
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "appAttemptId": "appattempt_1430424020775_0001_000001",
  "host": "localhost",
  "rpcPort": 56264,
  "trackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/",
  "originalTrackingUrl": "http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001",
  "diagnosticsInfo": "",
  "appAttemptState": "FINISHED",
  "amContainerId": "container_1430424020775_0001_01_000001"
}
 XML response
HTTP Request:
GET http://<timeline server http address:port>/ws/v1/applicationhistory/apps/application_1395789200506_0001/appattempts/appattempt_1395789200506_0001_000001 Accept: application/xml
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 488
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<appAttempt>
  <appAttemptId>appattempt_1430424020775_0001_000001</appAttemptId>
  <host>localhost</host>
  <rpcPort>56264</rpcPort>
  <trackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:8088/proxy/application_1430424020775_0001/</trackingUrl>
  <originalTrackingUrl>http://d-69-91-129-173.dhcp4.washington.edu:19888/jobhistory/job/job_1430424020775_0001</originalTrackingUrl>
  <diagnosticsInfo></diagnosticsInfo>
  <appAttemptState>FINISHED</appAttemptState>
  <amContainerId>container_1430424020775_0001_01_000001</amContainerId>
</appAttempt>
 Container List
利用Container List API，你能获取一个resource集合，其中每一个resource为一个container.运行GET operation，你能获取到一个container对象即可。
URI:
利用以下URI获取一个application attempt（通过appid和attemptid标识）的所有container对象
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps/{appid}/appattempts/{appattemptid}/containers
HTTP Operations Supported:
GET
Query Parameters Supported:
None
Elements of the containers (Container List) Object
当你请求container list时，会接收到container对象列表。
Item Data Type Description
container array of container objects(JSON)/zero or more container objects(XML) The collection of container objects
 
响应样例:
JSON response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts/appattempt_1430424020775_0001_000001/containers?
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "container": [
    {
      "containerId": "container_1430424020775_0001_01_000007",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 10,
      "startedTime": 1430424068296,
      "finishedTime": 1430424073006,
      "elapsedTime": 4710,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000007/container_1430424020775_0001_01_000007/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000006",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 20,
      "startedTime": 1430424060317,
      "finishedTime": 1430424068293,
      "elapsedTime": 7976,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000006/container_1430424020775_0001_01_000006/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000005",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 20,
      "startedTime": 1430424060316,
      "finishedTime": 1430424068294,
      "elapsedTime": 7978,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000005/container_1430424020775_0001_01_000005/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000003",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 20,
      "startedTime": 1430424060315,
      "finishedTime": 1430424068289,
      "elapsedTime": 7974,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000003/container_1430424020775_0001_01_000003/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000004",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 20,
      "startedTime": 1430424060315,
      "finishedTime": 1430424068291,
      "elapsedTime": 7976,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000004/container_1430424020775_0001_01_000004/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000002",
      "allocatedMB": 1024,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 20,
      "startedTime": 1430424060313,
      "finishedTime": 1430424067250,
      "elapsedTime": 6937,
      "diagnosticsInfo": "Container killed by the ApplicationMaster.\nContainer killed on request. Exit code is 143\nContainer exited with a non-zero exit code 143\n",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000002/container_1430424020775_0001_01_000002/zshen",
      "containerExitStatus": -105,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    },
    {
      "containerId": "container_1430424020775_0001_01_000001",
      "allocatedMB": 2048,
      "allocatedVCores": 1,
      "assignedNodeId": "localhost:9105",
      "priority": 0,
      "startedTime": 1430424054314,
      "finishedTime": 1430424079022,
      "elapsedTime": 24708,
      "diagnosticsInfo": "",
      "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000001/container_1430424020775_0001_01_000001/zshen",
      "containerExitStatus": 0,
      "containerState": "COMPLETE",
      "nodeHttpAddress": "http://localhost:8042"
    }
  ]
}
 XML response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts/appattempt_1430424020775_0001_000001/containers Accept: application/xml
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 1428
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<containers>
  <container>
    <containerId>container_1430424020775_0001_01_000007</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>10</priority>
    <startedTime>1430424068296</startedTime>
    <finishedTime>1430424073006</finishedTime>
    <elapsedTime>4710</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000007/container_1430424020775_0001_01_000007/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000006</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>20</priority>
    <startedTime>1430424060317</startedTime>
    <finishedTime>1430424068293</finishedTime>
    <elapsedTime>7976</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000006/container_1430424020775_0001_01_000006/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000005</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>20</priority>
    <startedTime>1430424060316</startedTime>
    <finishedTime>1430424068294</finishedTime>
    <elapsedTime>7978</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000005/container_1430424020775_0001_01_000005/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000003</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>20</priority>
    <startedTime>1430424060315</startedTime>
    <finishedTime>1430424068289</finishedTime>
    <elapsedTime>7974</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000003/container_1430424020775_0001_01_000003/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000004</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>20</priority>
    <startedTime>1430424060315</startedTime>
    <finishedTime>1430424068291</finishedTime>
    <elapsedTime>7976</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000004/container_1430424020775_0001_01_000004/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000002</containerId>
    <allocatedMB>1024</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>20</priority>
    <startedTime>1430424060313</startedTime>
    <finishedTime>1430424067250</finishedTime>
    <elapsedTime>6937</elapsedTime>
    <diagnosticsInfo>Container killed by the ApplicationMaster.
      Container killed on request. Exit code is 143
      Container exited with a non-zero exit code 143
    </diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000002/container_1430424020775_0001_01_000002/zshen</logUrl>
    <containerExitStatus>-105</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
  <container>
    <containerId>container_1430424020775_0001_01_000001</containerId>
    <allocatedMB>2048</allocatedMB>
    <allocatedVCores>1</allocatedVCores>
    <assignedNodeId>localhost:9105</assignedNodeId>
    <priority>0</priority>
    <startedTime>1430424054314</startedTime>
    <finishedTime>1430424079022</finishedTime>
    <elapsedTime>24708</elapsedTime>
    <diagnosticsInfo></diagnosticsInfo>
    <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000001/container_1430424020775_0001_01_000001/zshen</logUrl>
    <containerExitStatus>0</containerExitStatus>
    <containerState>COMPLETE</containerState>
    <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
  </container>
</containers>
 Container
利用Container API，你可以获取运行在YARN集群中的application的attempt特定container的信息。
URI:
使用如下URI获取由appid、apptemptid、containerid标识的container对象。
http(s)://<timeline server http(s) address:port>/ws/v1/applicationhistory/apps/{appid}/appattempts/{appattemptid}/containers/{containerid}
HTTP Operations Supported:
GET
Query Parameters Supported:
None
Elements of the container (Container) Object:
Item Data Type Description
containerId string The container Id
containerState string 来自ResourceManager的container state，有效的值为ContainerState的枚举。
containerExitStatus int container退出码
logUrl string 用于访问container聚合日志的URL。
diagnosticsInfo string 详细的诊断信息。
startedTime long container的started时间（从本epoch开始的毫秒数）
finishedTime long container的完成时间（从本epoch开始的毫秒数）
elapsedTime long container启动后的耗时（单位为毫秒）
allocatedMB int 分配到该container的内存数。
allocatedVCores int 分配到container计算核。
priority int container的优先级
assignedNodeId string containe分配的node的host和port
响应样例:
JSON response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts/appattempt_1430424020775_0001_000001/containers/container_1430424020775_0001_01_000001
Response Header:
HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked
Response Body:
{
  "containerId": "container_1430424020775_0001_01_000001",
  "allocatedMB": 2048,
  "allocatedVCores": 1,
  "assignedNodeId": "localhost:9105",
  "priority": 0,
  "startedTime": 1430424054314,
  "finishedTime": 1430424079022,
  "elapsedTime": 24708,
  "diagnosticsInfo": "",
  "logUrl": "http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000001/container_1430424020775_0001_01_000001/zshen",
  "containerExitStatus": 0,
  "containerState": "COMPLETE",
  "nodeHttpAddress": "http://localhost:8042"
}
 XML response
HTTP Request:
GET http://localhost:8188/ws/v1/applicationhistory/apps/application_1430424020775_0001/appattempts/appattempt_1430424020775_0001_000001/containers/container_1430424020775_0001_01_000001 Accept: application/xml
Response Header:
HTTP/1.1 200 OK Content-Type: application/xml Content-Length: 669
Response Body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<container>
  <containerId>container_1430424020775_0001_01_000001</containerId>
  <allocatedMB>2048</allocatedMB>
  <allocatedVCores>1</allocatedVCores>
  <assignedNodeId>localhost:9105</assignedNodeId>
  <priority>0</priority>
  <startedTime>1430424054314</startedTime>
  <finishedTime>1430424079022</finishedTime>
  <elapsedTime>24708</elapsedTime>
  <diagnosticsInfo></diagnosticsInfo>
  <logUrl>http://0.0.0.0:8188/applicationhistory/logs/localhost:9105/container_1430424020775_0001_01_000001/container_1430424020775_0001_01_000001/zshen</logUrl>
  <containerExitStatus>0</containerExitStatus>
  <containerState>COMPLETE</containerState>
  <nodeHttpAddress>http://localhost:8042</nodeHttpAddress>
</container>
 响应码：
  1. 查询中的domain，entity type，entity id 或者类似的不能解析的查询条件会返回HTTP 404 “Not Found” 的响应。
  2. 请求的path、parameter或者值如果无效，则响应结果为Bad Request, 400。
  3. 在一个安全的集群中，当调用者尝试执行没有权限的操作时，会生产一个401, “Forbidden”的响应。在查询一些entity时存在例外，如Domains；API在这里将权限拒绝的输出降级为empty和not-founds的应答。这隐藏了来自其他未授权调用方的domain的细节
  4. 如果timeline的entity PUT操作是无效的，这个失败不会返回在HTTP error code中，200 status code将会被返回，然而会有一个不能添加的entity的列表，其中的每个entity还有一个error code。

下一篇：hadoop 2.7.2 yarn中文文档——编写YARN Applications

  • 大小: 22.5 KB
分享到：
| hadoop 2.7.2 yarn中文文档——Web Applic ...
评论
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

    hadoop2.7.2(window版本).zip

    Hadoop 2.7.2引入了YARN（Yet Another Resource Negotiator），作为资源管理框架，负责调度和管理集群中的计算资源。相比早期版本，YARN提高了系统的可扩展性和安全性，降低了MapReduce的耦合度，使得其他计算框架如...

    hadoop2.7.2(Linux版本).zip

    1. YARN改进：在Hadoop 2.7.2中，YARN（Yet Another Resource Negotiator）进一步提升了资源管理效率，优化了任务调度算法，降低了作业启动延迟，增强了系统的整体性能。 2. HDFS增强：增加了对大文件的支持，改进...

    hadoop-2.7.2.rar

    Hadoop 2.7.2是Hadoop发展过程中的一个重要版本，它提供了许多增强特性和稳定性改进，使得大规模数据处理变得更加高效和可靠。在这个版本中，主要包含两个核心组件：HDFS（Hadoop Distributed File System）和...

    hadoop-yarn-api-2.5.1-API文档-中文版.zip

    标签：apache、hadoop、api、yarn、jar包、java、API文档、中文版； 使用方法：解压翻译后的API文档，用浏览器打开“index.html”文件，即可纵览文档内容。 人性化翻译，文档中的代码和结构保持不变，注释和说明精准...

    hadoop-yarn-client-2.6.5-API文档-中文版.zip

    标签：apache、client、hadoop、yarn、jar包、java、中文文档； 使用方法：解压翻译后的API文档，用浏览器打开“index.html”文件，即可纵览文档内容。 人性化翻译，文档中的代码和结构保持不变，注释和说明精准翻译...

    hadoop-2.7.2资源

    - YARN：作为Hadoop的资源管理器，YARN在2.7.2版本中进一步提升了多应用程序并行运行的效率，优化了资源调度算法，减少了任务调度的延迟。 - HDFS：增强了HDFS的容错性和可用性，如改进了数据块的复制策略，提高了...

    centos7 64位下编译Hadoop2.7.2源码库文件

    Hadoop 2.7.2是一个稳定版本，包含了分布式存储（HDFS）和计算框架（MapReduce），是大数据处理的基础。以下是关于如何在CentOS 7 64位系统上编译Hadoop 2.7.2源码库文件的详细步骤及相关的知识点： 1. **环境准备*...

    Hadoop2.7.2LIUNX集群(2)所需JDK1.8.gz

    Hadoop2.7.2LIUNX集群(2)所需JDK1.8.gzHadoop2.7.2LIUNX集群(2)所需JDK1.8.gzHadoop2.7.2LIUNX集群(2)所需JDK1.8.gzHadoop2.7.2LIUNX集群(2)所需JDK1.8.gz

    hadoop-2.7.2.tar.gz

    Hadoop 2.7.2 是一个开源框架，主要用于分布式存储和计算，是大数据处理领域的重要组成部分。这个版本的Hadoop在2015年发布，提供了许多改进和新特性，使得它能在各种Linux环境下稳定运行，从而满足企业对大规模数据...

    Hadoop2.7.2Windows10文件

    Windows10 环境下编译的Hadoop2.7.2 Windows10 环境下编译的Hadoop2.7.2 Windows10 环境下编译的Hadoop2.7.2

    hadoop2.7.2源码包

    Apache Hadoop 2.7.2 是一个广泛使用的开源框架，专为分布式存储和计算而设计，是大数据处理领域的重要工具。源码包提供了一窥Hadoop内部运作机制的机会，对于开发者、研究者以及想要深入理解Hadoop工作原理的人来说...

    hadoop-yarn-server-resourcemanager-2.6.0-API文档-中文版.zip

    赠送jar包：hadoop-yarn-server-...标签：server、apache、resourcemanager、hadoop、yarn、jar包、java、API文档、中文版； 使用方法：解压翻译后的API文档，用浏览器打开“index.html”文件，即可纵览文档内容。 人性

    Hadoop2.7.2伪分布部署

    较新版本的Hadoop2.x.y系列包括了YARN（Yet Another Resource Negotiator），这是一个资源管理平台，它改进了资源管理和作业调度的机制。老版本的Hadoop，如0.20或1.2.1版本，虽然在本教程中未详细涉及，但本教程的...

    hadoop2.7.2安装依赖文件

    hadoop2.7.2安装依赖文件,用于在window下调试hadoop！ hadoop2.7.2安装依赖文件,用于在window下调试hadoop hadoop2.7.2安装依赖文件,用于在window下调试hadoop

    hadoop2.7.2安装依赖文件.zip

    在这个名为“hadoop2.7.2安装依赖文件.zip”的压缩包中，包含了一系列在Windows环境下安装和运行Hadoop 2.7.2版本所必需的组件。下面我们将详细探讨这些文件及其在Hadoop生态系统中的作用。 首先，`hadoop.dll`是一...

    window版本下的hadoop 2.7.2版本的hadoop.dll和winutils.exe

    本文将详细介绍这两个组件以及如何在Windows系统下配置Hadoop 2.7.2版本的开发环境。 首先，`hadoop.dll`是Hadoop在Windows平台上的一个动态链接库文件，它包含了Hadoop运行所需的特定功能。由于Hadoop主要设计为在...

    hadoop-yarn-server-common-2.6.5-API文档-中文版.zip

    标签：server、apache、common、hadoop、yarn、jar包、java、中文文档； 使用方法：解压翻译后的API文档，用浏览器打开“index.html”文件，即可纵览文档内容。 人性化翻译，文档中的代码和结构保持不变，注释和说明...

    hadoop-yarn-client-2.7.3-API文档-中英对照版.zip

    包含翻译后的API文档：hadoop-yarn-client-2.7.3-javadoc-API文档-中文(简体)-英语-对照版.zip； Maven坐标：org.apache.hadoop:hadoop-yarn-client:2.7.3； 标签：apache、hadoop、yarn、client、中英对照文档、jar...

    hadoop2.7.2的hadoop.dll和winutils.exe

    8. **启动Hadoop服务**：最后，通过start-dfs.cmd和start-yarn.cmd脚本启动Hadoop的DataNodes、NameNodes以及ResourceManager等服务。 以上就是在Windows 7环境中安装和配置Hadoop 2.7.2所需的关键步骤，其中hadoop...

    hadoop2.7.2HA集群安装

    本文档将详细介绍如何搭建一个Hadoop 2.7.2版本的高可用（High Availability，简称HA）集群。此集群将包含五台服务器，分别命名为cancer01至cancer05，其中两台作为NameNode节点（活跃与备用），一台作为JournalNode...

Magicbox
Global site tag (gtag.js) - Google Analytics