`

hdfs-default配置说明等

阅读更多

为方便查询故在此列出

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Do not modify this file directly.  Instead, copy entries that you -->
<!-- wish to modify from this file into hdfs-site.xml and change them -->
<!-- there.  If hdfs-site.xml does not already exist, create it.      -->

<configuration>

<property>
  <name>dfs.namenode.logging.level</name>
  <value>info</value>
  <description>The logging level for dfs namenode. Other values are "dir"(trac
e namespace mutations), "block"(trace block under/over replications and block
creations/deletions), or "all".</description>
</property>

<property>
  <name>dfs.secondary.http.address</name>
  <value>0.0.0.0:50090</value>
  <description>
    The secondary namenode http server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.address</name>
  <value>0.0.0.0:50010</value>
  <description>
    The address where the datanode server will listen to.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.http.address</name>
  <value>0.0.0.0:50075</value>
  <description>
    The datanode http server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.ipc.address</name>
  <value>0.0.0.0:50020</value>
  <description>
    The datanode ipc server address and port.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.datanode.handler.count</name>
  <value>3</value>
  <description>The number of server threads for the datanode.</description>
</property>

<property>
  <name>dfs.http.address</name>
  <value>0.0.0.0:50070</value>
  <description>
    The address and the base port where the dfs namenode web ui will listen on.
    If the port is 0 then the server will start on a free port.
  </description>
</property>

<property>
  <name>dfs.https.enable</name>
  <value>false</value>
  <description>Decide if HTTPS(SSL) is supported on HDFS
  </description>
</property>

<property>
  <name>dfs.https.need.client.auth</name>
  <value>false</value>
  <description>Whether SSL client certificate authentication is required
  </description>
</property>

<property>
  <name>dfs.https.server.keystore.resource</name>
  <value>ssl-server.xml</value>
  <description>Resource file from which ssl server keystore
  information will be extracted
  </description>
</property>

<property>
  <name>dfs.https.client.keystore.resource</name>
  <value>ssl-client.xml</value>
  <description>Resource file from which ssl client keystore
  information will be extracted
  </description>
</property>

<property>
  <name>dfs.datanode.https.address</name>
  <value>0.0.0.0:50475</value>
</property>

<property>
  <name>dfs.https.address</name>
  <value>0.0.0.0:50470</value>
</property>

 <property>
  <name>dfs.datanode.dns.interface</name>
  <value>default</value>
  <description>The name of the Network Interface from which a data node should 
  report its IP address.
  </description>
 </property>
 
<property>
  <name>dfs.datanode.dns.nameserver</name>
  <value>default</value>
  <description>The host name or IP address of the name server (DNS)
  which a DataNode should use to determine the host name used by the
  NameNode for communication and display purposes.
  </description>
 </property>
 
 
 
<property>
  <name>dfs.replication.considerLoad</name>
  <value>true</value>
  <description>Decide if chooseTarget considers the target's load or not
  </description>
</property>
<property>
  <name>dfs.default.chunk.view.size</name>
  <value>32768</value>
  <description>The number of bytes to view for a file on the browser.
  </description>
</property>

<property>
  <name>dfs.datanode.du.reserved</name>
  <value>0</value>
  <description>Reserved space in bytes per volume. Always leave this much space free for non dfs use.
  </description>
</property>

<property>
  <name>dfs.name.dir</name>
  <value>${hadoop.tmp.dir}/dfs/name</value>
  <description>Determines where on the local filesystem the DFS name node
      should store the name table(fsimage).  If this is a comma-delimited list
      of directories then the name table is replicated in all of the
      directories, for redundancy. </description>
</property>

<property>
  <name>dfs.name.edits.dir</name>
  <value>${dfs.name.dir}</value>
  <description>Determines where on the local filesystem the DFS name node
      should store the transaction (edits) file. If this is a comma-delimited list
      of directories then the transaction file is replicated in all of the 
      directories, for redundancy. Default value is same as dfs.name.dir
  </description>
</property>
<property>
  <name>dfs.web.ugi</name>
  <value>webuser,webgroup</value>
  <description>The user account used by the web interface.
    Syntax: USERNAME,GROUP1,GROUP2, ...
  </description>
</property>

<property>
  <name>dfs.permissions</name>
  <value>true</value>
  <description>
    If "true", enable permission checking in HDFS.
    If "false", permission checking is turned off,
    but all other behavior is unchanged.
    Switching from one parameter value to the other does not change the mode,
    owner or group of files or directories.
  </description>
</property>

<property>
  <name>dfs.permissions.supergroup</name>
  <value>supergroup</value>
  <description>The name of the group of super-users.</description>
</property>

<property>
  <name>dfs.data.dir</name>
  <value>${hadoop.tmp.dir}/dfs/data</value>
  <description>Determines where on the local filesystem an DFS data node
  should store its blocks.  If this is a comma-delimited
  list of directories, then data will be stored in all named
  directories, typically on different devices.
  Directories that do not exist are ignored.
  </description>
</property>

<property>
  <name>dfs.replication</name>
  <value>3</value>
  <description>Default block replication. 
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
  </description>
</property>

<property>
  <name>dfs.replication.max</name>
  <value>512</value>
  <description>Maximal block replication. 
  </description>
</property>

<property>
  <name>dfs.replication.min</name>
  <value>1</value>
  <description>Minimal block replication. 
  </description>
</property>

<property>
  <name>dfs.block.size</name>
  <value>67108864</value>
  <description>The default block size for new files.</description>
</property>

<property>
  <name>dfs.df.interval</name>
  <value>60000</value>
  <description>Disk usage statistics refresh interval in msec.</description>
</property>

<property>
  <name>dfs.client.block.write.retries</name>
  <value>3</value>
  <description>The number of retries for writing blocks to the data nodes, 
  before we signal failure to the application.
  </description>
</property>

<property>
  <name>dfs.blockreport.intervalMsec</name>
  <value>3600000</value>
  <description>Determines block reporting interval in milliseconds.</description>
</property>

<property>
  <name>dfs.blockreport.initialDelay</name>  <value>0</value>
  <description>Delay for first block report in seconds.</description>
</property>

<property>
  <name>dfs.heartbeat.interval</name>
  <value>3</value>
  <description>Determines datanode heartbeat interval in seconds.</description>
</property>

<property>
  <name>dfs.namenode.handler.count</name>
  <value>10</value>
  <description>The number of server threads for the namenode.</description>
</property>

<property>
  <name>dfs.safemode.threshold.pct</name>
  <value>0.999f</value>
  <description>
    Specifies the percentage of blocks that should satisfy 
    the minimal replication requirement defined by dfs.replication.min.
    Values less than or equal to 0 mean not to start in safe mode.
    Values greater than 1 will make safe mode permanent.
  </description>
</property>

<property>
  <name>dfs.safemode.extension</name>
  <value>30000</value>
  <description>
    Determines extension of safe mode in milliseconds 
    after the threshold level is reached.
  </description>
</property>

<property>
  <name>dfs.balance.bandwidthPerSec</name>
  <value>1048576</value>
  <description>
        Specifies the maximum amount of bandwidth that each datanode
        can utilize for the balancing purpose in term of
        the number of bytes per second.
  </description>
</property>

<property>
  <name>dfs.hosts</name>
  <value></value>
  <description>Names a file that contains a list of hosts that are
  permitted to connect to the namenode. The full pathname of the file
  must be specified.  If the value is empty, all hosts are
  permitted.</description>
</property>

<property>
  <name>dfs.hosts.exclude</name>
  <value></value>
  <description>Names a file that contains a list of hosts that are
  not permitted to connect to the namenode.  The full pathname of the
  file must be specified.  If the value is empty, no hosts are
  excluded.</description>
</property> 

<property>
  <name>dfs.max.objects</name>
  <value>0</value>
  <description>The maximum number of files, directories and blocks
  dfs supports. A value of zero indicates no limit to the number
  of objects that dfs supports.
  </description>
</property>

<property>
  <name>dfs.namenode.decommission.interval</name>
  <value>30</value>
  <description>Namenode periodicity in seconds to check if decommission is 
  complete.</description>
</property>

<property>
  <name>dfs.namenode.decommission.nodes.per.interval</name>
  <value>5</value>
  <description>The number of nodes namenode checks if decommission is complete
  in each dfs.namenode.decommission.interval.</description>
</property>

<property>
  <name>dfs.replication.interval</name>
  <value>3</value>
  <description>The periodicity in seconds with which the namenode computes 
  repliaction work for datanodes. </description>
</property>

<property>
  <name>dfs.access.time.precision</name>
  <value>3600000</value>
  <description>The access time for HDFS file is precise upto this value. 
               The default value is 1 hour. Setting a value of 0 disables
               access times for HDFS.
  </description>
</property>

<property>
  <name>dfs.support.append</name>
  <value>false</value>
  <description>Does HDFS allow appends to files?
               This is currently set to false because there are bugs in the
               "append code" and is not supported in any prodction cluster.
  </description>
</property>

</configuration>
 
分享到:
评论

相关推荐

    hadoop 2.9.0 hdfs-default.xml 属性集

    Hadoop 2.9.0版本的HDFS配置文件hdfs-site.xml定义了分布式文件系统的主要配置参数,下面详细说明这些属性的关键知识点。 1. hadoop.hdfs.configuration.version 这是一个配置属性,用于记录当前使用的Hadoop HDFS...

    hadoop配置文件参数详解1

    第一种方法是选择相应版本的 Hadoop,下载解压后,搜索 \*.xml,找到 core-default.xml、hdfs-default.xml 和 mapred-default.xml,这些文件就是默认配置,可以参考这些配置的说明和 key 设置 Hadoop 集群。...

    core-site配置文件说明

    `core-site.xml` 是Hadoop配置文件中的核心组件之一,它定义了Hadoop集群的基本设置,包括但不限于HDFS的访问地址、临时目录以及一些基本的I/O配置。通过正确配置这些参数,可以确保Hadoop集群的稳定运行和高效处理...

    Hadoop 三个配置文件的参数含义说明

    同时,需要注意的是,随着Hadoop版本的更新,某些默认配置和参数可能会有所变化,因此建议根据实际使用的Hadoop版本查阅相应的默认配置文件,如Apache官网提供的`core-default.xml`、`hdfs-default.xml`和`mapred-...

    hadoop配置属性

    - 方法一:下载与Hadoop版本相对应的Hadoop压缩包,解压后通过搜索*.xml文件,可以找到core-default.xml、hdfs-default.xml和mapred-default.xml,这些文件中包含了默认的配置属性及说明,可以作为设置Hadoop集群时...

    Hadoop2.7.7

    在这些文件中设置HDFS的相关参数,例如`fs.defaultFS`用于指定HDFS的默认名称节点地址,通常设置为`hdfs://localhost:9000`。 4. 配置`mapred-site.xml`以启用MapReduce作业历史服务器。 5. 初始化HDFS。使用`hadoop...

    YARN相关配置总结

    - **配置说明**: - **Configuration Files**:根据需要调整`core-site.xml`、`hdfs-site.xml`、`yarn-site.xml` 和 `mapred-site.xml`中的参数。 - **Environment Configuration**:通过`hadoop-env.sh`和`yarn-...

    Java访问Hadoop分布式文件系统HDFS的配置说明

    其他配置如`dfs.namenode.servicerpc-address`、`dfs.https.address`等涉及HDFS的服务地址和端口,而`dfs.client.use.datanode.hostname`则控制是否使用数据节点的主机名进行通信。 为了使用Java API访问HDFS,你还...

    hadoop配置文件编辑

    本文将对 Hadoop 的配置文件进行详细的解释,并对每个配置文件中的关键参数进行详细的说明。 一、core-site.xml core-site.xml 是 Hadoop 集群的核心配置文件,用于配置 Hadoop 的基本参数。这里面有七个关键参数...

    hadoop安装部署说明文档

    在`etc/hadoop/core-site.xml`中配置默认文件系统为HDFS,`etc/hadoop/hdfs-site.xml`中配置副本数量等参数。 5. **格式化NameNode**:首次安装时需执行`hdfs namenode -format`。 6. **启动Hadoop**:依次启动...

    hadoop配置.zip

    以下是对配置过程的详细说明: 一、Hadoop HA概述 Hadoop HA主要通过在两个不同的节点上设置NameNode的热备来实现。这两个节点被称为活动NameNode(Active NN)和备用NameNode(Standby NN)。当活动NameNode出现...

    5_平台搭建文档.pdf

    - 配置core-site.xml:设置HDFS的默认文件系统(fs.defaultFS)和临时目录(hadoop.tmp.dir)。 - 配置hdfs-site.xml:设置数据副本数(dfs.replication)和NameNode的HTTP访问地址。 - 配置mapred-site.xml:...

    大数据运维技术第4章 Hadoop文件参数配置课件.pptx

    - **hdfs-site.xml**:此文件专门针对HDFS(Hadoop Distributed File System)进行配置,包括命名节点(NameNode)和数据节点(DataNode)的位置,文件副本数量,以及文件访问权限等。 - **mapred-site.xml**:...

    Vmware的Centos7中的Spark虚拟机配置

    - 配置`core-site.xml`,指定`fs.default.name`为HDFS的名称节点地址,`hadoop.tmp.dir`为临时目录。 - 配置`hdfs-site.xml`,例如设置`dfs.replication`为副本数量。 - 格式化HDFS:`hdfs namenode -format`。 ...

    Hadoop-2.8.5全面资料

    - **hdfs-site.xml**:配置HDFS的具体参数,如副本数量、数据块大小等。 - **mapred-site.xml**:配置MapReduce相关的参数,如JobTracker的地址等。 - **yarn-site.xml**:配置YARN的具体参数,如ResourceManager...

    hadoop 集群搭建说明书.rar

    - 在`core-site.xml`中,设置`fs.defaultFS`为HDFS的命名节点地址,通常为Master节点的IP。 - 在`hdfs-site.xml`中,配置副本因子(replication factor)和DFS相关参数,如`dfs.namenode.name.dir`和`dfs.datanode...

    在VWmare的CentO S系统中配置hadoop

    - 编辑`core-site.xml`、`hdfs-site.xml`、`mapred-site.xml`和`yarn-site.xml`等配置文件,根据实际需求设置相应的属性值。 - 特别注意`core-site.xml`中的`fs.defaultFS`应该设置为`hdfs://localhost:9000`...

    Hadoop-2.6.0分布式部署参考手册

    - **core-site.xml**:用于配置HDFS的基本信息,如NameNode的地址等。 - **hdfs-site.xml**:特定于HDFS的服务配置。 - **mapred-site.xml**:MapReduce框架的核心配置文件。 - **yarn-site.xml**:YARN资源管理器的...

    CSDN TUP第四期雅虎讲师PPT(.pdf)

    1. **默认配置**:包含`core-default.xml`、`hdfs-default.xml` 和 `mapred-default.xml` 文件,这些文件提供了Hadoop的基本配置项。 2. **站点特定配置**:包括 `core-site.xml`、`hdfs-site.xml` 和 `mapred-site....

Global site tag (gtag.js) - Google Analytics