[笔记]hdfs namenode FSNamesystem分析

NameNode在内存中维护整个文件系统的元数据镜像，用于HDFS的管理。
NameNode中元数据的管理主要由类FSNamesystem实现。

引用

/***************************************************
* FSNamesystem does the actual bookkeeping work for the
* DataNode.
*
* It tracks several important tables.
*
* 1)  valid fsname --> blocklist  (kept on disk, logged)
* 2)  Set of all valid blocks (inverted #1)
* 3)  block --> machinelist (kept in memory, rebuilt dynamically from reports)
* 4)  machine --> blocklist (inverted #2)
* 5)  LRU cache of updated-heartbeat machines
***************************************************/
public class FSNamesystem implements FSConstants, FSNamesystemMBean, FSClusterStats,
    NameNodeMXBean, MetricsSource

重要的成员：

引用

//
  // Stores the correct file name hierarchy
  //
  public FSDirectory dir;

FSDirectory实现了INode的管理。

引用

//
  // Mapping: Block -> { INode, datanodes, self ref }
  // Updated only in response to client-sent information.
  //
  final BlocksMap blocksMap = new BlocksMap(DEFAULT_INITIAL_MAP_CAPACITY,
                                            DEFAULT_MAP_LOAD_FACTOR);

BlockMap存放了整个文件系统所有的block，以及每个block对应的元数据，包括所属的INode（即文件）、每个副本的位置Datanode的信息、以及自身，通过BlockInfo封装，BlockInfo继承Block。

引用

/**
   * Stores the datanode -> block map. 
   * <p>
   * Done by storing a set of {@link DatanodeDescriptor} objects, sorted by
   * storage id. In order to keep the storage map consistent it tracks
   * all storages ever registered with the namenode.
   * A descriptor corresponding to a specific storage id can be
   * <ul>
   * <li>added to the map if it is a new storage id;</li>
   * <li>updated with a new datanode started as a replacement for the old one
   * with the same storage id; and </li>
   * <li>removed if and only if an existing datanode is restarted to serve a
   * different storage id.</li>
   * </ul> <br>
   * The list of the {@link DatanodeDescriptor}s in the map is checkpointed
   * in the namespace image file. Only the {@link DatanodeInfo} part is
   * persistent, the list of blocks is restored from the datanode block
   * reports.
   * <p>
   * Mapping: StorageID -> DatanodeDescriptor
   */
  NavigableMap<String, DatanodeDescriptor> datanodeMap =
    new TreeMap<String, DatanodeDescriptor>();

DatanodeDescriptor是类DatanodeInfo的子类，其封装了Datanode存放的block集合。

元数据主要类图：