HDFS磁盘调度策略

callstack:

 

FsVolumeList.getNextVolume(StorageType,long)----->FsVolumeList.chooseVolume(List,long)----->RoundRobinVolumeChoosingPolicy.chooseVolumn(List,long)

 

OR

 

FsVolumeList.getNextVolume(StorageType,long)----->FsVolumeList.chooseVolume(List,long)----->AvailableSpaceVolumeChoosingPolicy.chooseVolumn(List,long)

 

 

 

 

前者： Choose volumes in round-robin order.

后者： A DN volume choosing policy which takes into account the amount of free space on each of the available volumes when considering where to assign a new replica allocation. By default this policy prefers assigning replicas to those volumes with more available free space, so as to over time balance the available space of all the volumes within a DN.

 

 

默认采用RoundRobinVolumeChoosingPolicy

可以通过配置dfs.datanode.fsdataset.volume.choosing.policy参数来使用AvailableSpaceVolumeChoosingPolicy策略

 

数据请求过来时候，会依次轮训判断每块盘的可用空间大小是否大于请求的大小，如果请求写入数据的空间大于一台机器上所有磁盘的空间时候，会抛 out of space异常