region split时metascan出现regioninfo为空

最近测试94版本时，出现过多次 HRegionInfo was null or empty in Meta  的异常 

WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table:
java.io.IOException: HRegionInfo was null or empty in Meta for writetest, row=writetest,MNI07EGK9EOW7L7VG8TK,99999999999999
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:170)
at org.apache.hadoop.hbase.client.MetaScanner.access$1(MetaScanner.java:143)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:1)
at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:895)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:949)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:837)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:802) 
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:726)
at org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:82)
at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:162)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1326)

 

 

客户端偶然捕获到metascan时regioninfo为空的情况，此时region正在split过程中。代码上看应该不会有空的情况，是什么原因呢？

 

 

Client 端：
metaTable = new HTable(configuration, HConstants.META_TABLE_NAME);
Result startRowResult = metaTable.getRowOrBefore(searchRow,
HConstants.CATALOG_FAMILY);
if (startRowResult == null) { throw new TableNotFoundException("Cannot find row in .META. for table: " + Bytes.toString(tableName) + ", row=" + Bytes.toStringBinary(searchRow)); }
byte[] value = startRowResult.getValue(HConstants.CATALOG_FAMILY,
HConstants.REGIONINFO_QUALIFIER);
if (value == null || value.length == 0) { throw new IOException("HRegionInfo was null or empty in Meta for " + Bytes.toString(tableName) + ", row=" + Bytes.toStringBinary(searchRow)); }

Server端：
HRegionServer 
public Result getClosestRowBefore(final byte[] regionName, final byte[] row,
final byte[] family) throws IOException { // locate the region we're operating on HRegion region = getRegion(regionName); // ask the region for all the data Result r = region.getClosestRowBefore(row, family); return r; }
HRegion 
public Result getClosestRowBefore(final byte [] row, final byte [] family)
throws IOException {
try {
Store store = getStore(family);
// get the closest key. (HStore.getRowKeyAtOrBefore can return null)
KeyValue key = store.getRowKeyAtOrBefore(row);
Result result = null;
if (key != null) { Get get = new Get(key.getRow()); get.addFamily(family); result = get(get, null); }
return result;
} finally { closeRegionOperation(); }
}
HRegion.getClosestRowBefore 里面有两块：
（1）KeyValue key = store.getRowKeyAtOrBefore(row);获取到最近的KeyValue 
（2）get(get, null)，也就是scan的逻辑
在store.getRowKeyAtOrBefore时不考虑readPoint,所以memstore中任何信息都可以读取到
在scan逻辑中会考虑readPoint，只有小于等于readPoint才会被读取出来
在Region split时有一定概率会出现在查找一个key对应的region时，getRowKeyAtOrBefore可以找到新的region的keyvalue，但是scan时由于readPoint比较小，所以这个region的keyvalue都被过滤了，导致返回null，所以会出现上面的异常信息

 

90中不出现，主要是由于90中是先写hlog，后写memstore，然后更新mvcc，后两步很快，而94中顺序变为了先写memstore, 再写hlog,最后更新mvcc，当发生split时，如果正在写hlog的时候，客户端去访问，就会出现从memstore里面取出来的记录，在meta表里面因为mvcc的问题读取不到，从而报上述警告日志。

由于出现这个警告日志是在prefetch location中报出，实际在写的时候会再次去获取location，这时候发生问题的概率会变小，该问题对正常功能影响不是很大，先不改进。