Flume-ng的HdfsSink出現Lease mismatch錯誤

多台Flume-ng Agent做HA，實際環境中出現Lease mismatch錯誤，具體報錯如下：

11 Mar 2014 12:21:02,971 WARN  [SinkRunner-PollingRunner-DefaultSinkProcessor] (
org.apache.flume.sink.hdfs.HDFSEventSink.process:418)  - HDFS IO error
java.io.IOException: IOException flush:org.apache.hadoop.ipc.RemoteException(org
.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): Lease mismatch on /l
ogdata/2014/03/11/91pc/FlumeData.1394467200001.tmp owned by DFSClient_NONMAPREDU
CE_1235353284_49 but is accessed by DFSClient_NONMAPREDUCE_1765113176_43
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSName
system.java:2459)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSName
system.java:2437)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.fsync(FSNamesyste
m.java:3106)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.fsync(NameNo
deRpcServer.java:823)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTra
nslatorPB.fsync(ClientNamenodeProtocolServerSideTranslatorPB.java:714)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$Cl
ientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:4501
0)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.cal
l(ProtobufRpcEngine.java:453)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1408)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)

        at org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.ja
va:1643)
        at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:15
25)
        at org.apache.hadoop.hdfs.DFSOutputStream.sync(DFSOutputStream.java:1510
)
        at org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:
116)
        at org.apache.flume.sink.hdfs.HDFSDataStream.sync(HDFSDataStream.java:11
7)
        at org.apache.flume.sink.hdfs.BucketWriter$5.call(BucketWriter.java:356)

        at org.apache.flume.sink.hdfs.BucketWriter$5.call(BucketWriter.java:353)

        at org.apache.flume.sink.hdfs.BucketWriter$8$1.run(BucketWriter.java:536
)
        at org.apache.flume.sink.hdfs.BucketWriter.runPrivileged(BucketWriter.ja
va:160)
        at org.apache.flume.sink.hdfs.BucketWriter.access$1000(BucketWriter.java
:56)
        at org.apache.flume.sink.hdfs.BucketWriter$8.call(BucketWriter.java:533)

        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
        at java.lang.Thread.run(Thread.java:724)

這個是由於配置了多個Flume agent做HA,每個agent都有多個HDFS客戶端。多個HDFS客戶端同時向一個HDFS文件寫數據的時候造成Lease租約問題，Lease可以認為是一個文件寫鎖。解決方法是配置每個Agent的HDFS文件前綴後者後綴