kafka关注的issue

BrokerChangeListener computes inconsistent live/dead broker list https://issues.apache.org/jira/browse/KAFKA-3085 BrokerChangeListener missed broker id path ephemeral node deletion event. https://issues.apache.org/jira/browse/KAFKA-2448 Controller could miss a broker state change https://issues.apache.org/jira/browse/KAFKA-1120

在reduce InMemoryMapOutput OOM

2017-03-14 15:41:52,724 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in fetcher#1 at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56) at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46) at org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.<init>(InMemoryMapOutput.java:63)……

hive merge小文件

2.输出合并 set hive.merge.mapfiles = true #在Map-only的任务结束时合并小文件(默认开启) set hive.merge.mapredfiles = true #……

impala断开连接后释放session

代码在这里 https://github.com/cloudera/Impala/blob/cdh5-2.2.0_5.4.0/be/src/service/impala-server.cc 如果不想断开清除session,直接return void ImpalaServer::ConnectionEnd( const ThriftServer::ConnectionContext& connection_context) { return; unique_lock<mutex> l(connection_to_sessions_map_lock_); ConnectionToSessionMap::iterator it =……

HIVE-10956 HS2 leaks HMS connections

https://issues.apache.org/jira/browse/HIVE-10956 HMS链接泄露 https://issues.apache.org/jira/browse/HIVE-13002 HIVE-11934 Transaction lock retry logic results in infinite loop