Hadoop Do not use small blocks for balancing the cluster https://issues.apache.org/jira/browse/HDFS-8824 Allow Balancer to run faster https://issues.apache.org/jira/browse/HDFS-8818 平衡设置的文章：https://community.…… Hadoop 修改一下超时次数，具体代码在 DFSOutputStream.java private void completeFile(ExtendedBlock last) throws IOException { long localstart = Time.now(); long localTimeout = 400; boolean fileComplete = false; int retries = dfsClient.getConf().nBlockWriteLocateFollowingRetry; while (!fileComplete) { fileComplete…… Hadoop 记一次DataNode慢启动问题 ：启动过程中，datanode为了获取used siz…… Hadoop DiskChecker负责检查目录是否有建目录和判断权限。 datanode BlockPoolSlice.checkDirs() DataNode.checkDiskError() nodeManager LocalDirsHandlerService.checkDirs() Hbase create 'ucbrowser_imei2', {NAME=>'date', VERSIONS=>5} disable 'test_imei' drop 'test_imei' # 预先指定分区 create 'ucbrowser_utdid', 'date', { VERSIONS => '5', DATA_BLOCK_ENCODING => 'PREFIX', COMPRESSION => 'SNAPPY', SPLITS=>['03ufUX/IWscQABV+EwvaHDOo']} # 预先分区（200个reg…… Hadoop 之前发现在map任务里面经常看到Container killed by the ApplicationMas…… 未分类 划分VLAN 在网络->交换机里面，添加一个VLAN，看一下第二个vlan是怎么…… Hadoop 版本: hadoop cdh5.4 datanode jstack，很多这样的线程 "DataXceiver for client unix:/var/run/hdfs-sockets/dn [Waiting for operation #1]" daemon prio=10 tid=0x00007ffc42de9000 nid=0x68f8 waiting on condition [0x00007ffacbd1d000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method)…… Hadoop 下载hadoop编译好的包 安装cygwin 下载hadoop.dll和winutils…… Hadoop 我在namenode配置dfs.namenode.servicerpc-addres……