hbase 使用lzo压缩问题

zhaopingzi

安装好lzo测试

1. 
   
2. -bash-3.2$ hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://h1:9000/user/member1 lzo
   
3. 14/01/13 18:02:46 INFO util.ChecksumType: Checksum can use java.util.zip.CRC32
   
4. 14/01/13 18:02:46 DEBUG util.FSUtils: Creating file=hdfs://h1:9000/user/member1 with permission=rwxrwxrwx
   
5. 14/01/13 18:02:46 INFO util.FSUtils: FileSystem doesn't support getDefaultReplication
   
6. 14/01/13 18:02:46 INFO util.FSUtils: FileSystem doesn't support getDefaultBlockSize
   
7. 14/01/13 18:02:46 ERROR metrics.SchemaMetrics: Inconsistent configuration. Previous configuration for using table name in metrics: true, new configuration: false
   
8. 14/01/13 18:02:46 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
   
9. 14/01/13 18:02:46 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
   
10. 14/01/13 18:02:46 INFO compress.CodecPool: Got brand-new compressor
    
11. 14/01/13 18:02:46 DEBUG hfile.HFileWriterV2: Initialized with CacheConfig:disabled
    
12. 14/01/13 18:02:46 INFO compress.CodecPool: Got brand-new decompressor
    
13. SUCCESS
    
14. -bash-3.2$
    

复制代码

lzo应该是装好了，但是

create 'my123',{NAME => 'colfam', COMPRESSION => 'lzo'}

总是建不了表，还挂死：

hmaster的日志

1. 
   
2. ,1389605778967, region=eff5925e5f620356484a8c000196a460
   
3. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h4,60020,1389605778967
   
4. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h1,60020,1389605773227; 5 (online=5, available=4) available servers
   
5. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED event for eff5925e5f620356484a8c000196a460
   
6. 2014-01-13 18:08:30,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=my123,,1389607706152.eff5925e5f620356484a8c000196a460. state=CLOSED, ts=1389607716138, server=h4,60020,1389605778967
   
7. 2014-01-13 18:08:30,420 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x2438af49d2a0000 Creating (or updating) unassigned node for eff5925e5f620356484a8c000196a460 with OFFLINE state
   
8. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE, server=h1,60000,1389605772420, region=eff5925e5f620356484a8c000196a460
   
9. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h1,60020,1389605773227
   
10. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for region my123,,1389607706152.eff5925e5f620356484a8c000196a460.; plan=hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h1,60020,1389605773227
    
11. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region my123,,1389607706152.eff5925e5f620356484a8c000196a460. to h1,60020,1389605773227
    
12. 2014-01-13 18:08:30,425 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=h1,60020,1389605773227, region=eff5925e5f620356484a8c000196a460
    
13. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_FAILED_OPEN, server=h1,60020,1389605773227, region=eff5925e5f620356484a8c000196a460
    
14. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h1,60020,1389605773227
    
15. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h5,60020,1389605772968; 5 (online=5, available=4) available servers
    
16. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED event for eff5925e5f620356484a8c000196a460
    
17. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=my123,,1389607706152.eff5925e5f620356484a8c000196a460. state=CLOSED, ts=1389607710427, server=h1,60020,1389605773227
    
18. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x2438af49d2a0000 Creating (or updating) unassigned node for eff5925e5f620356484a8c000196a460 with OFFLINE state
    
19. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE, server=h1,60000,1389605772420, region=eff5925e5f620356484a8c000196a460
    
20. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h5,60020,1389605772968
    
21. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for region my123,,1389607706152.eff5925e5f620356484a8c000196a460.; plan=hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h5,60020,1389605772968
    
22. 2014-01-13 18:08:30,431 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region my123,,1389607706152.eff5925e5f620356484a8c000196a460. to h5,60020,1389605772968
    
23. 2014-01-13 18:08:30,434 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=h5,60020,1389605772968, region=eff5925e5f620356484a8c000196a460
    
24. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_FAILED_OPEN, server=h5,60020,1389605772968, region=eff5925e5f620356484a8c000196a460
    
25. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h5,60020,1389605772968
    
26. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h4,60020,1389605778967; 5 (online=5, available=4) available servers
    

复制代码

乔彦克

楼主这个问题解决了吗

wenhq

http://wiki.apache.org/hadoop/UsingLzoCompression 参考这个link看看。