免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 4058 | 回复: 2

[HBase] hbase 使用lzo压缩问题 [复制链接]

论坛徽章:
71
15-16赛季CBA联赛之同曦
日期:2018-08-23 15:41:42辰龙
日期:2014-08-15 09:07:43狮子座
日期:2014-06-03 13:55:33亥猪
日期:2014-06-02 11:17:08巨蟹座
日期:2014-05-06 10:02:03午马
日期:2014-05-04 08:18:27亥猪
日期:2014-04-29 11:11:32技术图书徽章
日期:2014-04-24 15:51:26技术图书徽章
日期:2014-04-17 11:01:53辰龙
日期:2014-04-15 12:45:46亥猪
日期:2014-04-11 09:06:23射手座
日期:2014-04-01 15:28:10
发表于 2014-01-13 18:20 |显示全部楼层
安装好lzo测试

  1. -bash-3.2$ hbase org.apache.hadoop.hbase.util.CompressionTest hdfs://h1:9000/user/member1 lzo
  2. 14/01/13 18:02:46 INFO util.ChecksumType: Checksum can use java.util.zip.CRC32
  3. 14/01/13 18:02:46 DEBUG util.FSUtils: Creating file=hdfs://h1:9000/user/member1 with permission=rwxrwxrwx
  4. 14/01/13 18:02:46 INFO util.FSUtils: FileSystem doesn't support getDefaultReplication
  5. 14/01/13 18:02:46 INFO util.FSUtils: FileSystem doesn't support getDefaultBlockSize
  6. 14/01/13 18:02:46 ERROR metrics.SchemaMetrics: Inconsistent configuration. Previous configuration for using table name in metrics: true, new configuration: false
  7. 14/01/13 18:02:46 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
  8. 14/01/13 18:02:46 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
  9. 14/01/13 18:02:46 INFO compress.CodecPool: Got brand-new compressor
  10. 14/01/13 18:02:46 DEBUG hfile.HFileWriterV2: Initialized with CacheConfig:disabled
  11. 14/01/13 18:02:46 INFO compress.CodecPool: Got brand-new decompressor
  12. SUCCESS
  13. -bash-3.2$
复制代码
lzo应该是装好了,但是

create 'my123',{NAME => 'colfam', COMPRESSION => 'lzo'}

总是建不了表,还挂死:

hmaster的日志

  1. ,1389605778967, region=eff5925e5f620356484a8c000196a460
  2. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h4,60020,1389605778967
  3. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h1,60020,1389605773227; 5 (online=5, available=4) available servers
  4. 2014-01-13 18:08:30,419 DEBUG org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED event for eff5925e5f620356484a8c000196a460
  5. 2014-01-13 18:08:30,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=my123,,1389607706152.eff5925e5f620356484a8c000196a460. state=CLOSED, ts=1389607716138, server=h4,60020,1389605778967
  6. 2014-01-13 18:08:30,420 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x2438af49d2a0000 Creating (or updating) unassigned node for eff5925e5f620356484a8c000196a460 with OFFLINE state
  7. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE, server=h1,60000,1389605772420, region=eff5925e5f620356484a8c000196a460
  8. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h1,60020,1389605773227
  9. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for region my123,,1389607706152.eff5925e5f620356484a8c000196a460.; plan=hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h1,60020,1389605773227
  10. 2014-01-13 18:08:30,422 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region my123,,1389607706152.eff5925e5f620356484a8c000196a460. to h1,60020,1389605773227
  11. 2014-01-13 18:08:30,425 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=h1,60020,1389605773227, region=eff5925e5f620356484a8c000196a460
  12. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_FAILED_OPEN, server=h1,60020,1389605773227, region=eff5925e5f620356484a8c000196a460
  13. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h1,60020,1389605773227
  14. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h5,60020,1389605772968; 5 (online=5, available=4) available servers
  15. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED event for eff5925e5f620356484a8c000196a460
  16. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=my123,,1389607706152.eff5925e5f620356484a8c000196a460. state=CLOSED, ts=1389607710427, server=h1,60020,1389605773227
  17. 2014-01-13 18:08:30,428 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x2438af49d2a0000 Creating (or updating) unassigned node for eff5925e5f620356484a8c000196a460 with OFFLINE state
  18. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE, server=h1,60000,1389605772420, region=eff5925e5f620356484a8c000196a460
  19. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h5,60020,1389605772968
  20. 2014-01-13 18:08:30,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for region my123,,1389607706152.eff5925e5f620356484a8c000196a460.; plan=hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h5,60020,1389605772968
  21. 2014-01-13 18:08:30,431 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region my123,,1389607706152.eff5925e5f620356484a8c000196a460. to h5,60020,1389605772968
  22. 2014-01-13 18:08:30,434 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=h5,60020,1389605772968, region=eff5925e5f620356484a8c000196a460
  23. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_FAILED_OPEN, server=h5,60020,1389605772968, region=eff5925e5f620356484a8c000196a460
  24. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan for my123,,1389607706152.eff5925e5f620356484a8c000196a460. destination server is h5,60020,1389605772968
  25. 2014-01-13 18:08:30,436 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for my123,,1389607706152.eff5925e5f620356484a8c000196a460. so generated a random one; hri=my123,,1389607706152.eff5925e5f620356484a8c000196a460., src=, dest=h4,60020,1389605778967; 5 (online=5, available=4) available servers
复制代码

论坛徽章:
0
发表于 2014-04-23 16:55 |显示全部楼层
楼主这个问题解决了吗

论坛徽章:
16
IT运维版块每日发帖之星
日期:2015-10-02 06:20:00IT运维版块每月发帖之星
日期:2015-09-11 19:30:52IT运维版块每周发帖之星
日期:2015-09-11 19:20:31IT运维版块每日发帖之星
日期:2015-08-26 06:20:00每日论坛发贴之星
日期:2015-08-20 06:20:00IT运维版块每日发帖之星
日期:2015-08-20 06:20:002015年辞旧岁徽章
日期:2015-03-03 16:54:15金牛座
日期:2014-05-04 16:58:09双子座
日期:2013-12-17 16:44:37辰龙
日期:2013-11-22 15:20:59狮子座
日期:2013-11-18 22:55:08射手座
日期:2013-11-12 10:54:26
发表于 2015-08-17 13:47 |显示全部楼层
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP