免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
› 论坛 › 操作系统 › Linux论坛 › 集群和虚拟机 › 为何LVS/TUN方式不能正常工作呢?
12下一页
最近访问板块 发新帖
查看: 3009 | 回复: 15
打印 上一主题 下一主题

为何LVS/TUN方式不能正常工作呢? [复制链接]

Linuxcn.com

论坛徽章:
0
跳转到指定楼层
1楼 [收藏(0)] [报告]
发表于 2006-06-30 12:15 |只看该作者 |倒序浏览
情况交代:
1>主机:win XP,本地连接的网卡LAN接局域网,IP地址:192.168.1.79/24,本地连接的Internet连接共享属性中,选择VMware Network Adapter VMnet1虚拟网卡,VMware Network Adapter VMnet1的IP地址:192.168.0.1/24

2>vm:ubuntu
Loadbalance:192.168.0.10/24
Realserver1:192.168.0.12/24
Realserver2:192.168.0.13/24

3>在win XP的DOS窗口下可以ping通192.168.0.10-192.168.0.13,在ubuntu下它们之间也可以相互ping通
4>相同的环境下,LVS/DR方式可以顺利工作
5>LVS/TUN方式下的一些配置文件内容摘要:
Loadbalance:
  1. echo "1" >/proc/sys/net/ipv4/ip_forward
  2. ifconfig eth0:0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20
  3. $IPVSADM -C
  4. $IPVSADM -A -t 192.168.0.20:80 -s rr
  5. $IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.12 -i
  6. $IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.13 -i
复制代码


Realserver1 & Realserver2:
  1. ifconfig tunl0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20
  2. route add -host 192.168.0.20 dev tunl0
  3. echo "1" > /proc/sys/net/ipv4/ip_forward
  4. echo "1" > /proc/sys/net/ipv4/conf/all/hidden
  5. echo "1" > /proc/sys/net/ipv4/conf/tunl0/hidden
复制代码


6>Realserver1 & Realserver2的kernel都已经支持了IP Tunnel,见图


7>Loadbalance的route表:
  1. root@Loadbalance:/etc/init.d# route -n
  2. Kernel IP routing table
  3. Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  4. 192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
  5. 10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth1
  6. 0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
复制代码

8>Realserver的route表:
  1. root@Realserver1:/etc/init.d# route -n
  2. Kernel IP routing table
  3. Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  4. 192.168.0.20    0.0.0.0         255.255.255.255 UH    0      0        0 tunl0
  5. 192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
  6. 0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
复制代码


9>但是,当我在win XP下从IE访问虚拟IP:http://192.168.0.20时,却不能够正常访问到测试的web页面
大家帮我看看我那里出错了,不胜感激!

[ 本帖最后由 Linuxcn.com 于 2006-6-30 12:21 编辑 ]
Linuxcn.com

论坛徽章:
0
2楼 [报告]
发表于 2006-07-03 09:35 |只看该作者
小顶一下
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
crazycash

论坛徽章:
0
3楼 [报告]
发表于 2006-07-03 17:10 |只看该作者

回复 1楼 Linuxcn.com 的帖子

在win XP下从IE访问虚拟IP:http://192.168.0.12／13，能够正常访问到测试的web页面么？
从loadbalance能够正常ping通192.168.0.2么？
使用tcpdump命令看看有没有包被转发过来。

另外可以把192.168.0.2配在tunl0:0上，这样如果有其他服务也作loadbalance，就不会引起冲突。
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
4楼 [报告]
发表于 2006-07-03 17:19 |只看该作者
原帖由 crazycash 于 2006-7-3 17:10 发表
在win XP下从IE访问虚拟IP:http://192.168.0.12／13，能够正常访问到测试的web页面么？
从loadbalance能够正常ping通192.168.0.2么？
使用tcpdump命令看看有没有包被转发过来。

另外可以把192.168.0.2配在tunl0:0上，这样如果有其他服务也作loadbalance，就不会引起冲突。


我开始也是想把虚拟IP在RS上配置到tunl0:0上来的， 但是配的时候它说设备不存在
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
5楼 [报告]
发表于 2006-07-04 14:07 |只看该作者
顶了
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
6楼 [报告]
发表于 2006-07-04 14:33 |只看该作者
当我做这个动作:从IE里输入:http://192.168.0.20访问web服务时,在Loadbalance机器上监视虚拟IP
  1. root@Loadbalance:/etc/init.d# tcpdump dst 192.168.0.20
  2. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  3. listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
  4. 11:01:32.495079 IP 192.168.0.1.3478 > 192.168.0.20.www: S 1562695239:1562695239(0) win 65535 <mss 1460,nop,nop,sackOK>
  5. 11:01:35.114333 IP 192.168.0.1.3478 > 192.168.0.20.www: S 1562695239:1562695239(0) win 65535 <mss 1460,nop,nop,sackOK>
  6. 11:01:38.467705 IP 192.168.0.1.3478 > 192.168.0.20.www: S 1562695239:1562695239(0) win 65535 <mss 1460,nop,nop,sackOK>
  7. 11:04:24.610365 arp who-has 192.168.0.20 tell 192.168.0.1
  8. 11:04:24.611726 IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK>
  9. 11:04:26.425108 IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK>
  10. 11:04:32.180984 IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK>
复制代码



在RS上监视Loadbalance的IP动作
  1. root@Realserver1:/home# tcpdump src 192.168.0.10
  2. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  3. listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
  4. 13:37:57.111593 IP 192.168.0.10 > 192.168.0.13: IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK> (ipip-proto-4)
  5. 13:37:57.118789 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 4234777010:4234777110(100) ack 295366668 win 8576
  6. 13:37:57.120787 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 100:264(164) ack 1 win 8576
  7. 13:37:57.122823 arp reply 192.168.0.10 is-at 00:0c:29:77:76:c7 (oui Unknown)
  8. 13:37:59.430390 IP 192.168.0.10 > 192.168.0.13: IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK> (ipip-proto-4)
  9. 13:37:59.430476 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 264:428(164) ack 1 win 8576
  10. 13:38:02.742802 arp who-has 192.168.0.13 tell 192.168.0.10
  11. 13:38:05.031489 IP 192.168.0.10 > 192.168.0.13: IP 192.168.0.1.3479 > 192.168.0.20.www: S 2187440212:2187440212(0) win 65535 <mss 1460,nop,nop,sackOK> (ipip-proto-4)
  12. 13:38:05.034128 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 428:592(164) ack 1 win 8576
  13. 13:38:22.508823 IP 192.168.0.10.netbios-dgm > 192.168.0.255.netbios-dgm: NBT UDP PACKET(138)
复制代码


不明白像这样的信息是什么意思:
13:37:59.430476 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 264:428(164) ack 1 win 8576

192.168.0.1是主机上的虚拟网卡IP,其他4台机器都是VM ware虚拟的

[ 本帖最后由 Linuxcn.com 于 2006-7-4 14:37 编辑 ]
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
7楼 [报告]
发表于 2006-07-04 14:47 |只看该作者
我LVS/DR方式时抓的包:
Loadbalance:
  1. root@Loadbalance:/etc/init.d# tcpdump dst 192.168.0.20
  2. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  3. listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
  4. 11:12:14.495745 IP 192.168.0.1.3579 > 192.168.0.20.www: S 1508679512:1508679512(0) win 65535 <mss 1460,nop,nop,sackOK>
  5. 11:12:14.515957 IP 192.168.0.1.3579 > 192.168.0.20.www: S 1508679512:1508679512(0) win 65535 <mss 1460,nop,nop,sackOK>
  6. 11:12:14.496939 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 2153076980 win 65535
  7. 11:12:14.497034 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 1 win 65535
  8. 11:12:14.496975 IP 192.168.0.1.3579 > 192.168.0.20.www: P 0:454(454) ack 1 win 65535
  9. 11:12:14.497110 IP 192.168.0.1.3579 > 192.168.0.20.www: P 0:454(454) ack 1 win 65535
  10. 11:12:14.600835 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 367 win 65169
  11. 11:12:14.600883 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 367 win 65169
  12. 11:12:32.695592 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 368 win 65169
  13. 11:12:32.696973 IP 192.168.0.1.3579 > 192.168.0.20.www: . ack 368 win 65169
  14. 11:12:33.626790 IP 192.168.0.1.3579 > 192.168.0.20.www: R 454:454(0) ack 368 win 0
  15. 11:12:33.626845 IP 192.168.0.1.3579 > 192.168.0.20.www: R 454:454(0) ack 368 win 0
  16. 11:12:43.782567 IP 192.168.0.1.3581 > 192.168.0.20.www: S 2172478390:2172478390(0) win 65535 <mss 1460,nop,nop,sackOK>
  17. 11:12:43.782879 IP 192.168.0.1.3581 > 192.168.0.20.www: S 2172478390:2172478390(0) win 65535 <mss 1460,nop,nop,sackOK>
  18. 11:12:43.783658 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 430936399 win 65535
  19. 11:12:43.783747 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 1 win 65535
  20. 11:12:43.783684 IP 192.168.0.1.3581 > 192.168.0.20.www: P 0:454(454) ack 1 win 65535
  21. 11:12:43.784286 IP 192.168.0.1.3581 > 192.168.0.20.www: P 0:454(454) ack 1 win 65535
  22. 11:12:43.819822 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 367 win 65169
  23. 11:12:43.819881 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 367 win 65169
  24. 11:13:01.135798 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 368 win 65169
  25. 11:13:01.135872 IP 192.168.0.1.3581 > 192.168.0.20.www: . ack 368 win 65169
复制代码


Realserver1:
  1. root@Realserver1:/etc/init.d# tcpdump src 192.168.0.10
  2. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  3. listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
  4. 13:45:55.017984 IP 192.168.0.10.1024 > dns-chj.sh.cncnet.net.domain:  46347+ PTR? 20.0.168.192.in-addr.arpa. (43)
  5. 13:45:55.033165 IP 192.168.0.10.1024 > dns-chj.sh.cncnet.net.domain:  14949+ PTR? 1.0.168.192.in-addr.arpa. (42)
  6. 13:45:55.049390 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 4234787174:4234787338(164) ack 295370700 win 9648
  7. 13:45:55.052651 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 164:328(164) ack 1 win 9648
  8. 13:45:55.059150 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 328:460(132) ack 1 win 9648
  9. 13:45:55.062662 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 460:576(116) ack 1 win 9648
  10. 13:45:55.065992 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 576:708(132) ack 1 win 9648
  11. 13:45:55.067946 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 708:840(132) ack 1 win 9648
  12. 13:45:55.142332 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 840:956(116) ack 1 win 9648
  13. 13:45:55.142343 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 956:1072(116) ack 1 win 9648
  14. 13:45:59.417299 arp who-has 192.168.0.13 tell 192.168.0.10
  15. 13:46:13.879055 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1072:1188(116) ack 1 win 9648
  16. 13:46:13.885049 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1188:1304(116) ack 1 win 9648
  17. 13:46:14.970845 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1304:1436(132) ack 1 win 9648
  18. 13:46:14.973078 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1436:1568(132) ack 1 win 9648
  19. 13:46:25.379295 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1568:1732(164) ack 1 win 9648
  20. 13:46:25.381318 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1732:1896(164) ack 1 win 9648
  21. 13:46:25.386048 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1896:2028(132) ack 1 win 9648
  22. 13:46:25.390993 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2028:2144(116) ack 1 win 9648
  23. 13:46:25.397327 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2144:2276(132) ack 1 win 9648
  24. 13:46:25.398270 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2276:2408(132) ack 1 win 9648
  25. 13:46:25.441275 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2408:2524(116) ack 1 win 9648
  26. 13:46:25.443290 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2524:2640(116) ack 1 win 9648
  27. 13:46:29.592310 arp who-has Realserver1 tell 192.168.0.10
  28. 13:46:41.591921 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2640:2756(116) ack 1 win 9648
  29. 13:46:41.595049 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2756:2872(116) ack 1 win 9648
  30. 13:47:50.542438 IP 192.168.0.10.netbios-ns > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
  31. 13:47:50.551414 arp reply 192.168.0.10 is-at 00:0c:29:77:76:c7 (oui Unknown)
复制代码


Realserver2:
  1. root@Realserver2:/etc/init.d# tcpdump src 192.168.0.10
  2. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  3. listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
  4. 14:15:09.051809 IP 192.168.0.10.1024 > dns-chj.sh.cncnet.net.domain:  46347+ PTR? 20.0.168.192.in-addr.arpa. (43)
  5. 14:15:09.070910 IP 192.168.0.10.1024 > dns-chj.sh.cncnet.net.domain:  14949+ PTR? 1.0.168.192.in-addr.arpa. (42)
  6. 14:15:09.098295 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 4234787174:4234787338(164) ack 295370700 win 9648
  7. 14:15:09.098340 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 164:328(164) ack 1 win 9648
  8. 14:15:09.098384 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 328:460(132) ack 1 win 9648
  9. 14:15:09.098597 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 460:576(116) ack 1 win 9648
  10. 14:15:09.098655 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 576:708(132) ack 1 win 9648
  11. 14:15:09.102670 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 708:840(132) ack 1 win 9648
  12. 14:15:09.195099 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 840:956(116) ack 1 win 9648
  13. 14:15:09.195115 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 956:1072(116) ack 1 win 9648
  14. 14:15:12.290227 arp who-has Realserver2 tell 192.168.0.10
  15. 14:15:25.805998 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1072:1188(116) ack 1 win 9648
  16. 14:15:25.812478 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1188:1304(116) ack 1 win 9648
  17. 14:15:27.440328 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1304:1436(132) ack 1 win 9648
  18. 14:15:27.440328 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1436:1568(132) ack 1 win 9648
  19. 14:15:37.461602 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1568:1732(164) ack 1 win 9648
  20. 14:15:37.466608 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1732:1896(164) ack 1 win 9648
  21. 14:15:37.469582 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 1896:2028(132) ack 1 win 9648
  22. 14:15:37.474732 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2028:2144(116) ack 1 win 9648
  23. 14:15:37.479656 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2144:2276(132) ack 1 win 9648
  24. 14:15:37.482315 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2276:2408(132) ack 1 win 9648
  25. 14:15:37.542983 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2408:2524(116) ack 1 win 9648
  26. 14:15:37.545591 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2524:2640(116) ack 1 win 9648
  27. 14:15:43.928563 arp who-has 192.168.0.12 tell 192.168.0.10
  28. 14:15:55.998471 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2640:2756(116) ack 1 win 9648
  29. 14:15:56.002589 IP 192.168.0.10.ssh > 192.168.0.1.3248: P 2756:2872(116) ack 1 win 9648
  30. 14:17:04.177823 IP 192.168.0.10.netbios-ns > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
  31. 14:17:04.185172 arp reply 192.168.0.10 is-at 00:0c:29:77:76:c7 (oui Unknown)
复制代码


LVS/DR方式能够正常工作
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
8楼 [报告]
发表于 2006-07-05 14:55 |只看该作者
那位大哥来指点一下啊
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
9楼 [报告]
发表于 2006-07-06 12:56 |只看该作者
大侠们伸个手
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
Linuxcn.com

论坛徽章:
0
10楼 [报告]
发表于 2006-07-07 10:33 |只看该作者
今天看了LVS的官网http://www.linuxvirtualserver.org中的LVS HOWTO FOR LVS/TUN中的一段:
  1. 7.1. You need a tunl0 device
  2. In LVS-Tun, the tunl0 device holds the VIP, just as the lo device holds the device for LVS-DR.

  4. You need to build the tunl0 device into the Linux kernel (in networking options - IP:tunneling) - it is turned off by default. tunl0 is a networking device like eth0, lo, dummy0. The tunnelling can be built as a module, in which case you'll have to insmod ipip before you can use it, or you can build it directly into the kernel.

  6. With a kernel built for tunneling, you should be able to see the unconfigured tunl0 device with ifconfig or with ip addr show (Feb 2004 - my ifconfig used to see the unconfigured tunl0, but it doesn't anymore.)
复制代码


按照它说的,我的IP:tunneling是配置成了模块方式的了,这样可以在ifconfig tunl0 vip network bcast前,要先通过命令insmod ipip来加载IP:tunneling这么模块,
可以,当我这样做的时候:
  1. root@Realserver1:/usr/src/linux# insmod ipip
  2. insmod: can't read 'ipip': No such file or directory
复制代码


这是为什么呢?

大家都不说话了吗?
实战分享：从技术角度谈机器学习入门| 【大话IT】RadonDB低门槛向MySQL集群下战书 | ChinaUnix打赏功能已上线！ | 新一代分布式关系型数据库RadonDB知多少？
12下一页
返回列表 发新帖
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号：11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们：huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP