centos 7 下的squid与openstack下的cdn实例
本帖最后由 cryboy2001 于 2014-09-29 16:33 编辑Contos 7 下最新squid设定
常用代理
透明代理
用户认证
强大的alc功能
Cdn介绍
Squid在cdn中的使用---反向代理
openstack下的cdn实例
bind实现view功能
环境
最小化安装contos7 ，关掉selinux，打开转发，防火墙就不关了，透明代理时要用到
两块网卡，ens6192.168.0.207与外网eth0：2.2.2.2
SELINUX=disabled
echo "1" > /proc/sys/net/ipv4/ip_forward
加装编译环境
yum -y install make gcc ccgcc+ gcc-c++
下载最新版squid
wget http://www.squid-cache.org/Versions/v3/3.4/squid-3.4.7.tar.gz
tar -zxvf squid-3.4.7.tar.gz
cd squid-3.4.7/
./configure --prefix=/usr/local/squid
make all
make install
ls /usr/local/squid/ #查看一下
加用户与权限
adduser squid -d /dev/null -s /sbin/nologin
chown -R squid.squid squid
编辑配置文件
vi /usr/local/squid/etc/squid.conf
visible_hostname squid.test.com #加主机名
pid_filename /var/run/squid.pid
cache_effective_user squid #用户与组
cache_effective_group squid
#下面是定义与应用acl，功能很强大
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/24 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
http_access deny all #最后一句拒绝其它所有
http_port 3128 #端口
cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256 #cache放的地方
coredump_dir /usr/local/squid/var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
检查配置文件一下对不对
./squid -k parse
./squid-z #初始化
运行
./squid
修改了配置后再运行
./squid -k reconfigure
加防火墙与以前的iptables不一样的
firewall-cmd --add-port=3128/tcp
客户端设置
成功的样子
access.log
透明代理
上面的代理要实现，要在客户端设置，为了方便不用在客户端设置，可以用透明代理
透明代理设置很简单，只在
把原来的
http_port 3128
改成 192.168.0.207是这台电脑的ip
http_port 192.168.0.207 3128transparent
关键是要改防火墙
firewall-cmd --zone=external --change-interface=eth0
firewall-cmd --zone=internal --change-interface=ens6
firewall-cmd --zone=internal --add-forward-port=port=80:proto=tcp:toaddr=192.168.0.207:toport=3128
firewall-cmd --zone=internal --list-all
客户端设置
不用设ie代理，只要把路由指向这台squid电脑就可以了，这可以通过dhcp设置。
cat var/logs/access.log
待续
用户认证
因为在上面没加认证模块所以要重新编译
cd squid-3.4.7/
./configure --enable-basic-auth-helpers="NCSA"
make
编译好了之后，不用现安装了，直接把要用的copy过去就可以了
cphelpers/basic_auth/NCSA/basic_ncsa_auth /usr/local/squid/bin/ncsa_auth
chown squid.squid ncsa_auth
修改配配
vi /usr/local/squid/etc/squid.conf
auth_param basic program /usr/local/squid/bin/ncsa_auth /usr/local/squid/etc/passwd
acl authuser proxy_auth REQUIRED
http_access allow authuser
#加上以上三句，注意一定要加在
http_access deny all
这一句前面
把透明代理的改回来
#http_port 192.168.0.207 3128transparent
http_port 3128
把前面
#acl localnet src 192.168.0.0/24
注消掉
完成的squid.conf如下
visible_hostname squid.test.com
pid_filename /var/run/squid.pid
cache_effective_user squid
cache_effective_group squid
http_port 3128
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
auth_param basic program /usr/local/squid/bin/ncsa_auth /usr/local/squid/etc/passwd
acl authuser proxy_auth REQUIRED
http_access allowauthuser
http_access deny all
cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256
coredump_dir /usr/local/squid/var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
用htpasswd生成密码文件
htpasswd -c /usr/local/squid/etc/passwd test1
htpasswd /usr/local/squid/etc/passwd test2
重新生效squid配置
./squid -k reconfigure
直接访问出现
要向开始一样改代理
改好后再访问出现
用户认证就成功了。
acl功能
在前面的两个应用中已用到了acl功能，acl应用分两步
1、定义acl 组如
acl all src 0.0.0.0/0.0.0.0 #定义一个所有的ip组合，般用在最后一句
以下是定义名为localnet的acl组，对这个组作同样的操作
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
下面是定义端口
acl SSL_ports port 443#定义安全端口，不作这代理。
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
2、应用acl组，用deny或者allow来做为动作
http_access deny !Safe_ports #加！号就是相反，拒绝不是以上定的Safe_ports组中的端口
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
http_access deny all #拒绝所有其它来源
常用的acl
1、不让192.168.0.22通过代理上网
acl test1 src 192.168.0.22
http_access deny test1
2、允许192.168.1.0/24通过
acl test2 src 192.168.1.0/24
http_access deny test2
3、不允许访问一个服务器
acl test3 dst 2.2.2.2
http_access deny test3
4、只有上班时间这台电脑才可以上网
acl pc1 src 192.168.0.22
acl worktime time 8:00-17:00
http_access allow pc1 worktime
5、限制一台电脑上网的连接数
acl pc1 src 192.168.0.22
acl maxconn maxconn 10
http_access denypc1 maxconn 本帖最后由 cryboy2001 于 2014-09-15 13:58 编辑
cdn架构
一句话：通过智能dns，根据用户来源，就近访问，代理缓存服务器，以达到快速的目的，同时可以增加源站的安全与稳定。
代理缓存服务器就是反向代理源站的内容。
组成部分：源站、缓存服务器、智能dns、用户
cdn的性能指标：并发量、吞吐量、响应时间、丢包率、命中率
前四个指标与其它服务器一样，命中率：客户端访问时，在cache中有，不用到源站抓起，这叫命中，否则就是不命中。
命中率高说明cdn系统的缓存策略越好，给源站分担的压力越多。
缓存的对象：静态、动态、流媒体等多种
下面就一般缓存方法：
客户端----------缓存服务器--------------源站
用Squid做为缓存服务器
#cat /usr/local/squid/etc/squid.conf
visible_hostname squid.test.com
pid_filename /var/run/squid.pid
cache_effective_user squid
cache_effective_group squid
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256
coredump_dir /usr/local/squid/var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
#主要就加了以下3句，
http_port 80 accel vhost vport #启用80端口，要不然squid是侦听3128端口的
cache_peer 2.2.2.2parent 80 0 #cache中没命中的话就到上一级或源站
http_access allow all #允许所有连接 要把之前的deny all 删除
使配置生效
#squid -k reconfigure
看看80端口谁在用
#lsof -i :80
是squidg正使用80端口
在客户端测试看看
查看日志
TCP_MISS 在cache中没找到，要连接到源站
TCP_MEM在cache中找到了，直接还回
当然以上只是最简单的测试，cdn的核心其实是智能dns，真正使用的cdn还包过各部分的集群，使用的最多的智能dns开源软件是bind view,有空做一个完整的cdn。
牛，学习了，先顶了再看 本帖最后由 cryboy2001 于 2014-09-29 16:37 编辑
用opentstack做cdn实验
架构
主机名与ip
一、源主机的安装
# #yum -y install httpd
产生一个测试页面
# cat /var/www/html/index.html
<html>
<body>
This is a test page!
</body>
</html>
#
打开防火墙
systemctl httpd.service start
systemctl start httpd.service
二、dns主机安装
安装bind
#yum -y install make gcc cc gcc+ gcc-c++ openssl openssl-devel perl
# wget ftp://ftp.isc.org/isc/bind9/9.10.1/bind-9.10.1.tar.gz
#tar zxvf bind-9.10.1.tar.gz
# cd bind-9.10.1
# ./configure --prefix=/usr/local/bind
#make && make install
制作配置文件
#/usr/local/bind/sbin/rndc-confgen >rndc.conf
#cat rndc.conf >rndc.key
#tail -n10 rndc.conf |head -n9 |sed -e s/#\ //g >named.conf
# cat etc/named.conf
key "rndc-key" {
algorithm hmac-md5;
secret "mHcJqq5mIIkvwZMwr5I5hg==";
};
controls {
inet 127.0.0.1 port 953
allow { 127.0.0.1; } keys { "rndc-key"; };
};
logging {
channel query_log {
file "query.log" versions 3 size 20m;
severity info;
print-time yes;
print-category yes;
};
category queries {
query_log;
};
};
options {
directory "/usr/local/bind/var";
pid-file "named.pid";
listen-on port 53 {any;};
allow-query {any;};
};
view "squid1" {
match-clients { 10.0.10.0/24; 10.0.1.0/24; };
recursion yes;
zone "." IN {
type hint;
file "named.ca";
};
zone "localhost" IN {
type master;
file "localhost.zone";
};
zone "0.0.127.in-addr.arpa" IN {
type master;
file "localhost.arpa";
};
zone "test.com" IN {
type master;
file "test1.com.zone";
allow-update {none;};
};
};
view "squid2" {
match-clients { 10.0.20.0/24;};
recursion yes;
zone "." IN {
type hint;
file "named.ca";
};
zone "localhost" IN {
type master;
file "localhost.zone";
};
zone "0.0.127.in-addr.arpa" IN {
type master;
file "localhost.arpa";
};
zone "test.com" IN {
type master;
file "test2.com.zone";
allow-update {none;};
};
};
view "other" {
match-clients { 0.0.0.0/0;};
recursion yes;
zone "." IN {
type hint;
file "named.ca";
};
zone "localhost" IN {
type master;
file "localhost.zone";
};
zone "0.0.127.in-addr.arpa" IN {
type master;
file "localhost.arpa";
};
zone "test.com" IN {
type master;
file "test.com.zone";
allow-update {none;};
};
};
相应目录下的zonefile
# ls var/test*
var/test1.com.zone var/test2.com.zone var/test.com.zone
# cat var/test*
$TTL 86400
@ IN SOA dns.test.com. hostmaster.test.com. (
20140930 ; serial
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
@ IN NS dns.test.com.
dns IN A 10.0.1.18
www IN A 10.0.10.102
$TTL 86400
@ IN SOA dns.test.com. hostmaster.test.com. (
20140930 ; serial
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
@ IN NS dns.test.com.
dns IN A 10.0.1.18
www IN A 10.0.20.101
$TTL 86400
@ IN SOA dns.test.com. hostmaster.test.com. (
20140930 ; serial
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
@ IN NS dns.test.com.
dns IN A 10.0.1.18
www IN A 10.0.1.19
squid1 IN A 10.0.10.102
squid2 IN A 10.0.20.101
设客户端dns
xp1上的dns查询
xp2上的查询
其它电脑上的查询
通过以上的查询，确认dns view已成功配好了。
三、代理缓存安装
看前面的squid设置内容
#cat squid.conf
visible_hostname squid2.test.com
pid_filename /var/run/squid.pid
cache_effective_user squid
cache_effective_group squid
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256
coredump_dir /usr/local/squid/var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
http_port 80 accel vhost vport
cache_peer 10.0.1.19 parent 80 0
http_access allow all
两台一样，除主机名外其它都是一样的
还有要打开80端口防火墙
四、客户机上试用
在客户机上的动作与日志
找一个不存在的网页
squid上的相关日志非常明显
加一台客户端，访问的是另一台squid
不存在的网页
日志很清楚
高手，，，收藏学习，，版主这个squid配置教程支持 rhel5.x 版本吗？ shell@ubuntu:~$ uname -a
Linux ubuntu 3.2.0-69-generic #103-Ubuntu SMP Tue Sep 2 05:02:14 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
shell@ubuntu:~$
我的应该支持吧 回复 7# cu_shell
@action08
自己编译的，什么系统都支持，配置都差不多的，只是其它操作稍有不同，比如启动程序，防火墙等
版主v587，求拜师。
页:
[1]
2