標籤: proxy

使用Squid 3.2 建立Fully Transparent Proxy（TPROXY4.1+）

環境中有使用NLTM驗證 + IIS7.5的應用服務並且會檢查Client端IP位址. 原來的transparent proxy無法通過檢查, 需改成TPROXYv4.1+ (Fully Transparent Proxy), 如此一來伺服器看到Client的IP才不會是Proxy Server的IP.

安裝過程發現Squid 3.2.0的ntlm支援有問題, IE會不段重複跳出要求使用者輸入帳號密碼, 查了論壇發現該版本有bug, 改成3.2.7就解決了.

 

系統：Centos 6.4

軟體：squid-3.2.7,bridge-utils,ebtables,libcap,libecap. Squid網路上都有包好的rpm檔,下載回來裝即可.若自行編譯核心或想自行編譯squid者請參考http://wiki.squid-cache.org/Features/Tproxy4. 其他package使用yum安裝.

網路架構：IPv4,主機為bridge模式接在gateway後

軟體設定

  1. 將主機設定為Bridge.記得先關閉NetworkManager以免設定衝突
    建立 /etc/sysconfig/network-scripts/ifcfg-br0 內容如下

    DEVICE=”br0″
    NM_CONTROLLED=”no”
    TYPE=”Bridge”
    BOOTPROTO=”none”
    IPADDR=”10.0.0.2″
    NETMASK=”255.255.255.0″
    BROADCAST=”10.0.0.255″
    NETWORK=”10.0.0.0″
    ONBOOT=”yes”
    GATEWAY=”10.0.0.1″
    DNS1=10.0.0.3
    DNS2=10.0.0.4
    DOMAIN=mydomain.com

    外部網卡 /etc/sysconfig/network-scripts/ifcfg-eth0 內容如下

    DEVICE=”eth0″
    NM_CONTROLLED=”no”
    ONBOOT=”yes”
    TYPE=”Ethernet”
    BRIDGE=”br0″

    內部網卡 /etc/sysconfig/network-scripts/ifcfg-eth1 內容如下

    DEVICE=”eth1″
    NM_CONTROLLED=”no”
    ONBOOT=”yes”
    TYPE=”Ethernet”
    BRIDGE=”br0″

    設定完成後重新開機或重起network服務.
  2. 確認 /etc/sysctl.conf的相關設定包含下面幾行

    net.ipv4.ip_forward = 1
    net.bridge.bridge-nf-call-ip6tables = 01
    net.bridge.bridge-nf-call-iptables = 01
    net.bridge.bridge-nf-call-arptables = 01
    net.ipv4.conf.default.rp_filter = 01
    net.ipv4.conf.all.rp_filter = 01
    net.ipv4.conf.eth0.rp_filter = 01
    net.ipv4.conf.eth1.rp_filter = 01

    執行以下指令使設定生效

    [root@centos6 etc]# sysctl –p

  3. 設定iptables及ebtables規則

    建立執行檔 /usr/local/bin/tproxy.init

    #!/bin/bash
    # IPv4-only
    # 在lo介面上建立代號為100的路由表格給iptables比對fwmark
    ip -f inet rule add fwmark 1 lookup 100
    ip -f inet route add local default dev lo table 100
    # 以下是不需要進入squid的例外規則, 例如本機有提供web服務
    iptables -t mangle -A PREROUTING -d 127.0.0.1 -p tcp –dport 80 -j ACCEPT
    #加入Divert規則將TCP80埠的流量重導到TCP3129
    iptables -t mangle -N DIVERT
    iptables -t mangle -A DIVERT -j MARK –set-mark 1
    iptables -t mangle -A DIVERT -j ACCEPT
    iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
    iptables -t mangle -A PREROUTING -p tcp –dport 80 -j TPROXY –tproxy-mark 0x1/0x1 –on-port 3129
    # 將橋接模式的封包移到路由模式
    ebtables -t broute -A BROUTING -i eth1 -p ipv4 –ip-proto tcp –ip-dport 80 -j redirect –redirect-target DROP
    ebtables -t broute -A BROUTING -i eth0 -p ipv4 –ip-proto tcp –ip-sport 80 -j redirect –redirect-target DROP

    賦予執行權限

    [root@centos6 bin]# chmod 755 tproxy.init

    將上面檔案加入開機程序 /etc/rc.d/rc.local

    #!/bin/sh
    #
    # This script will be executed *after* all the other init scripts.
    # You can put your own initialization stuff in here if you don’t
    # want to do the full Sys V style init stuff.
    touch /var/lock/subsys/local
    /usr/local/bin/tproxy.init

  4. 修改squid設定檔 /etc/squid/squid.conf .確認下面2行存在

    #限定IPv4, 避免LOG有錯誤訊息
    http_port 0.0.0.0:3128
    http_port 0.0.0.0:3129 tproxy

    重起squid服務.

  5. 雖然主機為橋接模式, 但為了讓Squid服務可以運作所以仍需給予伺服IP及相關路由, DNS以存取Web. 另外若使用selinux則另需以下設定：

    setsebool squid_connect_any=yes
    setsebool squid_use_tproxy=yes

  6. 檢查 /var/log/squid/access.log 若有client的瀏覽紀錄表示squid開始運作.

Netscreen + Squid (Transparent) + c-icap + ClamAV

架構圖：

squid

目的：
原始流往網際網路的WEB流量經過防火牆Netscreen-25重新導向內部的Squid proxy伺服器並提供掃毒需求.

方法：
透通式SQUID可以省去客戶端設定的麻煩, 但需將 TCP 80 流量重新導向. Netscreen 支援兩種導向方式, 介紹如下

  • NAT-DST(DNAT)
    NETSCREEN NAT-DST的限制如下(來源：ScreenOS參考指南)
    「注意 : 原始目的地IP位址及已轉譯的目的地IP位址必須在相同的安全區內」
    由於向外WEB流量在untrust (any), 因此SQUID伺服器也必須在DMZ或有MIP, 且必須允許untrust介面IP流向SQUID位址DMZ/MIP TCP:3128流量. 另外內部流量向untrust的TCP:80封包必須DNAT到SQUID伺服器MIP位址的TCP:3128. 該方式會造成統計SQUID流量時看到的都是防火牆作NAT介面的外部IP, 無法統計實際流量. 原本想把SQUID 內部IP加進untrust-vr, 然後DNAT到內部IP 也是不可行. 此方式可以在舊版ScreenOS(5.4之前)使用.
  • PBR(Policy Base Routing. Screenos 5.4.0以上支援)
    此方法可以直接將特定服務流量到向指定的IP但是不支援PAT, 也就是可以轉寄流量但是不能更改目的埠, 因此需要搭配在SQUID主機上執行IPTABLES以達成目的埠轉換..

考量流量分析精確性, Server安全性及架構, 這裡採用PBR以達到cache目的

  1. CentOS5主機安裝Squid 3.1
    上http://rpm.pbone.net搜尋下載squid3-3.1.8-1.el5.pp.i386.rpm
    [root@centos5]# rpm –Uvh squid3-3.1.8-1.el5.pp.i386.rpm
    修改 /etc/squid/squid.conf (只列出重點設定)

    #上層伺服器
    cache_peer proxy.hinet.net parent 80 0 no-query no-digest no-netdb-exchange

    #將連到youtube.com要求轉發給上層伺服器
    cache_peer_domain proxy.hinet.net .youtube.com

    #預設存取設定
    acl manager proto cache_object
    acl localhost src 127.0.0.1/32 ::1
    acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
    acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
    acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
    acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
    acl localnet src fc00::/7       # RFC 4193 local private network range
    acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

    # ACL for Youtube
    acl youtube dstdomain .youtube.com

    acl SSL_ports port 443
    acl Safe_ports port 80          # http
    acl Safe_ports port 21          # ftp
    acl Safe_ports port 443         # https
    acl Safe_ports port 70          # gopher
    acl Safe_ports port 210         # wais
    acl Safe_ports port 1025-65535  # unregistered ports
    acl Safe_ports port 280         # http-mgmt
    acl Safe_ports port 488         # gss-http
    acl Safe_ports port 591         # filemaker
    acl Safe_ports port 777         # multiling http
    acl CONNECT method CONNECT

    # 讓Youtube 使用上層, 其餘由squid本身直接存取
    never_direct allow youtube
    http_access allow manager localhost
    http_access deny manager
    http_access deny !Safe_ports
    http_access deny CONNECT !SSL_ports
    http_access allow localnet
    http_access allow localhost

    # ssl代理
    ssl_bump deny localhost
    ssl_bump allow all
    http_access deny all

    # 透通代理http
    http_port 3128 intercept

    # 透通代理https.基本上不建議在透通模式下啟用.必須置換憑證以解密ssl流量, 因此需要正式憑證, 若使用測試憑證client端會跳出此網站不安全的訊息
    https_port 3129 intercept ssl-bump cert=/etc/pki/tls/certs/testcert.pem

    hierarchy_stoplist cgi-bin ?

    # 將儲存區上限改為1gb.
    cache_dir ufs /var/cache/squid 1000 16 256
    coredump_dir /var/cache/squid
    refresh_pattern ^ftp:           1440    20%     10080
    refresh_pattern ^gopher:        1440    0%      1440
    refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
    refresh_pattern .               0       20%     4320

    #ICAP設定
    icap_enable on
    icap_preview_enable on
    icap_preview_size 128
    icap_send_client_ip on
    icap_service service_avi_req reqmod_precache 0 icap://localhost:1344/srv_clamav
    icap_service service_avi respmod_precache 1 icap://localhost:1344/srv_clamav
    adaptation_access service_avi_req allow all
    adaptation_access service_avi allow all

    #將錯誤顯示頁面設為中文
    error_default_language zh-tw

    設定開機即啟用服務

    [root@centos5]# chkconfig –add –level 2345 squid

  2. 設定iptables
    加入規則

    [root@centos5]# iptables -A PREROUTING -i eth0 -p tcp -m tcp -m multiport tcp -d 10.0.0.2 –dport 80,443 -j ACCEPT
    [root@centos5]# iptables -A PREROUTING -i eth0 -p tcp -m tcp ! -d localhost –dport 80 -j REDIRECT –to-port 3128
    [root@centos5]# iptables -A PREROUTING -i eth0 -p tcp -m tcp ! -d localhost –dport 443 -j REDIRECT –to-port 3129

    Squid伺服器本身有提供Web服務, 因此第一行需放行目的為本機IP的TCP 80,443封包, 第二行將連往網際網路TCP 80的封包重新導向Squid服務的TCP 3128. 第三行則將連往網際網路TCP 443的封包重新導向Squid服務的TCP 3129.
    啟動並儲存規則

    [root@centos5]# service iptables start
    [root@centos5]# iptables-save > /etc/sysconfig/iptables
    [root@centos5]# chkconfig –add –level 2345 iptables

  3. 設定NetScreen
    PBR流程
    policy3-1. 建立兩個acl存取名單. 第一組為來源IP是區域網路封包目標為網際網路網頁存取.第二組則為Squid伺服器本身的網際網路存取, 避免本身連向網路封包變成迴圈. CLI (指令模式)

    set access-list extended 1 src-ip 10.0.0.0/24 dst-port 80-80 protocol tcp entry 1
    set access-list extended 1 src-ip 10.0.0.0/24 dst-port 443-443 protocol tcp entry 2
    set access-list extended 2 src-ip 10.0.0.2/32 dst-port 80-80 protocol tcp entry 1
    set access-list extended 2 src-ip 10.0.0.2/32 dst-port 443-443 protocol tcp entry 2

    3-2. 第一組acl加入Http_Traffic群組(match-group), 第二組acl則加入Direct_Http_Traffic群組(match-group). CLI (指令模式)

    set match-group name Direct_Http_Traffic
    set match-group Direct_Http_Traffic ext-acl 2 match-entry 1
    set match-group name Http_Traffic
    set match-group Http_Traffic ext-acl 1 match-entry 1

    3-3. Tproxy的執行動作設定為當符合Http_Traffic群組的封包到達時直接送往Squid伺服器(10.0.0.2). Direct_Http則將符合Direct_Http_Traffic群組送往untrust介面(eth3). CLI (指令模式)

    set action-group name Tproxy
    set action-group Tproxy next-hop 10.0.0.2 action-entry 1
    set action-group name Direct_Http
    set action-group Direct_Http next-interface ethernet3 action-entry 1

    3-4. 建立政策(policy). 第一個先讓Squid本機連往網際網路, 若為其他客戶端則為順序2, 轉到Squid主機..

    set pbr policy name trust_policy
    set pbr policy trust_policy match-group Direct_Http_Traffic action-group Direct_Http 1
    set pbr policy trust_policy match-group Http_Traffic action-group Tproxy 2
    set pbr trust_policy
    exit

    3-5. 將政策連結到trust介面(eth1).CLI (指令模式)

    set interface ethernet1 pbr trust_policy

  4. 安裝Clamav
    #搭配MailScanner 因此安裝 Clamd

    [root@centos5]# yum install clamav clamav-db clamd

  5. 安裝c-icap
    上http://rpm.pbone.net搜尋下載c-icap , c-icap-modules的source rpm回來編譯

    [root@centos5]# rpmbuild –rebuild c-icap-0.1.1-1.el5.pp.src.rpm

    編譯完成後會有多個檔案主要安裝下面幾個
    c-icap-0.1.1-1.el5.pp.i386.rpm      # c-icap主程式
    c-icap-libs-0.1.1-1.el5.pp.i386.rpm   # library
    c-icap-progs-0.1.1-1.el5.pp.i386.rpm  # client程式
    c-icap-devel-0.1.1-1.el5.pp.i386.rpm  # 編譯c-icap-modules用
    下載的c-icap-modules並無開啟clamv, 需自行修改並編譯

    [root@centos5]# rpmbuild -i c-icap-modules-0.1.1-1.el5.pp.src.rpm
    [root@centos5]# cd /usr/src/redhat/SPECS

    更改c-icap-modules.spec檔案, 註釋掉」–without-clamav」

    CFLAGS=”${RPM_OPT_FLAGS} -fno-strict-aliasing” \
    –enable-shared                                \
    –with-bdb
    #       –without-clamav  # Still not in RedHat/CentOS

    重編

    [root@centos5]# rpmbuild -bb c-icap-modules-0.1.1-1.el5.pp.src.rpm
    [root@centos5]# cd ../RPMS/i386
    [root@centos5]# rpm –Uvh c-icap-modules-0.1.1-1.el5.rpm

    修改c-icap server設定 /etc/c-icap/c-icap.conf

    PidFile /var/run/c-icap/c-icap.pid
    CommandsSocket /var/run/c-icap/c-icap.ctl
    Timeout 300
    MaxKeepAliveRequests 100
    KeepAliveTimeout 600
    StartServers 3
    MaxServers 10
    MinSpareThreads     10
    MaxSpareThreads     20
    ThreadsPerChild     10
    MaxRequestsPerChild  0
    Port 1344
    ServerAdmin 你的管理員郵件
    ServerName 你的主機名稱
    TmpDir /var/tmp
    MaxMemObject 131072
    DebugLevel 1
    ModulesDir /usr/lib/c_icap
    ServicesDir /usr/lib/c_icap
    TemplateDir /usr/share/c_icap/templates/
    TemplateDefaultLanguage en
    LoadMagicFile /etc/c-icap/c-icap.magic
    RemoteProxyUsers off
    RemoteProxyUserHeader X-Authenticated-User
    RemoteProxyUserHeaderEncoded on
    #本機存取位址
    acl localhost src 127.0.0.1/255.255.255.255
    #所有位置
    acl all src 0.0.0.0/0.0.0.0
    #允許本機存取
    icap_access allow localhost
    #拒絕所有存取
    icap_access deny all
    #Log檔位置
    ServerLog /var/log/c-icap/server.log
    AccessLog /var/log/c-icap/access.log
    #以下三行設定病毒紀錄檔格式及位置
    acl VIRUS icap_resp_header{X-Infection-Found} /Type=.*/
    LogFormat viruslog “%tl, %>a %im %iu %is %{X-Infection-Found}<ih:”
    AccessLog /var/log/c-icap/viruses.log viruslog VIRUS
    Service echo srv_echo.so
    #開啟clamav模組掃瞄功能
    Include srv_clamav.conf

    修改c-icap-modules設定, 基本上維持預設值即可

    Service antivirus_module srv_clamav.so
    ServiceAlias  avscan srv_clamav?allow204=on&sizelimit=off&mode=simple
    srv_clamav.ScanFileTypes TEXT DATA EXECUTABLE ARCHIVE GIF JPEG MSOFFICE
    srv_clamav.SendPercentData 5
    srv_clamav.StartSendPercentDataAfter 2M
    srv_clamav.MaxObjectSize  5M
    srv_clamav.ClamAvTmpDir /var/tmp
    srv_clamav.ClamAvMaxFilesInArchive 0
    srv_clamav.ClamAvMaxFileSizeInArchive 100M
    srv_clamav.ClamAvMaxRecLevel 5

    此rpm所提供的init script以c-icap身份啟動, 經測試無法正常啟動.改以root啟動卻無問題, 因此修改/etc/sysconfig/c-icap

    #改掉下面這行, 將使用者改為root
    #RUN_AS=”c-icap”
    RUN_AS=”root”

    也可以到官網http://c-icap.sourceforge.net 下載最新0.1.2版自行安裝, 但原始package並未包含init script/syconfig/logrotate檔.
    設定開機啟動

    [root@centos5]# chkconfig –add –level 2345 c-icap

  6. 啟動相關服務

    [root@centos5]# service c-icap start

    [root@centos5]# service squid start

  7. 測試掃毒功能

    [root@centos5]# c-icap-client -f /path_to_clamav/test/clam.exe -s “srv_clamav?allow204=on&force=on&sizelimit=off&mode=simple”
    ICAP server:localhost, ip:127.0.0.1, port:1344
    <html>
    <head>
    <!–C-ICAP/0.1.2 srvClamAV module –>
    </head>
    <body>
    <H1>VIRUS FOUND</H1> You try to upload/download a file that contain the virus<br>
    ClamAV-Test-File
    <p>This message generated by C-ICAP/0.1.2 srvClamAV/antivirus module
    <!– And this is a silly HTML comment just to make this error bigger than 512 bytes
    is allowed to be displayed in IE. Yes, IE has a “feature” which does not allow
    error messages smaller than 512 bytes to be displayed, because they are not considered
    “friendly” enough.
    (Hmm…I think this stupid comment is better than embedding viruses or porn images in this error message like a bad guy suggested me!)–> </body>

    顯示掃到毒(clamav原始package內附的樣本)

  8. 結論：大致上需要的功能皆已完成, Squid部份可以進行校調以增加cache命中率.