SQUID Frequently Asked Questions (C) 2004 Team Squid, info@squid-cache.org Frequently Asked Questions (with answers!) about the Squid Internet Object Cache software. ______________________________________________________________________ Table of Contents 1. About Squid, this FAQ, and other Squid information resources 1.1 What is Squid? 1.2 What is Internet object caching? 1.3 Why is it called Squid? 1.4 What is the latest version of Squid? 1.5 Who is responsible for Squid? 1.6 Where can I get Squid? 1.7 What Operating Systems does Squid support? 1.8 Does Squid run on Windows? 1.9 What Squid mailing lists are available? 1.10 I can't figure out how to unsubscribe from your mailing list. 1.11 What other Squid-related documentation is available? 1.12 Does Squid support SSL/HTTPS/TLS? 1.13 What's the legal status of Squid? 1.14 Is Squid year-2000 compliant? 1.15 Can I pay someone for Squid support? 1.16 Squid FAQ contributors 1.17 About This Document 1.17.1 Want to contribute? Please write in SGML... 2. Getting and Compiling Squid 2.1 Which file do I download to get Squid? 2.2 How do I compile Squid? 2.3 What kind of compiler do I need? 2.4 What else do I need to compile Squid? 2.5 Do you have pre-compiled binaries available? 2.6 How do I apply a patch or a diff? 2.7 configure options 2.8 undefined reference to __inet_ntoa 2.9 How can I get true DNS TTL info into Squid's IP cache? 2.10 My platform is BSD/OS or BSDI and I can't compile Squid 2.11 Problems compiling libmiscutil.a on Solaris 2.12 I have problems compiling Squid on Platform Foo. 2.13 I see a lot warnings while compiling Squid. 2.14 Building Squid on OS/2 2.15 Building Squid on Cygwin 3. Installing and Running Squid 3.1 How big of a system do I need to run Squid? 3.2 How do I install Squid? 3.3 What does the squid.conf file do? 3.4 Do you have a squid.conf example? 3.5 How do I start Squid? 3.6 How do I start Squid automatically when the system boots? 3.6.1 From inittab 3.6.2 From rc.local 3.6.3 From init.d 3.7 How do I tell if Squid is running? 3.8 squid command line options 3.9 How do I see how Squid works? 3.10 Can Squid benefit from SMP systems? 3.11 Is it okay to use separate drives and RAID on Squid? 4. Configuration issues 4.1 How do I join a cache hierarchy? 4.2 How do I join NLANR's cache hierarchy? 4.3 Why should I want to join NLANR's cache hierarchy? 4.4 How do I register my cache with NLANR's registration service? 4.5 How do I find other caches close to me and arrange parent/child/sibling relationships with them? 4.6 My cache registration is not appearing in the Tracker database. 4.7 What is the httpd-accelerator mode? 4.8 How do I configure Squid to work behind a firewall? 4.9 How do I configure Squid forward all requests to another proxy? 4.10 I have dnsserver processes that aren't being used, should I lower the number in squid.conf? 4.11 My dnsserver average/median service time seems high, how can I reduce it? 4.12 How can I easily change the default HTTP port? 4.13 Is it possible to control how big each cache_dir is? 4.14 What cache_dir size should I use? 4.15 I'm adding a new cache_dir. Will I lose my cache? 4.16 Squid and http-gw from the TIS toolkit. 4.16.1 Firewall configuration: 4.16.2 Squid configuration: 4.17 What is ``HTTP_X_FORWARDED_FOR''? Why does squid provide it to WWW servers, and how can I stop it? 4.18 Can Squid anonymize HTTP requests? 4.18.1 Squid 2.2 4.19 Can I make Squid go direct for some sites? 4.20 Can I make Squid proxy only, without caching anything? 4.21 Can I prevent users from downloading large files? 5. Communication between browsers and Squid 5.1 Netscape manual configuration 5.2 Netscape automatic configuration 5.3 Lynx and Mosaic configuration 5.4 Redundant Proxy Auto-Configuration 5.5 Proxy Auto-Configuration with URL Hashing 5.6 Microsoft Internet Explorer configuration 5.7 Netmanage Internet Chameleon WebSurfer configuration 5.8 Opera 2.12 proxy configuration 5.9 How do I tell Squid to use a specific username for FTP urls? 5.10 Configuring Browsers for WPAD 5.11 Configuring Browsers for WPAD with DHCP 5.12 IE 5.0x crops trailing slashes from FTP URL's 5.13 IE 6.0 SP1 fails when using authentication 6. Squid Log Files 6.1 squid.out 6.2 cache.log 6.3 useragent.log 6.4 store.log 6.5 hierarchy.log 6.6 access.log 6.6.1 The common log file format 6.6.2 The native log file format 6.6.3 access.log native format in detail 6.7 Squid result codes 6.8 HTTP status codes 6.9 Request methods 6.10 Hierarchy Codes 6.11 cache/log (Squid-1.x) 6.12 swap.state (Squid-2.x) 6.13 Which log files can I delete safely? 6.14 How can I disable Squid's log files? 6.15 My log files get very big! 6.16 I want to use another tool to maintain the log files. 6.17 Managing log files 6.18 Why do I get ERR_NO_CLIENTS_BIG_OBJ messages so often? 6.19 What does ERR_LIFETIME_EXP mean? 6.20 Retrieving ``lost'' files from the cache 6.21 Can I use store.log to figure out if a response was cachable? 7. Operational issues 7.1 How do I see system level Squid statistics? 7.2 How can I find the biggest objects in my cache? 7.3 I want to restart Squid with a clean cache 7.4 How can I proxy/cache Real Audio? 7.5 How can I purge an object from my cache? 7.6 Using ICMP to Measure the Network 7.6.1 Supporting ICMP in your Squid cache 7.6.2 Utilizing your parents database 7.6.3 Inspecting the database 7.7 Why are so few requests logged as TCP_IMS_MISS? 7.8 How can I make Squid NOT cache some servers or URLs? 7.9 How can I delete and recreate a cache directory? 7.10 Why can't I run Squid as root? 7.11 Can you tell me a good way to upgrade Squid with minimal downtime? 7.12 Can Squid listen on more than one HTTP port? 7.13 Can I make origin servers see the client's IP address when going through Squid? 8. Memory 8.1 Why does Squid use so much memory!? 8.2 How can I tell how much memory my Squid process is using? 8.3 My Squid process grows without bounds. 8.4 I set cache_mem to XX, but the process grows beyond that! 8.5 How do I analyze memory usage from the cache manger output? 8.6 The ``Total memory accounted'' value is less than the size of my Squid process. 8.7 xmalloc: Unable to allocate 4096 bytes! 8.7.1 BSD/OS 8.7.2 FreeBSD (2.2.X) 8.7.3 OSF, Digital Unix 8.8 fork: (12) Cannot allocate memory 8.9 What can I do to reduce Squid's memory usage? 8.10 Using an alternate malloc library. 8.10.1 Using GNU malloc 8.10.2 dlmalloc 8.11 How much memory do I need in my Squid server? 9. The Cache Manager 9.1 What is the cache manager? 9.2 How do you set it up? 9.3 Cache manager configuration for CERN httpd 3.0 9.4 Cache manager configuration for Apache 9.5 Cache manager configuration for Roxen 2.0 and later 9.6 Cache manager ACLs in squid.conf 9.7 Why does it say I need a password and a URL? 9.8 I want to shutdown the cache remotely. What's the password? 9.9 How do I make the cache host default to my cache? 9.10 What's the difference between Squid TCP connections and Squid UDP connections? 9.11 It says the storage expiration will happen in 1970! 9.12 What do the Meta Data entries mean? 9.13 In the utilization section, what is Other? 9.14 In the utilization section, why is the Transfer KB/sec column always zero? 9.15 In the utilization section, what is the Object Count? 9.16 In the utilization section, what is the Max/Current/Min KB? 9.17 What is the I/O section about? 9.18 What is the Objects section for? 9.19 What is the VM Objects section for? 9.20 What does AVG RTT mean? 9.21 In the IP cache section, what's the difference between a hit, a negative hit and a miss? 9.22 What do the IP cache contents mean anyway? 9.23 What is the fqdncache and how is it different from the ipcache? 9.24 What does ``Page faults with physical i/o: 4897'' mean? 9.24.1 Ok, so what is unusually high? 9.25 What does the IGNORED field mean in the 'cache server list'? 10. Access Controls 10.1 Introduction 10.1.1 ACL elements 10.1.2 Access Lists 10.2 How do I allow my clients to use the cache? 10.3 how do I configure Squid not to cache a specific server? 10.4 How do I implement an ACL ban list? 10.5 How do I block specific users or groups from accessing my cache? 10.5.1 Ident 10.5.2 Proxy Authentication 10.6 Do you have a CGI program which lets users change their own proxy passwords? 10.7 Is there a way to do ident lookups only for a certain host and compare the result with a userlist in squid.conf? 10.8 Common Mistakes 10.8.1 And/Or logic 10.8.2 allow/deny mixups 10.8.3 Differences between src and srcdomain ACL types. 10.9 I set up my access controls, but they don't work! why? 10.10 Proxy-authentication and neighbor caches 10.11 Is there an easy way of banning all Destination addresses except one? 10.12 Does anyone have a ban list of porn sites and such? 10.13 Squid doesn't match my subdomains 10.14 Why does Squid deny some port numbers? 10.15 Does Squid support the use of a database such as mySQL for storing the ACL list? 10.16 How can I allow a single address to access a specific URL? 10.17 How can I allow some clients to use the cache at specific times? 10.18 How can I allow some users to use the cache at specific times? 10.19 Problems with IP ACL's that have complicated netmasks 10.20 Can I set up ACL's based on MAC address rather than IP? 10.21 Debugging ACLs 10.22 Can I limit the number of connections from a client? 10.23 I'm trying to deny foo.com, but it's not working. 10.24 I want to customize, or make my own error messages. 10.25 I want to use local time zone in error messages. 10.26 I want to put ACL parameters in an external file. 11. Troubleshooting 11.1 Why am I getting ``Proxy Access Denied?'' 11.2 I can't get local_domain to work; Squid is caching the objects from my local servers. 11.3 I get Connection Refused when the cache tries to retrieve an object located on a sibling, even though the sibling thinks it delivered the object to my cache. 11.4 Running out of filedescriptors 11.4.1 Linux 11.4.2 Solaris 11.4.3 FreeBSD 11.4.4 General BSD 11.4.5 Reconfigure afterwards 11.5 What are these strange lines about removing objects? 11.6 Can I change a Windows NT FTP server to list directories in Unix format? 11.7 Why am I getting ``Ignoring MISS from non-peer x.x.x.x?'' 11.8 DNS lookups for domain names with underscores (_) always fail. 11.9 Why does Squid say: ``Illegal character in hostname; underscores are not allowed?' 11.10 Why am I getting access denied from a sibling cache? 11.11 Cannot bind socket FD NN to *:8080 (125) Address already in use 11.12 icpDetectClientClose: ERROR xxx.xxx.xxx.xxx: (32) Broken pipe 11.13 icpDetectClientClose: FD 135, 255 unexpected bytes 11.14 Does Squid work with NTLM Authentication? 11.15 The default parent option isn't working! 11.16 ``Hot Mail'' complains about: Intrusion Logged. Access denied. 11.17 My Squid becomes very slow after it has been running for some time. 11.18 WARNING: Failed to start 'dnsserver' 11.19 Sending in Squid bug reports 11.19.1 crashes and core dumps 11.20 Debugging Squid 11.21 FATAL: ipcache_init: DNS name lookup tests failed 11.22 FATAL: Failed to make swap directory /var/spool/cache: (13) Permission denied 11.23 FATAL: Cannot open HTTP Port 11.24 FATAL: All redirectors have exited! 11.25 FATAL: file_map_allocate: Exceeded filemap limit 11.26 FATAL: You've run out of swap file numbers. 11.27 I am using up over 95% of the filemap bits?!! 11.28 FATAL: Cannot open /usr/local/squid/logs/access.log: (13) Permission denied 11.29 When using a username and password, I can not access some files. 11.30 pingerOpen: icmp_sock: (13) Permission denied 11.31 What is a forwarding loop? 11.32 accept failure: (71) Protocol error 11.33 storeSwapInFileOpened: ... Size mismatch 11.34 Why do I get fwdDispatch: Cannot retrieve 'https://www.buy.com/corp/ordertracking.asp' 11.35 Squid can't access URLs like http://3626046468/ab2/cybercards/moreinfo.html 11.36 I get a lot of ``URI has whitespace'' error messages in my cache log, what should I do? 11.37 commBind: Cannot bind socket FD 5 to 127.0.0.1:0: (49) Can't assign requested address 11.38 Unknown cache_dir type '/var/squid/cache' 11.39 unrecognized: 'cache_dns_program /usr/local/squid/bin/dnsserver' 11.40 Is dns_defnames broken in Squid-2.3 and later 11.41 What does sslReadClient: FD 14: read failure: (104) Connection reset by peer mean? 11.42 What does Connection refused mean? 11.43 squid: ERROR: no running copy 11.44 FATAL: getgrnam failed to find groupid for effective group 'nogroup' 11.45 ``Unsupported Request Method and Protocol'' for https URLs. 11.46 Squid uses 100% CPU 11.47 Webmin's cachemgr.cgi crashes the operating system 11.48 Segment Violation at startup or upon first request 11.49 urlParse: Illegal character in hostname 'proxy.mydomain.com:8080proxy.mydomain.com' 11.50 Requests for international domain names does not work 11.51 Why do I sometimes get ``Zero Sized Reply''? 11.52 Why do I get "The request or reply is too large" errors? 11.53 Negative or very large numbers in Store Directory Statistics, or constant complaints about cache above limit 11.54 Squid problems with WindowsUpdate v5 12. How does Squid work? 12.1 What are cachable objects? 12.2 What is the ICP protocol? 12.3 What is the dnsserver? 12.4 What is the ftpget program for? 12.5 FTP PUT's don't work! 12.6 What is a cache hierarchy? What are parents and siblings? 12.7 What is the Squid cache resolution algorithm? 12.8 What features are Squid developers currently working on? 12.9 Tell me more about Internet traffic workloads 12.10 What are the tradeoffs of caching with the NLANR cache system? 12.11 Where can I find out more about firewalls? 12.12 What is the ``Storage LRU Expiration Age?'' 12.13 What is ``Failure Ratio at 1.01; Going into hit-only-mode for 5 minutes''? 12.14 Does squid periodically re-read its configuration file? 12.15 How does unlinkd work? 12.16 What is an icon URL? 12.17 Can I make my regular FTP clients use a Squid cache? 12.18 Why is the select loop average time so high? 12.19 How does Squid deal with Cookies? 12.20 How does Squid decide when to refresh a cached object? 12.20.1 Squid-1.1 and Squid-1.NOVM algorithm 12.20.2 Squid-2 algorithm 12.21 What exactly is a deferred read? 12.22 Why is my cache's inbound traffic equal to the outbound traffic? 12.23 How come some objects do not get cached? 12.24 What does keep-alive ratio mean? 12.25 How does Squid's cache replacement algorithm work? 12.25.1 Squid 1.1 12.25.2 Squid 2 12.26 What are private and public keys? 12.27 What is FORW_VIA_DB for? 12.28 Does Squid send packets to port 7 (echo)? If so, why? 12.29 What does ``WARNING: Reply from unknown nameserver [a.b.c.d]'' mean? 12.30 How does Squid distribute cache files among the available directories? 12.31 Why do I see negative byte hit ratio? 12.32 What does ``Disabling use of private keys'' mean? 12.33 What is a half-closed filedescriptor? 12.34 What does --enable-heap-replacement do? 12.35 Why is actual filesystem space used greater than what Squid thinks? 12.36 How do positive_dns_ttl and negative_dns_ttl work? 12.37 What does swapin MD5 mismatch mean? 12.38 What does failed to unpack swapfile meta data mean? 12.39 Why doesn't Squid make ident lookups in interception mode? 12.40 dnsSubmit: queue overload, rejecting blah 12.41 What are FTP passive connections? 13. Multicast 13.1 What is Multicast? 13.2 How do I know if my network has multicast? 13.3 Should I be using Multicast ICP? 13.4 How do I configure Squid to send Multicast ICP queries? 13.5 How do I know what Multicast TTL to use? 13.6 How do I configure Squid to receive and respond to Multicast ICP? 14. System-Dependent Weirdnesses 14.1 Solaris 14.1.1 TCP incompatibility? 14.1.2 select() 14.1.3 malloc 14.1.4 DNS lookups and nscd 14.1.5 DNS lookups and /etc/nsswitch.conf 14.1.6 DNS lookups and NIS 14.1.7 Tuning 14.1.8 disk write error: (28) No space left on device 14.1.9 Solaris X86 and IPFilter 14.1.10 Changing the directory lookup cache size 14.1.11 The priority_paging algorithm 14.1.12 assertion failed: StatHist.c:91: `statHistBin(H, max) == H->capacity - 1' 14.2 FreeBSD 14.2.1 T/TCP bugs 14.2.2 mbuf size 14.2.3 Dealing with NIS 14.2.4 FreeBSD 3.3: The lo0 (loop-back) device is not configured on startup 14.2.5 FreeBSD 3.x or newer: Speed up disk writes using Softupdates 14.2.6 Internal DNS problems with jail environment 14.2.7 "Zero Sized Reply" error due to TCP blackholing 14.3 OSF1/3.2 14.4 BSD/OS 14.4.1 gcc/yacc 14.4.2 process priority 14.5 Linux 14.5.1 Cannot bind socket FD 5 to 127.0.0.1:0: (49) Can't assign requested address 14.5.2 FATAL: Don't run Squid as root, set 'cache_effective_user'! 14.5.3 Large ACL lists make Squid slow 14.5.4 gethostbyname() leaks memory in RedHat 6.0 with glibc 2.1.1. 14.5.5 assertion failed: StatHist.c:91: `statHistBin(H, max) == H->capacity - 1' on Alpha system. 14.5.6 tools.c:605: storage size of `rl' isn't known 14.5.7 Can't connect to some sites through Squid 14.6 HP-UX 14.6.1 StatHist.c:74: failed assertion `statHistBin(H, min) == 0' 14.7 IRIX 14.7.1 dnsserver always returns 255.255.255.255 14.8 SCO-UNIX 14.9 AIX 14.9.1 "shmat failed" errors with diskd 14.9.2 Core dumps when squid process grows to 256MB 15. Redirectors 15.1 What is a redirector? 15.2 Why use a redirector? 15.3 How does it work? 15.4 Do you have any examples? 15.5 Can I use the redirector to return HTTP redirect messages? 15.6 FATAL: All redirectors have exited! 15.7 Redirector interface is broken re IDENT values 16. Cache Digests 16.1 What is a Cache Digest? 16.2 How and why are they used? 16.3 What is the theory behind Cache Digests? 16.3.1 Adding a Key 16.3.2 Querying a Key 16.3.3 Deleting a Key 16.4 How is the size of the Cache Digest in Squid determined? 16.5 What hash functions (and how many of them) does Squid use? 16.6 How are objects added to the Cache Digest in Squid? 16.7 Does Squid support deletions in Cache Digests? What are diffs/deltas? 16.8 When and how often is the local digest built? 16.9 How are Cache Digests transferred between peers? 16.10 How and where are Cache Digests stored? 16.10.1 Cache Digest built locally 16.10.2 Cache Digest fetched from peer 16.11 How are the Cache Digest statistics in the Cache Manager to be interpreted? 16.12 What are False Hits and how should they be handled? 16.13 How can Cache Digest related activity be traced/debugged? 16.13.1 Enabling Cache Digests 16.13.2 What do the access.log entries look like? 16.13.3 What does a False Hit look like? 16.13.4 How is the cause of a False Hit determined? 16.13.5 Use The Source 16.14 What about ICP? 16.15 Is there a Cache Digest Specification? 16.16 Would it be possible to stagger the timings when cache_digests are retrieved from peers? 17. Interception Caching/Proxying 17.1 Interception caching for Solaris, SunOS, and BSD systems 17.1.1 Install IP Filter 17.1.2 Configure ipnat 17.1.3 Configure Squid 17.1.3.1 Squid-2 17.1.3.2 Squid-1.1 17.2 Interception caching with Linux 2.0 and ipfwadm 17.3 Interception caching with Linux 2.2 and ipchains 17.4 Interception caching with Linux 2.4 and netfilter 17.5 Interception caching with Cisco routers 17.5.1 possible bugs 17.6 Interception caching with LINUX 2.0.29 and CISCO IOS 11.1 17.7 Interception caching with FreeBSD 17.8 Interception caching with ACC Tigris digital access server 17.9 Interception caching with Foundry L4 switches 17.10 Interception caching with Cabletron/Entrasys products 17.11 The cache is trying to connect to itself... 17.12 ``Connection reset by peer'' and Cisco policy routing 17.13 WCCP - Web Cache Coordination Protocol 17.13.1 Does Squid support WCCP? 17.13.2 Configuring your Router 17.13.2.1 IOS Version 11.x 17.13.2.2 IOS Version 12.x 17.13.3 IOS 12.x problems 17.13.4 Configuring FreeBSD 17.13.4.1 FreeBSD-3.x 17.13.4.2 FreeBSD-4.0 through 4.7 17.13.4.3 FreeBSD-4.8 and later 17.13.5 Configuring Linux 2.2 17.13.5.1 Standard Linux GRE Tunnel 17.13.5.2 Joe Cooper's Patch 17.13.5.3 WCCP Specific Module 17.13.5.4 Common Steps 17.13.6 Configuring Others 17.14 Can someone tell me what version of cisco IOS WCCP is added in? 17.15 What about WCCPv2? 17.16 Can I use proxy_auth with interception? 17.17 Interception on Linux with Squid and the Browser on the same box 17.18 Interception caching with an Alcatel OmnySwitch 7700 18. SNMP 18.1 Does Squid support SNMP? 18.2 Enabling SNMP in Squid 18.3 Configuring Squid 2.2 18.4 Configuring Squid 2.1 18.5 How can I query the Squid SNMP Agent 18.6 What can I use SNMP and Squid for? 18.7 How can I use SNMP with Squid? 18.8 Where can I get more information/discussion about Squid and SNMP? 18.9 Monitoring Squid with MRTG 19. Squid version 2 19.1 What are the new features? 19.2 How do I configure 'ssl_proxy' now? 19.3 Empty placeholder This entry has been deleted. 19.4 Adding a new cache disk 19.5 Empty placeholder This entry has been deleted. 19.6 How do I configure proxy authentication? 19.7 Why does proxy-auth reject all users after upgrading from Squid-2.1 or earlier? 19.8 Delay Pools 19.8.1 How can I limit Squid's total bandwidth to, say, 512 Kbps? 19.8.2 How to limit a single connection to 128 Kbps? 19.8.3 How do you personally use delay pools? 19.8.4 Where else can I find out about delay pools? 19.9 Can I preserve my cache when upgrading from 1.1 to 2? 19.10 Customizable Error Messages 19.11 My squid.conf from version 1.1 doesn't work! 20. httpd-accelerator mode 20.1 What is the httpd-accelerator mode? 20.2 How do I set it up? 20.3 When using an httpd-accelerator, the port number for redirects is wrong 21. Related Software 21.1 Clients 21.1.1 Wget 21.1.2 echoping 21.2 Logfile Analysis 21.3 Configuration Tools 21.3.1 3Dhierarchy.pl 21.4 Squid add-ons 21.4.1 transproxy 21.4.2 Iain's redirector package 21.4.3 Junkbusters 21.4.4 Squirm 21.4.5 chpasswd.cgi 21.4.6 jesred 21.4.7 squidGuard 21.4.8 Central Squid Server 21.4.9 Cerberian content filter (subscription service) 21.5 Ident Servers 22. DISKD 22.1 What is DISKD? 22.2 Does it perform better? 22.3 How do I use it? 22.4 FATAL: Unknown cache_dir type 'diskd' 22.5 If I use DISKD, do I have to wipe out my current cache? 22.6 How do I configure message queues? 22.6.1 FreeBSD 22.6.2 OpenBSD 22.6.3 Digital Unix 22.6.4 Linux 22.6.5 Solaris 22.7 How do I configure shared memory? 22.7.1 FreeBSD 22.7.2 OpenBSD 22.7.3 Digital Unix 22.7.4 Linux 22.7.5 Solaris 22.8 Sometimes shared memory and message queues aren't released when Squid exits. 22.9 What are the Q1 and Q2 parameters? 23. Authentication 23.1 How does Proxy Authentication work in Squid? 23.2 How do I use authentication in access controls? 23.3 Does Squid cache authentication lookups? 23.4 Are passwords stored in clear text or encrypted? 23.5 How do I use the Winbind authenticators? 23.5.1 Supported Samba Releases 23.5.2 Configure Samba 23.5.3 Configure Squid 23.6 Authentication in accelerator mode 24. Terms and Definitions 24.1 Neighbor 24.2 Regular Expression 25. Security Concerns 25.1 Open-access proxies 25.2 Mail relaying ______________________________________________________________________ You can download the FAQ as HTML , PDF , compressed Postscript , plain text , linuxdoc SGML source or as a compressed tar of HTML . 1. About Squid, this FAQ, and other Squid information resources 1.1. What is Squid? Squid is a high-performance proxy caching server for web clients, supporting FTP, gopher, and HTTP data objects. Unlike traditional caching software, Squid handles all requests in a single, non- blocking, I/O-driven process. Squid keeps meta data and especially hot objects cached in RAM, caches DNS lookups, supports non-blocking DNS lookups, and implements negative caching of failed requests. Squid supports SSL, extensive access controls, and full request logging. By using the lightweight Internet Cache Protocol, Squid caches can be arranged in a hierarchy or mesh for additional bandwidth savings. Squid consists of a main server program squid, a Domain Name System lookup program dnsserver, some optional programs for rewriting requests and performing authentication, and some management and client tools. When squid starts up, it spawns a configurable number of dnsserver processes, each of which can perform a single, blocking Domain Name System (DNS) lookup. This reduces the amount of time the cache waits for DNS lookups. Squid is derived from the ARPA-funded Harvest project . 1.2. What is Internet object caching? Internet object caching is a way to store requested Internet objects (i.e., data available via the HTTP, FTP, and gopher protocols) on a system closer to the requesting site than to the source. Web browsers can then use the local Squid cache as a proxy HTTP server, reducing access time as well as bandwidth consumption. 1.3. Why is it called Squid? Harris' Lament says, ``All the good ones are taken." We needed to distinguish this new version from the Harvest cache software. Squid was the code name for initial development, and it stuck. 1.4. What is the latest version of Squid? Squid is updated often; please see the Squid home page for the most recent versions. 1.5. Who is responsible for Squid? Squid is the result of efforts by numerous individuals from the Internet community. Duane Wessels of the National Laboratory for Applied Network Research (funded by the National Science Foundation) leads code development. Please see the CONTRIBUTORS file for a list of our excellent contributors. 1.6. Where can I get Squid? You can download Squid via FTP from the primary FTP site or one of the many worldwide mirror sites . Many sushi bars also have Squid. 1.7. What Operating Systems does Squid support? The software is designed to operate on any modern Unix system, and is known to work on at least the following platforms: o Linux o FreeBSD o NetBSD o OpenBSD o BSDI o Mac OS/X o OSF/Digital Unix/Tru64 o IRIX o SunOS/Solaris o NeXTStep o SCO Unix o AIX o HP-UX o ``OS/2'' o ``Cygwin'' For more specific information, please see platforms.html . If you encounter any platform-specific problems, please let us know by registering a entry in our bug database . 1.8. Does Squid run on Windows? Starting from 2.5, official Squid sources will compile and run fine on Windows using the Cygwin development environment. A more complete native Windows port is provided by Acme Consulting . Guido Serassio maintains the native NT port of Squid and is actively working on having the needed changes integrated into the standard Squid distribution. Partially based on earlier NT port by Romeo Anghelache. The Cygwin ports reportedly also appears to run on Window 9x/ME but it is not recommended for as a server operating system. 1.9. What Squid mailing lists are available? o squid-users@squid-cache.org: general discussions about the Squid cache software. Subscribe via squid-users-subscribe@squid- cache.org. Previous messages are available for browsing at the Squid Users Archive , and also at theaimsgroup.com . o squid-users-digest: digested (daily) version of above. Subscribe via squid-users-digest-subscribe@squid-cache.org. o squid-announce@squid-cache.org: A receive-only list for announcements of new versions. Subscribe via squid-announce- subscribe@squid-cache.org. o squid-bugs@squid-cache.org: A closed list for sending us bug reports. Bug reports received here are given priority over those mentioned on squid-users. o squid@squid-cache.org: A closed list for sending us feed-back and ideas. o squid-faq@squid-cache.org: A closed list for sending us feed-back, updates, and additions to the Squid FAQ. We also have a few other mailing lists which are not strictly Squid- related. o cache-snmp@ircache.net: A public list for discussion of Web Caching and SNMP issues and developments. Eventually we hope to put forth a standard Web Caching MIB. o icp-wg@ircache.net: Mostly-idle mailing list for the nonexistent ICP Working Group within the IETF. It may be resurrected some day, you never know! 1.10. I can't figure out how to unsubscribe from your mailing list. All of our mailing lists have ``-subscribe'' and ``-unsubscribe'' addresses that you must use for subscribe and unsubscribe requests. To unsubscribe from the squid-users list, you send a message to squid- users-unsubscribe@squid-cache.org. 1.11. What other Squid-related documentation is available? o The Squid home page for information on the Squid software o Squid: The Definitive Guide to be published by O'Reilly and Associates January 2004. o The IRCache Mesh gives information on our operational mesh of caches. o The Squid FAQ (uh, you're reading it). o Oskar's Squid Users Guide . o Visolve's Configuration Guide . o Squid documentation in German , Turkish , Italian , Brazilian Portugese , and another in Brazilian Portugese . o Squid Programmers Guide . Yeah, its extremely incomplete. I assure you this is the most recent version. o Web Caching Resources o Squid-1.0 Release Notes o Squid-1.1 Release Notes o Tutorial on Configuring Hierarchical Squid Caches o RFC 2186 ICPv2 -- Protocol o RFC 2187 ICPv2 -- Application o RFC 1016 1.12. Does Squid support SSL/HTTPS/TLS? As of version 2.5, Squid can terminate SSL connections. This is perhaps only useful in a surrogate (http accelerator) configuration. You must run configure with --enable-ssl. See https_port in squid.conf for more information. Squid also supports these encrypted protocols by ``tunelling'' traffic between clients and servers. In this case, Squid can relay the encrypted bits between a client and a server. Normally, when your browser comes across an https URL, it does one of two things: 1. The browser opens an SSL connection directly to the origin server. 2. The browser tunnels the request through Squid with the CONNECT request method. The CONNECT method is a way to tunnel any kind of connection through an HTTP proxy. The proxy doesn't understand or interpret the contents. It just passes bytes back and forth between the client and server. For the gory details on tunnelling and the CONNECT method, please see RFC 2817 and Tunneling TCP based protocols through Web proxy servers (expired). 1.13. What's the legal status of Squid? Squid is copyrighted by the University of California San Diego. Squid uses some code developed by others . Squid is Free Software . Squid is licensed under the terms of the GNU General Public License . 1.14. Is Squid year-2000 compliant? We think so. Squid uses the Unix time format for all internal time representations. Potential problem areas are in printing and parsing other time representations. We have made the following fixes in to address the year 2000: o cache.log timestamps use 4-digit years instead of just 2 digits. o parse_rfc1123() assumes years less than "70" are after 2000. o parse_iso3307_time() checks all four year digits. Year-2000 fixes were applied to the following Squid versions: o squid-2.1 : Year parsing bug fixed for dates in the "Wed Jun 9 01:29:59 1993 GMT" format (Richard Kettlewell). o squid-1.1.22: Fixed likely year-2000 bug in ftpget's timestamp parsing (Henrik Nordstrom). o squid-1.1.20: Misc fixes (Arjan de Vet). Patches: o Richard's lib/rfc1123.c patch <../Y2K/patch3>. If you are still running 1.1.X, then you should apply this patch to your source and recompile. o Henrik's src/ftpget.c patch <../Y2K/patch2>. o Arjan's lib/rfc1123.c patch <../Y2K/patch1>. Squid-2.2 and earlier versions have a New Year bug . This is not strictly a Year-2000 bug; it would happen on the first day of any year. 1.15. Can I pay someone for Squid support? Yep. Please see the commercial support page . 1.16. Squid FAQ contributors The following people have made contributions to this document: o Dodjie Nava o Jonathan Larmour o Cord Beermann o Tony Sterrett o Gerard Hynes o Katayama, Takeo o Duane Wessels o K Claffy o Paul Southworth o Oskar Pearson o Ong Beng Hui o Torsten Sturm o James R Grinter o Rodney van den Oever o Kolics Bertold o Carson Gaspar o Michael O'Reilly o Hume Smith o Richard Ayres o John Saunders o Miquel van Smoorenburg o David J N Begley o Kevin Sartorelli o Andreas Doering o Mark Visser o tom minchin o Jens-S. Voeckler o Andre Albsmeier o Doug Nazar o Henrik Nordstrom o Mark Reynolds o Arjan de Vet o Peter Wemm o John Line o Jason Armistead o Chris Tilbury o Jeff Madison o Mike Batchelor o Bill Bogstad o Radu Greab o F.J. Bosscha o Brian Feeny o Martin Lyons o David Luyer o Chris Foote o Jens Elkner o Simon White o Jerry Murdock o Gerard Eviston o Rob Poe Please send corrections, updates, and comments to: squid-faq@squid- cache.org . 1.17. About This Document This document is copyrighted (2000) by Duane Wessels. This document was written in SGML and converted with the SGML-Tools package . Most current version of this document can always be found at http://www.squid-cache.org/Doc/FAQ/ in HTML, Plain Text, Postscript and SGML formats. 1.17.1. Want to contribute? Please write in SGML... It is easier for us if you send us text which is close to "correct" SGML. The SQUID FAQ currently uses the LINUXDOC DTD. Its probably easiest to follow examples in the this file. Here are the basics: Use the tag for links, instead of HTML Use for emphasis, config options, and pathnames: usr/local/squid/etc/squid.conf foo bar Use , just like HTML's 
 to show unformatted text.



  2.  Getting and Compiling Squid

  2.1.  Which file do I download to get Squid?

  You must download a source archive file of the form squid-x.y.z-
  src.tar.gz (eg, squid-1.1.6-src.tar.gz) from the Squid home page
  , or.  the Squid FTP site
  .  Context diffs are available for
  upgrading to new versions.  These can be applied with the patch
  program (available from the GNU FTP site
  ).


  2.2.  How do I compile Squid?

  For Squid-1.0 and Squid-1.1 versions, you can just type make from the
  top-level directory after unpacking the source files.  For example:

          % tar xzf squid-1.1.21-src.tar.gz
          % cd squid-1.1.21
          % make



  For Squid-2 you must run the configure script yourself before running
  make:

          % tar xzf squid-2.0.RELEASE-src.tar.gz
          % cd squid-2.0.RELEASE
          % ./configure
          % make



  2.3.  What kind of compiler do I need?

  To compile Squid, you will need an ANSI C compiler.  Almost all modern
  Unix systems come with pre-installed compilers which work just fine.
  The old SunOS compilers do not have support for ANSI C, and the Sun
  compiler for Solaris is a product which must be purchased separately.


  If you are uncertain about your system's C compiler, The GNU C
  compiler is available at the GNU FTP site .
  In addition to gcc, you may also want or need to install the binutils
  package.


  2.4.  What else do I need to compile Squid?

  You will need Perl  installed on your system.


  2.5.  Do you have pre-compiled binaries available?

  The developers do not have the resources to make pre-compiled binaries
  available.  Instead, we invest effort into making the source code very
  portable.  Some people have made binary packages available.  Please
  see our Platforms Page .


  The SGI Freeware  site has pre-compiled
  packages for SGI IRIX.



  Squid binaries for FreeBSD on Alpha and Intel
  .


  Squid binaries for NetBSD on everything
  



  Gurkan Sengun has some Sparc/Solaris packages
   available.



  Squid binaries for Windows
  .


  2.6.  How do I apply a patch or a diff?

  You need the patch program.  You should probably duplicate the entire
  directory structure before applying the patch.  For example, if you
  are upgrading from squid-1.1.10 to 1.1.11, you would run these
  commands:

          cd squid-2.5.STABLE3
          mkdir ../squid-2.5.STABLE4
          find . -depth -print | cpio -pdv ../squid-1.1.11
          cd ../squid-1.1.11
          patch -p1 < /tmp/squid-2.5.STABLE3-STABLE4.diff

  or alternatively

          cp -rl squid-2.5.STABLE3 squid-2.5.STABLE4
          cd squid-2.5.STABLE4
          zcat /tmp/squid-2.5.STABLE3-STABLE4.diff.gz | patch -p1


  After the patch has been applied, you must rebuild Squid from the very
  beginning, i.e.:

          make distclean
          ./configure ...
          make
          make install



  If your patch program seems to complain or refuses to work, you should
  get a more recent version, from the GNU FTP site
  , for example.


  2.7.  configure  options

  The configure script can take numerous options.  The most useful is
  --prefix to install it in a different directory.  The default
  installation directory is /usr/local/squid/.  To change the default,
  you could do:

          % cd squid-x.y.z
          % ./configure --prefix=/some/other/directory/squid



  Type

          % ./configure --help


  to see all available options.  You will need to specify some of these
  options to enable or disable certain features.  Some options which are
  used often include:
    --prefix=PREFIX         install architecture-independent files in PREFIX
                            [/usr/local/squid]
    --enable-dlmalloc[=LIB] Compile & use the malloc package by Doug Lea
    --enable-gnuregex       Compile GNUregex
    --enable-splaytree      Use SPLAY trees to store ACL lists
    --enable-xmalloc-debug  Do some simple malloc debugging
    --enable-xmalloc-debug-trace
                            Detailed trace of memory allocations
    --enable-xmalloc-statistics
                            Show malloc statistics in status page
    --enable-carp           Enable CARP support
    --enable-async-io       Do ASYNC disk I/O using threads
    --enable-icmp           Enable ICMP pinging
    --enable-delay-pools    Enable delay pools to limit bandwith usage
    --enable-mem-gen-trace  Do trace of memory stuff
    --enable-useragent-log  Enable logging of User-Agent header
    --enable-kill-parent-hack
                            Kill parent on shutdown
    --enable-snmp           Enable SNMP monitoring
    --enable-cachemgr-hostname[=hostname]
                            Make cachemgr.cgi default to this host
    --enable-arp-acl        Enable use of ARP ACL lists (ether address)
    --enable-htpc           Enable HTCP protocol
    --enable-forw-via-db    Enable Forw/Via database
    --enable-cache-digests  Use Cache Digests
                            see http://www.squid-cache.org/Doc/FAQ/FAQ-16.html
    --enable-err-language=lang
                            Select language for Error pages (see errors dir)



  2.8.  undefined reference to __inet_ntoa

  by Kevin Sartorelli  and Andreas Doering
  .


  Probably you've recently installed bind 8.x.  There is a mismatch
  between the header files and DNS library that Squid has found.  There
  are a couple of things you can try.


  First, try adding -lbind to XTRA_LIBS  in src/Makefile.  If -lresolv
  is already there, remove it.


  If that doesn't seem to work, edit your arpa/inet.h file and comment
  out the following:


          #define inet_addr               __inet_addr
          #define inet_aton               __inet_aton
          #define inet_lnaof              __inet_lnaof
          #define inet_makeaddr           __inet_makeaddr
          #define inet_neta               __inet_neta
          #define inet_netof              __inet_netof
          #define inet_network            __inet_network
          #define inet_net_ntop           __inet_net_ntop
          #define inet_net_pton           __inet_net_pton
          #define inet_ntoa               __inet_ntoa
          #define inet_pton               __inet_pton
          #define inet_ntop               __inet_ntop
          #define inet_nsap_addr          __inet_nsap_addr
          #define inet_nsap_ntoa          __inet_nsap_ntoa

  2.9.  How can I get true DNS TTL info into Squid's IP cache?

  If you have source for BIND, you can modify it as indicated in the
  diff below.  It causes the global variable _dns_ttl_ to be set with
  the TTL of the most recent lookup.  Then, when you compile Squid, the
  configure script will look for the _dns_ttl_ symbol in libresolv.a.
  If found, dnsserver will return the TTL value for every lookup.

  This hack was contributed by Endre Balint Nagy
  .


  diff -ru bind-4.9.4-orig/res/gethnamaddr.c bind-4.9.4/res/gethnamaddr.c
  --- bind-4.9.4-orig/res/gethnamaddr.c   Mon Aug  5 02:31:35 1996
  +++ bind-4.9.4/res/gethnamaddr.c        Tue Aug 27 15:33:11 1996
  @@ -133,6 +133,7 @@
   } align;

   extern int h_errno;
  +int _dns_ttl_;

   #ifdef DEBUG
   static void
  @@ -223,6 +224,7 @@
          host.h_addr_list = h_addr_ptrs;
          haveanswer = 0;
          had_error = 0;
  +       _dns_ttl_ = -1;
          while (ancount-- > 0 && cp < eom && !had_error) {
                  n = dn_expand(answer->buf, eom, cp, bp, buflen);
                  if ((n < 0) || !(*name_ok)(bp)) {
  @@ -232,8 +234,11 @@
                  cp += n;                        /* name */
                  type = _getshort(cp);
                  cp += INT16SZ;                  /* type */
  -               class = _getshort(cp);
  -               cp += INT16SZ + INT32SZ;        /* class, TTL */
  +               class = _getshort(cp);
  +               cp += INT16SZ;                  /* class */
  +               if (qtype == T_A  && type == T_A)
  +                       _dns_ttl_ = _getlong(cp);
  +               cp += INT32SZ;                  /* TTL */
                  n = _getshort(cp);
                  cp += INT16SZ;                  /* len */
                  if (class != C_IN) {



  And here is a patch for BIND-8:



  *** src/lib/irs/dns_ho.c.orig   Tue May 26 21:55:51 1998
  --- src/lib/irs/dns_ho.c        Tue May 26 21:59:57 1998
  ***************
  *** 87,92 ****
  --- 87,93 ----
    #endif

    extern int h_errno;
  + int _dns_ttl_;

    /* Definitions. */

  ***************
  *** 395,400 ****
  --- 396,402 ----
          pvt->host.h_addr_list = pvt->h_addr_ptrs;
          haveanswer = 0;
          had_error = 0;
  +       _dns_ttl_ = -1;
          while (ancount-- > 0 && cp < eom && !had_error) {
                  n = dn_expand(ansbuf, eom, cp, bp, buflen);
                  if ((n < 0) || !(*name_ok)(bp)) {
  ***************
  *** 404,411 ****
                  cp += n;                        /* name */
                  type = ns_get16(cp);
                  cp += INT16SZ;                  /* type */
  !               class = ns_get16(cp);
  !               cp += INT16SZ + INT32SZ;        /* class, TTL */
                  n = ns_get16(cp);
                  cp += INT16SZ;                  /* len */
                  if (class != C_IN) {
  --- 406,416 ----
                  cp += n;                        /* name */
                  type = ns_get16(cp);
                  cp += INT16SZ;                  /* type */
  !               class = _getshort(cp);
  !               cp += INT16SZ;                  /* class */
  !               if (qtype == T_A  && type == T_A)
  !                       _dns_ttl_ = _getlong(cp);
  !               cp += INT32SZ;                  /* TTL */
                  n = ns_get16(cp);
                  cp += INT16SZ;                  /* len */
                  if (class != C_IN) {



  2.10.  My platform is BSD/OS or BSDI and I can't compile Squid


          cache_cf.c: In function `parseConfigFile':
          cache_cf.c:1353: yacc stack overflow before `token'
          ...



  You may need to upgrade your gcc installation to a more recent
  version.  Check your gcc version with

          gcc -v


  If it is earlier than 2.7.2, you might consider upgrading.

  2.11.  Problems compiling libmiscutil.a  on Solaris

  The following error occurs on Solaris systems using gcc when the
  Solaris C compiler is not installed:

          /usr/bin/rm -f libmiscutil.a
          /usr/bin/false r libmiscutil.a rfc1123.o rfc1738.o util.o ...
          make[1]: *** [libmiscutil.a] Error 255
          make[1]: Leaving directory `/tmp/squid-1.1.11/lib'
          make: *** [all] Error 1


  Note on the second line the /usr/bin/false.   This is supposed to be a
  path to the ar program.  If configure cannot find ar on your system,
  then it substitues false.


  To fix this you either need to:

  o  Add /usr/ccs/bin to your PATH.  This is where the ar command should
     be.  You need to install SUNWbtool if ar is not there.  Otherwise,

  o  Install the binutils package from the GNU FTP site
     .  This package includes programs
     such as ar, as, and ld.


  2.12.  I have problems compiling Squid on Platform Foo.

  Please check the page of platforms  on which Squid is
  known to compile.  Your problem might be listed there together with a
  solution.  If it isn't listed there, mail us what you are trying, your
  Squid version, and the problems you encounter.


  2.13.  I see a lot warnings while compiling Squid.

  Warnings are usually not a big concern, and can be common with
  software designed to operate on multiple platforms.  If you feel like
  fixing compile-time warnings, please do so and send us the patches.



  2.14.  Building Squid on OS/2

  by Doug Nazar 


  In order in compile squid, you need to have a reasonable facsimile of
  a Unix system installed.  This includes bash, make, sed, emx, various
  file utilities and a few more. I've setup a TVFS drive that matches a
  Unix file system but this probably isn't strictly necessary.


  I made a few modifications to the pristine EMX 0.9d install.

  1. added defines for strcasecmp() & strncasecmp() to string.h

  2. changed all occurrences of time_t to signed long instead of
     unsigned long

  3. hacked ld.exe

     a. to search for both xxxx.a and libxxxx.a


     b. to produce the correct filename when using the -Zexe option


  You will need to run scripts/convert.configure.to.os2 (in the Squid
  source distribution) to modify the configure script so that it can
  search for the various programs.


  Next, you need to set a few environment variables (see EMX docs for
  meaning):

          export EMXOPT="-h256 -c"
          export LDFLAGS="-Zexe -Zbin -s"



  Now you are ready to configure squid:

          ./configure



  Compile everything:

          make



  and finally, install:

          make install



  This will by default, install into /usr/local/squid. If you wish to
  install somewhere else, see the --prefix option for configure.


  Now, don't forget to set EMXOPT before running squid each time. I
  recommend using the -Y and -N options.


  2.15.  Building Squid on Cygwin

  In order to compile squid, you need to have Cygwin fully installed.


  Unpack the source archive as usual and run configure:

          ./configure



  Compile everything:

          make



  and finally, install:

          make install



  This will by default, install into /usr/local/squid. If you wish to
  install somewhere else, see the --prefix option for configure.


  Now, add a new Cygwin user - see the Cygwin user guide - and map it to
  SYSTEM, or create a new NT user, and a matching Cygwin user and they
  become the squid runas users.


  Read the squid FAQ on permissions if you are using CYGWIN=ntsec.


  After run squid -z.  If that succeeds, try squid -N -D -d1, squid
  should start. Check that there are no errors. If everything looks
  good, try browsing through squid.


  Now, configure cygrunsrv to run squid as a service as the chosen
  usercode. You may need to check permissions here.



  3.  Installing and Running Squid

  3.1.  How big of a system do I need to run Squid?

  There are no hard-and-fast rules.  The most important resource for
  Squid is physical memory.  Your processor does not need to be ultra-
  fast.  Your disk system will be the major bottleneck, so fast disks
  are important for high-volume caches.  Do not use IDE disks if you can
  help it.


  In late 1998, if you are buying a new machine for a cache, I would
  recommend the following configuration:

  o  300 MHz Pentium II CPU

  o  512 MB RAM

  o  Five 9 GB UW-SCSI disks

     Your system disk, and logfile disk can probably be IDE without
     losing any cache performance.


  Also, see Squid Sizing for Intel Platforms
   by Martin Hamilton This
  is a very nice page summarizing system configurations people are using
  for large Squid caches.


  3.2.  How do I install Squid?

  After ``compiling Squid'', you can install it with this simple
  command:

          % make install


  If you have enabled the ``ICMP features'' then you will also want to
  type


          % su
          # make install-pinger



  After installing, you will want to edit and customize the squid.conf
  file.  By default, this file is located at
  /usr/local/squid/etc/squid.conf.


  Also, a QUICKSTART guide has been included with the source
  distribution.  Please see the directory where you unpacked the source
  archive.


  3.3.  What does the squid.conf  file do?

  The squid.conf file defines the configuration for squid.  the
  configuration includes (but not limited to) HTTP port number, the ICP
  request port number, incoming and outgoing requests, information about
  firewall access, and various timeout information.


  3.4.  Do you have a squid.conf  example?

  Yes, after you make install, a sample squid.conf file will exist in
  the ``etc" directory under the Squid installation directory.

  The sample squid.conf file contains comments explaining each option.



  3.5.  How do I start Squid?

  First you need to make your Squid configuration. The Squid
  configuration can be found in /usr/local/squid/etc/squid.conf and by
  default includes documentation on all directives.


  In the Suqid distribution there is a small QUICKSTART guide indicating
  which directives you need to look closer at and why. At a absolute
  minimum you need to change the http_access configuration to allow
  access from your clients.


  To verify your configuration file you can use the -k parse option

          % /usr/local/squid/sbin/squid -k parse


  If this outputs any errors then these are syntax errors or other fatal
  misconfigurations and needs to be corrected before you continue. If it
  is silent and immediately gives back the command promt then your
  squid.conf is syntactically correct and could be understood by Squid.


  After you've finished editing the configuration file, you can start
  Squid for the first time.  The procedure depends a little bit on which
  version you are using.


  First, you must create the swap directories.  Do this by running Squid
  with the -z option:


          % /usr/local/squid/sbin/squid -z



  NOTE: If you run Squid as root then you may need to first create
  /usr/local/squid/var/logs and your cache_dir directories and assign
  ownership of these to the cache_effective_user configured in your
  squid.conf.


  Once the creation of the cache directories completes, you can start
  Squid and try it out. Probably the best thing to do is run it from
  your terminal and watch the debugging output.  Use this command:

          % /usr/local/squid/sbin/squid -NCd1


  If everything is working okay, you will see the line:

          Ready to serve requests.


  If you want to run squid in the background, as a daemon process, just
  leave off all options:

          % /usr/local/squid/sbin/squid



  NOTE: depending on which http_port you select you may need to start
  squid as root (http_port <1024).


  NOTE: In Squid-2.4 and earlier Squid was installed in bin by default,
  not sbin.


  3.6.  How do I start Squid automatically when the system boots?

  Squid-2 has a restart feature built in.  This greatly simplifies
  starting Squid and means that you don't need to use RunCache or
  inittab.  At the minimum, you only need to enter the pathname to the
  Squid executable.  For example:

          /usr/local/squid/sbin/squid



  Squid will automatically background itself and then spawn a child
  process.  In your syslog messages file, you should see something like
  this:

          Sep 23 23:55:58 kitty squid[14616]: Squid Parent: child process 14617 started


  That means that process ID 14563 is the parent process which monitors
  the child process (pid 14617).  The child process is the one that does
  all of the work.  The parent process just waits for the child process
  to exit.  If the child process exits unexpectedly, the parent will
  automatically start another child process.  In that case, syslog
  shows:



          Sep 23 23:56:02 kitty squid[14616]: Squid Parent: child process 14617 exited with status 1
          Sep 23 23:56:05 kitty squid[14616]: Squid Parent: child process 14619 started



  If there is some problem, and Squid can not start, the parent process
  will give up after a while.  Your syslog will show:

          Sep 23 23:56:12 kitty squid[14616]: Exiting due to repeated, frequent failures


  When this happens you should check your syslog messages and cache.log
  file for error messages.


  When  you look at a process (ps command) listing,  you'll see two
  squid processes:

          24353  ??  Ss     0:00.00 /usr/local/squid/bin/squid
          24354  ??  R      0:03.39 (squid) (squid)


  The first is the parent process, and the child process is the one
  called ``(squid)''.  Note that if you accidentally kill the parent
  process, the child process will not notice.


  If you want to run Squid from your termainal and prevent it from
  backgrounding and spawning a child process, use the -N command line
  option.

          /usr/local/squid/bin/squid -N



  3.6.1.  From inittab

  On systems which have an /etc/inittab file (Digital Unix, Solaris,
  IRIX, HP-UX, Linux), you can add a line like this:

          sq:3:respawn:/usr/local/squid/sbin/squid.sh < /dev/null >> /tmp/squid.log 2>&1


  We recommend using a squid.sh shell script, but you could instead call
  Squid directly with the -N option and other options you may require.
  A sameple squid.sh script is shown below:



          #!/bin/sh
          C=/usr/local/squid
          PATH=/usr/bin:$C/bin
          TZ=PST8PDT
          export PATH TZ

          # User to notify on restarts
          notify="root"

          # Squid command line options
          opts=""

          cd $C
          umask 022
          sleep 10
          while [ -f /var/run/nosquid ]; do
                  sleep 1
          done
          /usr/bin/tail -20 $C/logs/cache.log \
                  | Mail -s "Squid restart on `hostname` at `date`" $notify
          exec bin/squid -N $opts



  3.6.2.  From rc.local

  On BSD-ish systems, you will need to start Squid from the ``rc''
  files, usually /etc/rc.local.  For example:

          if [ -f /usr/local/squid/sbin/squid ]; then
                  echo -n ' Squid'
                  /usr/local/squid/sbin/squid
          fi



  3.6.3.  From init.d

  Squid ships with a init.d type startup script in contrib/squid.rc
  which works on most init.d type systems. Or you can write your own
  using any normal init.d script found in your system as template and
  add the start/stop fragments shown below.


  Start:

          /usr/local/squid/sbin/squid



  Stop:

          /usr/local/squid/sbin/squid -k shutdown
          n=120
          while /usr/local/squid/sbin/squid -k check && [ $n -gt 120 ]; do
              sleep 1
              echo -n .
              n=`expr $n - 1`
          done



  3.7.  How do I tell if Squid is running?

  You can use the squidclient program:

          % squidclient http://www.netscape.com/ > test



  There are other command-line HTTP client programs available as well.
  Two that you may find useful are wget
   and echoping
  .


  Another way is to use Squid itself to see if it can signal a running
  Squid process:

          % squid -k check


  And then check the shell's exit status variable.


  Also, check the log files, most importantly the access.log and
  cache.log files.


  3.8.  squid  command line options

  These are the command line options for Squid-2:

     -a Specify an alternate port number for incoming HTTP requests.
        Useful for testing a configuration file on a non-standard port.

     -d Debugging level for ``stderr'' messages.  If you use this
        option, then debugging messages up to the specified level will
        also be written to stderr.

     -f Specify an alternate squid.conf file instead of the pathname
        compiled into the executable.

     -h Prints the usage and help message.

     -k reconfigure
        Sends a HUP signal, which causes Squid to re-read its
        configuration files.

     -k rotate
        Sends an USR1 signal, which causes Squid to rotate its log
        files.  Note, if logfile_rotate is set to zero, Squid still
        closes and re-opens all log files.

     -k shutdown
        Sends a TERM signal, which causes Squid to wait briefly for
        current connections to finish and then exit.  The amount of time
        to wait is specified with shutdown_lifetime.

     -k interrupt
        Sends an INT signal, which causes Squid to shutdown immediately,
        without waiting for current connections.

     -k kill
        Sends a KILL signal, which causes the Squid process to exit
        immediately, without closing any connections or log files.  Use
        this only as a last resort.
     -k debug
        Sends an USR2 signal, which causes Squid to generate full
        debugging messages until the next USR2 signal is recieved.
        Obviously very useful for debugging problems.

     -k check
        Sends a ``ZERO'' signal to the Squid process.  This simply
        checks whether or not the process is actually running.

     -s Send debugging (level 0 only) message to syslog.

     -u Specify an alternate port number for ICP messages.  Useful for
        testing a configuration file on a non-standard port.

     -v Prints the Squid version.

     -z Creates disk swap directories.  You must use this option when
        installing Squid for the first time, or when you add or modify
        the cache_dir configuration.

     -D Do not make initial DNS tests.  Normally, Squid looks up some
        well-known DNS hostnames to ensure that your DNS name resolution
        service is working properly.

     -F If the swap.state logs are clean, then the cache is rebuilt in
        the ``foreground'' before any requests are served.  This will
        decrease the time required to rebuild the cache, but HTTP
        requests will not be satisified during this time.

     -N Do not automatically become a background daemon process.

     -R Do not set the SO_REUSEADDR option on sockets.

     -V Enable virtual host support for the httpd-accelerator mode.
        This is identical to writing httpd_accel_host virtual in the
        config file.

     -X Enable full debugging while parsing the config file.

     -Y Return ICP_OP_MISS_NOFETCH instead of ICP_OP_MISS while the
        swap.state file is being read.  If your cache has mostly child
        caches which use ICP, this will allow your cache to rebuild
        faster.


  3.9.  How do I see how Squid works?


  o  Check the cache.log file in your logs directory.  It logs
     interesting (and boring) things as a part of its normal operation.

  o  Install and use the ``Cache Manager''.


  3.10.  Can Squid benefit from SMP systems?

  Squid is a single process application and can not make use of SMP.  If
  you want to make Squid benefit from a SMP system you will need to run
  multiple instances of Squid and find a way to distribute your users on
  the different Squid instances just as if you had multiple Squid boxes.


  Having two CPUs is indeed nice for running other CPU intensive tasks
  on the same server as the proxy, such as if you have a lot of logs and
  need to run various statistics collections during peak hours.

  The authentication and group helpers barely use any CPU and does not
  benefit from dual-CPU configuration.


  3.11.  Is it okay to use separate drives and RAID on Squid?

  RAID1 is fine, and so are separate drives.


  RAID0 (striping) with Squid only gives you the drawback that if you
  lose one of the drives the whole stripe set is lost. There is no
  benefit in performance as Squid already distributes the load on the
  drives quite nicely.


  Squid is the worst case application for RAID5, whether hardware or
  software, and will absolutely kill the performance of a RAID5. Once
  the cache has been filled Squid uses a lot of small random writes
  which the worst case workload for RAID5, effectively reducing write
  speed to only little more than that of one single drive.


  Generally seek time is what you want to optimize for Squid, or more
  precisely the total amount of seeks/s your system can sustain.
  Choosing the right RAID solution generally decreases the amount of
  seeks/s your system can sustain significantly.



  4.  Configuration issues

  4.1.  How do I join a cache hierarchy?

  To place your cache in a hierarchy, use the cache_peer directive in
  squid.conf to specify the parent and sibling nodes.


  For example, the following squid.conf file on childcache.example.com
  configures its cache to retrieve data from one parent cache and two
  sibling caches:


          #  squid.conf - On the host: childcache.example.com
          #
          #  Format is: hostname  type  http_port  udp_port
          #
          cache_peer parentcache.example.com   parent  3128 3130
          cache_peer childcache2.example.com   sibling 3128 3130
          cache_peer childcache3.example.com   sibling 3128 3130



  The cache_peer_domain directive allows you to specify that certain
  caches siblings or parents for certain domains:



          #  squid.conf - On the host: sv.cache.nlanr.net
          #
          #  Format is: hostname  type  http_port  udp_port
          #

          cache_peer electraglide.geog.unsw.edu.au parent 3128 3130
          cache_peer cache1.nzgate.net.nz          parent 3128 3130
          cache_peer pb.cache.nlanr.net   parent 3128 3130
          cache_peer it.cache.nlanr.net   parent 3128 3130
          cache_peer sd.cache.nlanr.net   parent 3128 3130
          cache_peer uc.cache.nlanr.net   sibling 3128 3130
          cache_peer bo.cache.nlanr.net   sibling 3128 3130
          cache_peer_domain electraglide.geog.unsw.edu.au .au
          cache_peer_domain cache1.nzgate.net.nz   .au .aq .fj .nz
          cache_peer_domain pb.cache.nlanr.net     .uk .de .fr .no .se .it
          cache_peer_domain it.cache.nlanr.net     .uk .de .fr .no .se .it
          cache_peer_domain sd.cache.nlanr.net     .mx .za .mu .zm



  The configuration above indicates that the cache will use
  pb.cache.nlanr.net and it.cache.nlanr.net for domains uk, de, fr, no,
  se and it, sd.cache.nlanr.net for domains mx, za, mu and zm, and
  cache1.nzgate.net.nz for domains au, aq, fj, and nz.


  4.2.  How do I join NLANR's cache hierarchy?

  We have a simple set of guidelines for joining
   the NLANR cache hierarchy.


  4.3.  Why should I want to join NLANR's cache hierarchy?

  The NLANR hierarchy can provide you with an initial source for parent
  or sibling caches.  Joining the NLANR global cache system will
  frequently improve the performance of your caching service.


  4.4.  How do I register my cache with NLANR's registration service?

  Just enable these options in your squid.conf and you'll be registered:

          cache_announce 24
          announce_to sd.cache.nlanr.net:3131



  NOTE: announcing your cache is not the same thing as joining the NLANR
  cache hierarchy.  You can join the NLANR cache hierarchy without
  registering, and you can register without joining the NLANR cache
  hierarchy.



  4.5.  How do I find other caches close to me and arrange par-
  ent/child/sibling relationships with them?

  Visit the NLANR cache registration database
   to discover other caches near
  you.  Keep in mind that just because a cache is registered in the
  database does not mean they are willing to be your
  parent/sibling/child.  But it can't hurt to ask...



  4.6.  My cache registration is not appearing in the Tracker database.


  o  Your site will not be listed if your cache IP address does not have
     a DNS PTR record. If we can't map the IP address back to a domain
     name, it will be listed as ``Unknown.''

  o  The registration messages are sent with UDP. We may not be
     receiving your announcement message due to firewalls which block
     UDP, or dropped packets due to congestion.


  4.7.  What is the httpd-accelerator mode?

  This entry has been moved to ``a different section''.


  4.8.  How do I configure Squid to work behind a firewall?

  Note: The information here is current for version 2.2.


  If you are behind a firewall then you can't make direct connections to
  the outside world, so you must use a parent cache.  Squid doesn't use
  ICP queries for a request if it's behind a firewall or if there is
  only one parent.


  You can use the never_direct access list in squid.conf to specify
  which requests must be forwarded to your parent cache outside the
  firewall, and the always_direct access list to specify which requests
  must not be forwarded.  For example, if Squid must connect directly to
  all servers that end with mydomain.com, but must use the parent for
  all others, you would write:

          acl INSIDE dstdomain .mydomain.com
          always_direct allow INSIDE
          never_direct allow all



  You could also specify internal servers by IP address

          acl INSIDE_IP dst 1.2.3.0/24
          always_direct allow INSIDE_IP
          never_direct allow all


  Note, however that when you use IP addresses, Squid must perform a DNS
  lookup to convert URL hostnames to an address.  Your internal DNS
  servers may not be able to lookup external domains.


  If you use never_direct and you have multiple parent caches, then you
  probably will want to mark one of them as a default choice in case
  Squid can't decide which one to use.  That is done with the default
  keyword on a cache_peer line.  For example:

          cache_peer xyz.mydomain.com parent 3128 0 default



  4.9.  How do I configure Squid forward all requests to another proxy?

  Note: The information here is current for version 2.2.

  First, you need to give Squid a parent cache.  Second, you need to
  tell Squid it can not connect directly to origin servers.  This is
  done with three configuration file lines:

          cache_peer parentcache.foo.com parent 3128 0 no-query default
          acl all src 0.0.0.0/0.0.0.0
          never_direct allow all


  Note, with this configuration, if the parent cache fails or becomes
  unreachable, then every request will result in an error message.


  In case you want to be able to use direct connections when all the
  parents go down you should use a different approach:

          cache_peer parentcache.foo.com parent 3128 0 no-query
          prefer_direct off


  The default behaviour of Squid in the absence of positive ICP, HTCP,
  etc replies is to connect to the origin server instead of using par-
  ents.  The prefer_direct off directive tells Squid to try parents
  first.


  4.10.  I have dnsserver  processes that aren't being used, should I
  lower the number in squid.conf ?

  The dnsserver processes are used by squid because the gethostbyname(3)
  library routines used to convert web sites names to their internet
  addresses blocks until the function returns (i.e., the process that
  calls it has to wait for a reply). Since there is only one squid
  process, everyone who uses the cache would have to wait each time the
  routine was called.  This is why the dnsserver is a separate process,
  so that these processes can block, without causing blocking in squid.


  It's very important that there are enough dnsserver processes to cope
  with every access you will need, otherwise squid will stop
  occasionally.  A good rule of thumb is to make sure you have at least
  the maximum number of dnsservers squid has ever needed on your system,
  and probably add two to be on the safe side. In other words, if you
  have only ever seen at most three dnsserver processes in use, make at
  least five.  Remember that a dnsserver is small and, if unused, will
  be swapped out.


  4.11.  My dnsserver  average/median service time seems high, how can I
  reduce it?

  First, find out if you have enough dnsserver processes running by
  looking at the Cachemanager dns output.  Ideally, you should see that
  the first dnsserver handles a lot of requests, the second one less
  than the first, etc.  The last dnsserver should have serviced
  relatively few requests.  If there is not an obvious decreasing trend,
  then you need to increase the number of dns_children in the
  configuration file.  If the last dnsserver has zero requests, then you
  definately have enough.



  Another factor which affects the dnsserver service time is the
  proximity of your DNS resolver.  Normally we do not recommend running
  Squid and named on the same host.  Instead you should try use a DNS
  resolver (named) on a different host, but on the same LAN.  If your
  DNS traffic must pass through one or more routers, this could be
  causing unnecessary delays.


  4.12.  How can I easily change the default HTTP port?

  Before you run the configure script, simply set the CACHE_HTTP_PORT
  environment variable.

          setenv CACHE_HTTP_PORT 8080
          ./configure
          make
          make install



  4.13.  Is it possible to control how big each cache_dir  is?

  With Squid-1.1 it is NOT possible.  Each cache_dir is assumed to be
  the same size.  The cache_swap setting defines the size of all
  cache_dir's taken together.  If you have N cache_dir's then each one
  will hold cache_swap / N Megabytes.


  4.14.  What cache_dir  size should I use?

  Most people have a disk partition dedicated to the Squid cache.  You
  don't want to use the entire partition size.  You have to leave some
  extra room.  Currently, Squid is not very tolerant of running out of
  disk space.

  Lets say you have a 9GB disk.  Remember that disk manufacturers lie
  about the space available.  A so-called 9GB disk usually results in
  about 8.5GB of raw, usable space.  First, put a filesystem on it, and
  mount it.  Then check the ``available space'' with your df program.
  Note that you lose some disk space to filesystem overheads, like
  superblocks, inodes, and directory entries.  Also note that Unix
  normally keeps 10% free for itself.   So with a 9GB disk, you're
  probably down to about 8GB after formatting.


  Next, I suggest taking off another 10% or so for Squid overheads, and
  a "safe buffer."  Squid normally puts its swap.state files in each
  cache directory.  These grow in size until you rotate the logs, or
  restart squid.  Also note that Squid performs better when there is
  more free space.  So if performance is important to you, then take off
  even more space.  Typically, for a 9GB disk, I recommend a cache_dir
  setting of 6000 to 7500 Megabytes:

  cache_dir ... 7000 16 256



  Its better to start out conservative.  After the cache becomes full,
  look at the disk usage.  If you think there is plenty of unused space,
  then increase the cache_dir setting a little.


  If you're getting ``disk full'' write errors, then you definately need
  to decrease your cache size.
  4.15.  I'm adding a new cache_dir .  Will I lose my cache?

  With Squid-1.1, yes, you will lose your cache.  This is because
  version 1.1 uses a simplistic algorithm to distribute files between
  cache directories.


  With Squid-2, you will not lose your existing cache.  You can add and
  delete cache_dir's without affecting any of the others.


  4.16.  Squid and http-gw  from the TIS toolkit.

  Several people on both the fwtk-users and the squid-users mailing
  asked about using Squid in combination with http-gw from the TIS
  toolkit .  The most elegant way in my opinion is
  to run an internal Squid caching proxyserver which handles client
  requests and let this server forward it's requests to the http-gw
  running on the firewall. Cache hits won't need to be handled by the
  firewall.


  In this example Squid runs on the same server as the http-gw, Squid
  uses 8000 and http-gw uses 8080 (web).  The local domain is home.nl.


  4.16.1.  Firewall configuration:

  Either run http-gw as a daemon from the /etc/rc.d/rc.local (Linux
  Slackware):

          exec /usr/local/fwtk/http-gw -daemon 8080


  or run it from inetd like this:

          web stream      tcp      nowait.100  root /usr/local/fwtk/http-gw http-gw


  I increased the watermark to 100 because a lot of people run into
  problems with the default value.


  Make sure you have at least the following line in
  /usr/local/etc/netperm-table:

          http-gw:        hosts 127.0.0.1


  You could add the IP-address of your own workstation to this rule and
  make sure the http-gw by itself works, like:

          http-gw:                hosts 127.0.0.1 10.0.0.1



  4.16.2.  Squid configuration:

  The following settings are important:



          http_port       8000
          icp_port        0

          cache_peer      localhost.home.nl parent 8080 0 default
          acl HOME        dstdomain .home.nl
          alwayws_direct  allow HOME
          never_direct    allow all


  This tells Squid to use the parent for all domains other than home.nl.
  Below, access.log entries show what happens if you do a reload on the
  Squid-homepage:


  872739961.631   1566 10.0.0.21 ERR_CLIENT_ABORT/304 83 GET http://www.squid-cache.org/ - DEFAULT_PARENT/localhost.home.nl -
  872739962.976   1266 10.0.0.21 TCP_CLIENT_REFRESH/304 88 GET http://www.nlanr.net/Images/cache_now.gif - DEFAULT_PARENT/localhost.home.nl -
  872739963.007   1299 10.0.0.21 ERR_CLIENT_ABORT/304 83 GET http://www.squid-cache.org/Icons/squidnow.gif - DEFAULT_PARENT/localhost.home.nl -
  872739963.061   1354 10.0.0.21 TCP_CLIENT_REFRESH/304 83 GET http://www.squid-cache.org/Icons/Squidlogo2.gif - DEFAULT_PARENT/localhost.home.nl



  http-gw entries in syslog:


  Aug 28 02:46:00 memo http-gw[2052]: permit host=localhost/127.0.0.1 use of gateway (V2.0beta)
  Aug 28 02:46:00 memo http-gw[2052]: log host=localhost/127.0.0.1 protocol=HTTP cmd=dir dest=www.squid-cache.org path=/
  Aug 28 02:46:01 memo http-gw[2052]: exit host=localhost/127.0.0.1 cmds=1 in=0 out=0 user=unauth duration=1
  Aug 28 02:46:01 memo http-gw[2053]: permit host=localhost/127.0.0.1 use of gateway (V2.0beta)
  Aug 28 02:46:01 memo http-gw[2053]: log host=localhost/127.0.0.1 protocol=HTTP cmd=get dest=www.squid-cache.org path=/Icons/Squidlogo2.gif
  Aug 28 02:46:01 memo http-gw[2054]: permit host=localhost/127.0.0.1 use of gateway (V2.0beta)
  Aug 28 02:46:01 memo http-gw[2054]: log host=localhost/127.0.0.1 protocol=HTTP cmd=get dest=www.squid-cache.org path=/Icons/squidnow.gif
  Aug 28 02:46:01 memo http-gw[2055]: permit host=localhost/127.0.0.1 use of gateway (V2.0beta)
  Aug 28 02:46:01 memo http-gw[2055]: log host=localhost/127.0.0.1 protocol=HTTP cmd=get dest=www.nlanr.net path=/Images/cache_now.gif
  Aug 28 02:46:02 memo http-gw[2055]: exit host=localhost/127.0.0.1 cmds=1 in=0 out=0 user=unauth duration=1
  Aug 28 02:46:03 memo http-gw[2053]: exit host=localhost/127.0.0.1 cmds=1 in=0 out=0 user=unauth duration=2
  Aug 28 02:46:04 memo http-gw[2054]: exit host=localhost/127.0.0.1 cmds=1 in=0 out=0 user=unauth duration=3



  To summarize:


  Advantages:

  o  http-gw allows you to selectively block ActiveX and Java, and it's
     primary design goal is security.

  o  The firewall doesn't need to run large applications like Squid.

  o  The internal Squid-server still gives you the benefit of caching.


  Disadvantages:

  o  The internal Squid proxyserver can't (and shouldn't) work with
     other parent or neighbor caches.

  o  Initial requests are slower because these go through http-gw, http-
     gw also does reverse lookups. Run a nameserver on the firewall or
     use an internal nameserver.



       --Rodney van den Oever 



  4.17.  What is ``HTTP_X_FORWARDED_FOR''?  Why does squid provide it to
  WWW servers, and how can I stop it?

  When a proxy-cache is used, a server does not see the connection
  coming from the originating client.  Many people like to implement
  access controls based on the client address.  To accommodate these
  people, Squid adds its own request header called "X-Forwarded-For"
  which looks like this:

          X-Forwarded-For: 128.138.243.150, unknown, 192.52.106.30


  Entries are always IP addresses, or the word unknown if the address
  could not be determined or if it has been disabled with the for-
  warded_for configuration option.


  We must note that access controls based on this header are extremely
  weak and simple to fake.  Anyone may hand-enter a request with any IP
  address whatsoever.  This is perhaps the reason why client IP
  addresses have been omitted from the HTTP/1.1 specification.


  Because of the weakness of this header, support for access controls
  based on X-Forwarder-For is not yet available in any officially
  released version of squid.  However, unofficial patches are available
  from the follow_xff  Squid development project and may be
  integrated into later versions of Squid once a suitable trust model
  have been developed.


  4.18.  Can Squid anonymize HTTP requests?

  Yes it can, however the way of doing it has changed from earlier
  versions of squid. As of squid-2.2 a more customisable method has been
  introduced.  Please follow the instructions for the version of squid
  that you are using.  As a default, no anonymizing is done.


  If you choose to use the anonymizer you might wish to investigate the
  forwarded_for option to prevent the client address being disclosed.
  Failure to turn off the forwarded_for option will reduce the
  effectiveness of the anonymizer. Finally if you filter the User-Agent
  header using the fake_user_agent option can prevent some user problems
  as some sites require the User-Agent header.


  4.18.1.  Squid 2.2

  With the introduction of squid 2.2 the anonoymizer has become more
  customisable.  It now allows specification of exactly which headers
  will be allowed to pass.  This is further extended in Squid-2.5 to
  allow headers to be anonymized conditionally.


  For details see the documentation of the http_header_access and
  header_replace directives in squid.conf.default.



  References: Anonymous WWW 



  4.19.  Can I make Squid go direct for some sites?

  Sure, just use the always_direct access list.

  For example, if you want Squid to connect directly to hotmail.com
  servers, you can use these lines in  your config file:

  acl hotmail dstdomain .hotmail.com
  always_direct allow hotmail



  4.20.  Can I make Squid proxy only, without caching anything?

  Sure, there are few things you can do.

  You can use the no_cache access list to make Squid never cache any
  response:

          acl all src 0/0
          no_cache deny all



  With Squid-2.4 and later you can use the ``null'' storage module to
  avoid having a cache directory:

          cache_dir null /tmp



  Note: a null cache_dir does not disable caching, but it does save you
  from creating a cache structure if you have disabled caching with
  no_cache.


  Note: the directory (e.g., /tmp) must exist so that squid can chdir to
  it, unless you also use the coredump_dir option.

  To configure Squid for the ``null'' storage module, specify it on the
  configure command line:

          ./configure --enable-storeio=ufs,null ...



  4.21.  Can I prevent users from downloading large files?

  You can set the global reply_body_max_size parameter.  This option
  controls the largest HTTP message body that will be sent to a cache
  client for one request.

  If the HTTP response coming from the server has a Content-length
  header, then Squid compares the content-length value to the
  reply_body_max_size value.  If the content-length is larger, the
  server connection is closed and the user receives an error message
  from Squid.

  Some responses don't have Content-length headers.  In this case, Squid
  counts how many bytes are written to the client.  Once the limit is
  reached, the client's connection is simply closed.

  Note that ``creative'' user-agents will still be able to download
  really large files through the cache using HTTP/1.1 range requests.



  5.  Communication between browsers and Squid

  Most web browsers available today support proxying and are easily
  configured to use a Squid server as a proxy.  Some browsers support
  advanced features such as lists of domains or URL patterns that
  shouldn't be fetched through the proxy, or JavaScript automatic proxy
  configuration.


  5.1.  Netscape manual configuration

  Select Network Preferences from the Options menu.  On the Proxies
  page, click the radio button next to Manual Proxy Configuration and
  then click on the View button.  For each protocol that your Squid
  server supports (by default, HTTP, FTP, and gopher) enter the Squid
  server's hostname or IP address and put the HTTP port number for the
  Squid server (by default, 3128) in the Port column.  For any protocols
  that your Squid does not support, leave the fields blank.

  Here is a screen shot  of the Netscape
  Navigator manual proxy configuration screen.



  5.2.  Netscape automatic configuration

  Netscape Navigator's proxy configuration can be automated with
  JavaScript (for Navigator versions 2.0 or higher).  Select Network
  Preferences from the Options menu.  On the Proxies page, click the
  radio button next to Automatic Proxy Configuration and then fill in
  the URL for your JavaScript proxy configuration file in the text box.
  The box is too small, but the text will scroll to the right as you go.

  Here is a screen shot  of the Netscape
  Navigator automatic proxy configuration screen.

  You may also wish to consult Netscape's documentation for the
  Navigator JavaScript proxy configuration
  


  Here is a sample auto configuration JavaScript from Oskar Pearson:



  ______________________________________________________________________
  //We (www.is.co.za) run a central cache for our customers that they
  //access through a firewall - thus if they want to connect to their intranet
  //system (or anything in their domain at all) they have to connect
  //directly - hence all the "fiddling" to see if they are trying to connect
  //to their local domain.

  //Replace each occurrence of company.com with your domain name
  //and if you have some kind of intranet system, make sure
  //that you put it's name in place of "internal" below.

  //We also assume that your cache is called "cache.company.com", and
  //that it runs on port 8080. Change it down at the bottom.

  //(C) Oskar Pearson and the Internet Solution (http://www.is.co.za)

      function FindProxyForURL(url, host)
          {
              //If they have only specified a hostname, go directly.
              if (isPlainHostName(host))
                      return "DIRECT";

              //These connect directly if the machine they are trying to
              //connect to starts with "intranet" - ie http://intranet
              //Connect  directly if it is intranet.*
              //If you have another machine that you want them to
              //access directly, replace "internal*" with that
              //machine's name
              if (shExpMatch( host, "intranet*")||
                              shExpMatch(host, "internal*"))
                  return "DIRECT";

              //Connect directly to our domains (NB for Important News)
              if (dnsDomainIs( host,"company.com")||
              //If you have another domain that you wish to connect to
              //directly, put it in here
                              dnsDomainIs(host,"sistercompany.com"))
                  return "DIRECT";

              //So the error message "no such host" will appear through the
              //normal Netscape box - less support queries :)
              if (!isResolvable(host))
                      return "DIRECT";

              //We only cache http, ftp and gopher
              if (url.substring(0, 5) == "http:" ||
                              url.substring(0, 4) == "ftp:"||
                              url.substring(0, 7) == "gopher:")

              //Change the ":8080" to the port that your cache
              //runs on, and "cache.company.com" to the machine that
              //you run the cache on
                      return "PROXY cache.company.com:8080; DIRECT";

              //We don't cache WAIS
              if (url.substring(0, 5) == "wais:")
                      return "DIRECT";

              else
                      return "DIRECT";
          }
  ______________________________________________________________________



  5.3.  Lynx and Mosaic configuration

  For Mosaic and Lynx, you can set environment variables before starting
  the application.  For example (assuming csh or tcsh):


          % setenv http_proxy http://mycache.example.com:3128/
          % setenv gopher_proxy http://mycache.example.com:3128/
          % setenv ftp_proxy http://mycache.example.com:3128/



  For Lynx you can also edit the lynx.cfg file to configure proxy usage.
  This has the added benefit of causing all Lynx users on a system to
  access the proxy without making environment variable changes for each
  user.  For example:

          http_proxy:http://mycache.example.com:3128/
          ftp_proxy:http://mycache.example.com:3128/
          gopher_proxy:http://mycache.example.com:3128/



  5.4.  Redundant Proxy Auto-Configuration

  There's one nasty side-effect to using auto-proxy scripts: if you
  start the web browser it will try and load the auto-proxy-script.


  If your script isn't available either because the web server hosting
  the script is down or your workstation can't reach the web server
  (e.g.  because you're working off-line with your notebook and just
  want to read a previously saved HTML-file) you'll get different errors
  depending on the browser you use.


  The Netscape browser will just return an error after a timeout (after
  that it tries to find the site 'www.proxy.com' if the script you use
  is called 'proxy.pac').


  The Microsoft Internet Explorer on the other hand won't even start, no
  window displays, only after about 1 minute it'll display a window
  asking you to go on with/without proxy configuration.


  The point is that your workstations always need to locate the proxy-
  script. I created some extra redundancy by hosting the script on two
  web servers (actually Apache web servers on the proxy servers
  themselves) and adding the following records to my primary nameserver:

  proxy    IN A    10.0.0.1 ; IP address of proxy1
           IN A    10.0.0.2 ; IP address of proxy2


  The clients just refer to 'http://proxy/proxy.pac'.  This script looks
  like this:



  function FindProxyForURL(url,host)
  {
          // Hostname without domainname or host within our own domain?
          // Try them directly:
          // http://www.domain.com actually lives before the firewall, so
          // make an exception:
          if ((isPlainHostName(host)||dnsDomainIs( host,".domain.com")) &&
                  !localHostOrDomainIs(host, "www.domain.com"))
                  return "DIRECT";

          // First try proxy1 then proxy2. One server mostly caches '.com'
          // to make sure both servers are not
          // caching the same data in the normal situation. The other
          // server caches the other domains normally.
          // If one of 'm is down the client will try the other server.
          else if (shExpMatch(host, "*.com"))
                  return "PROXY proxy1.domain.com:8080; PROXY proxy2.domain.com:8081; DIRECT";
          return "PROXY proxy2.domain.com:8081; PROXY proxy1.domain.com:8080; DIRECT";
  }



  I made sure every client domain has the appropriate 'proxy' entry.
  The clients are automatically configured with two nameservers using
  DHCP.


       --Rodney van den Oever 



  5.5.  Proxy Auto-Configuration with URL Hashing

  The Sharp Super Proxy Script page 
  contains a lot of good information about hash-based proxy auto-
  configuration scripts.  With these you can distribute the load between
  a number of caching proxies.


  5.6.  Microsoft Internet Explorer configuration

  Select Options from the View menu.  Click on the Connection tab.  Tick
  the Connect through Proxy Server option and hit the Proxy Settings
  button.  For each protocol that your Squid server supports (by
  default, HTTP, FTP, and gopher) enter the Squid server's hostname or
  IP address and put the HTTP port number for the Squid server (by
  default, 3128) in the Port column.  For any protocols that your Squid
  does not support, leave the fields blank.

  Here is a screen shot  of the Internet Explorer
  proxy configuration screen.

  Microsoft is also starting to support Netscape-style JavaScript
  automated proxy configuration.  As of now, only MSIE version 3.0a for
  Windows 3.1 and Windows NT 3.51 supports this feature (i.e., as of
  version 3.01 build 1225 for Windows 95 and NT 4.0, the feature was not
  included).

  If you have a version of MSIE that does have this feature, elect
  Options from the View menu.  Click on the Advanced tab.  In the lower
  left-hand corner, click on the Automatic Configuration button.  Fill
  in the URL for your JavaScript file in the dialog box it presents you.
  Then exit MSIE and restart it for the changes to take effect.  MSIE
  will reload the JavaScript file every time it starts.

  5.7.  Netmanage Internet Chameleon WebSurfer configuration

  Netmanage WebSurfer supports manual proxy configuration and exclusion
  lists for hosts or domains that should not be fetched via proxy (this
  information is current as of WebSurfer 5.0).  Select Preferences from
  the Settings menu.  Click on the Proxies tab.  Select the Use Proxy
  options for HTTP, FTP, and gopher.  For each protocol that enter the
  Squid server's hostname or IP address and put the HTTP port number for
  the Squid server (by default, 3128) in the Port boxes.  For any
  protocols that your Squid does not support, leave the fields blank.

  Take a look at this screen shot  if the
  instructions confused you.

  On the same configuration window, you'll find a button to bring up the
  exclusion list dialog box, which will let you enter some hosts or
  domains that you don't want fetched via proxy.  It should be self-
  explanatory, but you might look at this screen shot
   just for fun anyway.


  5.8.  Opera 2.12 proxy configuration

  Select Proxy Servers... from the Preferences menu.  Check each
  protocol that your Squid server supports (by default, HTTP, FTP, and
  Gopher) and enter the Squid server's address as hostname:port (e.g.
  mycache.example.com:3128 or 123.45.67.89:3128).  Click on Okay to
  accept the setup.


  Notes:

  o  Opera 2.12 doesn't support gopher on its own, but requires a proxy;
     therefore Squid's gopher proxying can extend the utility of your
     Opera immensely.

  o  Unfortunately, Opera 2.12 chokes on some HTTP requests, for example
     abuse.net .  At the moment I think it
     has something to do with cookies.  If you have trouble with a site,
     try disabling the HTTP proxying by unchecking that protocol in the
     Preferences|Proxy Servers... dialogue.  Opera will remember the
     address, so reenabling is easy.


       --Hume Smith 



  5.9.  How do I tell Squid to use a specific username for FTP urls?

  Insert your username in the host part of the URL, for example:

          ftp://joecool@ftp.foo.org/


  Squid should then prompt you for your account password.  Alterna-
  tively, you can specify both your username and password in the URL
  itself:

          ftp://joecool:secret@ftp.foo.org/


  However, we certainly do not recommend this, as it could be very easy
  for someone to see or grab your password.


  5.10.  Configuring Browsers for WPAD

  by Mark Reynolds 

  You may like to start by reading the Expired Internet-Draft
   that describes WPAD.


  After reading the 8 steps below, if you don't understand any of the
  terms or methods mentioned, you probably shouldn't be doing this.
  Implementing wpad requires you to fully understand:

  1. web server installations and modifications.

  2. squid proxy server (or others) installation etc.

  3. Domain Name System maintenance etc.

     Please don't bombard the squid list with web server or dns
     questions. See your system administrator, or do some more research
     on those topics.


  This is not a recommendation for any product or version. As far as I
  know IE5 is the only browser out now implementing wpad. I think wpad
  is an excellent feature that will return several hours of life per
  month.  Hopefully, all browser clients will implement it as well. But
  it will take years for all the older browsers to fade away though.


  I have only focused on the domain name method, to the exclusion of the
  DHCP method. I think the dns method might be easier for most people.
  I don't currently, and may never, fully understand wpad and IE5, but
  this method worked for me. It may work for you.


  But if you'd rather just have a go ...

  1. Create a standard ``netscape auto    proxy config file''.  The
     sample provided there is more than adequate to get you going.  No
     doubt all the other load balancing and backup scripts will be fine
     also.

  2. Store the resultant file in the document root directory of a handy
     web server as wpad.dat (Not proxy.pac as you may have previously
     done.)


     Andrei Ivanov  notes that you should
     be able to use an HTTP redirect if you want to store the wpad.dat
     file somewhere else.  You can probably even redirect wpad.dat to
     proxy.pac:

     Redirect /wpad.dat http://racoon.riga.lv/proxy.pac



  3. If you do nothing more, a url like
     http://www.your.domain.name/wpad.dat should bring up the script
     text in your browser window.

  4. Insert the following entry into your web server mime.types file.
     Maybe in addition to your pac file type, if you've done this
     before.
             application/x-ns-proxy-autoconfig       dat


  And then restart your web server, for new mime type to work.

  5. Assuming Internet Explorer 5, under Tools, Internet Options,
     Connections, Settings or Lan Settings, set ONLY Use Automatic
     Configuration Script to be the URL for where your new wpad.dat file
     can be found.  i.e.  http://www.your.domain.name/wpad.dat Test that
     that all works as per your script and network.  There's no point
     continuing until this works ...

  6. Create/install/implement a DNS record so that wpad.your.domain.name
     resolves to the host above where you have a functioning auto config
     script running. You should now be able to use
     http://wpad.your.domain.name/wpad.dat as the Auto Config Script
     location in step 5 above.

  7. And finally, go back to the setup screen detailed in 5 above, and
     choose nothing but the Automatically Detect Settings option,
     turning everything else off. Best to restart IE5, as you normally
     do with any Microsoft product... And it should all work. Did for me
     anyway.

  8. One final question might be 'Which domain name does the client
     (IE5) use for the wpad... lookup?' It uses the hostname from the
     control panel setting.  It starts the search by adding the hostname
     "WPAD" to current fully-qualified domain name.  For instance, a
     client in a.b.Microsoft.com would search for a WPAD server at
     wpad.a.b.microsoft.com. If it could not locate one, it would remove
     the bottom-most domain and try again; for instance, it would try
     wpad.b.microsoft.com next. IE 5 would stop searching when it found
     a WPAD server or reached the third-level domain,
     wpad.microsoft.com.



  Anybody using these steps to install and test, please feel free to
  make notes, corrections or additions for improvements, and post back
  to the squid list...


  There are probably many more tricks and tips which hopefully will be
  detailed here in the future. Things like wpad.dat files being served
  from the proxy server themselves, maybe with a round robin dns setup
  for the WPAD host.


  5.11.  Configuring Browsers for WPAD with DHCP

  You can also use DHCP to configure browsers for WPAD.  This technique
  allows you to set any URL as the PAC URL.  For ISC DHCPD, enter a line
  like this in your dhcpd.conf file:

          option wpad code 252 = text;
          option wpad "http://www.example.com/proxy.pac";



  Replace the hostname with the name or address of your own server.

  Ilja Pavkovic notes that the DHCP mode does not work reliably with
  every version of Internet Explorer. The DNS name method to find
  wpad.dat is more reliable.


  Another user adds that IE 6.01 seems to strip the last character from
  the URL.  By adding a trailing newline, he is able to make it work
  with both IE 5.0 and 6.0:<

          option wpad "http://www.example.com/proxy.pac\n";



  5.12.  IE 5.0x crops trailing slashes from FTP URL's

  by Reuben Farrelly 

  There was a bug in the 5.0x releases of Internet Explorer in which IE
  cropped any trailing slash off an FTP URL.  The URL showed up
  correctly in the browser's ``Address:'' field, however squid logs show
  that the trailing slash was being taken off.

  An example of where this impacted squid if you had a setup where squid
  would go direct for FTP directory listings but forward a request to a
  parent for FTP file transfers.  This was useful if your upstream proxy
  was an older version of Squid or another vendors software which
  displayed directory listings with broken icons and you wanted your own
  local version of squid to generate proper FTP directory listings
  instead.  The workaround for this is to add a double slash to any
  directory listing in which the slash was important, or else upgrade to
  IE 5.5.  (Or use Netscape)


  5.13.  IE 6.0 SP1 fails when using authentication

  When using authentication with Internet Explorer 6 SP1, you may
  encounter issues when you first launch Internet Explorer.  The problem
  will show itself when you first authenticate, you will receive a "Page
  Cannot Be Displayed" error. However, if you click refresh, the page
  will be correctly displayed.


  This only happens immediately after you authenticate.


  This is not a Squid error or bug.   Microsoft broke the Basic
  Authentication when they put out IE6 SP1.


  There is a knowledgebase article (KB 331906
  )
  regarding this issue, which contains a link to a downloadable "hot
  fix." They do warn that this code is not "regression tested" but so
  far there have not been any reports of this breaking anything else.
  The problematic file is wininet.dll. Please note that this hotfix is
  included in the latest security update.


  Lloyd Parkes notes that the article references another article, KB
  312176 .  He says that you must not have the registry entry that KB
  312176 encourages users to add to their registry.


  According to Joao Coutinho, this simple solution also corrects the
  problem:

  o  Go to Tools/Internet

  o  Go to Options/Advanced

  o  UNSELECT "Show friendly HTTP error messages" under Browsing.


  Another possible workaround to these problems is to make the
  ERR_CACHE_ACCESS_DENIED larger than 1460 bytes. This should trigger IE
  to handle the authentication in a slightly different manner.



  6.  Squid Log Files

  The logs are a valuable source of information about Squid workloads
  and performance. The logs record not only access information, but also
  system configuration errors and resource consumption (eg, memory, disk
  space). There are several log file maintained by Squid. Some have to
  be explicitely activated during compile time, others can safely be
  deactivated during run-time.


  There are a few basic points common to all log files. The time stamps
  logged into the log files are usually UTC seconds unless stated
  otherwise.  The initial time stamp usually contains a millisecond
  extension.


  6.1.  squid.out

  If you run your Squid from the RunCache script, a file squid.out
  contains the Squid startup times, and also all fatal errors, e.g. as
  produced by an assert() failure. If you are not using RunCache, you
  will not see such a file.


  6.2.  cache.log

  The cache.log file contains the debug and error messages that Squid
  generates. If you start your Squid using the default RunCache script,
  or start it with the -s command line option, a copy of certain
  messages will go into your syslog facilities. It is a matter of
  personal preferences to use a separate file for the squid log data.


  From the area of automatic log file analysis, the cache.log file does
  not have much to offer. You will usually look into this file for
  automated error reports, when programming Squid, testing new features,
  or searching for reasons of a perceived misbehaviour, etc.



  6.3.  useragent.log

  The user agent log file is only maintained, if


  1. you configured the compile time --enable-useragent-log option, and

  2. you pointed the useragent_log configuration option to a file.


  From the user agent log file you are able to find out about
  distributation of browsers of your clients. Using this option in
  conjunction with a loaded production squid might not be the best of
  all ideas.

  6.4.  store.log

  The store.log file covers the objects currently kept on disk or
  removed ones. As a kind of transaction log it is ususally used for
  debugging purposes. A definitive statement, whether an object resides
  on your disks is only possible after analysing the complete log file.
  The release (deletion) of an object may be logged at a later time than
  the swap out (save to disk).


  The store.log file may be of interest to log file analysis which looks
  into the objects on your disks and the time they spend there, or how
  many times a hot object was accessed. The latter may be covered by
  another log file, too. With knowledge of the cache_dir configuration
  option, this log file allows for a URL to filename mapping without
  recursing your cache disks. However, the Squid developers recommend to
  treat store.log primarily as a debug file, and so should you, unless
  you know what you are doing.


  The print format for a store log entry (one line) consists of eleven
  space-separated columns, compare with the storeLog() function in file
  src/store_log.c:


      "%9d.%03d %-7s %02d %08X %4d %9d %9d %9d %s %d/%d %s %s\n"



     time
        The timestamp when the line was logged in UTC with a millisecond
        fraction.


     action
        The action the object was sumitted to, compare with
        src/store_log.c:


        o  CREATE Seems to be unused.

        o  RELEASE The object was removed from the cache (see also
           ``file number'').

        o  SWAPOUT The object was saved to disk.

        o  SWAPIN The object existed on disk and was read into memory.


     dir numer

        The cache_dir number this object was stored into, starting at 0
        for your first cache_dir line.


     file number

        The file number for the object storage file. Please note that
        the path to this file is calculated according to your cache_dir
        configuration.


        A file number of FFFFFFFF denominates "memory only" objects. Any
        action code for such a file number refers to an object which
        existed only in memory, not on disk.  For instance, if a RELEASE
        code was logged with file number FFFFFFFF, the object existed
        only in memory, and was released from memory.


     status
        The HTTP reply status code.


     datehdr

        The value of the HTTP "Date: " reply header.


     lastmod
        The value of the HTTP "Last-Modified: " reply header.


     expires

        The value of the HTTP "Expires: " reply header.


     type
        The HTTP "Content-Type" major value, or "unknown" if it cannot
        be determined.


     sizes
        This column consists of two slash separated fields:


        1. The advertised content length from the HTTP "Content-Length:
           " reply header.

        2. The size actually read.


        If the advertised (or expected) length is missing, it will be
        set to zero. If the advertised length is not zero, but not equal
        to the real length, the object will be realeased from the cache.


     method
        The request method for the object, e.g. GET.


     key

        The key to the object, usually the URL.


  The timestamp format for the columns ``Date'' to ``Expires'' are all
  expressed in UTC seconds. The actual values are parsed from the HTTP
  reply headers. An unparsable header is represented by a value of -1,
  and a missing header is represented by a value of -2.


  The column ``key'' usually contains just the URL of the object. Some
  objects though will never become public. Thus the key is said to
  include a unique integer number and the request method in addition to
  the URL.



  6.5.  hierarchy.log

  This logfile exists for Squid-1.0 only.  The format is

          [date] URL peerstatus peerhost



  6.6.  access.log

  Most log file analysis program are based on the entries in access.log.
  Currently, there are two file formats possible for the log file,
  depending on your configuration for the emulate_httpd_log option. By
  default, Squid will log in its native log file format. If the above
  option is enabled, Squid will log in the common log file format as
  defined by the CERN web daemon.


  The common log file format contains other information than the native
  log file, and less. The native format contains more information for
  the admin interested in cache evaluation.


  6.6.1.  The common log file format

  The Common Logfile Format
   is used by numerous HTTP servers. This format consists of the
  following seven fields:

          remotehost rfc931 authuser [date] "method URL" status bytes



  It is parsable by a variety of tools. The common format contains
  different information than the native log file format. The HTTP
  version is logged, which is not logged in native log file format.



  6.6.2.  The native log file format

  The native format is different for different major versions of Squid.
  For Squid-1.0 it is:

          time elapsed remotehost code/status/peerstatus bytes method URL



  For Squid-1.1, the information from the hierarchy.log was moved into
  access.log.  The format is:

          time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type



  For Squid-2 the columns stay the same, though the content within may
  change a little.


  The native log file format logs more and different information than
  the common log file format: the request duration, some timeout
  information, the next upstream server address, and the content type.
  There exist tools, which convert one file format into the other.
  Please mind that even though the log formats share most information,
  both formats contain information which is not part of the other
  format, and thus this part of the information is lost when converting.
  Especially converting back and forth is not possible without loss.

  squid2common.pl is a conversion utility, which converts any of the
  squid log file formats into the old CERN proxy style output. There
  exist tools to analyse, evaluate and graph results from that format.



  6.6.3.  access.log native format in detail

  It is recommended though to use Squid's native log format due to its
  greater amount of information made available for later analysis. The
  print format line for native access.log entries looks like this:


      "%9d.%03d %6d %s %s/%03d %d %s %s %s %s%s/%s %s"



  Therefore, an access.log entry usually consists of (at least) 10
  columns separated by one ore more spaces:


     time
        A Unix timestamp as UTC seconds with a millisecond resolution.
        You can convert Unix timestamps into something more human
        readable using this short perl script:

                #! /usr/bin/perl -p
                s/^\d+\.\d+/localtime $&/e;



     duration
        The elapsed time considers how many milliseconds the transaction
        busied the cache. It differs in interpretation between TCP and
        UDP:


        o  For HTTP/1.0, this is basically the time between accept() and
           close().

        o  For persistent connections, this ought to be the time between
           scheduling the reply and finishing sending it.

        o  For ICP, this is the time between scheduling a reply and
           actually sending it.

        Please note that the entries are logged after the reply finished
        being sent, not during the lifetime of the transaction.


     client address
        The IP address of the requesting instance, the client IP
        address. The client_netmask configuration option can distort the
        clients for data protection reasons, but it makes analysis more
        difficult. Often it is better to use one of the log file
        anonymizers.

        Also, the log_fqdn configuration option may log the fully
        qualified domain name of the client instead of the dotted quad.
        The use of that option is discouraged due to its performance
        impact.


     result codes

        This column is made up of two entries separated by a slash. This
        column encodes the transaction result:


        1. The cache result of the request contains information on the
           kind of request, how it was satisfied, or in what way it
           failed. Please refer to section ``Squid result codes'' for
           valid symbolic result codes.


           Several codes from older versions are no longer available,
           were renamed, or split. Especially the ERR_ codes do not seem
           to appear in the log file any more. Also refer to section
           ``Squid result codes'' for details on the codes no longer
           available in Squid-2.


           The NOVM versions and Squid-2 also rely on the Unix buffer
           cache, thus you will see less TCP_MEM_HITs than with a
           Squid-1.  Basically, the NOVM feature relies on read() to
           obtain an object, but due to the kernel buffer cache, no disk
           activity is needed.  Only small objects (below 8KByte) are
           kept in Squid's part of main memory.


        2. The status part contains the HTTP result codes with some
           Squid specific extensions. Squid uses a subset of the RFC
           defined error codes for HTTP. Refer to section ``status
           codes'' for details of the status codes recognized by a
           Squid-2.


     bytes
        The size is the amount of data delivered to the client. Mind
        that this does not constitute the net object size, as headers
        are also counted. Also, failed requests may deliver an error
        page, the size of which is also logged here.


     request method
        The request method to obtain an object. Please refer to section
        ``'' for available methods.  If you turned off log_icp_queries
        in your configuration, you will not see (and thus unable to
        analyse) ICP exchanges. The PURGE method is only available, if
        you have an ACL for ``method purge'' enabled in your
        configuration file.


     URL
        This column contains the URL requested. Please note that the log
        file may contain whitespaces for the URI. The default
        configuration for uri_whitespace denies whitespaces, though.


     rfc931
        The eigth column may contain the ident lookups for the
        requesting client. Since ident lookups have performance impact,
        the default configuration turns ident_loookups off. If turned
        off, or no ident information is available, a ``-'' will be
        logged.


     hierarchy code
        The hierarchy information consists of three items:


        1. Any hierarchy tag may be prefixed with TIMEOUT_, if the
           timeout occurs waiting for all ICP replies to return from the
           neighbours. The timeout is either dynamic, if the
           icp_query_timeout was not set, or the time configured there
           has run up.

        2. A code that explains how the request was handled, e.g. by
           forwarding it to a peer, or going straight to the source.
           Refer to section ``'' for details on hierarchy codes and
           removed hierarchy codes.

        3. The IP address or hostname where the request (if a miss) was
           forwarded.  For requests sent to origin servers, this is the
           origin server's IP address.  For requests sent to a neighbor
           cache, this is the neighbor's hostname.  NOTE: older versions
           of Squid would put the origin server hostname here.


     type
        The content type of the object as seen in the HTTP reply header.
        Please note that ICP exchanges usually don't have any content
        type, and thus are logged ``-''. Also, some weird replies have
        content types ``:'' or even empty ones.


  There may be two more columns in the access.log, if the (debug) option
  log_mime_headers is enabled In this case, the HTTP request headers are
  logged between a ``['' and a ``]'', and the HTTP reply headers are
  also logged between ``['' and ``]''. All control characters like CR
  and LF are URL-escaped, but spaces are not escaped! Parsers should
  watch out for this.


  6.7.  Squid result codes

  The TCP_ codes refer to requests on the HTTP port (usually 3128). The
  UDP_ codes refer to requests on the ICP port (usually 3130). If ICP
  logging was disabled using the log_icp_queries option, no ICP replies
  will be logged.


  The following result codes were taken from a Squid-2, compare with the
  log_tags struct in src/access_log.c:



     TCP_HIT
        A valid copy of the requested object was in the cache.


     TCP_MISS
        The requested object was not in the cache.


     TCP_REFRESH_HIT
        The requested object was cached but STALE. The IMS query for the
        object resulted in "304 not modified".


     TCP_REF_FAIL_HIT
        The requested object was cached but STALE. The IMS query failed
        and the stale object was delivered.


     TCP_REFRESH_MISS
        The requested object was cached but STALE. The IMS query
        returned the new content.


     TCP_CLIENT_REFRESH_MISS

        The client issued a "no-cache" pragma, or some analogous cache
        control command along with the request. Thus, the cache has to
        refetch the object.


     TCP_IMS_HIT

        The client issued an IMS request for an object which was in the
        cache and fresh.


     TCP_SWAPFAIL_MISS

        The object was believed to be in the cache, but could not be
        accessed.


     TCP_NEGATIVE_HIT
        Request for a negatively cached object, e.g. "404 not found",
        for which the cache believes to know that it is inaccessible.
        Also refer to the explainations for negative_ttl in your
        squid.conf file.


     TCP_MEM_HIT
        A valid copy of the requested object was in the cache and it was
        in memory, thus avoiding disk accesses.


     TCP_DENIED
        Access was denied for this request.


     TCP_OFFLINE_HIT
        The requested object was retrieved from the cache during offline
        mode. The offline mode never validates any object, see
        offline_mode in squid.conf file.


     UDP_HIT
        A valid copy of the requested object was in the cache.


     UDP_MISS
        The requested object is not in this cache.


     UDP_DENIED
        Access was denied for this request.


     UDP_INVALID
        An invalid request was received.

     UDP_MISS_NOFETCH

        During "-Y" startup, or during frequent failures, a cache in hit
        only mode will return either UDP_HIT or this code. Neighbours
        will thus only fetch hits.


     NONE
        Seen with errors and cachemgr requests.


  The following codes are no longer available in Squid-2:


     ERR_*
        Errors are now contained in the status code.


     TCP_CLIENT_REFRESH
        See: ``TCP_CLIENT_REFRESH_MISS''.


     TCP_SWAPFAIL
        See: ``TCP_SWAPFAIL_MISS''.


     TCP_IMS_MISS
        Deleted, ``TCP_IMS_HIT'' used instead.


     UDP_HIT_OBJ
        Hit objects are no longer available.


     UDP_RELOADING
        See: ``UDP_MISS_NOFETCH''.


  6.8.  HTTP status codes

  These are taken from RFC 2616 
  and verified for Squid. Squid-2 uses almost all codes except 307
  (Temporary Redirect), 416 (Request Range Not Satisfiable), and 417
  (Expectation Failed). Extra codes include 0 for a result code being
  unavailable, and 600 to signal an invalid header, a proxy error. Also,
  some definitions were added as for RFC 2518  (WebDAV).  Yes, there are really two entries for
  status code 424, compare with http_status in src/enums.h:



   000 Used mostly with UDP traffic.

   100 Continue
   101 Switching Protocols
  *102 Processing

   200 OK
   201 Created
   202 Accepted
   203 Non-Authoritative Information
   204 No Content
   205 Reset Content
   206 Partial Content
  *207 Multi Status

   300 Multiple Choices
   301 Moved Permanently
   302 Moved Temporarily
   303 See Other
   304 Not Modified
   305 Use Proxy
  [307 Temporary Redirect]

   400 Bad Request
   401 Unauthorized
   402 Payment Required
   403 Forbidden
   404 Not Found
   405 Method Not Allowed
   406 Not Acceptable
   407 Proxy Authentication Required
   408 Request Timeout
   409 Conflict
   410 Gone
   411 Length Required
   412 Precondition Failed
   413 Request Entity Too Large
   414 Request URI Too Large
   415 Unsupported Media Type
  [416 Request Range Not Satisfiable]
  [417 Expectation Failed]
  *424 Locked
  *424 Failed Dependency
  *433 Unprocessable Entity

   500 Internal Server Error
   501 Not Implemented
   502 Bad Gateway
   503 Service Unavailable
   504 Gateway Timeout
   505 HTTP Version Not Supported
  *507 Insufficient Storage

   600 Squid header parsing error



  6.9.  Request methods

  Squid recognizes several request methods as defined in RFC 2616
  . Newer versions of Squid
  (2.2.STABLE5 and above) also recognize RFC 2518  ``HTTP Extensions for Distributed Authoring --
  WEBDAV'' extensions.

   method    defined    cachabil.  meaning
   --------- ---------- ---------- -------------------------------------------
   GET       HTTP/0.9   possibly   object retrieval and simple searches.
   HEAD      HTTP/1.0   possibly   metadata retrieval.
   POST      HTTP/1.0   CC or Exp. submit data (to a program).
   PUT       HTTP/1.1   never      upload data (e.g. to a file).
   DELETE    HTTP/1.1   never      remove resource (e.g. file).
   TRACE     HTTP/1.1   never      appl. layer trace of request route.
   OPTIONS   HTTP/1.1   never      request available comm. options.
   CONNECT   HTTP/1.1r3 never      tunnel SSL connection.

   ICP_QUERY Squid      never      used for ICP based exchanges.
   PURGE     Squid      never      remove object from cache.

   PROPFIND  rfc2518    ?          retrieve properties of an object.
   PROPATCH  rfc2518    ?          change properties of an object.
   MKCOL     rfc2518    never      create a new collection.
   COPY      rfc2518    never      create a duplicate of src in dst.
   MOVE      rfc2518    never      atomically move src to dst.
   LOCK      rfc2518    never      lock an object against modifications.
   UNLOCK    rfc2518    never      unlock an object.



  6.10.  Hierarchy Codes

  The following hierarchy codes are used with Squid-2:

     NONE
        For TCP HIT, TCP failures, cachemgr requests and all UDP
        requests, there is no hierarchy information.


     DIRECT
        The object was fetched from the origin server.


     SIBLING_HIT
        The object was fetched from a sibling cache which replied with
        UDP_HIT.


     PARENT_HIT
        The object was requested from a parent cache which replied with
        UDP_HIT.


     DEFAULT_PARENT
        No ICP queries were sent. This parent was chosen because it was
        marked ``default'' in the config file.


     SINGLE_PARENT
        The object was requested from the only parent appropriate for
        the given URL.


     FIRST_UP_PARENT
        The object was fetched from the first parent in the list of
        parents.



     NO_PARENT_DIRECT
        The object was fetched from the origin server, because no
        parents existed for the given URL.


     FIRST_PARENT_MISS
        The object was fetched from the parent with the fastest
        (possi