Setting up a transparent pass-through proxy with iptables

Update: Part 2 for https posted in separate post!

So for a very long now I’ve had a nagging issue with proxies. My primary source of internet is through my college HTTP Proxy and this adds a couple of issues whenever I am dealing with applications that don’t have proxy support coded in them. I have this issue often both on my laptop as well as on my android tablet (Youtube streaming!). Its a very distressing situation and I’ve always wanted to set-up a transparent proxy solution which could re-direct the traffic out of such applications to a sort of secondary proxy server which can interpret the requests and forward them to my college proxy server. Recently I managed to get this working πŸ˜€

The main tool used for this was iptables. For those of you who haven’t heard of iptables at a glance it is a flexible firewall which is now part of the Linux kernel by default. But iptables is actually much more powerful and flexible than just a simple firewall to block ports. iptables is capable of doing complicated re-routing of packets on several criteria. So much so that a lot of routers running linux (Yes quite a lot of them run Linux) use iptables to manage packets.

In my current setup I have a router which is connected to the IIIT LAN which has access to the proxy server. And laptop is connected to the router. I managed to install DD-WRT which is a linux based firmware for routers. So I can now SSH to my router and mess with the iptables on it. The idea was to redirect all traffic on port 80 which goes through the router back to a server running on my laptop.

iptables has rules across 5 tables : filter,nat,mangle,raw,security. The default table is filter which as its name suggests is used to filter traffic and block certain traffic etc. The nat table which is what we are interested in has to do with actual routing of a tcp stream when a new connection is created. In each of these tables iptables has a standard set of chains like “PREROUTING”,”POSTROUTING” in nat , which corresponds to sub-categories of packets. In this case before routing,etc.

Here are the rules I used based off DD-WRT Wiki – Transparent Web Proxy
PROXY_IP=192.168.128.2 # My Laptop's IP Adress
PROXY_PORT=1234 # Port number to redirect traffic to
LAN_IP=`nvram get lan_ipaddr` # This gets the IP Address of the router
LAN_NET=$LAN_IP/`nvram get lan_netmask` # 192.168.128.1/255.255.255.0
iptables -t nat -A PREROUTING -i br0 -s $LAN_NET -d $LAN_NET -p tcp --dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -i br0 -p tcp --dport 80 -j DNAT --to $PROXY_IP:$PROXY_PORT
iptables -t nat -I POSTROUTING -o br0 -s $LAN_NET -d $PROXY_IP -p tcp -j SNAT --to $LAN_IP
iptables -I FORWARD -i br0 -o br0 -s $LAN_NET -d $PROXY_IP -p tcp --dport $PROXY_PORT -j ACCEPT
iptables -t nat -I PREROUTING -i br0 -d 192.168.36.0/24 -j ACCEPT
iptables -t nat -I PREROUTING -i br0 -d 10.0.0.0/13 -j ACCEPT

So lets see what these mean.

  • The first rule is a rule which makes sure that all internal traffic on port 80 (from the router’s subnet to the router’s subnet) are not affected and continue to get processed normally.
  • The next rule is what actually re-routes traffic on port 80 to the my laptop on port 1234. the jump target “-j DNAT” tells iptables to modify the destination address and port of the packet matched by the criteria specified
  • The 3rd rule modifies the source address of the packet so that it appears as if it is coming from the router. This is necessary as otherwise traffic from my laptop when redirected back to my laptop will not be accepted by my laptop as it may consider it as illegal routing. (I am not completely sure about this one, but it didn’t work for me without it)
  • The 4th rule ensures that actual traffic intended directly for the PROXY_PORT on my laptop continues to reach my laptop.
  • The last 2 rules ensure that traffic directed to the IIIT Network are not re-directed to my laptop as I don’t need a proxy for them.

Note that iptables processes rules in chained fashion. so it stops with the first rule which matches. each command has a ‘-A’ or a ‘-I’ flag, which stands for Append at the end of the chain and Insert at the front.

This setup now ensures that all the traffic gets re-routed to a secondary proxy server I will run on my laptop. Initially I was planning on writing my own proxy handler to modify the HTTP headers as I get them to be compatible with the proxy. But eventually I decided its simpler and safer to go with an existing proxy server. I went with tinyproxy over well established ones like squid simply due to the tiny footprint of tinyproxy.  I simply had to add proxy.iiit.ac.in:8080 as an upstream proxy in tinyproxy’s configuration and voila , Its done!

A similar setup can more easily be done on a single system without the need for an external router. As we will simply have to re-direct outgoing traffic from the system on port 80 back to localhost. Although we now have transparent proxying this does not mean that every application will work. There’s still the major restriction that only HTTP requests on port 80 are handled. In my case the IIIT proxy server blocks other ports and proxying might not work in those cases. Keep in mind that all information about the target host is lost in this method and only the fact that the HTTP request actually contains the host name allows us to use this.

Hope you find this useful! I certainly did now that I can finally watch youtube videos on my android tablet and more importantly download updates for Minecraft πŸ˜€

Edit1 :
I came across a version of tinyproxy , pre-configured and built for the atheros architecture (the same as my router). You can find it at : https://downloads.openwrt.org/snapshots/trunk/ar71xx/packages/packages/tinyproxy_1.8.3-2_ar71xx.ipk Make sure this is the version compatible for your router, before trying it. Here’s the installation commands I used (on the router)
mkdir /tmp/tinyproxy
cd /tmp/tinyproxy
wget http://someserver/tinyproxy.ipk
#get the file here somehow, not proxy server won't work to get from original source
ipkg -d /tmp/tinyproxy tinyproxy.ipk
/tmp/tinyproxy/usr/sbin/tinyproxy -c tinyproxy.conf #specify ur config file somewhere

After a bit of tweaking and adding this on the startup script , we have it setup to work automatically on reboot from the router alone πŸ™‚

Edit 2 :
A lot of people where too lazy to come up with an iptables filter to work on a single laptop/computer. So here’s the iptable rules for the same:
#!/bin/bash
DEV=eth1
PROXY_IP=127.0.0.1
PROXY_PORT=1234
iptables -t nat -A OUTPUT -d 192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,127.0.0.1 -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -j DNAT --to $PROXY_IP:$PROXY_PORT

If you just want to get it working , without understanding any of it , I’ve made a all-in-one script just for you πŸ˜›
You can get it Here. It has tiny proxy included in it, and it automatically sets everything up including the iptables. Its currently configured for the IIIT proxy server , so you might want to change it depending on your proxy’s ip.

Relevant files posted at github repository

Advertisements
Posted in Uncategorized
13 comments on “Setting up a transparent pass-through proxy with iptables
  1. Sapiem says:

    Thanks for your blog comment..

    Please, I have a same situation. I use a laptop with windows and create in it a virtual hotspot. I connect to a proxy server via LAN, and connect my Android cel by WIFI to the virtual router hotspot. OK.

    I have my cel rooted and I need connect to internet by a proxy, so, I need do it by transparent proxy.

    My proxy server: 192.168.41.1:8080
    no pwd, no user

    How can I create iptables rules to obtein the connecion using ports 80, 443, etc…

    Please help me…

    Thanks

    • phinfinity says:

      The above method works using iptables which is available only in linux and will not work under windows since iptables is not available there. There might be an alternate method for windows which I am not aware of.
      On the other hand android itself is linux-based and since you have your device rooted there might be a way to setup the transparent proxying entirely on your phone. Several ROMs (cyanogen for e.g) come with iptables inbuilt, otherwise you will need to manually copy a binary version of iptables which will work on your rooted android device.

      The second probably more harder part is to get a proxy server running on your android device which will re-write HTTP headers accordingly before sending it to your actual proxy server (This is what tinyproxy does in my blog-post). This part will involve writing some sort of android application which I am unsure about, readymade components such as privoxy can probably simplify the task but this will require some considerable effort in terms of writing your own proxy server on android.

      After that its simply a matter of running similar iptables commands to redirect to the proxy server that you run on your device. Admittedly all this will take some effort and if you’re looking for a quick-fix you’ll have to search for existing android applications which will likely be paid. https://play.google.com/store/apps/details?id=com.mgranja.autoproxy_lite seems to be achieving the above result , however I have never used it myself and cannot say for sure.
      Hope you find this useful!

  2. Sapiem says:

    Thanks…
    I have kubuntu 12.10 too, but I can’t create the hotspot to connect the cel to it, and share the laptop connection to the proxy server by LAN.

    I use ProxyDroid, Autoproxy, ASProxy, etc, but none work. No body know why, but not work.

    I have access to iptables on android cel, but I add the chains, but don’t work neither…

    • phinfinity says:

      On Kubuntu you might want to try hostapd to host the wifi. (Ad-hoc connections usually don’t work with android). If you’re wifi card supports it , hostapd should be able to host a true master-mode hostpot. After that running tinyproxy and modifying the iptables (on kubuntu) should get your setup working.
      Alternatively when using windows and modifying iptables on android, I hope you have tinyproxy running somewhere? Possibly a windows version of it? Without tinyproxy (or an equivalent) the setup won’t work.

  3. Sapiem says:

    ProxyDroid and ASProxy add chains to iptables. For example, ASProxy work for only 10 sec, after that it down the service. So, the iptables work for only that time. ASProxy redirect to port 8080 and ProxyDroid (that work some time ago in my cel, but not now) to ports 8123, 8124, 8125.

    So I try add chains manually to the iptables to 8080, but don’t work. I think that the apk work with that internally, no only chaining the iptables to gloabl system..????

    I canΒ΄t understand how if ProxyDroid work very well for a lot of time, just one day, it stop working. I flash, reflash, rererereflash jajaja, root, reroot, rerereroot jajaja, but nothing.

    I have the ProxyDroid sourcecode but I can’t compile it becasuse I don’t have all import classes…

    About Kubuntu, I use very simple user level Linux, but if I have a reference, maybe I obtain that I will like.

    Thanks for all…

    I appreciate all help…

    • phinfinity says:

      The main thing to note here is that iptables chains are only one part of the work. You need a secondary proxy server which is capable of transparently forwarding requests to an upstream proxy. In my case I use tinyproxy. I’m guessing ProxyDroid,ASProxy etc. would come inbuilt with their own proxy servers. Which is why they chain their iptables to localhost:8080 (or 8123,8124 etc.) because they have a proxy server additionally running on the android device to handle the requests.
      The only two iptables rules that you need to add on android are :
      iptables -t nat -A OUTPUT -d 192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,127.0.0.1 -j ACCEPT
      iptables -t nat -A OUTPUT -p 6 –dport 80 -j DNAT –to YOUR_IP:YOUR_PORT
      here YOUR_IP and YOUR_PORT need to point to the secondary proxy server that you are running. I would suggest that you search for a version of tinyproxy for windows , and run it on your windows , then you can provide your windows IP (and port of tinyproxy running on windows) in the last iptables rule.

  4. Didn’t you faced problems in DNS resolution or do you guys have access to external dns servers.

    • phinfinity says:

      well DNS works over UDP and so can’t be tunnelled over a HTTP proxy. Usually set-ups involving a HTTP Proxy nearly always have a local DNS which resolves all IP addresses (as was my case) and so shouldn’t have any issues. Though the interesting thing is in order to use a HTTP proxy you don’t actually need a DNS server at all. So if you do have DNS issues (like your DNS does not resolve external names) you could actually run a local DNS server whose behaviours is to simply resolve all names to the ip address where the tinyproxy set-up runs (except local intranet names should still resolve correctly). This would serve the same purpose of what iptables does in the above set-up in catching packets. That would alleviate any issues in DNS not resolving external names.

  5. Pranav says:

    I was looking for this for more than an year now! I am in the exact same situation. Will try this and let you know.

    • Pranav says:

      Well, in openwrt I installed tinyproxy and was able to achieve the forwarding by HTTPS sites did not work.
      any possible reason/solutions for this?
      and I am now running DD-wrt again and I have ubuntu also.
      I tried various things like redsocks but I failed.

      What should I do now? flash openwrt and use tinyproxy again or do something with squid in my laptop? as I have never used squid before in my life and I am new tp linux.

      • phinfinity says:

        Yeah, the above setup only works for HTTP, HTTPS involves a bit more trickery because the HTTPS protocol is encrypted and you can’t read it transparently to obtain the hostname from the protocol (which tinyproxy does for HTTP requests right now).
        I have a rough working setup with a python script designed to run on a linux system (not dd-wrt since it is written in python): http://web.iiit.ac.in/~anish.shankar/phinfinity_downloads/transparent_https.py , but be warned that it is a very rough hacked up version. This one is standalone just for the HTTPS requests and does not require the rest of the instructions on this page, make sure to add iptables mentioned inside the python script.
        The above is written in python and so can’t run on the router, I’d have to re-write that in C and compile it for MIPS in order to get it running on my router (which is a bit painful as I haven’t been able to get hold of the right compiler toolchain)
        EDIT: this script , requires a working DNS already accessible. Without a DNS resolving external IP’s this won’t work. (You’d need a *MUCH* more convoluted harder setup for that IMO)

  6. […] proxy transparently. This post deals with extending the same for transparent HTTPS proxying. Click Here for my earlier post which deals with HTTP proxying. For a quick-fix solution and list of files […]

  7. Lukas says:

    Hi, I’m facing a similar problem, though I’m not using the router. I’m behind TGM proxy , connected via LAN on eth0 which goes to my laptop. Creating socks proxy tunneled with ssh via cntlm can bypass the proxy which is fine for my laptop. But I struggle to create a hotspot using wifi (wlan0) on my laptop and then route all the traffic from wlan0 to localhost:1234 and all the traffic arriving on localhost:1234 ( ssh tunnel ) to redirect to wlan0 where the hotspot is created. Any idea for a iptables config?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: