Bandwidth Limiting with IP Masquerade - Howto

Contents
Introduction
Requirements
Setting Bandwidth
Marking Packets for Limiting
Bandwidth Monitoring
Conclusion

Introduction

This document is meant for IP Masquerade users who want to limit specific host's bandwidth.
The example made throughout the document is a aDSL line (640Kbits download / 160Kbits upload) where
the DHCP hosts of the subnet are bandwidth limited and also forced through a cachine proxy.

My router specs are as follows:

Distribution:
Debian 2.2 (stable)
Technically, its Debian 3.0 (unstable), buy only the sysinit scripts, bind, sendmail, dhcp, nfs, samba and X are kept at the bleeding edge. (All the major 'Services').
Hardware:
CPU: 466Mhz Celeron
RAM: 192MB
DISK: 40GB 7200 RPM Maxtor ATA100, 2GB IBM 5400 RPM ATA33

Services:
DHCP Server
DNS Server
Mail Server
IP Masquerade (port fowarding also)
Caching Proxy (Squid)
Samba server
NFS Server

Requirements

The requirements here will only pertain to setting up bandwidth limiting and NOT IP Masquerading, DHCP Server, NFS Server, etc.

Kernel Requirements:

All IPTABLES support needed for IP Masquerade plus
CONFIG_IP_NF_CONNTRACK
CONFIG_IP_NF_TARGET_MARK --> This is for marking packets. We are going to mark the packets we want limited with this

QoS and/or fair queueing
CONFIG_NET_SCH_CBQ
CONFIG_NET_CLS_FW

Now I usually compile all the IPTABLES and QoS and/or fair queueing stuff as modules, but lsmod only shows those above as in use.

Software Requirements:

IPTABLES --> http://www.iptables.org/
iproute2 --> ftp://ftp.inr.ac.ru/ip-routing/ --> Most distributions come with this defaultly installed

Setting Bandwidth

I set my bandwidth in a shell script for the purpose of having it set on boot time and for ease and stopping and starting.

#!/bin/bash
#
# All Rates are in Kbits, so in order to gets Bytes divide by 8
# e.g. 25Kbps == 3.125KB/s
#
TC=/sbin/tc
DNLD=150Kbit # DOWNLOAD Limit
DWEIGHT=15Kbit # DOWNLOAD Weight Factor ~ 1/10 of DOWNLOAD Limit
UPLD=25KBit # UPLOAD Limit
UWEIGHT=2Kbit # UPLOAD Weight Factor

tc_start() {

    $TC qdisc add dev eth0 root handle 11: cbq bandwidth 100Mbit avpkt 1000 mpu 64
    $TC class add dev eth0 parent 11:0 classid 11:1 cbq rate $DNLD weight $DWEIGHT allot 1514 prio 1 avpkt 1000 bounded
    $TC filter add dev eth0 parent 11:0 protocol ip handle 4 fw flowid 11:1

    $TC qdisc add dev eth1 root handle 10: cbq bandwidth 10Mbit avpkt 1000 mpu 64
    $TC class add dev eth1 parent 10:0 classid 10:1 cbq rate $UPLD weight $UWEIGHT allot 1514 prio 1 avpkt 1000 bounded
    $TC filter add dev eth1 parent 10:0 protocol ip handle 3 fw flowid 10:1

}

tc_stop() {

    $TC qdisc del dev eth0 root
    $TC qdisc del dev eth1 root

}

tc_restart() {

    tc_stop
    sleep 1
    tc_start

}

tc_show() {

    echo ""
    echo "eth0:"
    $TC qdisc show dev eth0
    $TC class show dev eth0
    $TC filter show dev eth0
    echo ""

    echo "eth1:"
    $TC qdisc show dev eth1
    $TC class show dev eth1
    $TC filter show dev eth1
    echo ""

}

case "$1" in

start)

    echo -n "Starting bandwidth shaping: "
    tc_start
    echo "done"
    ;;

stop)

    echo -n "Stopping bandwidth shaping: "
    tc_stop
    echo "done"
    ;;

restart)

    echo -n "Restarting bandwidth shaping: "
    tc_restart
    echo "done"
    ;;

show)

tc_show()
;;

*)

    echo "Usage: /etc/init.d/tc.sh {start|stop|restart|show}"
    ;;

esac

exit 0

Now lets go through that.
At the top are some variables which make it easier to change the entire script with on change.
THIS SCRIPT IS ONLY ** ONE ** RULESET.
You can add as many rules as you like. I don't run a ISP, I only needed to restrict a few people.

This scripts assumes 2 things.
eth0 ==> is local network e.g. 192.168.0.0/24
eth1 ==> is Internet network e.g. Your ISP

Stop, Restart, and Show are self explanitory. I will not explain them.

Start:

$TC qdisc add dev eth0 root handle 11: cbq bandwidth 100Mbit avpkt 1000 mpu 64
$TC class add dev eth0 parent 11:0 classid 11:1 cbq rate $DNLD weight $DWEIGHT allot 1514 prio 1 avpkt 1000 bounded
$TC filter add dev eth0 parent 11:0 protocol ip handle 4 fw flowid 11:1

$TC qdisc add dev eth1 root handle 10: cbq bandwidth 10Mbit avpkt 1000 mpu 64
$TC class add dev eth1 parent 10:0 classid 10:1 cbq rate $UPLD weight $UWEIGHT allot 1514 prio 1 avpkt 1000 bounded
$TC filter add dev eth1 parent 10:0 protocol ip handle 3 fw flowid 10:1

First the top 3 three lines are for the Download bandwidth.

The first line creates and "PARENT" qdisc:
--> $TC qdisc add dev eth0 root handle 11: cbq bandwidth 100Mbit avpkt 1000 mpu 64
man tc for more information, but basically its the device definition. 100Mbit Netgear card.
Next line creates the child qdisc with will have the actual download limit in it.
--> $TC class add dev eth0 parent 11:0 classid 11:1 cbq rate $DNLD weight $DWEIGHT allot 1514 prio 1 avpkt 1000 bounded
man tc and friends again if you need any more definition than that. Personally I don't know what all it means or any optimal settings.
I just used the suggested man page defaults.
The last line and VERY important line, sets the handle on the child qdisc.
It sets the handle to 4. Later we will use iptables to MARK the download packets with a 4 mark, and thats how the child qdisc will know
which packets to limit and which to let alone.

The Bottom three lines are for the uploading bandwidth.
Its the same as the download, but the device has changed from eth0 to eth1, the variables have changed from $DNLD to UPLD etc, and
the handle has changed from 4 to 3.

Marking Packets for Limiting

Ok now you are ready to mark packets as they come through in order to limit them.
I just added the next few lines to my rc.firewall.

# Mark packets to route
# Upload marking
$IPTABLES -t mangle -A FORWARD -s 192.168.0.128/29 -j MARK --set-mark 3
$IPTABLES -t mangle -A FORWARD -s 192.168.0.6 -j MARK --set-mark 3

# Download marking
$IPTABLES -t mangle -A POSTROUTING -s ! 192.168.0.0/24 -d 192.168.0.128/29 -j MARK --set-mark 4
$IPTABLES -t mangle -A POSTROUTING -s ! 192.168.0.0/24 -d 192.168.0.6 -j MARK --set-mark 4

Here I mark the following hosts:
192.168.0.6
192.168.0.128/29 =>
        192.168.0.128
        192.168.0.129
        192.168.0.130
        192.168.0.131
        192.168.0.132
        192.168.0.133
        192.168.0.134
        192.168.0.135

I hope no one needs help with Subnet masks. :)
Downloads are marked 4 and uploads are marked 3

29 means 29 out of the 32 bits are marked. i.e. 11111111.11111111.11111111.11111000 is the subnet mask ==> 255.255.255.248 which means 00000111 hosts allowed in subnet or 8 hosts: 0 through 7 + 192.168.0.128 SIMPLE.

Bandwidth Monitoring

Freshmeat.
These following tools are recommended:
bwm ==> very simple, ncurses based, for quick and easy overall network summary.
iptraf ==> very robust, ncurses based, my favorite, without kernel patch it lets you monitor specific host based on MAC Addresses
connmon ==> ncurses and gtk interfaces. With kernel patch you can monitor individual ip bandwidths

Conclusion

It took me a long time to figure this crap out.
I learned plenty from the cbq-init script from freshmeat. thanx.
Email if you have anymore questions, fixes, complaints.

Written by Joe Roback <joe@roback.cc >