1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

bash script to check isp bandwidth and latency

Discussion in 'Other Scripting Languages' started by MaDeuce, Nov 30, 2010.

  1. MaDeuce

    MaDeuce Newbie

    Joined:
    Oct 24, 2008
    Messages:
    45
    Likes Received:
    16
    Location:
    Austin, TX
    I've been having ISP problems and wanted to monitor bandwidth and latency over a period of time. I couldn't find anything to do it, so I ended up writing a quick script to do it using speedtest.net hosts. I just chose ten of their hosts scattered across the US, but you can easily add others. Figured I'd share it in case it would be useful to anyone else.

    --Ma

    Code:
    [SIZE="2"][FONT="Lucida Console"]
    #!/bin/bash
    
    # The MIT License
    #
    # Copyright (c) 2010, MaDeuce
    #
    # Permission is hereby granted, free of charge, to any person obtaining a copy
    # of this software and associated documentation files (the "Software"), to deal
    # in the Software without restriction, including without limitation the rights
    # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    # copies of the Software, and to permit persons to whom the Software is
    # furnished to do so, subject to the following conditions:
    #
    # The above copyright notice and this permission notice shall be included in
    # all copies or substantial portions of the Software.
    # 
    # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
    # THE SOFTWARE.
    
    # This command measures latency and bandwidth of an internet connection from the command line.
    # It uses hosts that participate in speedtest.net.  Output is in the form of a csv file.
    # As designed, speedtest.net can be used to test latency and bandwidth only from a browser
    # and only then, manually.  I wanted to be able to test latency and bandwidth repeatedly
    # via cron, or similar, and to record historical data, so I created this script.  Output
    # is to stdout. If I'd have known at the beginning that I'd write this much code, I'd
    # have done it in python.
    
    # These are some participating hosts.  If you want to use a host from another location, you will
    # have to run the test from your browser and capture headers to determine the correct URL. Or, if
    # you are a flash hacker, you can get all of them from the swf file used to implement the test.
    # Note that URLs can differ between hosts, so you do need to determine the correct one to use
    # on a case by case basis.
    #
    AUSTIN_TX="austin.doublehorncommunications.com/speedtest"
    SANJOSE_CA="speedtest.smugmug.net"
    PALOALTO_CA="66.201.42.23/speedtest"
    MIAMI_FL="speedtest-cl.terra.com/speedtest"
    BOSTON_MA="64.119.128.142:9091/speedtest"
    RENTON_WA="speedtest.parcomweb.com/speedtest"
    CHICAGO_IL="chi.fastsoft.net/speedtest"
    CORVALIS_OR="speedtest.peakinternet.com/speedtest"
    RESTON_VA="speed1.iad2.inforelay.net/speedtest"
    # PORTLAND_OR="speed.opusnet.com/speedtest"
    # opusnet uses aspx for upload, as opposed to php. i don't want to implement aspx support, so don't use it
    # i'm sure there are other windows hosts. if you add hosts to this and see HTML output from upload(),
    # it means that the host is windows.  you should skip that host or implement aspx in addition to php.
    HOSTS="$AUSTIN_TX $PALOALTO_CA $MIAMI_FL $BOSTON_MA $RENTON_WA $CHICAGO_IL $CORVALIS_OR $RESTON_VA"
    
    # output a timestamp
    function tm {
        date "+%m%d,%H:%M:%S," | tr -d "\n"
    }
    
    function r13 {
        # 13 digit random number
        echo "${RANDOM}${RANDOM}${RANDOM}" | cut -c -13 -
    }
    
    function title {
    cat<<EOF
    #p = ping record
    #p,***d,HH:MM:SS,min,avg,max,stddev
    #l = latency record
    #l,***d,HH:MM:SS,host,totalT,namelookupT,connectT,starttransferT,size,speed
    #d = download bandwidth record
    #d,***d,HH:MM:SS,host,totalT,namelookupT,connectT,starttransferT,size,speed
    #u = download bandwidth record
    #u,***d,HH:MM:SS,host,totalT,namelookupT,connectT,starttransferT,size,speed
    EOF
    }
    
    # Redundancy in the next 4 functions should be factored out someday.  However,
    # I'll redo the script in python before I do that, I think.
    
    # test download speed
    function download {
        HST=$1
        SIZE=$2
        MAXSECS=$3
        FMT="%{time_total},%{time_namelookup},%{time_connect},%{time_starttransfer},%{size_download},%{speed_download}\n"
        RND=`r13`
        URL="$HST/random${SIZE}x${SIZE}.jpg?x=${RND}-1"
        HST=`echo $HST | sed -e s/\\\/.*$//g -e s/:.*$//g`
        echo -n 'd,'
        tm
        echo -n "${HST},"
        curl -m ${MAXSECS} -s -w "$FMT" $URL -o /dev/null
    }
    
    # test upload speed
    function upload {
        HST=$1
        SIZE=$2
        MAXSECS=$3
        # get random data from openssl. '-hex' will output two characters for each byte, therefore desired
        # size has to be divided by two. this is only approximate, but that's ok.
        DATA=`openssl rand $((SIZE/2)) -hex`
        FMT="%{time_total},%{time_namelookup},%{time_connect},%{time_starttransfer},%{size_upload},%{speed_upload}\n"
        RND=`r13`
        URL="$HST/upload.php?x=0.${RND}"
        HST=`echo $HST | sed -e s/\\\/.*$//g -e s/:.*$//g`
        echo -n 'u,'
        tm
        echo -n "${HST},"
        curl -m ${MAXSECS} -s -w "$FMT" -d $DATA -o /dev/null $URL
    }
    
    # test http latency by getting small file
    function latency {
        HST=$1
        MAXSECS=$2
        RND=`r13`
        URL="$HST/latency.txt?x=${RND}"
        HST=`echo $HST | sed -e s/\\\/.*$//g -e s/:.*$//g`
        echo -n 'l,'
        tm
        echo -n "${HST},"
        FMT="%{time_total},%{time_namelookup},%{time_connect},%{time_starttransfer},%{size_download},%{speed_download}\n"
        curl -m ${MAXSECS} -s -w "$FMT" $URL -o /dev/null
    }
    
    # test IP latency via ping
    function pingit {
        HST=$1
        MAXSECS=$2
        # just want hostname -- get rid of any http url fragment or port number
        HST=`echo $HST | sed -e s/\\\/.*$//g -e s/:.*$//g`
        echo -n "p,"
        tm
        echo -n "${HST},"
        # ping the host. hang on to the results so return code can be kept
        PV=`ping -t${MAXSECS} -c3 -Qq $HST`
        RC=$?
        if [ $RC -eq 0 ]
        then
    	# ping was successful
    	# output min/avg/max/stdev
    	echo $PV | fgrep round-trip | sed s/^.*=//g | sed -e 's/\//,/g' -e 's/ //g' -e 's/ms//g'
        else
    	# some speedtest.net hosts do not respond to pings
    	# ping was unsuccessful
    	echo "0,0,0,0"
        fi
    }
    
    # They use jpeg images with random content as a paylod for testing downloads.  The images are all
    # square (i.e., 'n x n').  There are nine fixed sizes of images.
    DSIZES="350 500 1000 1500 2000 2500 3000 3500 4000"
    
    # For some reason, they only use two sizes of payloads for upload testing.  25097 and 151325 bytes.
    # The payloads are simply random strings which are generated on the client side.  I don't think that their
    # exact size is important, nor do I think anything would prevent you from adding your own sizes.
    USIZES="25097 151325"
    
    title
    for HOST in $HOSTS
    do
        TMOUT=30                             # giveup after 30 seconds
        latency $HOST $TMOUT
        pingit  $HOST $TMOUT
        for SIZE in $DSIZES
        do
    	TMOUT=$((60*10))                 # giveup after 10 minutes
    	download $HOST $SIZE $TMOUT
        done
        for SIZE in $USIZES
        do
    	TMOUT=$((60*10))                 # giveup after 10 minutes
    	upload $HOST $SIZE $TMOUT
        done
    done
    [/FONT]
    [/SIZE]
    
     

    Attached Files:

    • Thanks Thanks x 2
    Last edited: Nov 30, 2010
  2. daltarak

    daltarak Newbie

    Joined:
    Oct 4, 2009
    Messages:
    22
    Likes Received:
    3
    Thank you for sharing. I noticed something in your r13 function, which $RANDOM doesn't always return 5 characters, maybe a better approach would be appending one more $RANDOM to echo.

    btw great work.
     
    • Thanks Thanks x 1
  3. MaDeuce

    MaDeuce Newbie

    Joined:
    Oct 24, 2008
    Messages:
    45
    Likes Received:
    16
    Location:
    Austin, TX
    nice catch. thanks!
     
  4. MaDeuce

    MaDeuce Newbie

    Joined:
    Oct 24, 2008
    Messages:
    45
    Likes Received:
    16
    Location:
    Austin, TX
    To fix the bug that daltarak identified, replace

    Code:
    function r13 {
        # 13 digit random number
        echo "${RANDOM}${RANDOM}${RANDOM}" | cut -c -13 -
    }
    with

    Code:
    function r13 {
        # 13 digit random number
        printf  '%013d' $((RANDOM*RANDOM))
    }

    Admittedly, this doesn't really create a full 13 digit random number, but this isn't crypto, and it's good enough to avoid name collisions in bandwidth testing.

    --Ma
     
  5. daltarak

    daltarak Newbie

    Joined:
    Oct 4, 2009
    Messages:
    22
    Likes Received:
    3
    After I ran the script I get some errors.

    1) ping -Qq says "bad value for TOS", did you mean -q to be silent?

    2) openssl rand function:

    Code:
    Usage: rand [options] num
    where options are
    -out file             - write to file
    -engine e             - use engine e, possibly a hardware device.
    -rand file:file:... - seed PRNG from files
    -base64               - encode output
    Version: OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
    Running CentOS 5.5
     
  6. MaDeuce

    MaDeuce Newbie

    Joined:
    Oct 24, 2008
    Messages:
    45
    Likes Received:
    16
    Location:
    Austin, TX
    >>daltarak

    I've only tested on OS-X; there are some differences between us.

    1) Yes, '-Qq' is supposed to be quiet. No idea about the 'Bad value for TOS', but see a) below. Here's the intent, so you can adjust for CentOS:

    a) I did not want ping to hang forever if it couldn't contact a host, so I set a timeout value via '-t'. On OS-X, the arg to '-t' is the max seconds to wait. You can either remove '-t${MAXSECS}' alltogether, or you can change it to work for CentOS.

    b) The only thing that's really important is that on OS-X, a ping result looks like the following:

    round-trip min/avg/max/stddev = 92.616/92.770/92.972/0.149 ms

    All the 'pingit' function really does is ping the host, grab the ping output, and then reformat into csv. Whatever you have to do to make that happen on your system should be fine.

    2) Here's the synopsis for 'openssl rand' on OS-X:

    openssl rand [-out file] [-rand file(s)] [-base64] [-hex] num

    'num' is the number of random bytes to output. All I'm doing here is getting SIZE/2 random bytes in hex format. Each byte prints as two hex characters, thus the SIZE/2.

    The random bytes are just the payload that is sent to the server for upload testing. You can use the '-base64' on CentOS to create printable random data, but it will be a little trickier to get the right size output. Or, there's no reason that you can't send binary data for the payload, so you could just 'openssl rand num' and leave it at that. In fact, that's probably the way I should have done it in the first place.

    --Ma
     
  7. hiderightnow

    hiderightnow Junior Member

    Joined:
    Jul 19, 2010
    Messages:
    104
    Likes Received:
    22
    Very nice, got my own script doing more or less what yours is. Thanks for the opportunity to a comparison between the results. Good job and thanks for sharing.
     
  8. crashus

    crashus Junior Member

    Joined:
    Feb 26, 2012
    Messages:
    196
    Likes Received:
    98
    If you have ability to utilize a LAMP you can install cacti and it will do the job for you
     
  9. member8200

    member8200 Regular Member

    Joined:
    Aug 9, 2014
    Messages:
    469
    Likes Received:
    33

    Hi! Thank you for sharing. This would be helpful. Modified some of the codes but still working fine. :)