Sunday, 1 February 2009

Hard drive throttling with VMware

wow its been, yet again waay to long since ive updated this, my excuse is that i've been wating till I had something unique.

In reality maybe this blogging this isnt for me :)

Anyway I've been working today with a client that had problems with a voip server running asterisk. The problem wasnt asterisk however it was the VMware server running on it :S.

One of the VM's is a file server that can get involved in heavy i/o from time to time, which when it does means that calls are heavily delayed from getting through to the Voip phones (about 5 secs).

The solution was to use "ionice"

You need a kernel of 2.6.13+
run this to see if you have it
-------------------------------------------------------------------
for d in /sys/block/sd[a-z]/queue/scheduler; do echo "$d => $(cat $d)" ; done
-------------------------------------------------------------------
output should be something like:
-------------------------------------------------------------------
/sys/block/sda/queue/scheduler => noop anticipatory deadline [cfq]
/sys/block/sdb/queue/scheduler => noop anticipatory deadline [cfq]
/sys/block/sdc/queue/scheduler => noop anticipatory deadline [cfq]
-------------------------------------------------------------------

ionice has lots too it (man ionice for more info)

Basically I wrote a little script to take input from a file which looks like this:

vmlist.txt:
---------------------------------------
win2003.vmx,3
Virtual Center.vmx,3
---------------------------------------

Script:

ionice.sh
---------------------------------------
#/bin/bash
#

OLDIFS=$IFS

cat $1 | while read LINE
do
IFS=,
# Set positional parameters - $1 is part of process we trigger on, $2 is renice value
set $LINE
IFS=$OLDIFS
# Now get PID for vmware-vmx-process
PID=$( ps -ef | grep "vmware-vmx" | grep -v grep | grep "$1" | tr -s ' ' ' ' | cut -d ' ' -f 2 )
if [ "$PID" == "" ] ; then
echo Failed to find vmware process for "$1" >&2
else
if ! ionice -c$2 -p $PID; then
echo Failed to renice pid $PID to $2 >&2
fi
fi
done

---------------------------------------


Run it like this-
---------------------------------------

./ionice.sh vmlist.txt



This then allowed the box to run disk defrag on all the VM's and still serve voip calls :)


(io)nice eh?

:)


REFs:
http://www.cyberciti.biz/tips/linux-set-io-scheduling-class-priority.html
http://communities.vmware.com/message/1084148#1084148

1 comment:

Andres Murillo said...

Another way to fix the issue is to edit /etc/init.d/vmware and change the following line:

vmware_bg_exec "`vmware_product_name` Host Agent" \

to:

vmware_bg_exec "`vmware_product_name` Host Agent" ionice -c 3 \