Python as a DTrace consumer using libdtrace
October 7th, 2011 • 3 CommentsSo DTrace is awesome – You can create nice analytics with it like Joyent does. But what I wanted is to access the output/aggregations from DTrace using Python to be able to parse the output as it comes. Using DTrace to Monitor Python or zones is already easy 🙂 Doing so it might be able to express up to date information to a (Cloud)Client (using of course sth. like OCCI)
So all I need is a Python based DTrace consumer which uses the dtrace library. To start with let’s reading the comments in the file /usr/include/dtrace.h – most notable is:
Note: The contents of this file are private to the implementation of the Solaris system and DTrace subsystem and are subject to change at any time without notice.
So we are already on our own – Next to that there is very limited documentation on writing DTrace consumers. A few quick searches might help you find some information. Probably the most up to date is to look into Bryan Cantrill’s consumer for node.js: https://github.com/bcantrill/node-libdtrace. And as mentioned a Python based consumer for libdtrace should be the goal of all this – so peeking to how others do it is probably a good idea :-P. For now let we will focus on a simple Hello World example.
First we need to understand how to use libdtrace – So let’s take a look at this diagram:
Source: http://www.macosinternals.com/images/stories/DTrace/drace_life_cycle.jpg
Following this life-cycle we can easily create some C code which will interface nicely with libdtrace. But since we can use it in C we can also use Python and ctypes to access the library. Here it is where we start the fun part.
To start with we will try to execute the following D script. It does nothing more than printing Hello World when executed using the dtrace command. But the output of this trace should now be received in Python code – so the output could be evaluated later on:
dtrace:::BEGIN {trace("Hello World");}
Now let’s write some python code – first we need to wrap some Structures which are defined in the dtrace.h file. Namely we will need dtrace_bufdata, dtrace_probedesc, dtrace_probedata and dtrace_recdesc. Since this is a blog post please refer to the source code at github for more details. We also need to define some types for callback functions – since we need a buffered writer, chew and chewrec functions as shown in the previous diagram:
CHEW_FUNC = CFUNCTYPE(c_int,
POINTER(dtrace_probedata),
POINTER(c_void_p))
CHEWREC_FUNC = CFUNCTYPE(c_int,
POINTER(dtrace_probedata),
POINTER(dtrace_recdesc),
POINTER(c_void_p))
BUFFERED_FUNC = CFUNCTYPE(c_int,
POINTER(dtrace_bufdata),
POINTER(c_void_p))
def chew_func(data, arg):
'''
Callback for chew.
'''
print 'cpu :', c_int(data.contents.dtpda_cpu).value
return 0
def chewrec_func(data, rec, arg):
'''
Callback for record chewing.
'''
if rec == None:
return 1
return 0
def buffered(bufdata, arg):
'''
In case dtrace_work is given None as filename - this one is called.
'''
print c_char_p(bufdata.contents.dtbda_buffered).value.strip()
return 0
The function called buffered will eventually write the Hello World string later on – The chew function will print out the CPU id.
With the basic stuff available new can load the libdtrace library and start doing some magic with it:
cdll.LoadLibrary("libdtrace.so")
LIBRARY = CDLL("libdtrace.so")
Now all there is left to do is follow the steps described in the diagram. First step is to get an handle an set some options:
# get dtrace handle
handle = LIBRARY.dtrace_open(3, 0, byref(c_int(0)))
# options
if LIBRARY.dtrace_setopt(handle, "bufsize", "4m") != 0:
txt = LIBRARY.dtrace_errmsg(handle, LIBRARY.dtrace_errno(handle))
raise Exception(c_char_p(txt).value)
Setting the bufsize option is important – otherwise DTrace will report an error. Now we’ll register the buffered function which we wrote in Python and for which we have a ctypes type:
buf_func = BUFFERED_FUNC(buffered)
LIBRARY.dtrace_handle_buffered(handle, buf_func, None)
Now we will compile the D script and run it:
prg = LIBRARY.dtrace_program_strcompile(handle, SCRIPT, 3, 4, 0, None)
# run
LIBRARY.dtrace_program_exec(handle, prg, None)
LIBRARY.dtrace_go(handle)
LIBRARY.dtrace_stop(handle)
If this exists correctly (The C file in the github repository has all the checks for the return codes in it) we can try to get the Hello World. The chew and chewrec functions are also implemented in Python and can now be registered.
If the second argument on the dtrace_work function is None DTrace will automatically use the buffered callback function described two steps ago. Otherwise a filename needs to be provided – but we wanted to get the Hello World into our Python code:
# do work
LIBRARY.dtrace_sleep(handle)
chew = CHEW_FUNC(chew_func)
chew_rec = CHEWREC_FUNC(chewrec_func)
LIBRARY.dtrace_work(handle, None, chew, chew_rec, None)
And last but not least we will print out any errors close the handle on DTrace:
# Get errors if any...
txt = LIBRARY.dtrace_errmsg(handle, LIBRARY.dtrace_errno(handle))
print c_char_p(txt).value
# Last: close handle!
LIBRARY.dtrace_close(handle)
Now this all isn’t perfect and not ready at all (especially the naming of functions could be updated, a nice abstraction layer be added, etc) – but it should give a nice overview of how to write DTrace consumers. And for the simple example here both the C and Python code at the previously mentioned github repository do seem to work – and do in fact output:
cpu : 0 Hello World Error 0
So maybe it’s time to combine pyssf (an OCCI implementation), pyzone (Manage zones using Python) and python-dtrace for monitoring and create a nice ‘dependable’ (Not my idea – these are the words of Andy) something…
[…] I bloged about how to use Python as a DTrace consumer with the help of ctypes. The examples in there are […]
[…] example of how to use Python as a DTrace consumer. This little program traces a Python program while is runs and shows you the flow of the code. The […]
[…] on the information you get from DTrace (using the Python consumer) you can draw life updating, up to date callgraphs of what is currently happening in the […]