python: non-defined function arguments

You know that you can get named arguments of the routine that are not in function definition with **kw parameter:

def func(**kw):
     for k,v in kw.items(): print "%s: %s" % (k,v)

kw is a dict here where key is parameter name and value is parameter value. So if you call

func(a=1, b=2)

your kw will be

{'a': 1, 'b': 2}

>>> func(a=1, b=2)
a: 1
b: 2

The same is for the unnamed arguments:

def func(*a):
     for v in a: print v

a is a list of arguments here

func(1, 2)

will produce tuple a

(1, 2)

>>> func(1, 2)
(1, 2)

Nice I should say. Let's view the other side. You have function

def func(a=1, b=2):
     print a, b

and you have a dict

kw = {'a': 3, 'b': 4}

Now you just can use some py magic

func(**kw)

You will get a=3 and b=4:

func(**kw)
3 4

And for the unnamed arguments

def func(a, b):
     print a, b

If you define a list of 2 values

a = (1, 2)

and pass it to the func

func(*a)

you will get a=1 and b=2

func(*a)
1 2

Some extra py magic for these tricks:

def func(a=1, b=2): pass
func(**{'a': 3, 'b': 4})

def func(a, b): pass
func(*(1, 2))

c/c++: dlclose

What should you know about dlclose? You should remember that it actually doesn't close the library. It decrements the reference counter. When the counter reaches zero value and no other libraries use symbols in it, then the library is unloaded. The next example will work:

    void *handle1 = dlopen("./obj", RTLD_LAZY);
    void *handle2 = dlopen("./obj", RTLD_LAZY);

    print p1 = (print)dlsym(handle1, "print");
    p1();
    dlclose(handle1);
    
    p1();

dlopen: performance

When you use dynamic library loading you probably open library each time you want to load routine. As for loading/unloading routines on demand you should open library every time to ensure that library is loaded and you can access routine you are interested. This will heavily reduce performance and memory usage. Probably you can control whether library was opened or not but this requires more code from you. In glibc from 2.2 had appeared 2 very useful flags to cover this situation: dlopen now may get RTLD_NOLOAD and RTLD_NODELETE flags. With RTLD_NOLOAD you can check if library was loaded and with RTLD_NODELETE you say dlclose not to unload library. I've made some tests. W/o RTLD_NOLOAD and RTLD_NODELETE flags:

        void *handle;
        int i; 
        for (i=0;i<1000000;++i)
        {
                handle = dlopen("/usr/lib/libdl.so", RTLD_LAZY);
                if (handle == NULL)
                        break;
                dlclose(handle);
        }

This gave me

for i in 1 2 3; do time ./test; done
real 0m0.567s
user 0m0.568s
sys 0m0.000s

real 0m0.566s
user 0m0.556s
sys 0m0.000s

real 0m0.562s
user 0m0.556s
sys 0m0.000s

Pretty slow, I should say. And w/ RTLD_NOLOAD and RTLD_NODELETE flags:

        void *handle;
        int i; 
        for (i=0;i<1000000;++i)
        {
                handle = dlopen("/usr/lib/libdl.so", RTLD_LAZY|RTLD_NOLOAD|RTLD_NODELETE);
                if (handle == NULL)
                        handle = dlopen("/usr/lib/libdl.so", RTLD_LAZY|RTLD_NODELETE);
                if (handle == NULL)
                        break;
                dlclose(handle);
        }

This one gave me

for i in 1 2 3; do time ./test; done
real 0m0.542s
user 0m0.536s
sys 0m0.000s

real 0m0.570s
user 0m0.572s
sys 0m0.000s

real 0m0.554s
user 0m0.528s
sys 0m0.004s

The same. libdl is pretty small. Let's try something bigger. Tests results for /lib/libc-2.7.so: w/o RTLD_NOLOAD and RTLD_NODELETE flags:

for i in 1 2 3; do time ./test; done
real 0m0.656s
user 0m0.656s
sys 0m0.000s

real 0m0.649s
user 0m0.644s
sys 0m0.000s

real 0m0.647s
user 0m0.644s
sys 0m0.000s

w/ RTLD_NOLOAD and RTLD_NODELETE flags:

real 0m0.611s
user 0m0.612s
sys 0m0.000s

real 0m0.610s
user 0m0.600s
sys 0m0.000s

real 0m0.608s
user 0m0.604s
sys 0m0.000s

We have won ~0.040s. Not bad. And another one for /usr/lib/libdb_cxx-4.5.so[note, I have changed circumstances: loop ran only 1000 times]: w/o RTLD_NOLOAD and RTLD_NODELETE flags:

for i in 1 2 3; do time ./test; done
real 0m1.270s
user 0m1.016s
sys 0m0.244s

real 0m1.290s
user 0m1.084s
sys 0m0.188s

real 0m1.269s
user 0m1.036s
sys 0m0.236s

w/ RTLD_NOLOAD and RTLD_NODELETE flags:

for i in 1 2 3; do time ./test; done
real 0m0.003s
user 0m0.004s
sys 0m0.000s

real 0m0.003s
user 0m0.004s
sys 0m0.000s

real 0m0.003s
user 0m0.000s
sys 0m0.000s

Yay! They differ for more than 1.2 seconds. That's because /usr/lib/libdb_cxx-4.5.so had to load some extra libraries.

• ldd /usr/lib/libdl.so
   linux-gate.so.1 =>  (0xb7fa2000)
   libc.so.6 => /lib/libc.so.6 (0xb7e53000)
   /lib/ld-linux.so.2 (0xb7fa3000)

• ldd /lib/libc-2.7.so
   /lib/ld-linux.so.2 (0xb7f7f000)
   linux-gate.so.1 =>  (0xb7f7e000)

• ldd /usr/lib/libdb_cxx-4.5.so
   linux-gate.so.1 =>  (0xb7f3b000)
   libpthread.so.0 => /lib/libpthread.so.0 (0xb7dcb000)
   libstdc++.so.6 => /usr/lib/gcc/i686-pc-linux-gnu/4.2.3/libstdc++.so.6 (0xb7ce0000)
   libm.so.6 => /lib/libm.so.6 (0xb7cbb000)
   libc.so.6 => /lib/libc.so.6 (0xb7b86000)
   libgcc_s.so.1 => /usr/lib/gcc/i686-pc-linux-gnu/4.2.3/libgcc_s.so.1 (0xb7b79000)
   /lib/ld-linux.so.2 (0x80000000)

Now we can see the difference. If your library has references to external library dlopen loads them each time if you haven't specified RTLD_NOLOAD and RTLD_NODELETE flags. I had better use RTLD_NOLOAD and RTLD_NODELETE. It requires some extra code from you and one extra open handle for the program instance per library. But it will produce much faster code. Be aware ;)

c/c++: binary representation of the numeric

I had some free time and remembered old task from the university: show numeric in binary representation. In my old days I'd probably write smth with divs and mods. The core of the algorithm was a mathematical transformation: 13 = 1101 = 2³*1+2²*1+2¹*0+2⁰*1. Now I just have thought about the problem from other side:

int
main()
{
    int n = 13; 

    for (int i=sizeof(n)*8-1;i>=0;--i)
        printf("%d", (n >> i)&1);

    return 0;
}

Here we are:

$gcc 1.c -o out -std=c99 -O2
$./out 
00000000000000000000000000001101

Just fun =).

netcat: debug your network applications

It is hard to debug client application w/o modifications in server(client) application or writing third-party tool. The resolution you may want - a ready-to-go application that emulates network listener to which you can connect client and interact with it. Such application exists - netcat. To run a telnet session on port 1234 just run

nc -l -p 1234

and connect to port 1234 with your client application. Now you can send messages to client and receive them from it. To use UDP protocol just add -u argument for nc. Usually it's easy to debug server application that uses TCP protocol just by connecting them using telnet and emulate client protocol. Using netcat you also can send UDP messages to server:

nc -u 1.2.3.4 1234

. Project home: http://netcat.sourceforge.net/