perl: arrays and hashes

This is mostly a reminder for me than an article for everybody as I haven't touched perl for ages.

Small reference on arrays and hashes in perl.

Arrays

Declaration

@array = (1, '1', (2));
@array = (1..20);# by range

Access to array members with index

$array[0];

Define reference to array

$array = \@array; #reference to another array
$array = [1, 3, 5, 7, 9]; #reference to anonymous array
$array = [ @array ]; #reference to anonymous copy
@$array = (2, 4, 6, 8, 10); #implicit reference to anonymous array

To deference reference to array put @ or $ before $

@array = @$array;
@array = $$array;

Access to members of array by reference with index

$array->[0];# using -> operator
@$array[0];# dereferencing
$$array[0];# dereferencing

Size of the array

$#array;# [note: size of an empty array is -1, so $#array is a number of elements - 1]

Here is a tricks to remove all elements from an array, add an element to array

$#array = -1;
$c[++$#c] = 'value';

Take a slice of an array

@array[0..2];# first, second and third elements
@array[0,2];# first and third elements

Hashes

Declaration

%hash = ('key0', 'value0', 'key1', 'value1');# amount of elements must be even
%hash = ('key0' => 'value0', 'key1' => 'value1');

Access to hash members with key

$hash{'key0'};

Define reference to hash

$hash = \%hash; #reference to another hash
$hash = {1 => 3, 5 => 7}; #reference to anonymous hash
$hash = {1, 3, 5, 7}; #reference to anonymous hash; amount of elements must be even
$hash = [ %hash ]; #reference to anonymous copy
%$hash = (2, 4, 6, 8); #implicit reference to anonymous hash; amount of elements must be even
%$hash = (2 => 4, 6 => 8); #implicit reference to anonymous hash

To deference reference to hash put % or $ before $

%hash = %$hash;
%hash = $$hash;

Access to members of hash by reference with key

$hash->{'key0'};# using -> operator
%$hash[0];# dereferencing
$$hash[0];# dereferencing

Size of the hash

scalar keys(%hash)

Take a slice of a hash

@hash{'key0','key1'};
@hash{@keys};

c++: separate members from their classes

In my post c++: separate methods from their classes I described how to call class method by reference. The similar staff you can do with class members. Assume you have a collection of class instances and you want to print the values of some members from them. Again you can define two lists - list of class instances and list of pointers to class members. Later you can iterate through these list to touch members of the classes.

#include <iostream>
#include <list>

class A
{
    public:
        int m0; 
        int m1; 
};

template<typename T>
void
print(const T &a, 
    int T::*p)
{
    std::cout << a.*p << std::endl;
}

int main(int argc, char **argv)
{
    std::list<A> ls;
    std::list<int A::*> lsm;

    int A::*p0 = &A::m0;

    lsm.push_back(p0);
    lsm.push_back(&A::m1);

    A a0, a1; 
    a0.*p0 = 0;
    a0.m1 = 1;
    a1.m0 = 2;
    a1.m1 = 3;
    
    ls.push_back(a0);
    ls.push_back(a1);
    
    for (std::list<A>::iterator i = ls.begin();i!=ls.end();++i)
        for (std::list<int A::*>::iterator j = lsm.begin();j!=lsm.end();++j)
            print(*i, *j);

    return 0;
}

With this piece of code you will get

0
1
2
3

This method to access class members can be combined with class methods references to achieve more power.

c++: Koenig lookup

Koenig lookup, named after Andrew Koenig, a former AT&T researcher and programmer known for his work with C++ is also known as argument dependent lookup.
Argument dependent lookup applies to the lookup of unqualified function through the namespaces according to the types of given arguments.
Let's go through a simple example.

#include <iostream>

namespace NS1 
{
    class A
    {   
        public:
            A() {std::cout << "NS1::A::A";}
    };  

    template<typename T>
    void f(T a)
    {   
    }   
}

int main(int argc, char **argv)
{
    NS1::A v;
    f(v);

    return 0;
}

Under normal conditions compiler will look for f in global namespace and fail as there is no such routine there. But modern compiler can be more intelligent: it can look at the arguments and exposes function from the namespace where arguments came. This will compile with modern compilers but will fail with old.

On the other hand the next piece of code might cause another kind problem.

#include <iostream>

namespace NS1 
{
    class A
    {   
        public:
            A() {std::cout << "NS1::A::A";}
    };  

    template<typename T>
    void f(T a)
    {   
    }   
}

template<typename T>
void f(T a)
{
}

int main(int argc, char **argv)
{
    NS1::A v;
    f(v);

    return 0;
}

The compilation will fail with compilers that do Koenig lookup with "call of overloaded function is ambiguous" error but will succeed with compilers that do not perform Koenig lookup. Old compilers will call function from the global namespace.

For portability the namespace should be declared explicitly.

linux: emergency reboot remote box

Once I've faced very unusual problem for me.
I started to reboot the remote box and was unable to do it, because kernel thread [pdflush] was in uninterruptible sleep.

For local machines I used to hit SysRq+b, but here it doesn't work for me.

The solution was quite simple. Send SysRq+b via /proc filesystem:

echo b > /proc/sysrq-trigger

python: tuple with one item

When you want to construct a tuple containing only one item you should write

variable = (1,)

You have to add extra coma after the item. If you forget this coma the item will be returned, not the tuple.
Weird =/

c++: exception in constructor

I wonder why people want to make all the stuff in constructor.

Constructor do not return anything, so it can't indicate that it failed to do something.

The only way is (please close your eyes here)to throw an exception(now you can open the eyes).

When you throw an exception in the constructor the destructor will not be called. Because compiler doesn't actually know whether the object had been constructed or not. So it's more safe to omit execution of destructor in this case.

So you should clean the stuff just before throwing the exception:

class A
{
    public:
        A()
        {
            do_stuff();

            if (smth_goes_wrong)
            {
                clean_the_shoes();

                throw ticket_to_hell;
            }  

            do_other_stuff();
        }  
}

Knowing that you can keep safe your code even anything else may throw an exception in your constructor:

class A
{
    public:
        A()
        {
            try
            {
                do_stuff();
            }
            catch (...)
            {
                clean_the_shoes();

                throw;
            }   

            do_other_stuff();
        }   
};

Or even better not to throw the caught exception upstairs and do not call anything that might throw an exception at all. Much better to have an init method that may fail safely and call it after the constructor had been called and you are able to safely handle any exceptions.

c/c++: embed binary data into elf v.2

In previos post I've described how to embed data into object.
The other opprotunity is to store data in the c/c++ array.
Again, I'll use data.txt:

$cat data.txt 
data file

To create a source file with this data I'll use xxd utility:

xxd -i data.txt data.c

$cat data.c
unsigned char data_txt[] = {
  0x64, 0x61, 0x74, 0x61, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x0a
};
unsigned int data_txt_len = 10;

Simple c source file to use this array will look like:

#include <stdio.h>

extern unsigned char data_txt[];
extern unsigned int data_txt_len;

int
main(int argc, char **argv)
{
    printf("%d", data_txt_len);
    printf("%s", data_txt);

    return 0;
}

To compile

gcc test.c data.c

c/c++: embed binary data into elf

It's great idea when you store program data somewhere outside the binary.
It can be modified for changing program's behaivior or for rebranding.

But sometimes you want to keep some data immutable, hidden into executable binary.

This can be help sections. If you don't want to have smth like

void
usage (status)
     int status;
{
  fprintf (status ? stderr : stdout, "\
Usage: %s [-nV] [--quiet] [--silent] [--version] [-e script]\n\
        [-f script-file] [--expression=script] [--file=script-file] [file...]\n",
       myname);
  exit (status);
}

and don't want this help section be stored in the separate file.You can simply embed binary data into your executable.

Consider you have data.txt:

$cat data.txt 
data file

You have to convert it to elf.
I know two ways:

• use linker:
  
  

  ld -r -b binary -o data.o data.txt

• use objcopy:
  
  

  objcopy -I binary -O elf32-i386 --binary-architecture i386 data.txt data.o

Both of these commands produce elf:

$readelf -a data.o 
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Intel 80386
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          96 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           40 (bytes)
  Number of section headers:         5
  Section header string table index: 2

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .data             PROGBITS        00000000 000034 00000a 00  WA  0   0  1
  [ 2] .shstrtab         STRTAB          00000000 00003e 000021 00      0   0  1
  [ 3] .symtab           SYMTAB          00000000 000128 000050 10      4   2  4
  [ 4] .strtab           STRTAB          00000000 000178 000043 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Symbol table '.symtab' contains 5 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 00000000     0 SECTION LOCAL  DEFAULT    1 
     2: 00000000     0 NOTYPE  GLOBAL DEFAULT    1 _binary_data_txt_start
     3: 0000000a     0 NOTYPE  GLOBAL DEFAULT    1 _binary_data_txt_end
     4: 0000000a     0 NOTYPE  GLOBAL DEFAULT  ABS _binary_data_txt_size

_binary_data_txt_size and _binary_data_txt_end contain 

Ok, you have data.o with your data in .data section and three symbols: _binary_data_txt_start, _binary_data_txt_end, _binary_data_txt_size

_binary_data_txt_end and _binary_data_txt_size have the same value here. So I'll use _binary_data_txt_size only.
Let's make a simple c program to use data from the object. It's a bit tricky.

#include <stdio.h>

extern int _binary_data_txt_start;
extern int _binary_data_txt_size;

int
main(int argc, char **argv)
{
    int size = (int)&_binary_data_txt_size;
    char *data = (char *)&_binary_data_txt_start;
    
    printf("%d", size);
    printf("%s", data);

    return 0;
}

_binary_data_txt_start and _binary_data_txt_size contain values in their addresses. So &_binary_data_txt_size contains not an address of the symbol but actually value of the symbol that holds the size of the data and &_binary_data_txt_start contains address of the data.

To compile

gcc test.c data.o

VM networking: QEMU and VMware

Sometimes you have to work with qemu and VMware virtual machines at the same time. Moreover you want these machines be visible to each other over the network.

To set up shared network environment for qemu and vmware you should prepare kernel to support TUN/TAP interfaces and bridge interfaces:
Enable TUN/TAP support:

Device Drivers  --->
   Networking support  --->
      <*> Universal TUN/TAP device driver support
Networking  --->
   Networking options  --->
      <*> 802.1d Ethernet Bridging #NOTE : at least for 2.6.20 series

Ensure that you have /dev/net/tun char device and it's writable and readable for qemu user.

Start vmnet(usually vmnet8) interface.

Set vmnet8 in promisc mode:

ifconfig vmnet8 promisc

Setup bridge interface:

brctl addbr br0

Add vmnet8 interface to the bridge:

brctl addif br0 vmnet8

Run vmware VM.

Create file /etc/qemu-ifup with:

#!/bin/sh
sudo /etc/qemu-ifup-sudo $@

Create file /etc/qemu-ifup-sudo with:

#!/bin/sh
/sbin/ifconfig $1 0.0.0.0 promisc up
/usr/sbin/brctl addif br0 $1

Make them executable and add qemu user to /etc/sudoers to run /etc/qemu-ifup-sudo in proper way.

Run qemu VM:

qemu -hda linux.img -net nic,macaddr=52:54:00:12:34:57 -net tap

For every new qemu VM instance you must set different macaddr!

source fetcher

Recently I've faced a problem with updating docs/code examples on libdodo's site. Each time I publish release I have to update docs and code examples. It takes time for formatting pages for web, updating each page and so on. I decided to write a wordpress plugin that fetches sources from the mercurial repository, formats the code and puts it on the wordpress page. The code you can find at source fetcher google code page. You have to install it to 'plugins' directory in the wordpress tree and edit two settings: URL to the repository and tag/revision. You can browse the results on the libdodo examples page.

grep: locale

I've spent almost an hour with "grep -RIE 'class [a-z]+[A-Z]+' *h" trying to find classes which have names beginning with lowercase letter and contain capitals.
That grep command ignored case and I got classes with all lowercase letters also.
I've dug into the man pages and found the next paragraph:

Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, [a-d] is equivalent to [abcd]. Many locales sort characters in dictionary order, and in these locales [a-d] is typically not equivalent to [abcd]; it might be equivalent to [aBbCcDd], for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.

Yes, "LC_ALL=C grep -RIE 'class [a-z]+[A-Z]+' *h" worked for me but I didn't expect such behaivior with UTF-8 locale.

Googling a bit I've found some pages contain:

• Collating symbols. These look like [.element.], where element is a collating element (i.e. a symbolic name for a multi-character string), and match the value of the collating element in the current locale. This doesn't seem to work in GNU grep.

• On some locales it might include both the uppercase and lowercase of a given character. In the POSIX locale, this always expands to only the character given. 

So '[A-Z]'  is only A,B,C,...,Z for POSIX/C locale.

c++: overriding virtualization

This is a well known technique for me but just recently I heard it was called as 'overriding virtualization'. That's why I decided to expose how calling method from the explicitly mentioned class can override virtualization in classes. Consider you have

class A
{
    virtual void m();
};

class B : public A
{
    virtual void m();
};

When you create instance of class B, you will call B::m in all cases:

A *a = new B;
a->m();/// B::m here

So, if you want to call m from A? Easy:

A *a = new B;
a->A::m();/// A::m here

In the example above I've overridden the virtual call.

c++: property

Recently I'd touched an article about properties in python. They were implemented using special function property and looked very elegant:

class A(object):
  def __init__(self):
    self._x = None

  def get_x(self):
    return self._x

  def set_x(self, x):
    self._x = x

  x = property(get_x, set_x)

obj = A()
obj.x = 6    # set
print obj.x  # get

We don't have such option in c++. But it's quite simple to create one. Some implementation I saw were ugly indeed. They used a lot of code and wanted to call some routines in constructor. My version of property in c++:

#include <iostream>

template<typename T, 
         T(*s)(T &, const T &), 
         T(*g)(T &)>
class property
{
    public:
        
        T operator =(const T &nt)
        {
            return s(t, nt);
        }

        operator T()
        {
            return g(t);
        }

    protected:

        T t;
};

class A
{
    public:

        static int setter(int &i, const int &ni)
        {
            cout << "S" << endl;
            i = ni;

            return i;
        }

        static int getter(int &i)
        {
            cout << "G" << endl;

            return i;
        }

        property<int, setter, getter> p;
};

int main(int argc, char **argv)
{
    A a;

    a.p = 6;
    std::cout << a.p;

    return 0;
}

You have to define 2 static class methods(or just functions, doesn't matter) for setter and getter of the property. The modified version of class property may have default setter and getter:

template<typename T>
T setter(T &i, const T &ni)
{
    i = ni;

    return i;
}

template<typename T>
T getter(T &i)
{
    return i;
}

template<typename T, 
         T(*s)(T &, const T &) = setter<T>, 
         T(*g)(T &) = getter<T> >
class property
{
    public:
        
        T operator =(const T &nt)
        {
            return s(t, nt);
        }

        operator T()
        {
            return g(t);
        }

    protected:

        T t;
};

Now you can customize only one of them:

int getter(int &i)
{
    cout << "G" << endl;

    return i;
}

class A
{
    public:

        A(int a)
        {
            p = a;
        }


        property<int, setter<int>, getter> p;
};