peter.makholm.net

Tim Bunce mentioned my blogpost about Benchmarking serialization modules in a post on the perl5-porters mailing list. He wished that someone would make that benchmark into a distribution on CPAN.

How can I refuse. So here it is, Benchmark::Serialize is just uploaded to CPAN. (Might be some time before it appears).

Besides making the script into a module I also added a list of the size of the serialized data to the output. A replacement of the original script is available in the examples directory.

(I planned on naming the module Benchmark::Serialization, but my fingers slipped. Should I rename it?)

It would be nice if I could get my ORLite performance boost without really changing the (undocumented) API. So I got one more idea: Do the slicing in the ORLite generated code. It’s available in a new branch on GitHub.

To benchmark all three solutions I used a variant of CPANDB::Dependecy::csv():

sub csv {
    my $class = shift;
    for my $edge ( $class->select ) {
        my $foo = $edge->distribution . "\t" . $edge->dependency . "\n";
    }
}

My::Plain->begin;
My::Unsliced->begin;
My::SelfSlice->begin;

cmpthese( -30, {
    plain => sub { csv("My::Plain::Dependency") },
    unsliced => sub { csv("My::Unsliced::Dependency") },
    selfslice => sub { csv("My::SelfSlice::Dependency") },
});

Unfortunately it seems like both having DBI doing the slicing and doing it myself costs roughly the same:

            Rate selfslice     plain  unsliced
selfslice 1.61/s        --       -1%      -41%
plain     1.64/s        2%        --      -40%
unsliced  2.71/s       68%       66%        --

So I probably end up making some sort of ORLite subclass as Adam Kennedy suggested in a comment.

While profiling some code making heavy use of Adam Kennedy’s ORLite module for accessing a SQLite database I found that most of my time was spend in DBI.pm.

Sorted by inclusive time (ie. including time spent in subroutines) two non-XS functions stood out: selectall_arrayref and fetchall_arrayref. Looking a the code both of these functions had a comment stating that a C implementation existed in Drivers.xst and the comment at selectall_arrayref further said that the Perl version is used as a fallback if a slice is given

So could I get away with the slicing?

Turned out to quite easy. Just use array refs as objects instead of the usual hash refs.

The run time of the select() method provided by ORLite went from 344µs/call to 162µs/call on average. And as my test data makes at roughly 100000 select() calls this is a quite noticeable speedup. All included my running time (under Devel::NYTProf) improved from 400 seconds to 340 seconds.

Unfortunately I have to be able to update the state of my ORLite generated objects. This was easy while the objects was blessed has refs. Array refs are not as easy to update. The easy solution was to make the simple non-fk accessors to be lvalue subroutines.

For simple web-based services I usually just use Perl and CGI.pm. After having read a bit about HTTP::Engine I tried it for a simple project yesterday. Beside my own logic it only took a few line of code to have a stand alone HTTP server for my service.

My colleague needed it to be served from the same Apache server as the rest of his webapplication. Some tiny changes and my stand alone server was transformed into a plain CGI script. When we going to deploy the script I’m guessing we make some tiny changes and have it running as a mod_perl module.

Try it for you next project! Even if you usually just use CGI.pm.

It is common knowledge that you can’t have private functions and methods in Perl5. But it turns out that you can do it, one way is to use namespace::clean. Using this module you can either declare all the names of private functions at the top or use a serie of non-obvious “use namespace::clean“, “no namespace::clean” calls.

Wouldn’t it be much nicer just to be able to write:

sub foo :Private {
    ...
}

You can, with my brand new Sub::Private module. It is actually quite simple:

use Attribute::Handlers;

use namespace::clean     qw();
use B::Hooks::EndOfScope qw(on_scope_end);
use Sub::Identify        qw(get_code_info);

sub UNIVERSAL::Private :ATTR(CODE,BEGIN) {
    my ($package, $symbol, $referent, $attr, $data) = @_;

    on_scope_end {
        namespace::clean->clean_subroutines( get_code_info( $referent ) );
    }
}

Putting the attribute handler in the UNIVERSAL namespace isn’t nice. I have to find a solution for that for the next version.

At this years YAPC::EU I’m giving a talk about some of the more advanced features of regular expressions.

Slides are available at http://hacking.dk/talks/yapceu2009/ – There is a pretty regexp validating dates (including February 29th) in it!

Over the years I have collected a few shell snippets that have to be sourced or called as shell functions to work correctly. Having all these function definitions in a monolithic file sourced from my .bashrc isn’t cool, so for some time I’ve had the following in my .bashrc:

for i in $HOME/.lib.sh/* ; do
    source $i
done

This has at least one obvious downside. Each time I change a function definition I have to reload the function i each and every open shell if I want a consistent work environment. Today I came up with a nifty solution. Each file in ~/.lib.sh which previously contained a function definition isedited to only contain the function body. The I add the following new function called load_functions:

FUNCTIONS=$HOME/.lib.sh

eval $(
    find $FUNCTIONS -type f | \
        while read file ; do echo function $(basename $file)\(\) { source $file\; }\; ; done
)

For each file under ~/.lib.sh a new function is automatically defined which just sources the real file. This means I can edit the files without having to reload the function in every open shell. ~/.lib.sh/load_files itself is just sourced from my .bashrc.

One of my functions wraps the wonderful Perl5 module local::lib. This module makes it easy to manage multiple sandboxes with locally installed Perl modules. It works by setting a couple of environment variables including $PERL5LIB. Instead of using local::lib directly I used a function called perllibs:

# Defaults:
PERL_LOCAL_LIB="home"
DIR=$HOME/.perl

if [ -n "$1" ]; then
    PERL_LOCAL_LIB=$1
fi

export PERL_LOCAL_LIB

case $PERL_LOCAL_LIB in
    alpha)
        DIR=$HOME/projects/alpha/ ;;
    mercury)
        DIR=$HOME/subversion/mercury/perl ;;
    tmp)
        mkdir -p /tmp/makholm/perl
        DIR=/tmp/makholm/perl ;;
esac

eval $( perl -Mlocal::lib=$DIR )

Adding new projects to this list would be a hassle if I had to reload the file in multiple shells each time.

I’m a happy vim-user and I don’t see that change any time soon. But recently I have been hacking around with Padre, an editor writen in Perl and for Perl programmers. And even though I like my non-GUI editor I think Padre has one advantage: It is quite hackable inPerl – my favourite language for hacking.

One thing I wanted to play with was debugging from my editor. It is probably posible to write in vim-perl scripts, but having easy access to all the internals without language barriers made the task of writing a debugger plugin much easier and more importantly, much more fun.

So if you have ever thought about nice features you wanted while editing Perl, then please join the Padre project and keep hacking perl while improving you tools.

I often write code like this:

    $_->store() for @objects;

and quite often it actually works. But suddenly a piece of code doing this in a loop broke with this error message: Can’t call method “store” without a package or object reference. And quite right, sometimes @objects would contain plain integers.

Unfortunately it wouldn’t be quite easy to track down the relevant change, so enter the perl debugger. Declaring a watch on ‘@objects’ isn’t useful as it triggers each time @objects enters or leaves scope. But saving a reference to @objects in $my::objects and the watching $my::objects->[0] worked.

I had reimplemented the store() method using Data::Walk for walking some structure instead of doing it by hand. And Data::Walk sets $_ to the current node. A due to the aliasing implied in the for-modifier each element in my @objects array was garbled.

Two solutions: The store() method can localize $_ by adding local $_; before using Data::Walk – this works in legacy perl interpreters. The routine looping through @objects can make $_ a lexical variable by adding my $_; before the loop – This is a new feature in Perl 5.10.

An even more robust solution would be to have Data::Walk localize $_ itself AND use a lexical $_ in code where $_ is aliased to important data. See RT #47309.

I’ve just uploaded a new version of Padre-Plugin-Debugger to CPAN. It has been stuck in github for a while as I meant to write some documentation and start calling it version 1.0. But as I haven’t written any documentation yet, it is still just a puny 0.3 version.

The major update is how the interpreter is called. Now it will actually find the modules you are using, if you add the correct directories to ‘Edit -> Preferences -> Run Paramerters’