Post-hackathon thoughts

It has been a few days, and we have (sort of) landed.

I shall call this year's Oslo.pm Perl 6 Patterns hackathon a success, judging by how the participants seemed to enjoy themselves during the hackathon, and the inspired blogging afterwards.

Salve (sjn) and Rune (krunen) have, on behalf of Oslo.pm, delivered to standards that I am a bit worried that guests now will come to expect ;), and although I did not have capacity to help with much, I am happy with how everything turned out, except for one thing:

Damian caught a cold, and was essentially out of the loop for most of the hackathon.

Darnit.

But he delivered a kick-ass talk on Thursday, and was very helpful and a very good resource before he somewhat reluctantly accepted the thrown towel.

I did not do much hacking this year myself, but I managed to revive some of my Perl 6 skills. I even fiddled a bit with the cookbook (dormant for three years or so, and therefore slightly out-of-date in some respects). I hope to contribute more to that piece of documentation in the coming months.

The bonus if I manage to work my way through it, is that I will be pretty much up to speed with the language. Win!

My thanks go to all the participants of the Hackathon, without you, Perl 6 would be much the poorer!

Oslo.pm Patterns Hackathon pictures

Hacking: sjn, moritz (back), pmichaud, jnthn, masak, tadzik, sergot, bjarneh

---

Hacking: imarcusthis, infosophy

---

Posing, back to front, left to right:
infosophy, moritz, arnsholt, sjn, krunen, masak
tadzik, jnthn, frettled, pmichaud

Oslo.pm Perl 6 Patterns Hackathon 2012

In one week (2012-04-20 – 2012-04-22), a bunch of bright people will attend the Perl 6 Patterns Hackathon in Oslo.

Perl 6 – both its specification and implementations – will become one of the Great Ones. I admit that we are not quite there yet, but to me, Perl 6 is a language for the long term.

In one of the meetings krunen, sjn and I had this winter, we discussed this, and how to get there. I hope the Perl 6 Patterns Hackathon will contribute significantly, and I also hope that in 30-60 years, we will look back and be happy about most of the choices made around this time! I truly believe that we can get there!

Call for sponsor matching

In this spirit, Ingvoldstad IT decided to sponsor this hackathon with NOK 5,000 (≈ EUR 650), and I hereby call for other companies who develop software* to match this amount. Oslo.pm will put those money to good use!


I hope to see you in Oslo in a week!

---

* I did not write "Perl" here, because I think that is largely irrelevant. Perl 6 is an amalgam of many of the most interesting programming language features, and offers a very compelling path to those who want a next-generation language after the current versions of e.g. Perl, Python, and Ruby.

Community boost

It is now three days since the release of Rakudo Star.

Everything has met my personal expectations, except for one major thing:

There is a bunch of new people on #perl6, who are actively engaged in trying out Rakudo Star/Perl 6, in testing, submitting bug reports, patch suggestions, asking for help, providing help, helping themselves…

I really do not have anything profound to say, but I can say this:

Now we are really getting somewhere!

Rakudo Star is here

The wait is over, and a usable Rakudo-based Perl 6 distribution is ready for early adopters. Yay!

More and more of Perl 6 is implemented in Perl 6. That is a very good tendency.

Missing feature

A few hours ago, I suddenly had a bright(?) idea, or desire if you will:

Proper (Unicode) exponents in Perl.

That is, I want to be able to write 2², 4¹³⁷, 3^-9, etc. and have Perl understand them.

For Perl 5, I suspect someone would use a source filter to implement it.

For Perl 6, PerlJam++ suggested introducing each of the exponents as postfix operators, using this example for squaring:

our &postfix:<2> := &infix:<**>.assuming(b => 2);

But then a negative exponent would complicate things a bit.

It's a thought, anyway, and not one that I'd want to distract more pressing implementation concerns.

And ifwhen someone decides that this is a good idea to have in the language core, I'll start nagging about Knuth's up-arrow notation. Not that I'd want anyone attempt calculating 4↑↑↑↑4.

Rakudo ng - what will it mean for us?

If you've been hanging around the right blogs and the #perl6 IRC channel on Freenode, then you've probably seen references to a slightly mysterious "ng", or "Rakudo ng".

That's the upcoming (next) version (generation) of Rakudo, which will form the basis for Rakudo *.

In essence, this is a refactoring/rewrite of Rakudo for the purpose of better compliance with the specification and performance improvements (yay). The old Rakudo master made it difficult — if not impossible — to implement several essential parts of the Perl 6 spec and top priorities on the Rakudo roadmap.

In January, this has led to less focus on the current Rakudo version's bugs and gotchas, and instead on working to prepare ng as the new master branch — that is, the Rakudo that you will be downloading the next time.

For those of us who do some Perl 6 coding in Rakudo, this means that we can expect a nice little bunch of incompatibilities as compared to the current master. And yes, it's very close, so it's time to prepare.

Here's a list of the blindingly obvious things I think we need to watch out for:

• Older Rakudo was not in line with parts of the spec that ng will be.
• The spec has changed. (ng development has uncovered several necessary changes.)
• Older Rakudo is in line with parts of the spec that ng perhaps isn't.
• Rakudo ng is, of course, not feature complete when it replaces older Rakudo as the master.

In other words: let's not fool ourselves into thinking that we all of a sudden have a new Rakudo that's both compatible with the older as well as being spec compliant.

The good news about Rakudo ng

If you judge by the above paragraphs, you'd think that Rakudo ng was bad for Perl 6 developers. But that's far off the mark. I prodded #perl6 and Patrick Michaud before publishing this post, and here's a brief summary of (most of) the improvements we can see coming with Rakudo ng as opposed to the current implementation.

• Most of the top priorities of the Rakudo roadmap will be implemented!
• Laziness will mostly work (the spec is undergoing change)
• Performance improvements, many due to laziness
• Array/List/Parcel/etc. will be compliant with the updated spec
• Protoregexes
• Better longest token matching
• Meta-operators are really meta, and generated on demand
• The base object metamodel is far closer to the spec than before
• Major portions of the metamodel are implemented in Perl 6
• Array and hash vivification will work properly
• Lexical subs and variables work properly
• Operators have the correct names (with angles)
• Subs have the correct sigils (with ampersands)
• Phasers work, and the phaser model is much improved

Our programs will need a bit of attention. I recommend subscribing to perl6-language for up-to-date information about changes to the specification and language discussions.

There's still a lot of work to be done, and I'm sure the Perl 6 developers are happy for any help they can get.

feather.perl6.nl - a Sysadminish Tale

<@Juerd> frettled: Blog about the mess you found when
         you first logged in on feather yesterday :)

---

Sure.

This will, incidentally, also explain why Trac is kindof unavailable now.

feather.perl6.nl is a Xen guest (a virtual machine, hereafter "VM") that's hosting several important services for the Perl 6 community. There's SVN web access, a Trac installation, and a bunch of other stuff I honestly don't know the half of.

Recently, the VM started running out of memory too often for comfort. What was going on? Juerd asked for help in tracking down the problem, as he didn't have the time to do so himself. And needing some distraction from work -- something to help me procrastinate -- I volunteered.

By now, you're probably banging your head on your keyboard in sympathy with me for saying something that may have been slightly less than brilliant. You know the feeling; Matt Trout reaches his right hand towards you, the world is suddenly in slow-motion, you see his hand closing in on you, his grin widening, and a voice saying "thhhhhaaaaannnnnkssss ffffooooorrr vooooluuunnnteeeerrriiiinnnnng", and you're basically up that creek with all the mud and dirt in it.

After handing over an SSH public key and getting sudo access (yeah, yeah, I know), I had a look anyway.

First, I went on a brief but wild goose chase, finding some error messages regarding ConsoleKit which appeared to be more frequent just before the server went out of memory, checked the Debian version (an unholy mix of Debian unstable and Debian experimental with lots of package updates pending someone's attention), and generally tried to get a feel of how the system was configured.

We already knew that Apache somehow might be responsible for gobbling up available memory, so my first action was to have a look at the last 100,000 lines of the Apache access log, using a simplistic log analysis script.

But which log? There were three Apache log directories to choose from. I (correctly) guessed that the one called simply /var/log/apache2 might be the interesting one, the others seemed to be legacy directories which should have been removed ages ago.

According to the script, there were 0 accesses in the last 100,000 lines.

Knowing the script, that was not so strange, because it makes a few assumptions regarding the log format, using a regexp belonging to the days before named captures and whatnot:

while (<>) {
  if (/^(\S+) (\S+) - - \[[^\]]*\] \"(GET|POST) \S*
       HTTP(|\/1\.[01])\" \d{3} (\d+) \"/) {

The regexp line has been split for the sake of the line width of this blog. There's nothing to be proud of here.

Anyway, I first had to remove the first capture; feather's logs weren't showing the virtualhost as the first column, and access types were most certainly not limited to only GET and POST:

frettled@feather:~$ sudo awk '{print $6}' /var/log/apache2/access.log|
sort -u
"CHECKOUT
"CONNECT
"DELETE
"GET
"HEAD
"MERGE
"MKACTIVITY
"OPTIONS
"POST
"PROPFIND
"PROPPATCH
"PUT
"REPORT

Right.

After straightening that up (and adding %v to the LogFormat specifications in the Apache config for future use), I got the following result:

Use of uninitialized value $size in addition (+)
at /usr/local/sbin/bandwidthips line 39, <> line 1002.
Use of uninitialized value $size in addition (+)
at /usr/local/sbin/bandwidthips line 40, <> line 1002.
Use of uninitialized value $size in addition (+)
at /usr/local/sbin/bandwidthips line 44, <> line 1002.

AAAARRGH! Idiot! Imbecile! Inept half-wit! Yep, I'd forgotten to renumber my captures. See, this is why Perl should be in version 5.10.1 or 6 when fiddling with those bloody annoying regexps.

frettled@feather:~$ sudo tail -100000 /var/log/apache2/access.log|
/usr/local/sbin/hitips|head
193.200.132.146: Bytes = 14329487 (3.44%), Hits = 51503 (51.82%)
66.249.71.2: Bytes = 132116084 (31.73%), Hits = 18111 (18.22%)
66.249.71.37: Bytes = 50846948 (12.21%), Hits = 6236 (6.27%)
93.158.149.31: Bytes = 54880221 (13.18%), Hits = 1894 (1.9%)
71.194.15.106: Bytes = 460200 (0.11%), Hits = 1894 (1.9%)
209.9.237.232: Bytes = 433388 (0.1%), Hits = 1686 (1.69%)
193.200.132.135: Bytes = 1726871 (0.41%), Hits = 1635 (1.64%)
193.200.132.142: Bytes = 429358 (0.1%), Hits = 1609 (1.61%)
208.115.111.246: Bytes = 8461415 (2.03%), Hits = 1238 (1.24%)
67.218.116.133: Bytes = 18945415 (4.55%), Hits = 1126 (1.13%)

So, uhm, around 52% of the hits come from feather3.perl6.nl, and nearly 25% from Google's indexer. Lovely.

Looking at the accesses from feather3, I quickly saw that they mostly had to do with svnweb.

Juerd had already stopped Apache, but someone -- I don't know who -- started it again at 12:00, probably anxious that SVN and such didn't work.

I then followed the running processes using the top command, updating each second (top d1), sorting by memory usage (typing M while top was running), hoping to catch some quickly growing processes.

Nopes. None, zilch, nada. Nothing that appeared horribly wrong. Sure, the apache2 processes used some memory (30-60 MB resident set, 50-100 virtual), but nothing appeared to be out of the ordinary. I changed the update frequency to each third second -- top sometimes uses an inordinate amount of CPU, depending on magic -- and waited. After a while, a couple of apache2 processes were using more CPU and memory than the others, around 60-90 MB resident. And they were growing. And according to lsof, they were active in the svnweb directory (and used a metric shitload of libraries). And after growing, they didn't release memory, they just kept on using it. But it wasn't enough to use up memory, there was still a bunch of free RAM.

So that was perhaps svnweb's fault, then?

Maybe.

But then my time ran out, and I had to drop the ball, leaving the top process running.

Five minutes later, the memory ran out again. It's just as if someone was waiting for me to go idle in order to produce the problem that I was looking for.

Sigh.



svnweb kindof remained the main suspect, until Juerd caught whatever was happening at the right time.

And catching what happens at the right time is bloody important.

Here's what he found, using Apache's server status:



Well, that's not svnweb. That's Trac. And the IP addresses belong to Google.



And that's spam, effectively creating a DoS or DDoS attack on our services as a side effect when search engines try to index the Trac webpages. It probably isn't intentional, but spammers just don't care.

So, what can we do to protect feather from suffering from such attacks in the future?

There's a lot that can be done. It takes effort. It takes time. It takes someone.

Here are a few suggestions on how to improve the robustness of the kind of services feather provides:

• Add a captcha to the web form. The disadvantage is that this does not really save processing resources, but it probably should be done anyway.
• Add an unnecessary and bogus input field to the web form, e.g. "Phone number". This input field should be hidden with CSS so that web browsers don't display it, and if someone submits anything with data with that field's name, then you can be nearly 100% certain it's spam from someone who's used a web scraper before automatically filling the form. Filter it out.
• Change the webserver delegation architecture, so that each Apache process isn't loading tens of megabytes of libraries and keeping them in memory. Off-loading to shorter-living FastCGI daemons or similar solutions, or even sacrificing program startup speed by using CGI+suexec, etc., may be decent starting points.
• Consider using a front-end proxy like Varnish to gloss over underlying nastiness.
• Start with a new VM and migrate services to that one, gradually.
• Document configuration choices and what each web service does/is there for, so that the next sysadmin coming along can make educated guesses quicker. :)

These tasks can rather easily be split into manageable one-person projects.

Does this sound interesting to you, or did I lose you at the third line of this blog entry?

Pop in on #perl6 on the freenode IRC network and say so.

Typing More Or Less

Not too long ago, there was a bit of minor cleanup in the Perl 6 specification regarding the use of whatever (*); there were some inconsistencies in how it behaved, depending on context.

The net result is that you now must use @arr[*-1] to get the last element, you cannot get away with simply using @arr[*].

Some may feel that this extra typing is bothersome, especially if you have a Unicode-friendly keyboard setup.

However, we can sneak our way past this problem by using a constant.

This also works with the current release of Rakudo, so it's not quite science fiction:

constant Ω = *-1;

Or, if you're feeling Cyrillic rather than Greek:

constant Ѡ = *-1;

Now we can substitute our nice constant for *-1 anywhere in the following code:

my @letters = 'a'..'z';
say '→'~@letters[*-1];
→z

Like so:

constant Ω = *-1;
my @letters = 'a'..'z';
say '→'~@letters[Ω];
→z

And, of course, you can do this with other things that are so tedious to type when you're dealing with maths:

constant π = pi;
say '→'~π
→3.14159265358979

2009 In Perl

Repeat after me: I will not pretend to be an analyst or doomsayer, even though the end (of 2009) is nigh.

In 2009, Perl grew up a bit more, both as a language and as a community.

Language Development

Perl 5.10.1 came with a pony to those of us who fear the .0 releases.

The Perl 5.11 development tree got started, and it looks like it is rolling on rails. At this rate, we will see 5.12.0 quicker than you can say antidisestablishmentarianism.

... Perl 6 has made progress both on the specification side and in implementations -- yep, that is plural. It is sometimes confusing when naming changes under your feet, but it is acceptable while the spec is still settling.

Community

In 2009, I think I saw more openness regarding the internal conflicts in the Perl community as a whole; there were abundant admissions that we were not communicating nearly as well as we should, that there was at least a small amount of internal bickering over the present and future state of the onion -- onions, I must inject, tend to come in many shapes and flavours, and are not always the same inside -- really, which way we are going, are we having a conflict or not (yes we are -- no we are not -- huh, are we talking? -- pass the chips), and get off my lawn before I shoot or hug you.

In brief, it looks to me like 2009 was the year when the community showed renews signs of self-awareness.

But much more happened. We got a closer focus on Perl visibility, from my POV mainly owing to Matt S. Trout's lightning talk challenge from NPW 2009, plus a whole range of people working on other PR aspects for ourselves. And mst still keeps his hair colour. Wow.

Other Stuff

I made new friends, I learned a lot, I even got to help out a bit, and I hope that this will continue in 2010.

I hope you will too.

Happy new year!

Dice Roller Deconstructed

As promised, here are the elements of last week's dice rolling code:

use v6;

This is a nice way to say that we are in Perl 6 land.

subset D10 of Int where 1..10;

A "D10" is a 10-sided die, and it can only have integer values in the range 1..10. Subtyping Int is an acceptable way of taking care of that.

sub is_success (D10 $roll, D10 $target) {

Here, I am already using the subtype D10 of Int. This subroutine compares the rolled die $roll with the target number $target, and is called from the subroutine roll() for each die in the dice pool. I chose to create an explicit subroutine because it seems a bit clearer what happens in the special case of a rolled 10, which means that you get to re-roll that die for a potential new success.

    my $n = 0;
    if ($roll == 10) {
        say "10 again";
        $n += roll 1,$target;
    }

If we roll a 10, then the roll() subroutine is called with a dice pool of 1 and the same target number as we got originally for determining success.

    $roll >= $target ?? $n + 1 !! $n;
}

We always return the number of successes from the roll for the "10 again" rule (if it happened), and in case this roll was a success, we return an additional success.

sub roll (Int $poolsize where { $_ > 0 }, D10 $target? = 8) {

The dice pool size can of course not be negative, but it also cannot be zero; you always get to roll a die, so I have added a type constraint for that. The target number is optional, defaults to 8, and has to be possible with a D10.

    my D10 @rolls = (1..10).pick($poolsize, :replace);

From left to right:

• @rolls is an array that will contain the results of the normal die rolls
• (1..10).pick($poolsize is a way of picking $poolsize dice having possible values in the range 1..10 and "rolling" (randomizing) each of them.
• pick($poolsize, :replace) means that we not only pick a result, but we also make it possible to achieve the same result again. Specifically, it is important for us that each die can have ANY value, not just values that have not been picked before. The semantics of pick() are explained in .pick your game (the 15th gift in the Perl 6 Advent Calendar).

    say "Roll: " ~ @rolls.sort.join(",");

@rolls.sort.join(",") sorts the elements of the @rolls array and stringifies them joined with a comma, e.g. "1,2,3,3,4" for @rolls = 4,1,3,2,3

    [+] @rolls.map: { is_success $_,$target };
}

This piece of code maps is_success $_,$target on every value in the @rolls array and creates a sum of those results. In other words, it sums up the number of successfull die rolls.

given @*ARGS.elems {

The @*ARGS array contains the command line arguments to the program, and .elems therefore is the number of arguments used.

    when 2   {
               say "Target number: " ~ @*ARGS[1];
               continue;
             }

This block only runs in case we have two arguments, but it explicitly says that we may not be done yet: the continue statement counters the default implisit break to ensure that we can match the input value against other tests.

    when 1|2 {
               my $n = roll |@*ARGS>>.Int;
               say "Successes rolled: " ~ $n;
               $n >= 5 and say "Exceptional success!";
             }

We start off with a junction to say that either 1 or 2 is fine by us, we want both to match. Then we call roll() with the same arguments we got in, but each converted to Int. White magic. We store the value, and exclaim that the result is an exceptional success if it is.

    when *   {
               $*ERR.say("roll.p6 poolsize [target]");
               exit(64);
             }
}

This is the equivalent of C's default, the catch-all that handles remaining uncaught cases. We print a helpful usage string to STDERR ($*ERR in Perl 6) and exit with the correct Unix exit code, praying that nobody uses a different kind of system.

Dice Rolls for Role-Players

I realize that the title of this post is a bit of an oxymoron, because a Real Role-Player of course doesn't roll dice often. ;)

But in the cases where the Real Role-Player does roll dice, wouldn't it be nice to have a computer program to forget at home rather than some even more easily mislaid dice?

The Perl 6 Advent Calendar provided some inspiration for this post.

A problem with many minor programming examples you see on the net, is that they do not take into account the needs of a role-player. Role-players play many different systems, with different criteria for success in dice rolls. D6 (the regular six-sided cubic dice used for playing Monopoly, Yahtzee, etc.) are not used much in the majority of systems.

Therefore, I'll look at the Storyteller System, which is used in the World of Darkness series of games.

The general principle is that you have a pool of dice to roll, and you count your successes, which in this system is the number of dice that have a value greater than or equal to a given target number for the roll. The standard target number is 8 in most implementations. Five successes in the same roll is an exceptional success. Obviously, it's nice to have many dice to roll!

Here's a real Perl 6 program that works with Rakudo today: it accepts two command line parameters, the first being the size of the dice pool, the optional second parameter defines the target number for success:

---

use v6;

subset D10 of Int where 1..10;

sub is_success (D10 $roll, D10 $target) {
    my $n = 0;
    if ($roll == 10) {
        say "10 again";
        $n += roll 1,$target;
    }
    $roll >= $target ?? $n + 1 !! $n;
}

sub roll (Int $poolsize where { $_ > 0 }, D10 $target? = 8) {
    my D10 @rolls = (1..10).pick($poolsize, :replace);
    say "Roll: " ~ @rolls.sort.join(",");
    [+] @rolls.map: { is_success $_,$target };
}

given @*ARGS.elems {
    when 2   {
               say "Target number: " ~ @*ARGS[1];
               continue;
             }
    when 1|2 {
               my $n = roll |@*ARGS>>.Int;
               say "Successes rolled: " ~ $n;
               $n >= 5 and say "Exceptional success!";
             }
    when *   {
               $*ERR.say("roll.p6 poolsize [target]");
               exit(64);
             }
}

---

Thanks to moritz++ for ironing out two annoying mistakes!

Here are a few usage examples:

---

$ perl6 roll.p6
roll.pl poolsize [target]

$ perl6 roll.p6 5
Roll: 1,2,7,8,9
Successes rolled: 2

$ perl6 roll.p6 5 2
Target number: 2
Roll: 1,2,2,4,9
Successes rolled: 4

$ perl6 roll.p6 5 4
Target number: 4
Roll: 6,8,9,10,10
10 again
Roll: 8
10 again
Roll: 2
Successes rolled: 6 - Exceptional success!

---

There are no comments in this piece of code, I want people to try to understand it as-is, based on the Perl 6 Advent Calendar. If you have any questions, comments, corrections, etc., don't hesitate, just write!

In my next blog entry, I'll pick the program apart and comment on what I've done and why, and who knows, maybe someone has come up with an elegant solution to the same problem.

GCD - A Small Language Enthuser

fun gcd (x:int,y:int) : int =
    case x of 0 => y
  | _ => if x < 0 then gcd(y,0-x) else
         if y < 0 then gcd(0-y,x) else
         if y > x then gcd(y-x,x) else gcd(x-y,y);

"But that's not Perl!"

Yeah, yeah, I hear you.

I'll rectify that minor detail in a bit.

But first, an anecdote.

Back in the late nineteennineties, I was studying computer science, and one of the classes was about program specification and verification.

Several of the students already had a background with several programming languages, some were functional, some were imperative, and other languages were a bit confused about what they really were.

When studying program specification and verification, you either become rather obsessed with program correctness -- and hopefully elegance -- or you fail spectactularly.

There are several ways to muster enthusiasm when dealing with such studies; they can be rather, ehrm, theoretical.

I therefore flitted about, flirting with various programming languages, comparing them with the eagerness that young idealists do.

For some reason, I found Euclid's GCD algorithm to be particularly fascinating, for reasons unknown to men to this day.

The Perl version I saw was rather awful, and technically incorrect:

sub gcd {
    if (!$_[0]) {
        return $_[1];
    }
    if ($_[1] > $_[0]) {
        return gcd ($_[1]-$_[0],$_[0]);
    }
    return gcd ($_[0]-$_[1],$_[1]);
}

Yikes. I mean, eep. And Perl does have a modulo operator.

sub gcd {
    my ($x, $y) = @_;
    $y ? gcd ($y, $x % $y) : abs $x;
}

I won't claim that the above code is the epitome of elegance, but it solves the problem in a general and easily read way (I admit a prejudice against $_[N]), while retaining correctness.

This is, BTW, one place where some golfers miss the boat; the GCD cannot be a negative integer. That's why the ML code at the top is so verbose.

Small challenges like these kept me going, and it can be an inspiring way to learn details in a new language. So, what would it look like in Perl 6?

sub gcd (Int $x, Int $y) {
    $y ?? gcd($y, $x % $y) !! $x.abs;
}

What's your favourite algorithm for playtesting languages?

What stops me from using Perl 6, today?

Since I got hooked on the Perl community, and got a taste of Perl 6, I've been wondering about:

1. what, exactly, is it that I could use Perl 6 for, right now?
2. why am I not actively using Perl 6 now?

Those are easy questions, but answering is hard, so this may be a long post.

Sure, the points listed below are not exactly Perl 6 specific; I could probably have picked some other programming language, but I somehow feel more comfortable in the way that Perl 6 still is Perl.

What I could use Perl 6 for right now

I think it's fair to say that using Perl 6 today mostly means using Rakudo, and that I wouldn't use it in what we popularly call a "production setting". But many of us programmers, sysadmins, geeks and nerds have perfectly suitable hobby projects, where we won't have clients wringing our necks if there is three minutes of downtime in a month, or if we don't deliver the Speedy Gonzalez of services; we have projects that are neither computing performance constrained or stability constrained.

So that's where I could have started using Perl 6 half a year ago, and of course still can.

I know I can use Perl 6 for e.g. a fairly complex web site using Web.pm and Squerl for a SQLite backend. It will probably work just fine, for a lot of projects.

I know I can use it for lots of one-liner scripts.

I know that in some regards, Perl 6 will outperform classic Perl 5 in terms of programmer time spent. An example is the given-when control structure, which (to me) is semantically superior to if-elsif-elsif-elsif. Programmer time is important to me, I hate coding too much for menial tasks. And I'm sorry to say that Perl 5.10 doesn't do it yet for me, as I cannot rely on its presence, even for hobby projects.

And I know I can use Perl 6 to refresh some of the knowledge about programming language specifics (terminology, technique, methodology, etc.) that I've allowed to rust since I left university in 2001.

Concrete projects, in no particular order

• web page for registration of pool billiards tournament results; it's not performance critical, and the users could check and verify the dataset themselves after input


• conversion of historical results data in CSV format to a database; one-time job, needs manual verification no matter what programming language I use to do it


• contributing to the Temporal.pm specification and implementation in Perl 6


• personal web gallery generation; I positively loathe most of the online galleries, because they sooner rather than later are discovered to have HUGE, GAPING security vulnerabilities


• blogging tool; I'm not very comfortable with blog software running on servers, either, and whatever blogging I do, it's not actual work

That's quite a lot, isn't it? It ought to have been enough to get me going in a jiffy!

Why I'm not actively using Perl 6 now

This may be a surprise to some: it's not because of a lack of matureness in the tools, a lack of confidence in the language or tools, stability issues, etc. As I tangentially mentioned above, I believe there is no technical hindrance for me to start coding on a hobby project.

I have plenty of hobby projects to choose from. They are also quite manageable in terms of eventual lines of code.

However, there is something holding me back, and that's a certain degree of perfectionism mixed with procrastination fever.

mst mentioned during the NPW hackathon this spring that perfectionism was a barrier against getting started. If you're too obsessed with getting things right at first, at wanting to avoid failure, procrastinating is too easy. Getting slightly intoxicated (yup, drinking alcohol, which of course is only a recourse for adults) is a way of reducing your own perfection anxiety. This is almost exactly what Randall Munroe's xkcd calls the Ballmer Peak:



But I don't sleep too well after drinking alcohol, and I also tend to do hobby projects in my "running breaks" during work hours, in which case alcohol intake may be a very bad idea.

In addition, my time at work is a series of interruptions, which really isn't conductive to sitting down and learning something new and complex.

When I get home from work, I'm usually so fed up with computers that I don't want to have anything to do with them.

So my spare time, whatever is left of it, usually isn't spent on programming. Note that I don't even do these projects in a programming language I already know well; they are on hold regardless of that.

All in all, there's nothing much wrong with Perl 6.

Blaming the immaturity of Rakudo would just be a silly excuse. There's something wrong with my capacity for finding the time to get down and dirty with it, that's what; I'm apparently not currently capable of saying honestly:

This is my Perl 6 hour. This hour, I'm going to do Perl 6 stuff, and this time is sacred.

What the #perl6 IRC bots do

Do you feel like a n00b on #perl6, like I do, and wonder what the different bots do?

I keep forgetting what they are, so here's a list for you and me both:

dalek

Announces commits (mainly to rakudo, nqp-rx and the Perl 6 book)

hugme

Used for hugging another user without "direct" contact:

hugme: hug masak

ilbot2

Near-realtime IRC logs with automatic link generation to irclog.perlgeek.de. The original ilbot sucked, according to moritz.

ilogger2

Another logging bot

lambdabot

Keeps track of karma ("moritz++" adds one to moritz's karma score, "frettled--" subtracts one from mine)

lisppaste3

Announces entries pasted to http://paste.lisp.org/new/perl6 (which is where we paste code and other stuff, so that we avoid spamming the channel too much, and also don't have to worry about creating our own temporary web pages)

masak

Submits rakudo bugs. Aw, okay, then, he's not a bot, just a really nice guy!

mubot

Also tracks karma, attempting to be slightly less annoying than lambdabot. mubot is clever enough to recognize that your nick may vary slightly from time to time and channel to channel. mubot is written in Perl 6!

p6eval

Perl 6 code evaluation bot. We use this for live testing of code that may be of interest to others; it chats back to the channel. perl6: my $a; will result in a test against several Perl 6 interpreters (elf, mildew, mildew-js, pugs, rakudo, sprixel), nqp: say('foo') tests nqp-rx, std: my $a will parse the expression with STD.pm.

phenny

Our secretary. Sample usage:

phenny, tell frettled I'll get back to you on that

phenny will then let me know when I become active on the channel again.

pointme

Provides links to projects tracked by proto. Example usage:

< carlin> pointme: rssbot
< pointme> carlins's rssbot is at http://github.com/carlins/rssbot

pointme is written in Perl 6!

pugs_svn

Tracks commits to the pugs repository, most of which are changes to the test suite and spec.

zaslon

Tracks blog posts from a certain group of bloggers. Zaslon is written in Perl 6!

Thanks to carlin, Juerd, jnthn and moritz for late night clarifications!

Some ways that Perl 6 is grand, part 3 of ?

I'll be really brief this time, I promise!

Several things about Perl 6 are there to save programmer time. Some of them even will do so. ;) This week's favourite is pure laziness. Let's imagine we have one of those lists with month names in them again, and for some odd reason only want to do something to each third month.

for ^12 :by(3) {
  say @months[$_];
}

I think that's just neat.

The caret notation is the "upto" operator, and shorthand for a range from 0 and up to its argument. So in the example above, we're asking for a range from 0 and up to 12.

The by adverbial is new in the specification. It denotes the increments for the upto operator, and allows e.g. real numbers, so that you can increment by 0.25 if it makes sense for your code to do so.

The above example then essentially iterates over 0, 3, 6, 9.

The by adverbial isn't yet implemented in Rakudo (well, duh, the spec just saw it added), where you have to settle for this for now:

for ^4 {
  say @months[$_*3];
}

… and something different if you want to increment by 0.25. I hope you don't want to increment through the list of months by 0.25.

Print-and-log in Perl 6

In one of my early blog entries, "Simple print-and-log subroutine", I shared a small piece of code that has been a nice, every-day tool - in Perl 5.

Today, I set about converting that to a naïve Perl 6 version, and being quite the Perl 6 n00b still, I ran into a few hurdles along the way.

The hurdles were easy enough to avoid, once masak++ and moritz++ had bonked my head sufficiently.

I'll walk through the code, step by step, to illustrate what I learned today, and what others might need to know.

use v6;

Ah, first, remember this statement. It's a nice hint for other software if you want to continue using the .pl file suffix instead of .p6l etc.

my $verbose = 1;
my $logfile = "/tmp/test.log";

my $*PREFIX = "plog";

Variables starting with the twigil $* is what we call contextual variables, and they just started working yesterday in Rakudo. Contextual variables follow a dynamic call path. When we reuse this variable later, it will depend on which call path we followed.

sub plogwarn ($msg)
{ 
  my $*PREFIX = "plogwarn";

$*PREFIX now has a different value only for the cases where we've called the plogwarn subroutine, and the subsequent call to plog inherits this value.

  plog $msg,1;

Oh, BTW, it's really, REALLY important to keep track of where you add whitespace or not. I'd been sleeping and class, and forgotten completely that plog($msg,1) would call plog with the parameters $msg and $1, while plog ($msg,1) would call plog with the single parameter ($msg,$1) (yep, a list). It may be safer to avoid parentheses altogether when you're not dealing with lists - except when you have to.

}

sub plogdie ($msg)
{ 
  my $*PREFIX = "plogdie";
  plog $msg,2;
  exit 1;
}

sub plog ($msg is copy, $level?)

The trait is copy means that I want to make a copy of the incoming parameter $msg, so that I can modify it inside of plog. Normally, parameters in Perl 6 are read-only and pass-by-reference (though not "reference" as you know it from Perl); the parameter as named is merely an alias for the actual variable. I can change the parameter in the caller if I use is rw, but that's not what I want to do here. The question mark in the second parameter - $level? - means that the parameter is optional, and I've used that in both plogwarn and plogdie above.

{ 
  my $dt = Time.gmtime.date.iso8601
         ~ " "
         ~ Time.gmtime.time.iso8601;

Normally, I'd use localtime, but this is NYI (not yet implemented) in Temporal.pm.

  $msg = ($dt,"[$*PREFIX]",$msg).join(" ");

Here, I abuse that is copy to save myself one precious variable, just as an example.

  if $verbose {
    given $level {
      when 1 { warn $msg; }
      when 2 { die $msg; }
      default { say $msg; }
    }

The given…when construct is similar to the switch…case construct in other languages, but subtly different, as it allows more possible values and value types than most. Coming from a world of Perl 5.8 and older, this is simply lovely.

  }
  # Append message to $logfile
  my $*OUT = open $logfile, :a;

Here's another use of a contextual variable, but this time it's the one normally associated with STDOUT. print FILEHANDLE $msg is no longer necessary, because you can always assign $*OUT in its own scope. The open statement has :a to indicate that I'm opening for append (so the syntax is less unixy than Perl 5's >>).

  say $msg;
  $*OUT.close;
}

This is how we can use the three subroutines, and the output should illustrate the different contextuals nicely:

plog("Oh, hello, there, sweetie.");
plogwarn("Consider yourself warned");
plogdie("Die, frackin' monster, die!");

…outputs…

2009-09-06 19:59:32 [plog] Oh, hello, there, sweetie.
2009-09-06 19:59:32 [plogwarn] Consider yourself warned
2009-09-06 19:59:32 [plogdie] Die, frackin' monster, die!

And here's the complete example code, which works with the current Rakudo:

use v6;

my $verbose = 1;
my $logfile = "/tmp/test.log";
my $*PREFIX = "plog";

sub plogwarn ($msg)
{ 
  my $*PREFIX = "plogwarn";
  plog $msg,1;
}

sub plogdie ($msg)
{ 
  my $*PREFIX = "plogdie";
  plog $msg,2;
  exit 1;
}

sub plog ($msg is copy, $level?)
{ 
  my $dt = Time.gmtime.date.iso8601
         ~ " "
         ~ Time.gmtime.time.iso8601;

  $msg = ($dt,"[$*PREFIX]",$msg).join(" ");

  if $verbose {
    given $level {
      when 1 { warn $msg; }
      when 2 { die $msg; }
      default { say $msg; }
    }
  }
  # Append message to $logfile
  my $*OUT = open $logfile, :a;
  say $msg;
  $*OUT.close;
}

Some ways that Perl 6 is grand, part 2 of ?

Okay, this is really part 1b of ?, but…

In my earlier post, I used the zip operator to join two lists into a hash.

There was one obvious use of the operator that escaped me at the time, and that was how I sometimes need to create a new hash from the keys of two hashes, or keys and values. And now I think it's starting to look neat:

my %A = { a => 1, b => 2 };
my %B = { z => 9, y => 8 };

my %AB = %A.keys Z %B.keys;
# { "a" => "z", "b" => "y" }

%AB = %A.keys Z %B.values;
# { "a" => 9, "b" => 8 }

However, this is a bit unpredictable, since the hash key order is undefined. So if you expect sorted keys, do that at the same time:

%AB = %A.keys.sort Z %B.keys.sort;
# { "a" => y, "b" => z }

# Sort by B's values - two variants
%AB = %A.keys.sort Z map { %B{$_} }, %B.keys.sort;
%AB = %A.keys.sort Z %B.sort.map( { .value } );
# { "a" => 8, "b" => 9 }

The equivalent Perl 5.10 version would be:

use List::MoreUtils qw/zip/;
my @k = sort(keys(%A));
my @v = map { $B{$_} }, sort(keys(%B));
%AB = zip @k, @v;

I now have a nice-ish argument for upgrading to Perl 5.10.1 on $workplace's servers. :D

masak++ for helping a tired me with the map expression.
Chas. Owens++ for spotting the missing use statement for Perl 5.
Pm++ for another way of sorting by value, just what I was hoping for!
isec++ for spotting a missing sort() for Perl 5.

Autovivification - a reminder

Most of you already know this by heart, but the odd reader may have forgotten.

Autovivification is what we call the process of automatically creating entries in built-in data structures (Perl 5: array/list and hash), usually at the time we check whether an inner element exists or not.

This can be a royal PITA, if you don't pay attention to the problem. That's why it keeps being mentioned.

Here's a simple example:

my %hash;
my $n;
while (!exists ($hash{x}) && $n < 5) {
    $n++;
    if (!exists ($hash{x}{y}) {
        print "hash{x}{y} does not exist: $n\n";
    }
}

Q: How many times does the above while loop run in Perl 5?
A: Once.

The simple matter of checking the existence of the inner hash resulted in an entry being created for $hash{x}.

That means that tests like these should be written more carefully:

…
    if (defined ($hash{x})) {
        if (!exists ($hash{x}{y}) {
            print "$n: hash{x}{y} does not exist.\n";
        }
    } else {
        print "$n: hash{x} is undefined.\n";
    }
…

This prints $n: hash{x} is undefined. (with an incrementing $n) five times.

---

Edit 2009-08-24 18:49 UTC: MST commented that there is an autovivification module on CPAN that lets us say no autovivification; - and it's even lexically scoped! That's just $notreallyanexpletive brilliant! Thanks, Matt, and thanks, Vincent!

---

Oh, BTW, Perl 6 has a useful specification for autovivification, which illuminates the problem further.

Hash key sort order - a Perl 6 community sunshine story

The order of hash keys is implementation dependent and arbitrary. Unless %hash is altered in any way, successive calls to .keys, .kv, .pairs, .values, or .iterator will iterate over the elements in the same order.

S09 - Hashes

This is new. It may not actually say much, but it does say what was implicit before, so that there is little room for doubt.

So here's my little sunshine story about how easy it is to clarify a part of the spec.

In my previous post, I used an imaginary case for showing off some features of Perl 6 - some of which also are available in Perl 5.10, as mentioned by Robert 'phaylon' Sedlacek in a comment.

This wasn't the only useful comment, I think there's a bit to be learned by reading those, so please do.

But I digress from the point of this post, which is a question that was raised in another comment to last Sunday's post:

By the way, do you know where the behavior of ~%h is spec'ed? I keep getting the keys back in the same order I put them in and don't know if that is an implementation quirk or a feature.

- Chas. Owens

The spec wasn't very clear about this; S32/Containers - Hash said that certain iterator methods iterate "… the elements of %hash in no apparent order, but the order will be the same between successive calls to these functions, as long as %hash doesn't change."

S09 - Hashes didn't say anything about it at all.

I said I would ask around. Thanks to the excellent community channel #perl6 on Freenode, I got an answer similar to this: no, this is unspecified/undefined behaviour, but feel free to come up with a better way of saying it, and update the synopses.

And how hard is it to update the synopses? Not at all! If we want to contribute, we get access. It's as easy and simple as that.

First, you need to check out the (part of) the svn repository that you want to contribute to:

svn co http://svn.pugscode.org/pugs/docs/Perl6

Then you change whatever you want to change, preferably discuss it with some of the experienced souls on #perl6 or e.g. the Perl 6 language mailing list, show diffs on e.g. gist.github.org or paste.lisp.org, and if you think you're doing the right thing - commit the change.

"But I can't commit, I only have read access" you might say. Just ask in any of the mentioned fora for a "commit bit" and state your e-mail address and preferred username, and someone will help you out with that part.

As we say on #perl6: community++

I got started, will you join me?