Introducing new tools

It is exciting to get into a new project and inject new ideas and tools. The project that I am talking about is a generic report runner pulling its information from a database.

In its current incarnation it spins up from cron, and runs reports one at a time, but it now needs to run multiple reports including bulk reports. To do that I am making heavy use of POE, and a worker process pool. And, I am getting to use my own tools such as POEx::Role::SessionInstantiation, in addition to other great tools like MooseX::Declare.

Not only that, but I pulled in local::lib for dev purposes, and Dist::Zilla to handle the disitribution creation.

It's like being a kid in a toy store.

Of POE + Moose Awesomeness

So Florian finally dropped the bomb. Devel::Declare, MooseX::Method::Signatures, and MooseX::Declare were all released with all of the wholesome developed goodness that I wrote before YAPC::NA. So what does that mean? It means that I was able to finally release to CPAN a few modules.

POEx::Role::SessionInstantiation is the core of it all. It turns your Moose objects into POE::Sessions and manages the ugly for you.

POEx::Role::TCPClient builds on POEx::Role::SessionInstantiation and rolls up the common SocketFactory/ReadWrite wheels and gives you awesome functionality with minimal effort.

POEx::Role::TCPServer does the same thing as TCPClient, but in reverse.

I chose to stay the lower level for the roles because I found those pieces of the POE world ripe for Moosifying. The upper level Components are a bit unwieldy and I am not a big fan of them.

And once I work the kinks out of POEx::ProxySession, I'll be doing a release of that. I decided to add some more functionality with regards to getting notified from the POEx::ProxySession::Client when TCPClient fires socket errors and connect errors.

And then I'll be able to release an alpha of Voltron. Florian brought the parsed signature attribute on MooseX::Method::Signatures::Meta::Method up to public so that means I can rely on it now for doing signature comparisons instead of dumb string comparsons that I am doing now.

Expect to hear more soon on this front. I'm rather excited about getting to release my modules finally.

Working with Rakudo and Parrot

So after pmichaud had a call for applications for Rakudo, I came to a small epiphany that perhaps I should work on a naive port of POE to Rakudo. So this past weekend came and I started to do a little investigative work on what it would take to do POE on Rakudo. The first step is to get the core loop running. In Perl5 terms this means that select(2) needs to exist. In Perl5 world, select() is for the most part a C equivalent. There is another version of select() but that doesn't concern us.

Anyhow, select() let's us do synchronous multiplexing of IO across multiple filedescriptors. This is how POE does it's magic. As things come back from select() placed into the things-to-watch queue via POE::Kernel->select_read and select_write, those are placed into the Queue and get mingled in with all of the rest of the events that need to be dispatched. This is an obviously very simplified explanation.

So the first thing I did was go into various IRC channels to ask some very pertinent questions such as where is select() implemented? I dug around in the classes defined by Rakudo but could not find a definition of select. But someone pointed me in a couple of different directions and referred me to a couple of different people that were interested in the same topic.

Ultimately, it came down to this. I took a look-see into the Parrot source and found that, no, select was not implemented as a generic plain-old C-like call you could make. Instead, each of the Parrot socket objects have a method, poll, that will return to do it's status, if any. Internally, it uses select(2) to do this for the unixy source file anyhow.

Unfortunately, Rakudo has no parallel in the IO classes because it was decided to /remove/ select(2) (including the other version) from the spec. Apparently, people wanted to roll that up into the concurrency layer that doesn't really exist yet, because it is pretty instrumental in event loops. In the mean time, I'm for the most part stuck. Sure I could do a IO::Select port to Rakudo (and had planned to do just that), but I would have to implement it using the non-portable Q:PIR construct and call down to the Parrot sockets methods. I'd have to learn PIR, ontop of any augmentation to Perl6 that I barely know. Plus, it would only work on the Rakudo implementation due to the use of inline PIR.

With that said, I decided to stop looking at doing a naive port until the IO layer is fleshed out a bit more and I have safer abstractions from which I can do simple things like use a select-like idiom that is extremely common for writing socket based network servers. The Parrot and the Rakudo projects have come a very long way in their years of development, but I just don't have the time to be writing core infrastructure pieces and also maintain and further develop my other modules. I really understand the catch-22 that Rakudo and Perl6 in general have when it comes to specs vs implementations, but I can't really do much to fix it without investing a lot of time and effort coming up to speed on the thoughts behind a lot of the decisions (such as removing select()).

I tried to approach this project from the perspective of bringing current Perl5 tools forward to the Perl6 world. That means attempting a naive, get it running, port that doesn't bother with any of the new fancy idioms. At most, I wanted to adjust the sigils, and do some light source filtering by hand to cover the large breakages. Document what I found for other projects looking to do the same. And go from there. I couldn't really get to square one because of the specification vs. implementation situation.

So while I certainly sympathize with these projects, I just don't feel they are ready yet for the casual developer (eg. me) to even take the first steps. I would like to at least revisit the issue for the Rakudo Star release, however, given more of the IO layer has been implemented and I can do what I want without resorting to platform specific inline assembly.

Let the flames begin :-/

Not enough time

"It would have been really cool to use module X because I could have used the same config for both things. Just didn't have enough time." Or "there just isn't any time to go back and redo some of it." Or "this thing is hacked together because we didn't have any time". And that is kind of hard to wrap my head around. Yes, time is always against us when the customer is yelling for feature X, complaining about bug Y and deadline Z is looming. They are writing the checks, after all. But, there has to be a point where something needs to be more "right" than "right now." I've worked on projects with some tight time lines. I've had to compromise on features, workaround bugs, and ship something I felt wasn't 100%, but I don't think I've kludged something together without ever revisiting it and correcting it.

As a programmer who takes pride in his work, it is hard to hear these kinds of things. Because to me, it sounds like the people making those statements are resigning to accept that the hackery and the shortcuts are worth more than the increase in maintenance time down road.

So how does all of this apply to Perl? Backticks. Network daemons that don't use POE. Manual parameter validation. Etc. All of the things that make Perl a swiss army chainsaw, also make it very easy to forget to engineer your software. That is why I like things like POE, Moose, Catalyst and others. Those modules help you to engineer your software. They discourage you from doing things like using backticks or reinventing wheels.

And using those tools to begin with decreases your development time. Those projects have solved a lot of the hard problems for you. They are forward thinking and keep their userbases in mind.

I wish I had a solid answer for people with the perpetual time problem.