Of the long road to usable POEx::Roles

This past week was punctuated by lots of frustration. But the good kind really. As of yesterday, I have a working POEx::Role::TCPServer. That is, the simple POE::Wheel::SocketFactory and POE::Wheel::ReadWrite bundle on top of POEx::Role::SessionInstantiation(pxrsi) works as expected without surprises. It took all week to get there really.

Pxrsi had problems. A lot of problems. While in the simple cases, it would work as designed, as soon as you threw in a Wheel that fiddles with the states of the instantiating session, it go sideways, off the cliff, into the valley of lava, inhabited by flesh eating zombies. The biggest problem was that my advised methods like _start were getting called in an infinite loop, especially if the first _start executed touch the states and kicked off a _clone_self. Chatting with Matt Trout about the problem had a startling revelation: when constructing an anonymous class, roles that you give to the method are applied. Which makes sense if you if you step away from my crack filled idea of what I was doing (anonymous class clones to enable per instance method level changes). So my methods were getting advised multiple times.

And so that led to lots of hilarity. in TCPServer, I am instantiating a SocketFactory on _start. Well if there are multple _starts, the second will fail because the first has already bound to the port. But of course, that isn't obvious at first. In the depths of the debugger, I was noticing that as roles were being applied that the instantiated SocketFactory was /going out of scope/. WTF? So yeah, I chased that one for a little while.

So I made a few changes to pxrsi: only clone once when needed, revert tracing from fancy advised methods to simple statement before invocation, etc. The only clone once thing made a lot of sense. It would only happen if an instance received a _register_state event, and from that point on, you are working with an anonymous clone class specific to that instance anyhow. So that in the end is a big performance gain not having to clone the class every time a state is registered or removed.

Awesome, but then I came across another problem. I was making use of Moosex::AttributeHelpers in POEx::Role::TCPServer for wheels management and I had defined a number of "provides" methods. Of course, the _invoke_state heuristic for determining events from non-events was failing. And that is the problem with implicitness, in this case.

But the day was saved when Cory Watson announced work to support method traits. Holy crap that is exactly what I needed. MooseX::POE (mxp) had added a declarative keyword for defining events. And it made sense because of how mxp was architected around POE::Session. I wanted something a little more magical. So it wasn't a day or so before Cory had the method traits stuff working. A couple of git remote updates later, and I had declarative events. So _invoke_state and _register_state were rewritten to handle checking for a POEx::Role::Event role and like magic, the simple cases were working.

So I scurried back to POEx::Role::TCP and it was still failing. The coderefs installed by the various wheels shipped with POE obviously expect a POE::Session API compatibility. Hrm. Then reworked part of the method responsible for wrapping coderefs delivered from wheels (a horrible hack, I might add).

Then mysteriously another failure cropped up in pxrsi that took me awhile to figure out. My test wouldn't even run. Of course, this was after a week of changes from Devel::Declare, to MooseX::Method::Signatures, to MooseX::Declare (basically the whole chain of dependencies under pxrsi). The message I was getting was rather cryptic. It (Moose) was complaining that it couldn't find one of the advised methods in the inheritance hierarchy for the Role. It kind of blew my mind. It was working before! But then I remember that I had made a small change to the test to advise one of the methods in the class being defined rather than in a class defined from an event handler.

The way MooseX::Declare works with roles definition is that it defers application of those roles until after the class has finished parsing. And with my changes, if you remember, method modifiers don't get executed until after roles have been applied. So I was a little perplexed. I don't know if this is a bug in Class::MOP/Moose or what, but the roles being applied weren't actually loaded, yet. But once you had an instance of the class, your role was ready to go. So I patched MooseX::Declare to call load_class as the roles were encountered to ensure they are loaded before being applied. And that fixed my advising.

So what is the result of all of this? POEx::Role::TCPServer works and is very simple. Scarily simple. And you can advise any portion of it. This means that you could around advise handle_on_connect to SSLify a socket before letting TCPServer do the mundane stuff, etc. It is pretty cool.

But ultimately, why am I reinventing the wheel, you may ask. I came across a tiny little problem when working on Voltron last weekend. I need a much, much simpler model for session proxying than what IKC provides. Basically, I want to be able to "subscribe" to a remote session and have it instantiate a local, persistent proxy session. So I need to write that. And well, if I am writing things like this, I might as well eat my own dog food and build up all of the toolchain to make that happen. So that includes things like TCP server and client roles, etc. It worries me somewhat because I only have a month left before YAPC and that means, I need to get this thing working as quickly as possible. Hell of a time to decide that I need to replace one of the core components.

I really feel that writing these roles will decrease complexity in developing POE applications. For me, it always felt like there was this tension in writing good, clean OO code and making use of POE. Oh sure you have object states and whatever, but the machinations of POE would shine through for every constructor that instantiated a POE::Session object. I want to make the encapsulation complete, eventually. Ricardo Signes is working on some MRO::Magic module that promises some awesomeness. It would be very cool if all your method invocations, magically, behind the scenes carried out the POE stuff without having to be too concerned about it. So posting, yielding, and calling would look like ordinary method invocations on an object, but the object itself would essentially be a proxy. Still a long way off from making that happen, but we're getting there.

MooseX::Declare and MooseX::Method::Signatures

In my on going efforts to have code written before YAPC::NA for my Voltron talk, I came across an odd problem. Well, first, let's back up and talk a little about Voltron. Basically, I am combiningPOE, IKC, PubSub and Moose to create an application server that can introspect into connected pieces and automagically setup applications and participants to each other. The introspection is the key part.

Anyhow, MooseX::Declare supports a method syntax that is pretty cool. Basically it gives you a Perl 6 feel in Perl 5 land. That said, it was missing an important feature which is return type constraints which I had found out experimentally. I had made a naive assumption that the method syntax had feature parity with MooseX::Method::Signatures. It didn't.

So I make mention of this in #devel-declare to Florian Ragwitz, Robert Sedlacek, and the others. Come to find out MXD /reimplemented/ bits from MXMS, instead of making use of the module directly. So I peek inside the code for both modules. Ultimately, both modules make use of Devel::Declare to work their magic, so that helps. But they way they go about using that module is slightly different.

MXD actually does most of it's processing via a Devel::Declare::Context::Simple object. The context is contained, and lots of stuff is delegated to it. If I was going to make MXMS parse methods inside of MXD, I was going to need to pass the current context to MXMS and have it operate on that.

MXMS doesn't instantiate a Context::Simple object, it is the object. Or rather a subclass once removed. So instead of delegating operations to a separate object, it is making calls to $self. Obviously that isn't going to fly.

First, MooseX::Method::Signatures needed to have it's context split out from the rest of the code. MXMS was actually subclassing Devel::Declare::MethodInstaller::Simple which is a subclass of Devel::Declare::Context::Simple and it was doing that to get a method for parsing attributes (not the Moose kind, the Perl 5 kind). Handy, but if I was to split out the context and absorb a context from MXD, it needed to be Context::Simple and nothing more. So I copy/pasted the attribute parsing code in to MXMS and tweaked it to operate on a separate context. The rest of the module needed all of the instances of $self->foo turned into $ctx = $self->context; $ctx->foo. But once that was done, it was easy street from then on out.

Then for MXD to make use of MXMS, all I had to do was instantiate it, pass it a context, and tell it to do it's thing. Rockin'.

Except now, my previous work to defer method modifiers until after roles consumption broke. MXMS had no clue that it was supposed to defer method modifier application. Nor should it. So how is that reconciled? Well, what if MXMS had a coderef to call upon method application, so that you could override the default $meta->add_method or $meta->add_method_modifier? So that is what I implemented. Now in Keyword::MethodModifier, I simply pass in a closure around what should happen (pushing method modifer application coderefs onto the modifiers_store for later execution).

Awesome. Then Ash noted that applying the around method modifier to something that provided an invocant would break things because you would end up with /two/ invocants (the $orig coderef and the original invocant) and that would break Parse::Method::Signature. And because the old MXD code was playing with the method signature and creating a Meta::Method object itself I had cargo culted that behavior since I didn't want to fundamentally change anything. So my shiny was broken. Hrm.

So we discuss options on how to fix. I suggest that perhaps we make modifications to Parse::Method::Signature::Sig to allow for creating and manipulating them, because that is what I was doing anyhow, but with raw strings, so I could change the invocant and futz with the arguments. Then Ash provided a Good Enough solution which was to dig into Meta::Method inside MXMS, and inject the $orig parameter. And so I did. Anytime the declarator was 'around', I injected $orig both into the lexicals and into the positional args. Clean fix.

So after all that what is the next result? Full support of MXMS style methods inside of MXD including return value type constraints. So now in Voltron, I have full introspection ability which will be need for autowiring participants to applications.

Naming things is hard

So this past week, I finished up the evil arcana known as POEx::Role::SessionInstantiation. Well, at least that is how I ultimately named it. But that decision to name it as such wasn't an easy one to make, largely because I was straddling two separate fiefdoms and neither could seem to agree.

Originally, I had thought of naming the module POE::Session::Moose. At the time, it made sense. It was a Session implemented in Moose, except it was more than that. And it wasn't a subclass of Session per se. So I queried around for possible name ideas with somewhat neutral persons. Fresh on everyone's mind was the idea of using POEx as a root namespace. Matt Cashner actually pushed the idea to the forefront recently with a /topic change in #poe. And since my module was really a Moose::Role, POEx::Role seemed like a great starting point.

Then we kicked around ideas for the last name. Matt Trout suggested readability within the Moose system (ie. ->does(RoleName)) which tickled all the right language processors the right way. Sessionify? Sessionize? Kind of hard to pronounce. SessionInstantiation came up. ->does(SessionInstantiation) reads really well. So it was settled: POEx::Role::SessionInstantiation. Even shortens to something goofy: pxrsi or PEH-ZERCK-SEE. But to make sure I wasn't going to be stepping on anyone's toes by leaping into the POEx namespace first, I broached the topic on #poe.

It was not well received. Many reasons were given, but the biggest reason was control. The POEx namespace from previous conversations on the matter years previous, was supposed to be a reboot and salvation from the POE::Component "swamp." Where the POEx namespace and its descendents essentially define an interlocking, guaranteed behavior. A laudable goal. One that I thought my carefully considered name aligned with. There are many aspects in the POE world that could easily be reimplemented as Roles. As we know from chromatic's evangelizing, Roles == behaviors.

My efforts to get the ball rolling on the conversation, to get a decision made, to get some sort of boundaries drawn around the problem (at least defining the various aspects of the POE world that would be better represented inside a POEx::* namespace) were futile. And finally it was decided that perhaps it would better fit in a Moose namespace (such as MooseX).

Hrm. Not discourage, I started the conversation about naming in #moose. I was met with resistance. "This seems more POE specific than Moose specific." And in the near past, the discussion on having a MooseX::Role namespace was just line noise. So there was no where for me to go. It was offered to perhaps nestle inside MooseX::POE, but that didn't feel right largely because Chris Prather's approach and mine are so completely different.

Eventually, I engage both camps at the same time and let them duke it out. Back and forth on naming happened more or less like a tennis match with each side reasoning against it being in their space. Talk about feeling the love, heh. More suggestions came up to put it in the Class::* namespace, but that doesn't work because it is a Role. Someone thought maybe start a new Top Level Name: Role, but that doesn't suggest Moosiness. But then again, Moose is the only object system that implements Roles in p5. All in all there was a lot of mental masturbation.

And in the end, what was decided? Nothing. Consensus was never reached. So after a few days of that, my new, shiny code was still sitting in a git repo without CPAN distribution. So I was left to decide on my own really. The way that the CPAN indexer works, there are no real constraints on names so long as there isn't a collision. My efforts were mostly to not piss in people's cheerios. And that didn't pan out. Decision by committee simply doesn't occur with any kind of efficiency or expediency. Especially when that committee is made up of two differing groups each with their own ideas and designs on naming things.

POEx::Role::SessionInstantiation stuck. I liked it. It was generated with outside opinions before engaging each camp. It reads nicely in practice. It properly describes the behavior bestowed upon the composed class. And so it was uploaded to the CPAN.

The only good, I feel, that came of this was an absolute refutation of the concept that Perl is a dying language. There are a plethora of cabals out there, each with their own fiefdom in the Perl kindgom. And that is healthy. There are so many expansions of technology within the Perl space, that while we may all not agree on certain things (names for example), we can all agree that our language of choice rocks.

POE + Moose with POEx::Role::SessionInstantiation | You, too, can do evil dark magic with Perl | IRC is a better friend than Google

And deeper down the rabbit hole we go.

So it all started in IRC the other day this past week, with Martijn van Beers talking about using MooseX::POE to rebuild Client::TCP. And I had a kind of epiphany. What if POE::Session were actually a Moose::Role. Think about it. What if you could take any object and turn it into a Session with out invoking the arcana yourself? That would be pretty cool right? Suddenly, the object_states of POE::Session become a first class concept. Subclassing a Session because you just want to make a couple of tweaks to a couple of methods becomes as simple of providing around advice for those methods. Your object becomes the ultimate POE cyborg. But how could a Role be so transmutative. With lots of fuckin' poking and prodding anything is possible.

So I shared this little epiphany and started hacking on it that night. Until dawn. And by then, I had a very simple working solution. After consulting some great POE::Session implementations on the CPAN such as POE::Session::Cascading and POE::Session::PlainCall (and stealing their ideas), I figured out that I needed to call session_alloc to register with the Kernel, and also to implement _invoke_state to receive events. So I did.

Inside the Role, I provide a dummy BUILD sub and then after advice for said BUILD sub (in case the consuming class has a BUILD and overrides my dummy one) to register with the Kernel. That took a little bit of brain power, but was oddly enough provided by Martijn again by regurgitating that exact information I needed from the #moose channel about this very topic. Pure awesomesauce. Next was _invoke_state. I spent a little time digesting the Class::MOP documentation (which is quite good), in order to figure out how take a given string for a state/event to be invoked and actually do that. And it was pretty dumb simple: find_method_by_name(). Follow the normal POE::Session behavior of sending things to _default if there is no event found and we are set.

Proof of concept complete. Time to flesh it out a bit. I wanted to be able to change events at runtime. So what does the Kernel do to make that happen? It calls _register_state on the Session. Groovy. I go back to the Class::MOP docs to figure out how to add a method to a class: add_method(). Well that's just too damn easy! I expand my little proof of concept tests to include using $poe_kernel->state('new_state' sub {}); and everything is peachy keen. Except, what about removing? Ah, look. remove_method(). Awesome.

I go off to parade my effort around and that is when Rocco Caputo points out a giant glaring naivety: I am changing the actual class, not the instance that Session represents. Adding a state to one instance effectively adds it to EVERY other instance. Holy leaky Sessions, batman! So back to the drawing board.

What about subclassing? Anonymous subclasses per instance. Sounds evil. Will it work though? I give a whirl. My 'new_state' test is passing. My 'remove_method()' test fails. WTF? I double check the meta object. Yup, has_method() tells me that my anonymous subclass does not have that method. But of course, further up the inheritance tree, my parent does. So find_method_by_name and, hell, invoking the method /works/. At this point, I am bitching. Both ruby and ecmascript support per instance manipulation of methods. Sigh. So I am desperate. I travel to the land of #moose and wail and generally gnash my teeth. That is when Chris Prather had this bang up idea, so evil, so magical, so awesome, that he deserves full credit for it: anonymous class CLONES. Not subclasses, but actual, full out, clones. I consult Class::MOP::Class and Moose::Meta::Class. I see all of the tools available for what I need.

Queue generic john williams evil march music.

So while I was doing this, I stumbled upon a horrible side effect to my evil, evil, subclassing/cloning ways: it breaks POE. So I do some spelunking into the guts of POE to figure out what is going on. Come to find out, POE uses the stringification of your Session object to basically track it internally. And since my cloning creates an anonymous class and I rebless the Session into said class, my stringification changes. Not all of it mind you, because the memory address is still the same. I still have the same object, it is just wearing a bunny suit instead of a santa claus suit.

Ultimately I was breaking POE encapsulation anyhow because I needed to store the newly reblessed Session object into the current slot, but down the road, the mismatch between the stringifications was killing me. And so the evil gets deeper. I overload "". Add an attribute to the Role to store the original stringification. But everything was still broken. After some deduction (print statements instead of the debugger because the debugger was giving me a SEGV, for the lose), I determine that while my sub routine for overloading stringification was indeed composed into the class, the magic of its execution was not happening. Hrm. Querying #moose again returns positive: Shawn Moore notices in the source of overload.pm that it does some symbol table hackery to enable it's magic:

${__PACKAGE__ . "::OVERLOAD"}{dummy}++;
*{__PACKAGE__ . "::()"} = sub {};

And obviously, Role composition doesn't invoke this particular incantation. So I throw it into "after 'BUILD'", and make sure that the symbol table hackery takes place in the anonymous clone classes, too. Now the magic happens in my composed class, which means now my stringification evil works, which means that POE now works, which means it all works! Well, in the simple cases.

There is always more. I wanted to add tests to make sure that composing an anonymous class with the Role wouldn't fail for some weird reason. And so I added that while inside one of the events for the test Session. And, surprise, it fails. But not because it is an anonymous class. I was trying to set an alias on the new anonymous Session from the constructor (the alias attribute has a trigger to call alias_set() on the Kernel) and that alias was getting added to another pre-existing Session, my parent. Silly me. I was still inside my parent's context. POE helpfully switches contexts between Sessions so that when you are constructing a Session, its _start will fire within it's own context, but I was jumping the gun. I didn't have a valid context yet and therefore my alias was getting added to my parent. Add some checks. Make sure to clear/restore my own context after each invocation (to handle parent/child situations). And I am set.

And along the while I also added tracing that looks just like POE::Sessions tracing, but a bit more complicated. Tracing happens on a per method basis and each method actually has around advice applied to it dynamically. I know, not exactly the most performance oriented way to do it. It will likely change to a simple statement just before invocation.

Whew. It works though. You can see the evil arcana incarnate out on github or clone the repo from me here. After I finish up the docs and get it prepared and ready for the CPAN, I'll likely look at doing Real Work with it next. Maybe start an experimental branch of POE::Component::Jabber using it.