CloudPAN

It is funny that my returning post would concern clouds as I am actually writing this while above real clouds over nine kilometers up. While attending the Orlando Perl Workshop (aka. Perl Oasis), I was a selected speaker for a short talk about one of my modules: CloudPAN. I figured I would do a little bit of a write up about CloudPAN and why I think it is useful and how it could be even more useful.

But before getting to the point of talking about CloudPAN we need to take a slight detour and talk about the awesomeness that is MetaCPAN. If you've been living under rock for the past couple of years, you might not know that MetaCPAN is quite possibly one of best things to evolve out of the Perl world recently. MetaCPAN is important to the Perl community for a number of reasons including: it provides a really simple API for interrogating the CPAN indicies, it also provides an open source web platform for displaying CPAN information.

The project was born out of frustration at search.cpan.org. I don't have all of the history available to me at the moment, but it is sufficient to say that not every one was happy with how search.cpan.org worked. Not only that, people couldn't provide patches or even fork it if they wanted.

So how does MetaCPAN provide the magic that it does? It actually mostly relies upon ElasticSearch. ElasticSearch is a fantastic tool built on top of Lucene that provides all sorts of awesome scalability via clustering, multiple indexes, and basically takes Lucene and applies it in a tremendous way. In fact, the API that MetaCPAN provides is really just a thin wrapper around what ElasticSearch expects with a few bells and whistles that let us do things that enable modules like CloudPAN.

CloudPAN was really just a silly idea that came about during the QA Hackathon 2012 in Paris. David Golden wanted to enable the CPAN.pm client to talk to MetaCPAN in order to reduce it's footprint. If you can directly query some external service about available distributions and modules, there is no need to download large gzipped files and build out local caches. But! There was a snag. The MetaCPAN::API module actually had a bunch of deps that aren't exactly core. So part of my time spent was implementing MetaCPAN::API::Tiny that only relied upon HTTP::Tiny to talk to MetaCPAN. With HTTP::Tiny getting into core, that meant the CPAN.pm client could grow up without external deps.

And it was during that work that I noticed a rather curious method to the MetaCPAN API: file. This is how the source display works on the site, basically. The file api basically opens up the distribution, finds the file, and slurps in for you. The gears started turning at that point. Could I really write a dumb INC hook that made a call to MetaCPAN for the source to modules that weren't installed locally? Thus, CloudPAN was born. I aptly demonstrated pulling in Moo, building classes, using those classes, etc. all without having actually installing Moo.

It took me sometime to do a little clean up and throw together some docs. After all, it was just a useless hack (the most useless at the QA Hackathon). But then I found myself using it on a regular basis when trying out modules to get a feel for how they worked and if they provided the right solution to the problem I had. I figured people would also like the ability to try out modules. So it ended up on CPAN.

The current version even has the option to persist what you've downloaded previously. And when using it again, it will load from that location. No need to download the entire depenency graph again for a particular module just because you accidentally CTRL-D'd your re.pl shell.

I'd like to grow it up further and perhaps teach it do other things, such as fetching from your own local MetaCPAN installation. And maybe even have it do authentication and verification of the remote. This could eventually end up as a way to distribute pure Perl deps for scripts on the first run.

But I'll save that for another day. Give CloudPAN a try if you find yourself wanting to evaluate modules, but don't want to clutter your local perl installation with modules you'll never use.

Returning to the fray

Expect me to start updating this blog again. I plan on contributing to the Moe project (http://github.com/stevan/moe). I am also doing some courses from Coursera. Some work stuff might bleed into this blog as well (including when my posts are published on our official work blog: http://blog.booking.com)

Quick update: I've moved to the Netherlands. If you remember, I had quite an atrocious experience at the hands of the TSA. That experience stuck. From that point on, I knew my country had given up on its citizenry. I eventually sought out employment elsewhere-- As in, not within the bounds of the US.

I am now a team lead at Booking.com working on some of the more critical infrastructure pieces. We live in beautiful Amsterdam. I had taken a break of sorts of doing any serious open source work. I am now going to change that a little bit.

With the announcement of Moe (and my photobomb making a cameo in Stevan Little's slide deck[https://speakerdeck.com/stevan_little/perl-is-not-dead-it-is-a-dead-end?slide=164]), I've started the trek to learning enough Scala to be dangerous. OPW2013 really seemed to energize me if only because getting together with a bunch of really smarty-pants hackers always seems to have that effect.

My inaugural return post will be about CloudPAN if only because it is fresh on my mind. I gave a talk at OPW2013 about this nifty little hack that fell out of my attending the QA Hackathon last year. And from that point on, I'll try to do a weekly update.

It might be more often. It might be less often. But I want to at least average one post a week.

So expect a post later this week. I have a long flight tomorrow and I intend to do some hacking and also some writing.

Sensible Letter

I just contacted my House representative with the following letter. It is a more properly measured correspondence, filled with less ragespeak and more call-to-action words. I wait patiently with baited breath for my form letter response thanking me for contacting him about gun control laws. Sigh.


----

Regarding TSA experience:

Mr. Joe Barton:

After refusing to be have my body irradiated and naked pictures shown to "highly trained security professionals", I was given an "enhanced pat down" which had no patting involved, only lots of unwanted rubbing and general molestation. This is not security. This is blatant violation of my person. Then when I voiced my dissent on this process using strong language (as any reasonable person would do under this circumstance), I was "invited" to have a conversation with a manager in a suit and a uniformed law enforcement officer. At this point, I asked if I was being detained or if I was free to go. My question was not answered until after the LEO and manager arrived where they attempted to intimidate me into surrendering my first amendment protected right to free speech. I was then accused of causing a "scene" and was given the threat of "not flying." I argued for my protected right to free speech, but then realized that this was not going to change anything. I reiterated my previous detainment questions and was told I was free to go so I left immediately.

After having such a terrible experience at the hands of these poorly trained "professionals," I urge you not to further support any bills that will increase funding to the TSA for any additional staff or equipment and to vote for any bills that limit the TSA's governance. Situations like mine are not increasing airport security, they are violating individuals rights and persons.

Sincerely,

Nicholas Perez
[contact information redacted]

Fuck you TSA

I refused to be irradiated, so instead I was molested. And because I had the unmitigated gall to speak my fuckin' mind about this I was then "invited" to have a conversation with a couple of fuckin' goons. Did you know that these assholes think they are protecting this country? And that because I said "Mother Fucker" to one of them that I was causing a "scene." Of course I am you baffoon in a blue shirt. Causing a "scene" is not against the law. MOTHER FUCKER is constitutionally protected speech. And to educate TSA a bit more, saying FUCK is not the same thing as shouting fire in a theater. Learn the difference before trotting out that tired old line to justify your horrible argument for censorship. A note to anyone else ever in a similar situation: take pictures and remember the magic words: "Am I being detained? Am I free to go?" if they answer no to the first or yes to the second, just walk away. These fucktwats are simply not worth the time.

Pittsburg Perl Workshop and Stuff

So this past weekend was neat. I love coming to workshops even if it is a very compressed time frame. My take away from this workshop is that I have wanted to do some hardware hacking for quite sometime, but never got around to purchasing an arduino board with some addons. So perhaps in the next few months I'll start working on something. Something network-enabled and practical. We'll see. Not that there is any direct Perl that is running on these boards (the code is some kind of C-derivative with a compiler/IDE), but you can use Perl to talk to it over a serial port or whatever.

And while I had set off to write a POE::Filter that spoke the Minecraft SMP protocol this weekend, it never materialized. Instead, I started working on modernizing yet another one of my modules: POE::Filter::XML::RPC. The problem is that I am subclassing XML::LibXML::Element and it doesn't play nice with subclassing. So I wanted to extend it, using Moose and advise all of the methods I could find that returned other Elements or Nodes. That was going to be a very very large copy/paste job. MooseX::Declare didn't support shortcut method modifiers (around [qw/foo bar baz/] {...}). So last night and this morning, I hacked that support into MooseX::Method::Signatures (and its use is transparent to MooseX::Declare). All so I could have a single line of copy paste instead of a lot more. I'm so god damn lazy sometimes.

And interesting side effect of adding this functionality into MXMS: you can have stringified array references as method names, heh. In MXD, the declarators are given specific meaning and functionality via callbacks so that the array references are absorbed and passed on to the meta munging methods that add the method modifications.

Anyhow, I've made the pull requests to rafl, and just waiting for him to incorporate and release.

If anyone is ever on the fence on whether or not to attend Perl workshops, let me settle that for you: GO. It is an awesome time. Everything from learning to networking opportunities await any avid Perl programmer. And this gave me ideas for next year's set of talks that I need to write (I didn't speak at this workshop, I was all tuckered out from speaking earlier in the year).

Hope to see new faces at OPW in January.

Of Modernizing My Own Modules

It is amazing how much bitrot accumulates over the course of a year and a half. And also amazing to see the state of the art move so rapidly.

Dist::Zilla has a break-neck pace of development. While my old dist.ini files still run without issue, so many new and awesomer features have crept into the core. This makes it worth while to revisit your dist.ini files even if changes are minor in your project. AutoPrereqs is smart enough now that only a few MooseX::Declare edge cases are missed. PodWeaver is just plain awesome and if you aren't using it to generate your POD something is wrong. The introduction of the Basic PluginBundle means you have even more fine grained control over your distributions. If you are looking for a basic template of a dist.ini that you can use in your own projects, take a gander at mine http://xrl.us/bh3y2k (Link to nickandperla.net). And if you've looked at my documentation for modules and like my POD style, here is my weaver.ini too: http://xrl.us/bh3y2x (Link to nickandperla.net)

And while I like modernizing my code, this latest push for several modules wasn't entirely voluntary. Newer perls have broken MooseX::Method::Signatures (and likely Devel::Declare itself). This affects a large chunk of my code as I have grown very accustomed to the idea of having constraints on my methods. The brokenness lies in the parsing itself. Seems it doesn't like there to be a newline between the method declaration and the opening brace. In other words, to avoid this problem you have to use ugly K&R-style braces. So that is what I have been doing for a lot of modules, moving the braces.

But, other modules, it was good-old-fashioned modernization. A new version of POE::Filter::XML was released last night that brings it into the modern era. It was an old module in the sense that the distribution still had distribution artifacts stored in source control. POD was all done by hand and at the bottom of the files. POE::Filter::XML::Handler didn't even have any documentation. I wrote my own god damn accessors. It was ugly.

So I put in the time to make things right. It should be backwards compatible (to an extent). Node was updated to be a proper Moose class (using MooseX::NonMoose::InsideOut) that extends XML::LibXML::Element. This lets us, among other things, override methods and call super() when appropriate (making sure that methods that return Elements actually return Nodes). All of the custom constructors went away. And the logic was greatly simplified by using attributes with native traits. The code simply /looks/ modern.

Next on the chopping block is POE::Component::Jabber. I plan on stripping the functionality down to a mere Role. Also, I am going to remove support for pre-XMPP connections, and server specific connection types for components. I want a mean, lean Role that can be composed cleanly and easily. I also want it to be easily extensible too, so if other enterprising developers /want/ those other connection features, they can implement them and use them without monkey-patching my code.

In other words, POE::Component::Jabber is disappearing and will be replaced with POEx::Role::XMPPClient in the not too distant future.

So don't be afraid of looking back on your modules and vomiting a little in your mouth. It doesn't take /that/ much time to modernize them. That way the next time you want to use something and find a bug in it, you won't cringe when you need to fix it and do a quick release.

Of Perl Jobs

So I've been on the receiving end of a couple of unsolicited emails regarding open positions in a couple of companies. And it is odd to get these kinds of emails, especially when they aren't from recruiters, but from the hiring manager themselves. It is a little flattering, to say the least.

That said, the one thing these emails/companies/etc have in common is that the culture that makes me the most productive just isn't there, mentioned, or encouraged. Now, one could say my workplace is a little unorthodox since we are all virtual, but in the grand scheme of things, it isn't the virtual part that matters the most. What matters the most is that our company isn't isolated from the community.

We actively participate on CPAN, IRC, message boards, and most of the major modern Perl projects mailing lists. In our day to day, our communications circle encompasses the community. We contribute to important projects. We release generic solutions to problems we've solved in the course of our business. And we make use of others' solutions as they have released them. Frankly, I don't see how we could get much done if we /didn't/ invest as heavily in the community as we do.

So when I get emails or see job postings from companies that fail to mention any level of community participation, they are placed into the round file holder. And I am not alone in this. The people that I would consider my peers, other community members, the "rockstars" for which these job postings are seeking, aren't about to jump ship from a culture of shared commons and productivity to one of isolation and take-but-not-give-back.

So in the future, if you recruiters or hiring managers wish to cater to truly senior level Perl developers, Perl developers working with the state-of-the-art, please take the time to understand that there are deeper motivations than merely salary, location, or even the business domain.