DBIC::API

With work deadlines, holiday baking, and playing in the snow (in Texas no less), I got sidetracked and never wrote the advent article I promised for the Catalyst folks. I feel horrible.

As if that wasn't bad enough, apparently there are more complex configurations out there that I didn't account for in my DBIC::API branch (that is now merged in and is trunk), so it breaks people's apps.

So I know what I will be doing next week: fixing the brokenness.

Short post. See you next year.

Of more nerdhackery

I admit I am a dork. So I recently started playing Magic: The Gathering again with real-live cards. And it is rather a pain in the ass to figure what I need to build a bad ass deck with out shelling out for some stupid guide that lists all of the cards in a set.

Luckily, someone was awesome enough to type up all of the cards in various sets (including the activation text, if applicable). But you can tell the goofballs manage this data as a flatfile of text. It is very regular though. If only that were a database...

So step one is to build a schema using DBIx::Class that represents the various fields present in the file (with some of them broken down further so you could separate things like power and toughness instead of being a single field). Next, write the line-by-line parser using simple regex. Convert parsed data into created rows. And like magic, we have a database.

So what can we now do with this database beyond the boring crap like searching for cards by casting cost? Write a deck analyzer of course!

I haven't executed this part yet. But I will. And it will be most excellent. See, I figure I can take this data and throw rules engine at it. Provide a Cat app for building decks and running it through the rules engine to determine things like "Not enough land to support this deck."

Doing a quick search on CPAN returns a number of rules engines to use, but I think FSA::Rules shows the most promise. It is simple enough to not get in the way, and lets you define your own internal data.

The Cat app could obviously do more like build out dumb starter decks based some rules too. So far I haven't found any thing on the internets that does what I am wanting to build. This is good.

Hooray nerdhackery!

DBIC::API Hackin'

I've just finished up my branch of Catalyst::Controller::DBIC::API. What a beast. It is functionally complete and backward compatible, but I will likely continue to tune it, write more tests, etc.

For my work project, I was bumping up against some pain in the ass things when trying to make DBIC::API play nice with Extjs's grid stuff. I needed some more configurability in what the data looked like going to and from the server. This included properly handling JSON booleans as well.

So I embarked on an ambitious task to add those features in, plus Moosify it. I also wanted to do more validation up front before ->search() was ever called. You wouldn't even know if your ->config() parameters actually fit your ResultSet until the first request came in and you watched it bomb.

The validation part took the longest as it involved the most yak shaving. The outcome from that though, is Data::DPath::Validator. Making use of Yuval's wonderful Data::Visitor, I wrote a parser that generates Data::DPaths, which are then used (ala Data::DPath->dpath) to validate data. So you feed it a template (replacing the allowed, variable pieces of the data structure with asterisks) and it will create all of the valid paths for the template, then feed it a data structure to see if it is valid. By default it operates in loose mode so only one path needs to succeed for the data structure to be valid, but you can easily flip it into strict mode where /all/ paths must pass for the data structure to be valid.

What this tool gained me in DBIC::API was a way to verify everything from search parameters, to prefetch parameters based on the provided config options (which were templates). I admit that my tool was more general purpose than the limited use in DBIC::API, so I had to subclass and flatten some aspects of the generate DPaths.

Beyond that kind of validation, I shifted goodly chunks of the request munging stuff into its own class with triggers for validating against the schema of the model. So most of the aspects of validation happen in one place, at one time, and very early in the request.

Anyhow, you can check out branch of DBIC::API here.

And you can take a look at Data::DPath::Validator on CPAN here.

Or even the github repo here.

Of getting closer

So the work project has take a giant leap forward through burning some midnight oil last night. Through my failings, I've come to better understand Catalyst, how jQuery and it's plugins are nice, but not the right tool for the job, and how reverting back to trust Template::Toolkit, while ugly, gets the job done.

That said, I have a working datagrid. Some values are hardcoded at the moment to match the schema, but eventually, I will have it all automagical, down to the generation of the edit fields based on either data_type or additional metadata stuff into column_info.

So what was it that finally let me do the grid thing that I wanted? Ext. Now, I know some of you really disdain Ext as a company. And you have every right to dislike them. They commit the worst licensing FUD ever. Head on out to their site and see what I mean. Basically telling people that they must pay for a license if they are developing "commercial" software. Except that completely ignores the fact of how the GPL (version 3 specifically) works. in the grand scheme of things, I am developing "commercial" software in that it serves a business purpose and money is changing hands, but what Ext fails to make clear is that use of GPL software in those cases is covered. I am not distributing anything. I can eat up all of the GPL'd code I could ever want and never release a damn thing. The terms of the license are awesome like that. And yet it is "commericial."

Anyhow, beyond the bullshit with Ext licensing FUD, they have a rather solid UI model in their javascript library that very closely fits how desktop toolkits work. You can have windows, GTK/TK style layouts, decent events model, etc. Not only that, they have *gasp* a GridPanel. Throwing in some abstractions for "proxying" data, storing it, and providing a consistent interface for the UI to use, you have a very solid paradigm for doing web UI.

But there have been some bumps along the way. Catalyst::Controller::DBIC::API, while providing a fantastic set of functionality, leaves much of the customization up to the end developer to figure out. So I peaked inside the module and about went blind, heh. No, it really isn't that bad, it just needs some love and more of the steps it takes broken down some more into finer grained portions to allow the kind of customization that I required. For example, Ext's JsonReader and JsonWriter expect things in what I could consider a more normal format. Except DBIC::API doesn't allow you to make those changes. Oh sure you can override the format_list method, except you have no clue what to do without opening up the source. And there are lots of places like that. Where I had to explicitly dig into the source to figure out how to apply customizations. but in the end, I have the two talking to each other and even delivering metadata to the JsonReader/Store instead of hard coding things on the javascript side.

So after I get the work project squared away I will revisit DBIC::API and give it some lovin'. And then take that, with some Ext GPLed code, and build the real automagical datagrid that I wanted to build from the beginning.