"The DarkPAN is too big and scary to change Perl 5's defaults all even by the time of Perl 5.14" -- chromatic
DarkPAN was discussed quite a bit at YAPC::NA. But really, what is it? How is it defined? Who are these people that have a vested interested in Perl and yet do not participate? I think attempting to appease a faceless entity with no defined boundaries has fail written all over it. It is fear mongering.
If we break it, they can choose not to upgrade. What about bug fixes?, I hear you say. That is the joy of open source. There are lots of unemployed hackers out there that would be willing to backport those fixes for you. Oh heavens forbid! Actually paying open source hackers to write code! The sky is falling!
We need to stop this meme of DarkPAN. Perl does not belong to DarkPAN. It belongs to us who participate in its wellbeing. And if the amorphous corporate overlords are angered by our changes, then they need to get involved and list their grievances, just like Joe OpenSource Hacker.
But until we hear /anything/ from this unholy DarkPAN, I say we make the decisions best for our language of choice. I once had a professor that was fond of saying "Silence is tacit consent." And in this case, I would say that is extremely apropos of Perl stewardship.
Subscribe to:
Post Comments (Atom)
Amen brother.
ReplyDeleteSome of the people at YAPC NA worked for companies that are part of the DarkPAN. I know I have worked for those sort of companies before (and I was there). Remember, DarkPAN is usually maintained by organizations, not individuals. Anything that increased the cost of upgrading decreases the likelyhood of a corporation upgrading. For instance, Abigail's company is still using 5.8.3 because the cost to retest the code with 5.8.9 is too high. The answer here is to encourage people to write more tests for their own code and Perl. Better tests means less fear of upgrading.
ReplyDelete"Anything that increased the cost of upgrading decreases the likelihood of a corporation upgrading."
ReplyDeleteAnd that is fine. There is nothing wrong with that. It fits their business model. More power to them for making the choice that benefits them the most.
So with that said, how does that affect continuing development of new, exciting features by default in newer perls? It shouldn't. 5.10.0 should have had all of the magic stuff turned on by default, but it didn't. And that is horribly broken.
"Remember, DarkPAN is usually maintained by organizations, not individuals."
And those organizations can appoint someone to speak for them. They do it all the time for representatives in various consortia. But if they don't have anyone advocate for them or contribute back to the community in a meaningful way, then they shouldn't get this implicit say in what is default for Perl because it might break their undertested code, especially if they aren't going to upgrade /anyway/.
We should not indefinitely leave out the good features from the default because it might upset some curmudgeon code. The previous versions of perl will always exist and will always remain free. Those organizations that have anchored themselves to a particular version because of business decisions should not affect Perl going forward.
I think it's a deeper problem. Retesting/certifying software for new versions and platforms is a normal part of IT. Certainly some companies are slower to adopt new versions, but for the core, supported bits there is a process in place for upgraded to newer version of Linux, updates to J2EE, newer versions of Spring, etc.
ReplyDeleteFor some reason Perl doesn't get on this track. This points to a reluctance and a deep apathy toward supporting Perl at all levels in many companies. This is way it's acceptable for many companies to simple order IT to continue using a 7-10 year old version of Perl (or worse) because there is some vague plan to get rid of all the Perl apps someday, which never happens because Perl is so useful.
So Perl lives in this sort of perpetual twilight, the dark and semi abandoned child of corporate IT.
Let me summarize here. This summary is the result of talking to lots of folks at YAPC, ranging from corporate perl users (darkpan victims) to core developers.
ReplyDelete*Fuck DarkPAN*
As a developer, if you don't talk to me, I really can't be overly concerned with your needs. well, unless you're paying me gobs of cash to be concerned.
So, let me say this again so my thoughts on the matter aren't misunderstood.
Fuck DarkPAN.
Hi. I am the DarkPAN.
ReplyDeleteWe have 250k lines of Perl code supporting a billion dollars worth of ordering and logistics, which delivers everything from the Google swag in your closet to hospital drug order forms, and the national emergency body bag supply contract for major emergencies.
We hire 7 full time Perl developers, at good salaries, including 2 (soon to be 3) CPAN authors. We even have an open source policy, sponsor local conferences. At least 5-10 CPAN modules exist primarily because we allow spin offs from work originally done to support our business. And those recent releases to the Perl SAP connector that remove all the memory leaks? We funded those.
And yes, we care when some module we critically depend on arbitrarily breaks (we depend on 85 CPAN modules directly).
We care when anything just changes behaviour with no warning, like when sudo changed a security setting and broke a few pieces of our infrastructure.
We run an offline CPAN mirror, so your breakages are buffered and don't hurt us immediately, sure.
But when you make big changes, they absolutely cost us time and money, and slow down our release schedule.
And we can tolerate some motion from time to time, but for a select few modules we'll have 20k or 50k lines piled on top, and it's no obvious from your perspective which ones those are.
But let me confirm that we absolutely do appreciate the stability and back-compatibility of Perl, and the CPAN.
It helps us stay bespoke and productive, helps justify Perl as a language, and helps us resist suggestions that we throw the whole thing away and rebuild everything in SAP or WebSphere or something else painful.
Genuinely, but unofficially, yours.
Adam K
(Comments do not represent the official position of the company etc etc)
@AdamK: It sounds more like you are dealing with CPAN breakages rather than changes in the core interpreter. It's a lot to track. Sounds like you do a great job of doing it too. And I empathize when things change behavior without warning.
ReplyDeleteBut how does any of that affect new, major, releases of perl including new semantics by default? Are the deltas not delta-y enough?
Do you use 5.10.0? Do you have the new features enabled? If you don't use 5.10.0, or if you do and don't have the new features enabled, why bother with the upgrade if it buys you nothing (except a slow down with a common my ($foo) = @_; idiom)?
If the change in core perl is that suddently the sigils start to be invariant as will happen with Perl 6 then this certainly not good news to companies like the one AdamK is working for.
ReplyDeleteIf the change is that suddenly there is a new say() or class {} keyword in the language then it might break things but if the in-house developers pay a bit attention to what's going on with perl then it can be easily avoided or fixed.
If the problem is that 5.12 suddenly has "use strict" enabled by default and your code was not working under use strict then you will be indeed in trouble. But frankly, in that case you are already in trouble.
We don't use 5.10.0. In fact, we don't really get to choose the Perl we use.
ReplyDeleteUnless there is an absolutely critical reason we need it, we have to use a supported Perl. And that means, for us, whatever Perl comes with RedHat.
We've got a lot more freedom about which CPAN modules we use, but we don't abuse Perl the language enough (and we don't burn enough CPU) to justify rolling our own Perl.
Support is more important than language features.
That said, when the core language changes, it's absolutely going to impact us. Every CPAN module that breaks because of a change adds to the likelyhood something we depend on will break. And who knows if in a codebase this large whether or not there's some language thing we use.
What we need, really, is some way to make it possible (and easy) for us to analyse our codebase in advance for expressions that might be changing or causing problems.
Change is tolerable, but change that requires us to hand-audit the entire codebase is extremely problematic, and change that doesn't have a quick (preferably automated) method for upgrading our current code is also problematic.
@AdamK So, I have to ask, how good are your test scripts? I have found that the level of fear is inversely proportional to the quality of the organizations test suite.
ReplyDelete@NPEREZ As opposed to yelling at people for fearing change, we should be showing people how to write good, automated test suites that will reassure them that nothing bad will happen to them, or where the bad things are if they exist.
Breaking things that people depend on without warning is a drag, sure. But, when is there not warning?
ReplyDeleteDon't all the Perl releases go through a long development process where any organization is free to pull down the latest changes for their internal testing?
If you depend on the Perl that is shipped with an OS release, then you might need to work with the OS Vendor to determine their upcoming plans wrt versions so you can test appropriately.
I note that Oracle on HP/UX ships with a Perl that is used for management scripts. That's an alternative; support your own Perl. Perl is NOT hard to install. I understand that you want to be able to get Perl support from Red Hat or Sun, though, but no decision like this doesn't have some costs.
I started to reply and then realized my reply is too long winded for comments. See my reply in this blog post: How we should address the DarkPan Problem
ReplyDelete@AdamK a propos "analyse our codebase in advance for expressions that might be changing or causing problems" - something like deprecated code analyzer?
ReplyDeleteBreaking things that people depend on without warning is indefensible. Breaking things that people depend on, with or without warning, increases the cost of upgrading. Test suites allow some of the cost to be avoided but an unavoidable cost will remain. It is not fear that makes one wish to avoid this unavoidable cost but common sense in the face of limited resources.
ReplyDeleteThe DarkPAN contributes to the acceptance of Perl, even if those who maintain it don't contribute code to Perl directly.