Jun 26, 2010

Solving code generation problems in dzil

Firstly I want to clarify a bit on my opinions of PluginBundle::USERNAME modules, as some comments there have inspired this post. I don't think you should use them because it makes it harder to disable plugins, and I think Robin Smidsrød put it best:
Mostly it is because the Dist::Zilla::PluginBundle::USERNAME doesn't actually say anything about its intention. It only says use Dist::Zilla as this person does, but what does that actually mean? If you don't know the person it doesn't really tell you anything.

I'd much more prefer PluginBundles that actually advocate certain types of standards or behaviors.
...

Essentially Bundle's like @Git and @Basic don't cause problems because they're generic well defined and contained within their distributions. A bundle that wasn't contained within its dist might not cause a problem if it's for a generic well defined purpose. I'd be in favor of a BundleQATests or similar so long as the author was considerate that some tests (PodSpellingTests) don't work well on some *nix distributions and isn't included in the bundle. But Something like @Xeno (my cpan username) would be my own special settings... who wants to use something that's only for me? and more importantly why would they ever think they have a right to bug something that the name itself implies it's only for me.

But another problem was brought up... code generation. Here's what Nilson said:
dzil is very nice, but I don't if I like the idea of different line numbers and files in the repository vs. the CPAN release. I'm still trying to fully digest this idea.

I remember one of the main mentioned drawbacks of source filters were the possibilities of error messages in the wrong lines. And now, everyone seems to embrace this without hesitation.

So to start dzil's generation isn't quite as bad as a source filter which will give you the wrong line number period, using this generation will still give you the right line number for the module being run at the time. It just means the cpan line number may not match the repo line number. I'm going to show you how to fix this, but you don't always have to. The most common place I've found problems with in code generation has been tests created by extending InlineFiles. In these cases you just want to go and bug the author of the Plugin. I think using these plugins is much better than writing your own EOL Tests, NoTabs, Critic, and Kwalitee etc. It helps keep your dist quality up without extra work, ultimately leaving you with just the responsibility of writing good tests that are specific to just your distribution.

Now on to making sure that your repository matches your cpan dist. The first thing you want to do is use [Git::CommitBuild] (if you're not using git look for something like it or write something like it) This will take all the generated output and commit it to another branch each time you build. Then make sure this branch is pushed to your public repository. I think the biggest problem this solves is having things like your README and your LICENSE actually be in your public repository (if your repo doesn't have a LICENSE... what's the legal situation?). If you're using github you can also change this to be the default branch to be displayed in the repo admin settings.

Next thing is don't use ::Plugin::Prepender This will evil-y insert lines at the beginning of your code which will definitely throw off the line numbers being output. It even seems to suggest using it to insert use strict; use warnings;. If you need output prepended to all files, I suggest writing a plugin that takes advantage of dzil new and maybe just write a script that you can fire off whenever you need to create a new file. This is the only module I know for sure that does this, but avoid ANY that insert actual code or prepend lines to your files (in a way that isn't added to your actual 'master'/'trunk' that should be patched).

POD can also screw up your line numbers, if you're using PodWeaver (or module that has similar side effects) with dzil, which you likely are. Perl Best Practices page 140 will save you here (actual quote pg 475 a summary chapter).
  • Keep all user documentation in a single place within your source file. [Contiguity]
  • Place POD as close as possible to the end of the file. [Position]
This includes the # ABSTRACT: my abstract here line. If you put the # ABSTRACT and any actual pod after the code then all the pod generated will be after the code in your build, and thus any line number errors will be correct. Of course this doesn't save you if your error is in the output pod, but I suspect that's not the original complaint anyhow, and there are lots of dzil plugins to help you keep your pod sane.

Essentially don't do anything that will change line numbers for the code in the resulting build output. Following this will ease contribution, and debugging; I do not believe it significantly increases maintainer load. Happy Hacking!

UPDATE:
just remembered... dzil inserts a BEGIN block... sigh... can't win for nothing.

UPDATE:
I recommend using Dist::Zilla::Plugin::OurPkgVersion to avoid dzil's BEGIN block / VERSION insertion.

10 comments:

  1. With regard to the "Second Branch" for built distributions, I completely agree with this attitude, I have been doing so for quite some time, just the overhead of maintaining it is a bit pesky.

    It gives me nice guarantees for users, that in the event I remove something off CPAN, it will always be available on Github, even in the event the given version was so short-lived it missed being on Backpan ( it hasn't happened to me recently, but I've seen it in the past ).

    Additionally, the convenience of seeing the exact differences between any 2 released dists with good ol' 'Git Diff' trumps that search.cpan.org diff tool by miles.

    I've started work on a way to generate the other branch with Git, but its inordinately complicated, especially if you want to do it the RightWay, the RightWay being generating the other branch without ever switching from the current one. ( Which requires a bit of Git mastery, and using Git::PurePerl to do all your dirty work ).

    As for the making the release branch the "Default" I have to recommend against this strongly simply from the contributor perspective. It makes contribution *harder*, especially when combined with source-rewritery.

    When somebody forks your dist to patch it, it defaults to the same branch you defaulted to, and when they locally checkout, it again, defaults to this same branch, and they will by instinct, develop patches vs the release branch instead of the current master, and that's highly counter-productive. Reintegrating their patch then becomes a little more challenging beyond the simple "git rebase" and "git merge", and can make proper history preservation impossible.

    ReplyDelete
  2. If they'd half a brain and read your submitting patches doc (that I'm sure you don't actually have) it wouldn't be a problem. If anyone can't figure out not to patch master... and sends me a patch that doesn't apply to master I'll send them right back to the SubmittingPatches doc.

    ReplyDelete
  3. I think you may be missing the point of @USERNAME plugin bundles.

    I (and I suspect most CPAN authors with @USERNAME bundles) do not actually expect people to use these bundles in their own modules. I put my own bundle (::PluginBundle::MSCHOUT) on cpan only so that if people want to build *my* CPAN projects using the stock dist.ini, they can get everything they need to do that with "cpanm Dist::Zilla::PluginBundle::MSCHOUT". If I didn't publish the plugin bundle on CPAN, people would have to install a slew of plugin modules by hand just so that they could build my projects. I publish the @MSCHOUT bundle on CPAN for the sole purpose of making it easier to contribute to my Dist::Zilla enabled projects. I do not expect anyone else to use the bundle @MSCHOUT in their own projects. To me, this is very different from "feature" plugin bundles like @Git.

    I can't name my plugin bundle based on its behaviour for various reasons. First, Dist::Zilla and its plugin ecosystem are rapid moving targets. The behaviour of @MSCHOUT changes as git changes. Second, @MSCHOUT encompases so many other plugins, that it is impossible to come up with a name that fits.

    ReplyDelete
  4. s/as git changes/as Dist::Zill changes/. *sigh* :)

    ReplyDelete
  5. You should have a handful of bundle's not 1. If you only want to simplify installation ... isn't that what a Task is for?

    ReplyDelete
  6. Simplification if installation is not the point.

    Having the same dist.ini in all of my modules is the reason for using a plugin bundle. I disagree about using a handful of bundles. If I did that, then later decide to add another plugin to my dist::zilla style, I would have to update the dist.ini in *every* CPAN module that I maintain. I have more than a handful of modules, so that is not fun. However, with my own plugin bundle, I just update the bundle and I am done.

    I don't *have* to put the bundle on CPAN. But doing so is a courtesy to anyone that might want to contribute to my projects.

    ReplyDelete
  7. So you don't think that any of your modules should have different configurations... No wonder I don't do this. One module I like doesn't work with all of them TestSynopsis is a good module imo. But it doesn't work with my dzil plugins because their synopsis is usually a snippet from an ini file and thus not compile-able perl code. I have NoTabsTests in another module (not my project) because I like tabs but need a reminder not to use them.

    ReplyDelete
  8. Almost all of my modules simply need [@MSCHOUT]. For the 2 or 3 that differ, I use something like:

    [@Filter]
    bundle = @MSCHOUT
    remove = MakeMaker

    Still much shorter than listing out all of the plugins. It works for me. Again, I don't expect anyone to use @MSCHOUT other than me, unless you are contributing to my projects.

    ReplyDelete
  9. uh... you obviously didn't read my previous article...

    ReplyDelete
  10. @vendull: when you change your PluginBundle you might break the build of several of your modules without knowing it.
    In the end it all gets down to the same old question: update your module every time a new version of Module::Install comes out which fixes some bugs or use another module that doesn't include itself in the dist but relies on the installed version.

    ReplyDelete

No trolling, profanity, or flame wars :: My Blog, my rules! No crying or arguing about them.