shlomif

[	Tags	|	blogging, cpan, cpanhq, feature, news, perl, perl 5, perlbuzz	]
[	Current Location	|	Home	]
[	Current Mood	|	tired	]
[	Current Music	|	Carmen and Camille - Shine 4U	]

W00t! Andy Lester published a feature I wrote about CPANHQ on Perlbuzz. This is in continuation to my previous blog entry about the topic, but much more focused.

This publicity seemed to have work, because we've been visited by two new Perl people on #cpanhq, who tore my hacky Catalyst code to pieces. Anyway, check the article out, and if you like it, upvote it on reddit.

shlomif

A few times in the past, I wanted to find perl 5 CPAN distributions that only had a Module-Build-based Build.PL file and not a fallback Makefile.PL file. Yesterday, after some trial and error, I was able to formulate Yahoo Search query, to do just that.

Here it is - the shortened URL with a hyperlink to the full one:

http://xrl.us/bezbkx - Yahoo Search to find Build.PL only distributions

It works by looking for specific phrases in the /dist pages, looking for "Build.PL" and specifically excluding "Makefile.PL". I hereby place this URL and whatever associated techniques under CC0 / Public Domain. Enjoy!

shlomif

Limbic Region recently wrote a post on his use Perl journal titled "Improving search.cpan.org (Advanced Search)". The post concludes by saying that "Feel free to add your own [ideas] - I know someone who may actually be motivated to implement some of them as an alternate to search.cpan.org is reading :-)". Since I talked with L~R about CPANHQ, a seacrch.capn.org alternative in question, that I have been contributing to, I can assume meant me.

Before I reply to L~R's points, let me introduce CPANHQ. As documented in the first and only wiki page it has so far (which I wrote two days ago), "CPANHQ aims to be a community-driven, meta-data-enhanced alternative to such sites as http://search.cpan.org/ and http://kobesearch.cpan.org/. Currently, the functionality is very basic, but we have great ambitions."

I learned about CPANHQ by overhearing a conversation on Freenode's #perl channel. I joined the effort, and made several enhancements to the Catalyst and DBIx-Class-based code, which was a useful learning experience. The functionality is still very basic, and there's a lot of work to do, so any help would be appreciated. Just fork the github repository ( or my repository, which I tend to keep somewhat more up-to-date ), and play with the code.

Now back to L~R's suggestions. At first I was a bit sceptical of them and they seemed like an overkill, and not really useful. When I'm using search.cpan.org to search for a distribution, either I know the name of the module that I'm looking for and just perform a search to be directed to the location of the exact page, or I usually find the order of the search sufficient for my needs.

However, on a second reading, it seems like what Limbic suggests (like being able to sort results by Kwalitee, creation date, number of downloads, last update date, rating, etc., providing a code search, or recommendations) actually has some merit and should be considered. One thing I'm sure about is that the default CPAN search should remain simple, while only giving an option for an advances search using a different form, where it might be possible to formulate a more complex query using a specialised language. Otherwise, if we complicate the existing search, it could prove confusing to many users.

L~R's suggestions remind me of my CPAN Module-Rank thought experiment, that aimed to define an automatically calculated metric for the relevance of CPAN modules.

L~R later referred me to a post to Perlmonks.org titled "Advanced CPAN Search?" where someone asks a similar question.

To sum up, please consider monitoring or even contributing to CPANHQ if you want to improve the future of finding stuff and interacting with CPAN. It seems that some people have issue with the current CPAN search state-of-the-art, and would like to see it improved. We'd be happy to hear what you think.

shlomif

The Tel Aviv Open Source Club will host a talk by Yaron Meiry ("Sawyer") about "Programming Red Flags") on 28-June-2009.

The meeting will take place at Tel Aviv University, at the Schreiber Maths&CS building, room 008 on 18:30. So mark your calendars. More information can be found on the wiki page for the meeting.

Attendance is free, it is not necessary to RSVP and everyone are welcome.

With any other problems, feel free to contact me.

Abstract

Some programming habits should be avoided at all costs; some should be considered with care and some are just divine. Red flags tell us what to try and avoid and what to inspect more closely. They are a sign of code that might become problematic or straightforwardly dangerous. The talk will be Perl-oriented (with Perl examples) but the programming practices are generic and can be applied to most programming languages, so if you program in Ruby, Python, PHP or other languages, this talks to you as well.

Hopefully by the time we're done, you'll be able to write cleaner, more maintainable, less error-prone, and better code. If not, your money back guarantee!

shlomif

Just for the fun of it, I decided to join the Iron Man Blogging Challenge, so here's my first post.

I recently released a new version (1.0300) of HTML-Widgets-NavMenu, my CPAN module for generating navigation menus. This release fixes a bug in the "HeaderRole" type of navigation menus (which I believe is not widely used) and also cleans up the internals. Instead of my ($self, %args) = @_;, there's now my ($self, $args) = @_;, which is cleaner and should also be a little faster.

You can see HTML-Widgets-NavMenu in action on my homepage. By going to various pages, you can see the navigation menu expands or unexpands in certain places, and the breadcrumbs' trail and next/previous/up links updated.

The story of how I created this module is interesting. It started with a lot of ugly and crufty Website Meta Language logic that evolved to generate my homesite's navigation menu. I didn't really like it so I look for an alternative. I looked for one on the CPAN and found two modules that didn't seem too inviting. Then when I mentioned my quest on Perl-Israel, a fellow Perler said he created a hack based on his module for generating JavaScript menus to do just that and uploaded it to CPAN. That module seemed to have the most user-friendly interface, so I decided to use it. However, I patched it, inherited from one of its classes, and created a lot of code of my own.

Eventually, I ended up with only original code, and so I wrapped things up, licensed it under the MIT/X11 licence and uploaded it to CPAN. Since then, it was improved to some extent, including adding 100% Automated Test coverage at one point (which didn't prevent the HeaderRole bug).

So far I received two five star reviews of the module on CPAN Ratings, and also heard from other people using the bug tracker and on E-mail. As a result, I believe this module is moderately popular, and fills a nice niche.

If you're looking for a module to generate navigation menus for your site, then please take a look at it. And if you have ideas for improvements, I'll be glad to hear from you.

shlomif

Before I start with the entry itself, I should note that someone copied my "Recent Hacktivity" meme. So I am going to tell my team intimidating lawyers to send him a "Cease and Desist" letter for Copyrights/trademarks/patents/trade-secrets/whatever violation. (Just kidding)

OK, I did a lot of work on Perl-Begin. I went over all the pages and updated all the out-of-date information I could find, and then also implemented a decent "Testimonials" sidebar instead of the half-assed "These companies/projects are using" Perl that there was there earlier. Next, I plan to have to create a tools/IDEs/etc. section.

I also did some work on the Better-SCM site. Most of it was done by going over the emails in my relevant folder and implementing their suggestions.

I did a lot of work on Freecell Solver. I implemented Simple Simon support in Games-Solitaire-Verify, and verified all the FCS-generated Simple Simons solutions up to PySol deal No. 5,000. I also prepared updated Simple Simon statistics. I also did some Perl and PDL work on creating a meta-scan that minimises the average solutions' lengths generated by the solver. This worked out pretty well. Finally the code in the Freecell Solver trunk is now worth over a million U.S. Dollars (at least according to SLOCCount by David A. Wheeler).

I did some extensive work on refactoring and cleaning up the Statistics-Descriptive CPAN module, and released version 3.0000 to the CPAN. Next I'd like to create a perltidy theme to correspond with my personal style, so I'll be able to reformat the code there properly. At the moment, a lot of it is still using 2-whitespace-wide indentation levels, which is annoying.

Next I tended to this akregator bug, where the "Copy Link Location" did not copy to the middle-mouse button paste. Fixing it was pretty easy because there was a line doing it commented-out. The patch was accepted and is now in both KDE-4.2 and KDE-trunk.

I helped sort out the Perl-related mailing lists page at the Perl 5 Wiki. The effort was started by szabgab, and I did most of the sorting. In the process, I learned about many interesting mailing lists, and of many almost dead ones. After I was through, I did not know what to do with the rest of my life.

I ran into this bug with Amarok 2.x on my system, which I fixed, by removing some weird junk from the end of /etc/fstab. The fact that I could not use Amarok irritated me a lot, and I was glad that I resolved it.

I released a new version of the Config-IniFiles CPAN module with some fixes and a t/pod.t and t/pod-coverage.t files to prevent POD problems.

Finally, one evening after an intensive biking session, I worked on forward-porting my Amarok per-song-volume script to Amarok 2.x. Now, the original Perl script could not be used because running arbitrary UNIX process scripts is no longer supported by Amarok 2.x, which now requires all scripts to be written in JavaScript using QtScript. Porting it took all evening, but I had it working eventually. I still resent the Amarok developers for only allowing JavaScript there now, which I don't hold in high esteem.

shlomif

One thing that always bothered me about the Perl debugger is the fact that it didn't rememeber / preserve the GNU readline command history from previous sessions like a shell does for example. Today I discovered how to enable the preservation of the command history from previous sessions of the debugger. To do so, add something like the following the the file "~/.perldb" (under your home directory):

&parse_options("HistFile=$ENV{HOME}/.perldb-history HistSize=5000");

I explictly set the history size to a large value, but this is optional. You can also set the path to the history file to a different value.

Now when you start perl -d it will recall all the latest values from previous invocations.

shlomif

When using Kopete, a unified instant-messaging client for the KDE desktop environment, one can recall previous messages using Ctrl+Up and Ctrl+Down. A friend, who is a KMess developer, told me it was also a KMess feature.

This wasn't very discoverable, and I discovered it only by accident. But it seems useful.

shlomif

Apparently, the source for E Text Editor, a multi-platform Textmate clone was released under a not-entirely-open-source licence. This is good news despite the fact that it is not a fully-open-source licene, because now Textmate-like functionality is available for Linux and other platforms, and not only on the Mac.

I'm still sticking with Vim, though, which is fully cross-platform, open-source and even GPL-compatible.

shlomif

I worked a lot on Freecell Solver, doing refactoring and optimisations. On my Birthday (5-May, Tuesday) I tried to do several optimisations, and most of them did not work, so it was a bit frustrating. But some of them did work, and at least I know not to repeat the other ones.

After using it for Freecell Solver, I had decided to adopt the Statistics-Descriptive CPAN module, and eventually got the necessary permissions. I uploaded version 2.7 which featured some bug fixes, cleanups and a conversion to Module-Build. This release still failed some CPAN Testers tests, mainly on 64-bit architectures, due to rounding problems, so I uploaded a new release - 2.8 - yesterday, with more robust tests and a few other improvements.

I worked a bit on the Perl Beginners' Site, fixing some broken links, adding more external links, and fixing some other problems. While working on it, I found a validation problem caused by HTML-Widgets-NavMenu, which I corrected on its trunk, but did not upload to CPAN yet.

One day, I did not have a lot to do, so I decided to convert Test-Run from the old and deprecated NEXT.pm to MRO::Compat. I ran into a few problems because my inheritance graph was incompatible with the C3 guidelines, but mst helped me (again). Thanks! So I ended up uploading Test-Run along with all of its plugins with the converted code.

I also worked on a solution for this KMail bug that a friend referred me to. It took me a lot of experimentation to get a Perl script to be able to properly filter the Postfix outgoing e-mail, and after everything seemed to work, my ISP's email-server rejected the message. Either I'm doing something wrong, or I need to use a different outgoing SMTP server.

Happy Hacking and happy birthday to Alan Haggai Alavi!

shlomif

I have collected several vim tips in the past weeks, so here they are:

1. One can use the :sp(lit) command to split a window into two. :new splits and starts editing a new buffer. Finally, both of these commands followed by a filename stats editing the specified file in the new viewport.

2. This tip is long overdue. You can make Vim behave more like a Windows-like editor by adding source $VIMRUNTIME/mswin.vim to your .vimrc. Reportedly, it causes many problems and is not recommended, but I still like to use it, afer all these years.

3. One can match any char including newline using \_.. The "\_" construct works for other character classes, and allows you to match all of them as well as newline.

   It took me a long time to find that, back when I did, so I'm documenting it here.

4. After one indents or unindents several lines using "<", ">", "<<", ">>", etc. and the indentation level needs to be promoted or demoted further, one can use the "." (= repeat) command to repeat the shifting again (and again) until it is right.

   I discovered this trick by accident, but immediately found it useful.

Hope all Israelis had a nice Independence Day. Happy Vimming!

shlomif

Today I discovered iotop ( on Freshmeat). It's a (Linux-only) tool to find which processes are doing the most hard-disk activity at present. Useful for the cases where your hard-disks' LED is active and you don't know why.

The 0.2.1 version (which is the latest) is available in the Mandriva Cooker repositories, and it was a single urpmi command away from installation. I assume installing it on Debian, Ubuntu, etc. should be just as easy.

I learned about iotop in this blog post (in Hebrew) by Ilan Shavit, which I read today. Thanks Ilan, for the tip!

shlomif

I'm administrating several instances of MediaWiki. Up to several months ago, it had been the case that every few days, there was a spam attack on one or more of them, which required my attention. Lately however, they haven't been spammed at all. So I wonder why.

The anti-spam measures we employ are:

1. We do not allow anonymous commits, and instead require registration. Nevertheless, the registration does not require an E-mail-handshake, which means it can be implemented more easily by pure-HTTP bots.

2. We utilise the MediaWiki SpamBlacklist plugin, which blacklists edits based on URLs matching certain regular expressions. We use both the Wikimedia-maintained blacklist and our own blacklist, where we add URLs that we were spammed with.

   Lately, our custom blacklist has not been updated, since we didn't encounter any spam.

3. All external URLs in the wikis are using rel="nofollow" to make sure they do not gain page rank. This did not prevent wiki spam, but it makes sure that spammers don't benefit from spamming.

Possible explanations to why there was no wiki spam lately is that the wikimedia-blacklist has been more promptly maintained and prevented most spam, or that spam-bots check for the rel="nofollow" attribute and in case they find it, do not spam. That, or something changed in the configuration of the wikis, which I wasn't made aware of.

I'm happy that I have less work to do to combat the spam, but am worried that I may become negligent. But I guess I should enjoy it while it lasts.

shlomif

For all those who are interested, I have written a review of the Winter 2008 issue of the journal "The Perl Review" on the Israeli Perl Mongers wiki. I was given a copy of this issue as a present by szabgab in one of the recent Tel Aviv Open Source club meetings, and felt that I should review it.

shlomif

This is a small warning for open-source enthusiasts in Israel not to rely on the Linmagazine Events Calendar. It's no longer actively maintained, and my two submissions of Tel Aviv Open Source Club meetings were not published. For one of them, I sent a request to the editor to accept my event via Drupal's messaging mechanism, and he didn't.

For alternatives you may consider the Whatsup.org.il post calendar ( available on the front page) and hopefully we'll be able to arrange an aggregated Google Calendar/iCalendar for FOSS events in Israel soon. But if you've relied on Linmagazine.co.il's calendar alone - you shouldn't.

shlomif

[	Tags	|	amarok, bug, bugs, c preprocessor macros, cmake, cpan, fcfs, fcs, freecell, gtk, gui, hacking, hacktivity, inline-ruby, kde, kde 4, kdebase, kdelibs, llvm, log, mod, module files, patch, perl, programming, qt, recent, ruby, rwlock, s3m, solver, wxwidgets, xine, xm	]
[	Current Location	|	Home	]
[	Current Mood	|	tired	]
[	Current Music	|	Jesus Christ Superstar	]

This is another log of my (= Shlomi Fish) hacktivity.

1. xine-lib patch got applied I mentioned my xine-lib patch for fixing the MOD/S3M/XM/IT/etc. mime types, and after pinging the mailing list discovered that it has been applied, and later on corrected.

   After one of the recent Mandriva updates, I can now listen to Module files on Amarok 2.x comfortably. Thanks to the Amarok developers for applying and correcting my patch.

2. Building the KDE 4.x trunk - I decided to try the KDE 4.x-trunk to help with the wallpaper-per-virtual-desktop feature that was incorporated there. It took me quite a long time to build it, because I did it wrong and the KDE build process did not warn me about it. Here's the executive summary of how to do it right: you need to build the following modules from trunk (in order): kdesupport, kdelibs, kdepimlibs, and kdebase, using a script such as the following:

   #!/bin/bash
   k4d="/opt/kde4-trunk"
   k4sd="$k4d/share"
   k4bd="$k4d/bin"
   PATH="$k4bd:$PATH"
   test ! -e build && mkdir build
   cd build
   rm -f CMakeCache.txt
   export CMAKE_PREFIX_PATH="$k4d:$CMAKE_PREFIX_PATH"
   cmake -DCMAKE_INSTALL_PREFIX="$k4d" \
       -D"PYTHON_SITE_PACKAGES_DIR:PATH"="$k4d/lib" \
       -DCMAKE_BUILD_TYPE=debugfull \
       ..
   

   In kdebase, you may see that "make install" tries to install some python files under /usr. I workarounded it by editing the appropriate .cmake files and deleting the installation instructions for them, but there's a patchset in the works for that.

3. KDE trunk bug - In any case, I noticed a few bugs in kde4-trunk (as expected) and reported a bug about the picture names in the wallpaper selection. I spent the rest of the day searching the code and running it under the debugger, in order to try to fix the bug, and was eventually able to supply a patchset for that (in the bug report). The patches were not committed yet.

4. First-Come First-Served Readers/Writers Lock - A Chinese programmer contacted me by email and reported the fact that the First-Come First-Served Readers/Writers Lock did not implement the _destroy() function, which was documented there. He was right, and it was funny because I haven't touched this code for many years (2002 according to Freshmeat), and didn't expect it to contain any problems. Apparently, they were one of the first to want to use it in production.

   So I went to work. I converted the build system to CMake, because GNU Autohell gave me a weird error (again), that I couldn't care to fix. I fixed this problem, and also fixed some more issues with the code and API. Then I uploaded a new version and published it on Freshmeat.

   I guess Murphy was right by saying that every computer programme contains one more bug than has been discovered so far.

5. CPAN - In the XML::RSS realm, I applied a patch that Simon Wistow sent me for a test file, and corrected it for style, and then uploaded a new version (1.44) to the CPAN on 1-April-2009. I also corrected some bugs in HTML-Links-Localize and WWW-Search-MSN.

6. Freecell Solver - I investigated a build problem of recent Freecell Solvers with CMake-2.6.2, and discovered that it was a bug in CMake-2.6.2, which was corrected in 2.6.3. I also applied some cleanups to the code, and extracted some incredibly complicated C-preprocessor-macros into functions, and afterwards wrote a makefile for targetting LLVM bitcode, and implemented a working "profile" CMAKE_BUILD_TYPE.

7. GUI for a Ruby Library - as an exercise in learning the Ruby programming language, I started writing a library, and then I needed a GUI to visualise it. My first thought was using wxRuby, but Mandriva didn't have a package for it. After contemplating on using gtk+, I decided against it, and started prepared an .src.rpm for wxRuby, while trying to adapt an existing wxPerl GUI to use the Ruby library using Inline::Ruby. I discovered that the CPAN version was incredibly broken and completely non-functional on recent perl's, but looking at the rt.cpan.org bug-report, I found a an up-to-date version on gitorious, which I was able to use.

   So I started adapting the wxPerl library to use the Ruby code. It worked very nicely at first, but then I ran into a strange crash. I reproduced this with a simple non-GUI Perl Inline::Ruby script that used my code, while the equivalent Ruby script ran flawlessly. So I reported this as a bug in the gitorious Inline::Ruby. Meanwhile, wxRuby failed to build, and I have some bugs to report.

   I realised I needed to write the GUI in Ruby, and so, decided to use QtRuby. I started from the final tutorial example, and now have a mostly working code. All of this wasted a lot of time that could otherwise be invested on the library itself. But at least I uncovered some bugs.

shlomif

MySQL sports the REGEXP to enable search using a regular expression pattern. However, it does not contain a built-in function (such as Perl's quotemeta) to escape the special characters in such patterns in order to prevent injecting malicious regular expression code.

Some time ago, I needed to search for a string within word-boundaries in MySQL in a PHP site I help maintain. Here's the solution I came up with for constructing this search:

function enbackslash($matches)
{
    return '\\' . $matches[1];
}

$safe_word = $mysqli->real_escape_string(
    preg_replace_callback('/([^\\w\\s])/', "enbackslash", $word)
    );
      
$sql .= "text regexp '[[:<:]]${safe_word}[[:>:]]'";

Basically what happens is that I prefix a backslash to any non-alphanumeric and non-whitespace character, and then escape the string again using the standard RDBMS-specific escaping function.

I would appreciate any ideas for improving upon this code, but it seems to work for me.

shlomif

While cleaning up an old hard-disk, I found a Subversion working copy of PONIE - the Perl 5 port to Parrot, and found out the URL of its long-lost Subversion repository. Now I see it's the first hit on a Google search for "ponie repository", but still I wondered what it was for a long time.

shlomif

It's official! I'm going to publish a new book soon, and I placed a new interview with me about it. Read it so you'll understand why "Mastering cat" would be an essential addition to your collection of technical books.