en_dmitriid

Past Friday I got myself acquainted to Python.

Different people view Python as "nothing to learn" or as "if only I had time to learn" or even as "very difficult to learn".

I had to learn it quick for a very simple reason. I have lots of photos taken at different resolutions. I need to resize them to different sizes (much like Flickr does), create several thumbnails of each image, upload them to a host (Amazon S3) and load information on uploaded images to a database.

The process must be automated, of course.

It all started when I convinced myself that Python Imaging Library is the best library to manipulate images. Indeed:



The line with im.thumbnail((800, 800), Image.ANTIALIAS) is absolute genius. If your image is, say, 1024х768px, then what you get in the end is not a skewed 800х800px, but a proportional 800x600px.

Combined with MySQLDb and Boto this yields a very easy solution to my task. However, that's not the point.

As it turns out, getting to know many languages (even superficially) may give you a huge head start in learning a new programming language. I've had an overview or, sometimes, more than an overview of such languages as Lisp, Haskell, Erlang... How does that help me with Python? Well, I got an imeddiate understanding of the following:



Tuples and destructuring assignment

A tuple is a fixed-length list. This list may contain elements that are very different in nature:

Looks similar, doesn't it? :)

Let now get data out of our tuples:

Certainly, the principles of theses examples are very different. Haskell and Erlang use pattern matching that Python doesn't have. However, it looks very similar and is very easy to understand once you know the other examples. PHP, by the way, has a similar thing:

This, of course, is very different and the inner workings are absolutely different :) One thing about this is quite peculiar, though. Once you get used to these sorts of destructuring assignments, you start using PHP's list and Python's tuples much more often. Because it's so convenient.

Named parameters

I once talked about named parameters. As it turns out, Pythons has them.

Anonymous functions (lambdas)



Very similar to code in other languages:



List comprehensions

Remember how we used to define lists in mathematics? For instance:

M is all x's where x belongs to S and x is even. Nice languages have ways to define lists in a very mathematical way. This is called list comprehensions. One downside to Python's comprehensions is the syntax. Here are previous examples rewritten to use list comprehensions instead of lambdas:





Anyhow, thanks to other languages the new language (a more or less mainstream language, not K :)) ) is not just easy, it's extreamly easy and can be learned in, say, 15 minutes.

Ever since I saw Lisp I've sorely missed named function parameters. What the heck is that? Ok, here goes...

Lisp examples are quoted directly from Practical Common Lisp

Let's refresh our knowledge of WinAPI - the dearly beloved API of all Windows developers. For instance, ShellExecute:

HINSTANCE ShellExecute(      
    HWND hwnd,
    LPCTSTR lpOperation,
    LPCTSTR lpFile,
    LPCTSTR lpParameters,
    LPCTSTR lpDirectory,
    INT nShowCmd
);

On the whole I think I remember which parameters this function requires and why. Not too clearly though. I do remember the first three. The fourth might spring to mind while writing the function. The last too however, are quire elusive and I'm never quite sure which goes where and when. The problem gets worse if you remember that som parameters, hwnd, lpParameters, lpDirectory in our case, are optional and can be replaced with NULLs. So, we would quite often invoke this function like this::

 hResult = ShellExecute(NULL, "open", "path/to/file", NULL, NULL, SW_SHOWDEFAULT);

Scary, huh? Now imagine if we could write this function like this:

 hResult = ShellExecute(operation="open", file="path/to/file");

Beautiful. Now, this is what named parameters are all about.

Let's go back to Lisp. Let's consider the following function:

 (defun foo (&key a b c) (list a b c))

This function accepts three parameters and simply listst them in their order. This is how this function behaves when we invoke it in numerous ways:

(foo)                ==> (NIL NIL NIL)
(foo :a 1)           ==> (1 NIL NIL)
(foo :b 1)           ==> (NIL 1 NIL)
(foo :c 1)           ==> (NIL NIL 1)
(foo :a 1 :c 3)      ==> (1 NIL 3)
(foo :a 1 :b 2 :c 3) ==> (1 2 3)
(foo :a 1 :c 3 :b 2) ==> (1 2 3)

Note the last line. Even though the order or parameters is incorrect, the function still behaves as designed. Now, that's the beauty and power of named parameters. A developer now only needs to know which parameters a function requires, not their not their order.

Unfortunately none of the mainstream languages support named parameters, not natively at least. That is why invoking a function (especially a WinApi function) is a long and tedious process whereas we could simply make do with passing a limited number of named arguments (as my example with ShellExecute shows).

I am currently developing in ColdFusion and it supports named parameters natively (though you cannot honestly call it a mainstream language):

<cffunction name="function" access="public">
    <cfargument name="arg1" type="numeric" required="yes">
    <cfargument name="arg2" type="string" required="no">
    <!--- и так далее --->


</cffunction>

This function is invoked as follows:

<cfset function(arg1=1, arg2='string')>

As in Lisp, the order of arguments is not important as long as they are named.

Some languages (scripting languages, mostly) emulate this behaviour through the use of hashes. PHP anyone?

function a_func($params)
{
    if(isset($params['arg1'])) /* do stuff */
    if(isset($params['arg2'])) /* do stuff */
}
 
/* Calling the function */
$result = a_func(array('arg1' => 123, 'arg2' => 'a string'));

That is, we actually pass an array, but it serves our purpose well enough. Ruby employs exactly the same technique:

# find from ActiveRecord
      def find(*args)
        options = extract_options_from_args!(args)
        validate_find_options(options)
        set_readonly_option!(options)

        case args.first
          when :first then find_initial(options)
          when :all   then find_every(options)
          else             find_from_ids(args, options)
        end
      end

 
# Calling the function:
Person.find(:first, :order => "created_on DESC", :offset => 5)
Person.find(:all, :group => "category")
Person.find(:all, :offset => 10, :limit => 10)
# where Person is an object of type Activerecord

However, none of these can even compare to Lisp's implementation. ColdFusion, for instance, doesn't accept an arbitrary number of parameters (see Lisp's &rest). Hashes both in PHP and Ruby are rather cumbersome because implementation of the function may become convoluted (or you have to resort to using less-than-safe extract or extract_options_from_args! functions). I'm not even talking other languages like С/С++, C# and Java :). Some languages though, like Haskell, Erlang and Nemerle, have pattern matching allows the developer to work arounв named parameters in a very elegant manner, but that is a story for another day.

Web programmers are generally viewed as "less-than-programmers". Some even think that they are not programmers at all, come to think of it. Indeed, some think that creating something like Codeproject is a piece of cake. Those same people also think that creating a yet another accounting program in Visual Basic (or, god forbid, VBA) is uber-cool and is worthy of all praise.

Now, why would anybody think that? Huh?

Let's consider the so-called "application programmer". Or even the so-called "systems programmer". All they ever need to know is their favorite programming language (C/C++/Java/Object Pascal...). Well, quite often something like a passing knowledge of SQL comes along and a declarative language to store application settings in (usually the INI file format or XML). In toto - at most three technologies: programming language, SQL, XML, and chances are the programmer will use datasets and MSXML to work with the latter to.

What about the web programmer, then?

• Favorite programming language - one of PHP/Ruby/Perl/Java/C#/Python


• SQL


• HTML


• CSS


• JavaScript (sometimes with an added bonus of JSON)


• and quite often XML

Also throw in the bag the fact that web programming is originally and immediately a client-server programming. Most of application programmers don't even have a clue about what's that all about.

So, what we have in the end is that the despised web programmers actually know and do the following:

• They program client-server applications easily and effortlessly


• They combine imperative and declarative programming seamlessly


• They have a more-than-passing knowledge about about non-compatible and non-intersecting technologies and know how to combine them

And as soon as the project starts to grow... We have data replication, distributed servers and son on and so forth...

And still web programming is perceived to be a child's play whereas grown-ups are off working on their "serious" technologies which are supposedly to hard for a web-programmer to comprehend. Well, think again, lads. It is we, web porgrammers, who look down on you because most of the technologies most of you are just beginning to toy with have already been seen, learned and taken to use by us.

BTW, this discrimination against web programming is exactly the reason why web programmers are so behind on tools and libraries. It is only recently that decent tools have sprung up all over the place (such as Ruby on Rails, Django, Seam, Prado and others)

Now that's a bloody shame if you ask me.

Well, I have finally been coerced into working on a project which is, at least partly, related to Adriyatik (I am the sole administrator and developer of that thing, BTW). The details are under NDA, but I can tell you it's related to online catalogs and e-commerce.

So, the question is - what shall I use? The requirements for this project are such that any standard solution that's out there in the wild needs a lot of work before it starts resembling what we want. Besides, as a frequent reinventor of the wheel, I decided to develop everything from the scratch up (or almost).

www.adriyatik.com has a TYPO3 underneath. This - shall I say framework? - is comfortable, easy to use, and, most importantly, extensible every which way. And written in PHP. A quick glance at extension writers' manual convinced me that extending the system my way would be slow and ineffective at best. Even though there is a tt_products extension, I really hate the thought of digging into its code and bending it to my will (and there is no guarantee that it would bend).

So, what a guy has to do? I needed a toolset that would be both low-level enough to allow bending and extending and high-level enough for me not to worry about the details concerning parameter passing, headers, cookies, sql injection, XSS and so on and so forth.

My original impulse was to develop everyting usin PHP.

I've been using PHP since 2001. I've written quite a lot of code in PHP (including the beginnings of a personal framework), I "loved it tenderly" and I would always stand by its side in quite a lot of holy wars (I even have an article on PEAR (in Russian)). However, the release of PHP5 has killed all hope in me for that language. I simply cannot see why a developer would want to move from version 4 to version 5. The same old castrated OOP, iconv as an extension, not as a part of the core (that's why TYPO3, fo instance, carries around it's own Unicode tables and a whole lot of home-brewed code to handle encodings), total lack of unified database access (talks on adding an extension akin to PEAR's MDB have stalled in the past, are stalled now and pray that the extension makes it into PHP6).

Creating anything from scratch in PHP is a major pain in the ass. Even if you decide to go with PEAR there is simply no guarantee that something you need has left alpha already. Moreover, integration of these packages is another issue that could sometimes prove troublesome.

Anyway, PEAR will not help me achieve the main goal - implementation of business logic, separation of business logic and presentation and the million of other things that will spring up inevitably. Because scalability and extensibility are not some mythical words, but rather a harsh reality of life. Because tomorrow I will definitely need to conect a product A to a product B through a product C and a bunch of conditional constraints.

No one says it can't be in PHP. See Yahoo! Auctions for an example. Do I need all the hassle though?

As a result I've been seduced by hype and decided I should check out Ruby-on-rails. You know what? I like it so far.

Certainly, the rigid implementation of the MVC pattern does not guarantee bliss. However...

RoR is low-level enough to bend every which way. It is also high-level enough to implement, say, validation of input data like this:

# This code is in the model
class BlogPost < ActiveRecord::Base
  validates_presence_of :title
  validates_presence_of :author
end

What's happened there? When you try to wtite a record into the BlogPosts table the model checks if there are non-null non-empty fileds title and author and presents us with an error otherwise. A detailed explanation would be rather tedious, so check out tutorials and books instead.

What has realy sold me on Ruby is the incredible mix of imperativeness, declarativeness and functionalism in a language which remains object-oriented. What gas sold me on Rails is that you can often get away with writing just a line of code to do your job, but you can go ahead and rewrite everything if you like. Or extend. Or chenge something.

Ruby certainly has its issues. A slow interpreter is the one that immediately stands out (especially in Windows). However, some folks have proven that Ruby is suitable for large projects. For instance, alistapart.com was (re)written with RoR. Or, as an extreme example, here's the story of eins.de (a German portal which claims to have over a million of page impressions a day):
http://poocs.net/articles/2006/03/13/the-adventures-of-scaling-stage-1
http://poocs.net/articles/2006/03/18/q-a-the-adventures-of-scaling-stage-1
http://poocs.net/articles/2006/03/20/the-adventures-of-scaling-stage-2
http://poocs.net/articles/2006/03/27/the-adventures-of-scaling-stage-3
http://poocs.net/articles/2006/04/03/the-adventures-of-scaling-stage-4

Well, the future looks bright :) Now I should really go ahead and make myself learn Ruby...