Recently in Web design/development Category
Our good friend Jesse recently posted an article on his blog saying, "NOW is the time [for Perl] to step up!!!" Jesse mentioned a discussion going on inside the Ruby on Rails community in which at least one significant member expressed frustration with the lack of intelligent software architecture methodology put into Rails. I think he called other RoR contibutors "a bunch of half-trained PHP morons," which goes a long way -- in my book, anyway -- toward describing something akin to building an automobile out of toothpicks and rubber cement.
Jesse suggests this is the perfect time for Catalyst to make a entrance. I couldn't agree more.
Catalyst is a Perl web application development framework that compares, in some ways, to Ruby on Rails. Catalyst does a fine job of providing developers with a solid MVC framework for developing web applications, but I think what makes Catalyst so formidable is that it also leverages much of the excellent Perl code available from CPAN, the global distributed repository of reusable Perl component code.
Yeah, Catalyst is awesome. I'm thrilled to be using it for a project at work right now. That may come as a surprise to people who have heard me describe KnowledgeBlue as purely-Java shop, but the company is adapting to better exploit the skills available just as I am taking steps to learn more about Java development.
Catalyst needs a lot more (well-written) documentation. There is an excellent tutorial -- Catalyst::Manual::Tutorial -- that is distributed in POD format as part of the Catalyst::Manual package, but even after going through this tutorial, a Catalyst newbie is likely to still be doing some head scratching.
The tutorial is great, really. It walks through setting up a connection to a database backend with DBIx::Class, creating templates using the all-powerful Perl Template Toolkit, and using the Catalyst tools to magically provide authentication and program flow.
The problem, however, is that Catalyst, can do so much, it can be difficult to grasp how to do simple things. The complexity of deploying a simple application approaches what a JSP developer must do. The difference is that after a JSP developer edits several configuration, source, and HTML files, he or she has a simple web application that says "Hello World." The Catalyst developer might spend the same amount of time and end up with a "Hello World" application in a very extensible MVC framework. From there, it requires a minimal amount of work to extend the application, for example, to send its output in PDF format or to get its "Hello World" message from a web service.
In addition to wrapping your mind around all that Catalyst can do and how to do it is the large number of Perl packages you must install. This is less of a burden than it used to be because a lot of Linux distributions provide the fundamental packages necessary for Catalyst development like Catalyst::Runtime, Catalyst::Devel, and Catalyst::Manual, but to really develop kick-ass applications, you've still got to install other packages like HTML::FormFu, Template::Alloy, DBIx::Class, and others in addition to their corresponding Catalyst glue modules.
I think this blog posting may be the first in a long series of brain dumps on Catalyst. I hope I can make it easier for others to transition into Catalyst development.
Great news for Catalyst developers using Fedora 7 or Fedora Core 6 Linux distributions: Core Catalyst modules are NOW available in FC6 extras and F7 repositories!
Yes, it's true! Just do a yum install perl-Catalyst-Devel and BAM! You'll be taking web development to a new level while staying in the comfortable world of managable packages.
(Note: Thanks to redbeard2 for pointing out my stupid typo: "are not available.")
A couple evenings ago, I had to move some data around for a client. One of their Samba servers was overutilized and was running out of space while another was underutilized and had plenty of space -- more than enough to contain what was on the overutilized server and future growth. The plan was to swap the shares between the servers. The total data I needed to move around was about 300G. I knew, even with a gigabit network, that was going to take a while.
So, while I waited for the rsync process to proceed, I decided to walk myself through an open source JavaScript framework called Dojo.
Now, I should back up here. I've always been sort of anti-JavaScript. I've been witness too many times to what happens when someone designs security for an application using JavaScript and that security goes out the window when you simply turn off JavaScript in your web browser. I've always been a big proponent of "graceful degradation" and my use of JavaScript in web applications has been mostly cosmetic. If I ever developed anything using client-side scripting to add functionality to the application, I made sure there was some server-side logic to back it up in case script interpretation was not available on the client side.
Plus, I would always cite the mobile phone as my example of the user agent that could not process a web application that was JavaScript-heavy.
With all that said, I've started to see the light, so to speak, with regards to JavaScript. With so much of What's Cool on the Web being built using AJAX-type code and with mobile devices like smartphones coming out which support JavaScript, it is starting to make sense to me to get my feet wet and at least get in up to my knees.
A few weeks back, there was a thread posted on a Catalyst mailing list I subscribe to which posed the question of which JavaScript/AJAX framework was preferred by Catalyst developers.
(You can see the archived thread here.)
The answers list members gave included Dojo, jQuery, and the Yahoo! User Interface library.
When Matt Trout -- DBIx::Class and Catalyst guru -- indicated that Dojo was his preference, I made a mental note then and there to check it out.
In summary, it's very cool. I like that it seems to almost agree with the philosophy of graceful degradation. With the Dojo libraries loaded, you can decorate your forms with enhanced widgets simply by adding parameters to the HTML tags. For example, if you have a text field that asks for a date: <input type="text" name="date" />. Dojo lets you turn that into a fancy "select your date from this pop-up calendar" widget simply by adding a few parameters: <input type="text" name="date" dojoType="dijit.form.DateTextbox" />.
Now that's pretty dang cool.
I've been playing with the 0.9(beta) and 0.43 versions of Dojo. The documentation for 0.9 is pretty sparse at this point, but I'm impressed with what it can do.
The one thing I'm a bit disappointed that it can't do is visual effects many other frameworks support like drag and drop, element opacity tricks, etc. Maybe I just haven't run across it yet.
There does seem to be good support for abstracted AJAX functionality including interactive file uploads -- something many of the people I've done work for in the past have asked for.
Fozzout.
Installing Catalyst on a Fedora/RedHat system can be a challenge - especially if you insist on keeping all installed software manageable via RPM. This is because many of the packages required to run Catalyst are not available from any common repositories and therefore must be built as RPM packages.
Eventually, I believe these packages will either show up in fedora-extras or a new third-party repository will be created for Perl modules required by Catalyst that are provided neither by fedora-core or fedora-extras.
In the meantime, I've created a straightfoward shell script called catalystinstaller.sh that drives yum and cpan2rpm to install everything you need to get a good start developing Catalyst apps on a Fedora Core 6 system.
I'm aware of one problem with this script, so far. When building Catalyst::Runtime. the module's configuration routines complain that the optional package Catalyst::Engine::Apache is not installed. It appears to want version 1.05 which is an older version than what is currently available. The currently available version is installed by this point in the process. So, just hit ENTER when it asks if you want to install it (the default is No).
This is a first release (v0.01). Run at your own risk. Please report any issues you run across and, of course, any suggestions you come up with.
I would love to set up a yum repository for these packages that are not available yet in the standard repositories, so, if someone wants to help with that project, please let me know. The hard part won't be setting up the repository, but keeping packages up to date.
Interested parties can download the catalystinstaller.sh script here.
I thought I'd share a short demonstration of some very cool Perl technology.
About a week ago, I got back from taking my family on a short vacation to beautiful southern Utah. I documented our trip and posted copies of a few photos we took in the Events section of my website. You can read the Sept. Mini-Vacation page yourself.
In the past, when preparing images for publication on the web, I've either hand-coded the HTML -- usually a mess of table elements -- or I've used a homegrown shell script called buildgallery.sh that generates the HTML table code around a set of images. After generating the scaffolding of tabl code for the image organization, I'd go in and populate various cells with image captions, descriptions, and links to different sizes.
This works, but tables are ugly. CSS-only presentation is so much more elegant. Plus, it'd be better if I didn't have to jump around the HTML to populate the caption and description text for all the images.
Perl to the rescue!
I'm already using the Perl Template Toolkit to generate static pages for each page on my website. I use the ttree command at the command line on my staging server to generate the pages by "wrapping" the main content.
Since the page about our southern Utah vacation was just being fed to a template processing engine to generate a resulting HTML file, it made sense to use the power of the Perl Template Toolkit to generate the HTML around my gallery of images.
By the time I've gotten to this point, I've already run a shell script that uses ImageMagick's, uhm, magic, to create sensibly-sized thumbnail and web-friendly sized versions of each image. Those images are stored in files with .med and .thumb in the names before the filename extension.
For each group of images, I declared a data structure called images which is essentially an anonymous list of anonymous hashes. Each anonymous hash contains information about an image such as the base filename, image dimensions, title, and description text.
[% images = [
{ basename => "fiesta_fun-all_3-2",
width => 156,
height => 117,
title => "Our kids driving the kiddie-carts",
description => "Maya, Lucy, and Eli on the kiddie-carts course."},
{ basename => "fiesta_fun-lucy_eli-4",
width => 156,
height => 117,
title => "Lucy and Eli",
description => "Lucy and Eli on the kiddie-carts course."} ] %]
For those unfamiliar with the Perl Template Toolkit syntax, it is similar (but not too similar ;-)) to PHP or ASP in that template code can be embedded with HTML as long as each piece of template code begins with [% and ends with %]. This is, of course, overrideable -- You can change those start/end sequences to anything you want.
Next, I declared a block called doimage which takes one set of information about an image -- one hash from our list of hashes -- and generates HTML for it.
This BLOCK section only needs to be declared once in the file and could be placed in its own file for re-use later, if I chose to do so.
[% BLOCK doimage %]
[% DEFAULT img_path = "images" %]
<div class="picture">
<div class="imagecontainer">
<a href="[% img_path %]/[% img.basename %].med.jpg" target="_new">
<img width="[% img.width %]" height="[% img.height %]"
src="[% img_path %]/[% img.basename %].thumb.jpg"
alt="[% img.title %]"
title="[% img.title %]"/></a>
</div>
<div class="textcontainer">
<div class="picture_caption">[% img.title %]</div>
[% IF img.description %]
<div class="picture_description">[% img.description %]</div>
[% END %]
</div>
<div class="floatbreak"></div>
</div>
[% END %]
It should be pretty obvious by looking at the code (unless you're reading from an RSS feed of this document, in which case, the above example will be all messed up) how the values of the individual hash elements are interpolated into the HTML.
Next, I need to loop through all the images by iterating through the list of hashes I declared earlier. For each one of these hashes, I want to process the information in the hash using the block I declared.
[% FOREACH img = images %]
[% PROCESS doimage img_path = "images" %]
[% END %] Finally, I dressed up the HTML by adding some properties to the stylesheet:
.picture {
margin: 5px;
margin-left: 20px;
border: solid #aaa 1px;
}
.imagecontainer {
text-align: center;
float: left;
width: 170px;
}
.picture img {
margin: 5px;
}
.textcontainer {
margin-left: 200px;
}
.textcontainer .picture_caption {
font-family: sans-serif;
font-size: large;
font-weight: bold;
}
.textcontainer .picture_description {
font-family: sans-serif;
font-size: medium;
font-weight: medium;
}
.floatbreak {
clear: both;
height: 0;
}(A big shout-out of kudos to Tene for helping me with that CSS.)
In summary, I don't think I'll be using any shell scripts to build HTML table scaffolding around my simple image galleries anymore.
I've already thought of some ways to improve upon this. For example, I could use the Perl Template Toolkit's Image plugin to examine each image file and automagically extract information such as image dimensions, EXIF data, etc. as it iterates over the list of image information.
I've been so busy lately, but tonight I took some time to get caught up on the Utah Open Source Planet and, I must say, there was lots of good stuff to read. Thanks to y'all sharing your knowledge. You rock.
I thought I'd pick on one of my favorite UOSSP bloggers, Aaron Toponce, but not in a negative way. I read his semi-recent entry about using HTML entities to obfuscate web site data in an attempt to foil robots -- particularly robots intent on harvesting e-mail addresses and other information.
Some years ago, I implemented this technique on several sites, personal and professional. It seemed to make sense the average spammer/data-harvester, was not going to implement the code necessary to de-entity-ize the content in search of e-mail addresses. In retrospect, however, I think that's a poor assumption.
See, spammers have money and they give their money to poor souls who will write code for money and, in many cases, have the smarts to pull it off. So, semi-smart coders tasked with maximizing the pool of e-mail addresses gleamed from a vast array of websites will very quickly implement techniques to foil the simplest of data obfuscation techniques. Converting text to HTML entities has got to be one of the first obfuscation techniques they are faced with circumventing.
After that, they probably implement simple OCR techniques to gleam data from sites that convert all their e-mail addresses into text rendered as image files.
That said, this HTML entity-based obfuscation technique is better than nothing, right? Because spammers like their pools of e-mail addresses to be fresh, it usually only takes a couple of weeks to see if any anti-spam technique results in a significant reduction of incoming spam, so it's easy to verify your technique is working. When we implemented the HTML-entity based obfuscation technique, there was a decrease in the amount of spam, but there was still plenty of spam.
If you're interested in playing with ways of automating the process of converting text data to a string of HTML entities, check out the HTML::Entities Perl module -- part of the comprehensive HTML::Parser distribution of modules.
Once you have this installed, you can do something like this:
perl -MHTML::Entities -ne 'print encode_entities($_, "\32-\255")'
For the Perl head-scratchers, this is a one-liner that loads the HTML::Entities module, wraps a loop around reading from STDIN or a filename parameter, and prints the result of the encode_entities() function call for each line of input read. Hit Control+D to get out of it.
[foo] /home/fozz 19 % perl -MHTML::Entities \
-ne 'print encode_entities($_, "\32-\255")'
Aaron Toponce
Aaron Toponce
When it was clear the HTML entity-based obfuscation simply did not have what it takes to win against increasingly smart harvesting bots, we deployed a CAPTCHA solution using the Authen::Captcha Perl module for our clients that really needed/wanted to publish e-mail addresses on their websites. This solution has worked out much better and, paired with educating users about the risks of leaving your e-mail address on websites, we've seen more significant decreases of incoming spam.
I've got brand new blogging software up on my site now. I guess we'll call it Fozzolog 1.99. It seems to be working pretty well. The only issue that has arisen is that the change in RSS propogation has caused the Utah Open Source Planet to include all of the most recent 25 entries in their aggregation. Fun.
I've also put up a new site design and this one is completely table-free. All the layout is done with positional CSS and seems to be pretty cross-browser friendly.
I'm using the original content from my old site throughout, so there are still some tables littered about, but the general layout -- the "wrapping", if you will -- is all CSS. The design is based on one I saw on the openwebdesign.org site called A20. I changed the color scheme and tweaked some of the components, but it's still largely intact.
It takes a lot of work to produce a truly functional table-free design and I applaud the folks who put forth the effort to do it.
I've been working on the next generation of the Fozzolog weblog framework for a while now. This new generation is likely to be worth showing the rest of the world (*crosses fingers*).
The new framework makes use of mod_perl and many of my latest favorite Perl modules including, but probably not limited to:
One issue I ran into in development was how long it took to render a listing of all weblog entries, especially if there are many entries. My thoughts first went to the database, so I tried a couple things to optimize data retrieval. First, I dropped the body text from the entries in the SQL query so the results were smaller. That seemed to make things a tiny bit faster, but not much. Next, I used Cache::Cache to cache the results because a listing of all weblog entries doesn't change very often. That made a difference, but it still took six seconds to list all my entries.
Visiting Jennie's blog just fills me with a ton of guilt for not popuating this space in a more frequent manner. So much so that... I'm doing something about it.
One reason I haven't been blogging is because I've been working on overhauling the Fozzolog code - the code that drives this blog (and a few others). I should have been done with that a couple weeks ago, but I've been so busy with other stuff, it hasn't happened.
Other stuff? Oh, just work, mostly. Iodynamics is very busy right now. It's gratifying to look at the "Big Board" (a reference to Buffy The Vampire Slayer - Season 7) at our Salt Lake office -- it is chock full of active (Hot & Spicy) projects.
There have been three Sons Of Nothing shows in January. One in Park City and two in Colorado. These shows mark a landmark for me and the band: the first time we've run visuals and audio effects completely without the Windows operating system. Everything is running Linux now.
I'm very pleased.
We recorded the audio and video from these recent shows and I have been editing it all into some multimedia that may be usable as promotional material for the band.
Meanwhile, the rest of the band -- the actual musicians -- have been busy recording the next CD. I've been up to Matt's house to observe and document a couple of times and it's sounding great.
Google had their Summer of Code, I just had my...
Weekend... Of... CODE
(Say it like The Muppet Show's "Pigs In Space.")
It's been kind of nice, really. First of all, I enhanced the way gig images are viewed over at the Sons Of Nothing site. Instead of images opening up in a new browser window when you clicked on a thumbnail, the images will open up "inside" the site along with some handy "next image" and "previous image" navigation.
It wasn't hard. Just create a new template for the full-size image page and give it some meta data about the next and previous images in the set. Done.
Perl, mod_perl, and Template Toolkit just rock... like Sons Of Nothing rocks... but different.
For example, check out this recently-posted rare glimpse of me from our September jaunt to Steamboat Springs, Colorado.
Secondly, Iodynamics is currently involved in a project for a client that involves gluing a bunch of facilities together within one website including auctions, weblogs, news articles, file downloads, and more. Call in the next 20 minutes and you'll also get this handy dandy nosehair trimmer.
Anyway, the original project specification had us doing everything in PHP - not our first choice for anything. We weren't too worried about spending too much time writing PHP code because our plan was to use "off-the-shelf" components or applications and glue them together into one seamless website.
It never quite works that way, does it?
The "RSS feed" we were supposed to be using for the news articles turned out to be an XML format which borrowed haphazardly from RSS, but is completely different, thereby requiring some ground-up parser development.
The file download repository is specific enough, we hadn't been able to find anything that would do the trick admirably enough.
Oh, did I mention the client was seriously hoping to have it all done last week?
Fortunately, the client succumbed and said, "Go ahead and do it in Perl." Whatever it takes, just get it done quick. Man, after searching for something akin to CPAN's XML::Simple for PHP, it was so refreshing to be able to just use versatile Perl modules like XML::Simple, LWP::UserAgent, DBI, and others.
I was able to crank out all kinds of magic to do the news articles and the file download repository in an amazingly small amount of code and still maintain a strict separation of concerns (i.e. MVC) -- something that you have to work really hard to do when programming with PHP.
