The experiences discussed herein were valuable in their own right, but the challenge itself is rewarding as well. How often do we pause to reflect on what we've learned, and more importantly, how it has changed us? Because of that, I recommend you perform the exercise as well.

I freely admit that some of this isn't necessarily caused by my experiences with the language alone - but instead shaped by the languages and my experiences surrounding the times.

One last bit of administrata: Some of these memories are over a decade old, and therefore may bleed together and/or be unfactual. Please forgive the minor errors due to memory loss.


How QBASIC Made Me A Programmer

As I've said before, from the time I was very young, I had an interest in making games. I was enamored with my Atari 2600, and then later the NES. I also enjoyed a playground game with Donald Duck and Spelunker.

Before I was 10, I had a notepad with designs for my as-yet-unreleased blockbuster of a side-scrolling game that would run on my very own Super Sanola game console (I had the shell designed, not the electronics).

It was that intense interest in how to make a game that led me to inspect some of the source code Microsoft provided with QBASIC. After learning PRINT, INPUT, IF..THEN, and GOTO (and of course SomeLabel: to go to) I was ready to take a shot at my first text-based adventure game.

The game wasn't all that big - consisting of a few rooms, the NEWS directions, swinging of a sword against a few monsters, and keeping track of treasure and stats for everything - but it was a complete mess.

The experience with QBASIC taught me that, for any given program of sufficient complexity, you really only need three to four language constructs:

1. Input
2. Output
3. Conditional statements
4. Control structures

Even the control structures may not be necessary there. Why? Suppose you know a set of operations will be performed an unknown but arbitrary amount of times. Suppose also that it will be performed less than X number of times, where X is a known quantity smaller than infinity. Then you can simply write out X number of conditionals to cover all the cases. Not efficient, but not a requirement either.

Unfortunately, that experience and its lesson stuck with me for a while. (Hence, the title of this weblog.)

Side Note: The number of language constructs I mentioned that are necessary is not from a scientific source - just from the top of my head at the time I wrote it. If I'm wrong on the amount (be it too high or too low), I always appreciate corrections in the comments.


What ANSI Art taught me about programming

When I started making ANSI art, I was unaware of TheDraw. Instead, I opened up those .ans files I enjoyed looking at so much in MS-DOS Editor to see how it was done. A bunch of escape codes and blocks came together to produce a thing of visual beauty.



Since all I knew about were the escape codes and the blocks (alt-177, 178, 219-223 mostly), naturally I used the MS-DOS Editor to create my own art. The limitations of the medium were strangling, but that was what made it fun.

And I'm sure you can imagine the pain - worse than programming in an assembly language (at least for relatively small programs). Nevertheless, the experience taught me some valuable lessons:

• Even though we value people over tools, don't underestimate the value of a good tool. In fact, when attempting anything new to you, see if there's a tool that can help you. Back then, I was on local BBSs, and not the 1337 ones when I first started out. Now, the Internet is ubiquitous. We don't have an excuse anymore.
  
  
• I can now navigate through really bad code (and code that is limited by the language) a bit easier than I might otherwise have been able to do. I might have to do some experimenting to see what the symbols mean, but I imagine everyone would. And to be fair, I'm sure years of personally producing such crapcode also has something to do with my navigation abilities.
  
  
• Perhaps most importantly, it taught me the value of working in small chunks and taking baby steps. When you can't see the result of what you're doing, you've got to constantly check the results of the latest change, and most software systems are like that. Moreover, when you encounter something unexpected, an effective approach is to isolate the problem by isolating the code. In doing so, you can reproduce the problem and problem area, making the fix much easier.

The Middle Years (included for completeness' sake)

The middle years included exposure to Turbo Pascal, MASM, C, and C++, and some small experiences in other places as well. Although I learned many lessons, there are far too many to list here, and most are so small as to not be significant on their own. Therefore, they are uninteresting for the purposes of this post.

However, there were two lessons I learned from this time (but not during) that are significant:

1. Learn to compile your own $&*@%# programs (or, learn to fish instead of asking for them).
2. Stop being an arrogant know-it-all prick and admit you know nothing.

As you can tell, I was quite the cowboy coding young buck. I've tried to change that in recent years.


How ColdFusion made me a better programmer when I use Java

Although I've written a ton of bad code in ColdFusion, I've also written a couple of good lines here and there. I came into ColdFusion with the experiences I've related above this, and my early times with it definitely illustrate that fact. I cared nothing for small files, knew nothing of abstraction, and horrendous god-files were created as a result.

If you're a fan of Italian food, looking through my code would make your mouth water.

DRY principle? Forget about it. I still thought code reuse meant copy and paste.

Still, ColdFusion taught me one important aspect that got me started on the path to Object Oriented Enlightenment: Database access shouldn't require several lines of boilerplate code to execute one line of SQL.

Because of my experience with ColdFusion, I wrote my first reusable class in Java that took the boilerplating away, let me instantiate a single object, and use it for queries.


How Java taught me to write better programs in Ruby, C#, CF and others

It was around the time I started using Java quite a bit that I discovered Uncle Bob's Principles of OOD, so much of the improvement here is only indirectly related to Java.

Sure, I had heard about object oriented programming, but either I shrugged it off ("who needs that?") or didn't "get" it (or more likely, a combination of both).

Whatever it was, it took a couple of years of revisiting my own crapcode in ColdFusion and Java as a "professional" to tip me over the edge. I had to find a better way: Grad school here I come!

The better way was to find a new career. I was going to enter as a Political Scientist and drop programming altogether. I had seemingly lost all passion for the subject.

Fortunately for me now, the political science department wasn't accepting Spring entrance, so I decide to at least get started in computer science. Even more luckily, that first semester Venkat introduced me to the solution to many my problems, and got me excited about programming again.

I was using Java fairly heavily during all this time, so learning the principles behind OO in depth and in Java allowed me to extrapolate that for use in other languages. I focused on principles, not recipes.

On top of it all, Java taught me about unit testing with JUnit. Now, the first thing I look for when evaluating a language is a unit testing framework.


What Ruby taught me that the others didn't

My experience with Ruby over the last year or so has been invaluable. In particular, there are four lessons I've taken (or am in the process of taking):

• The importance of code as data, or higher-order functions, or first-order functions, or blocks or closures: After learning how to appropriately use yield, I really miss it when I'm using a language where it's lacking.
  
  
• There is value in viewing programming as the construction of lanugages, and DSLs are useful tools to have in your toolbox.
  
  
• Metaprogramming is OK. Before Ruby, I used metaprogramming very sparingly. Part of that is because I didn't understand it, and the other part is I didn't take the time to understand it because I had heard how slow it can make your programs.
  
  Needless to say, after seeing it in action in Ruby, I started using those features more extensively everywhere else. After seeing Rails, I very rarely write queries in ColdFusion - instead, I've got a component that takes care of it for me.
  
  
• Because of my interests in Java and Ruby, I've recently started browsing JRuby's source code and issue tracker. I'm not yet able to put into words what I'm learning, but that time will come with some more experience. In any case, I can't imagine that I'll learn nothing from the likes of Charlie Nutter, Ola Bini, Thomas Enebo, and others. Can you?

What's next?

Missing from my experience has been a functional language. Sure, I had a tiny bit of Lisp in college, but not enough to say I got anything out of it. So this year, I'm going to do something useful and not useful in Erlang. Perhaps next I'll go for Lisp. We'll see where time takes me after that.

That's been my journey. What's yours been like?

Now that I've written that post, I have a request for a post I'd like to see:

What have you learned from a non-programming-related discipline that's made you a better programmer?

1. JetBrains' IntelliJ IDEA: An awesome IDE for Java. So great, I don't mind spending the $249 (US) and using it over the free Eclipse. The Ruby plugin is not too shabby either, the license for your copy is good for your OSX and Windows installations, and you can try it free for 30 days. Martin Fowler thinks IntelliJ changed the IDE landscape. If you work in .NET, they also have ReSharper, which I plan to purchase very soon. Now if only we could get a ColdFusion plugin for IntelliJ, I'd love it even more.
   
   
2. Programming Ruby, Second Edition: What many in the Ruby community consider to be Ruby's Bible. You can lower the barrier of entry for your favorite programmer to using Ruby, certainly one of the funner languages a lot of people are loving to program in lately. Sometimes, I sit and think about things to program just so I can do it in Ruby.
   
   If they've already got that, I always like books as gifts. Some of my favorites from this year have been: Code Complete 2, Agile Software Development: Principles, Patterns, and Practices which has a great section on object oriented design principles, and of course, My Job Went to India. I have a slew of books I've yet to read this year that I got from last Christmas (and birthday), so I'll have to list those next year.
   
   
3. Xbox 360 and a subscription to XNA Creator's Club (through Xbox Live Marketplace - $99 anually) so they can deploy their games to their new Xbox. This is without a doubt the thing I'd want most, since I got into this whole programming thing because I was interested in making games. You can point them to the getting started page, and they could make games for the PC for free, using XNA (they'll need that page to get started anyway, even if you get them the 360 and Creator's Club membership to deploy to the Xbox).
   
   
4. MacBook Pro and pay for the extra pixels. I love mine - so much so, that I intend to marry it. (Ok, not that much, but I have been enjoying it.) The extra pixels make the screen almost as wide as two, and if you can get them an extra monitor I'd do that too. I've moved over to using this as my sole computer for development, and don't bother with the desktops at work or home anymore, except on rare occasions. You can run Windows on it, and the virtual machines are getting really good so that you ought not have to even reboot to use either operating system.
   
   Even if you don't want to get them the MacBook, a second or third monitor should be met with enthusiasm.
   
   
5. A Vacation: Programmers are notoriously working long hours and suffering burnout, so we often need to take a little break from the computer screen. I like SkyAuction because all the vacations are package deals, there's often a good variety to choose from (many different countries), most of the time you can find a very good price, and usually the dates are flexible within a certain time frame, so you don't have to commit right away to a certain date.

Happy Thanksgiving to those celebrating it, and thanks to all you who read and comment and set me straight when I'm wrong - not just here but in the community at large. I do appreciate it.

Do you have any ideas you'd like to share (or ones you'd like to strike from this list)?

What we really need here is a way to filter the collection while looping over it. Move that extra complexity and indentation out of our code, and have the collection handle it.

In Ruby we have each as a way to loop over collections. In C# we have foreach, Java's got for(ElemType elem : theCollection), and Coldfusion has <cfloop collection="#theCollection#"> and the equivalent over arrays. But wouldn't it be nice to have an each_where(condition) { block; } or foreach(ElemType elem in Collection.where(condition))?

I thought for sure someone would have implemented it in Ruby, so I was surprised at first to see this in my search results:


However, after a little thought, I realized it's not all that surprising: it is already incredibily easy to filter a collection in Ruby using the select method.

But what about the other languages? I must confess - I didn't think long and hard about it for Java or C#. We could implement our own collections such that they have a where method that returns the subset we are looking for, but to be truly useful we'd need the languages' collections to implement where as well.

Of these four languages, ColdFusion provides both the need and opportunity, so I gave it a shot. First, I set up a collection we can use to exercise the code:


Then, I exercised it:


Obviously, that breaks because don't have a cf_loop tag. So, let's create one:


It works fine for me, but you might want to implement it differently. The particular area of improvement I see right away would be to utilize the item name in the where attribute. That way, you can use this on simple arrays and not just assume arrays of structs.

Thoughts anybody?

Chad mentions a measure he used to assess a job candidate's depth of knowledge in Java: a question of how to make the JVM crash.

I'm definitely lacking in this regard. I've got a pretty good handle on Java, Ruby, and ColdFusion. I've done a small amount of work in .NET and have been adding to that recently. I can certainly write a program that will crash - but can I write one to crash the virtual machine (or CLR)?

I can relunctantly write small programs in C/C++, but I'm unlikely to have the patience to trace through a large program for fun. I might even still be able to figure out some assembly language if you gave me enough time. Certainly in these lower level items it's not hard to find a way to crash. It's probably harder to avoid it, in fact.

In ColdFusion, I've crashed the CF Server by simply writing recursive templates (those that cfinclude themselves). (However, I don't know if that still works.) In Java and .NET, I wouldn't know where to start. What about crashing a browser with JavaScript?

So Chad mentions that you should know the internals of JVM and CLR. I should know how JavaScript works in the browser and not just how to getElementById(). With that in mind, these things are going on the to-learn list - the goal being to find a way to crash each of them.

Ideas?

But back then, and even until relatively recently when tabs were confined to the realm outside of Internet Explorer, the amount of people who used the approach in web applications was small. So small, in fact, we felt we could be lazy and hold what should have been repeated as hidden form fields within sessions instead.

Now Internet Explorer has tabs, and people are starting to use them. That changes things as people start to internalize the concept and use the productivity-boosting feature in ways that break our applications. Now, instead of presenting a list of customers and expecting our user to click one and edit the associated records until he's complete; our users are opening three or four customers (customerA through customerD) at a time in separate tabs. If you had stored form state in the session, when they think they are editing customerA, they will in fact be changing the record for customerD with a form that is pre-filled with values from customerA. Oops.

Luckily, the fix is relatively easy: just start storing the state in the form instead of the session. Of course, that's only "easy" if the path through your forms is linear. What about the case where many different forms can be traversed in just about any order, sometimes they appear based on what other forms said, and some are required while others can be skipped?

It's still easy if you've got automated tests that cover the possible scenarios. Just strip out the session and add the data to the forms until the tests pass. If you don't have tests, and you are not familiar enough with the different paths to create 100% coverage (or its been so long you forgot the paths), it's not looking good for you. Chances are, you don't have tests. This is a relic from the days before tabbed browsing, and who was doing automated testing then?

But, there is still one way out for you: inject the needed fields into your form after it's been crested but before it has been sent to the browser. I've not yet come across a situation in Rails or Javaland where I needed to do this, so I haven't investigated how to do it there (and Rails is new in the first place, so its unlikely it would be a problem in any application if you've thought to avoid the practice now that tabbed browsing is popular). But in ColdFusion, it is easy. Since Application.cfm is run before the request, you can check what page is being requested from there and intercept it if the page is one on which you need to do this processing. Wrap a cfinclude of that page in cfsavecontent tags, and now you have a string of what will be sent to the browser. Just find the spot you need to insert your data, insert it, and output the resultant string to the browser.

In my case it was especially easy because fortunately, we were only storing the id of the record in the session and I could be sure it needed to be in every form on the page. Thus, my code looked like this (in Application.cfm):


If you can get at the data before it is output to the browser, this strategy works well. However, I don't like the magic nature of it when you'll be editing the individual files later and wondering why in the world it works. It's only a hack, so expect to learn the application well enough to test it and put the right fix in later. But, this works well as a temporary solution as you can be more sure it works than you can be sure about the flow of the application when it gets quite complex.

For this post, I just wanted to give an overview of the symposium in general. As soon as I walked in and found my name badge, I knew these guys were a seriously professional conference, and they would pay attention to the small details to make our experience great.

Why? The badge was inside a nice, hard plastic (think something you might store a baseball card in, but bigger) and worn around a neck clip - the kind you used to see people wearing to hold their keys around their necks. But that wasn't the impressive part - the most impressive was that they put the schedule inside too, upside down on the back side, so all you had to do was flip over the badge and you could read the schedule and room assignments easily.

We also got a nice laptop backpack, a leather notebook where we could store pre-printed slides and code samples from the sessions we attended, as well as a CD with all the presentation material from every session.

Now, that's all useful for going back and having something to reference, but it's also great to have so you can flip through and see a good overview of sessions before you attend, so you know if it will be valuable for you.

The rooms were outstanding. We stayed in the Marriott Austin Airport South. I got the feeling the hotel staff greatly appreciated us being there - I got even better service from them than I normally expect from Marriott. Now, I'm not sure of the weekend rates there, but Thursday they wanted 200 dollars (US), and Sunday they wanted $189 (if I recall correctly). But NFJS was able to negotiate $99 rooms for us for Friday and Saturday. It was certainly nice to get such great service at such a bargain. Of course, it would have been even nicer if the hotel provided free wireless Internet access. NFJS provided it in the conference rooms, but that didn't reach all the way up to our rooms. I'm getting free wireless next door in another Marriott-owned hotel for over 100 dollars less than I would be paying at Marriott proper tonight!

The food was amazing as well. Just good all-around, and even included options for vegetarians that didn't include side-dishes only. I was fairly impressed by that. Also, I think the hotel bar is the cheapest bar in Austin. =)

Everything went extremely smooth from what I could tell. Jay Zimmerman and any other staff (including the hotel) did an excellent job of ensuring that. Oh yeah (read: "most importantly"), the speakers were awesome. They really knew how to entertain and be informative. But, for being a Java-based conference, it was interesting (exciting?) to see how many bashes Java got - and not just from the speakers. I'll probably blog some funny quotes later.

The first night most of the speakers made themselves available (aside from in-between sessions) at the hotel bar. I had dinner with a friend of mine (who I didn't know was going to be there) and Scott Davis, who is an amazingly charismatic guy. The second night a lot of people went to Bruce Tate's house. Someone later told me the group I was hanging out with in the dynamic languages BOF had been invited, but by that time we didn't feel like going to crash the party (not to mention a second-hand invite of "I heard" isn't that strong a thing to go on in the first place).

Also, one of the presenters had Sean Corfield's blog on his bookmarks toolbar, so that was exciting to see (I'll have to go through my notes to remember who).

In any case, I've got tons of sessions to blog about, some of which may contain useful information for you. I'm staying another night in Austin so I won't start tonight (other than this one), but keep a look out for more detailed posts about the actual content of the conference in the coming days.

PS: I'm also looking forward to catching up with all the other blogs and seeing how CFUnited went. But if I see one more post about the iPhone I'm going to have to duct-tape my head like a mummy gets wrapped to avoid exploding.

The first was regarding case (as in upper/lower). The XML elements needed to have the same case as described in the documentation (which contained at least one element that wasn't in the real specification anyway). That's fine - I can live with that. But it annoyed me a little that they saw fit to use camelCase. (this is only a machine, after all. I don't think it cares if it can easily read the differentWordsAllBunchedUpTogether.)

At least if they had chosen all upper or all lower I can easily switch the case of my own, but requiring camelCase made my code quite a bit nastier than it needed to be. Now, I have to provide a lookup mechanism for each element in the XML. Of course, if the language I was using was case sensitive, this wouldn't be a big deal at all. However, even though it is not sensitive to case, this is still not that bad of an issue.

But then restriction number two comes along: all elements must be sent in the order they appear in the specification. We're just passing data around here. The XML describes the data in the first place, so what use is ordering it? I understand that parent/child relationships must remain intact, but I cannot see how there could possibly be a good reason that the "firstName" element should come before the "lastName" element. (Can you? I'd love to know!) Why don't we just go to a flat file and forget about the angled brackets altogether?

I might as well have just hard-coded the XML and had a variable for every element/attribute!

Can you spot the problem? Its easy to miss: I forgot to var my loop variable, item. I normally remember to put most variables in the var scope, but I often forget to do so when variables are being defined in or by a tag, or in a loop. Things like item, index, and query names. So I spent some time trying to figure out why ColdFusion's stack didn't seem to be working properly: instead of closing the tag with the correct element name, it was closing it with the last used one on recursive calls.

I was about to ask if anyone could help me spot the flaw in my logic when it hit me - the var scope!

Later, he moves on to describe four different types of design for your DSL. This was something I hadn't really thought about before, therefore it was the most interesting for me. Here are the four types:

• Instantiation: The DSL consists simply of methods on an object. This is not much different from normal programming. He says it is "DSLish," perhaps the way Blaine Buxton looks at it.
  
  
• Class Macros: DSL as methods on some ancestor class, and subclasses can then use those methods to tweak the behavior of themselves and their subclasses. This follows a declarative style of programming, and is the type of DSL followed by Ruby on Rails (and cfrails) if I understand him correctly.
  
  
• Top-level methods: Your application defines the DSL as a set of top-level methods, and then invokes load with the path to your DSL script. When those methods are called in the configuration file, they modify some central (typically global) data, which your application uses to determine how it should execute. This is like scripting, and (again) if I understood correctly, this is the style my (almost working correctly) Partial Order Planner uses.
  
  
• Sandboxing (aka Contexts): Similar to the Instantiation style, but with more magic. Your DSL is defined as methods of some object, but that object is really just a "sandbox." Interaction with the object's methods modifies some state in the sandbox, which is then queried by the application. This is useful for processing user-maintained scripts kept in a database, and you can vary the behavior by changing the execution context.

Note: These are "almost" quotes from the slides/talk (with a couple of comments by myself), but I didn't want to keep rewinding and such, so they aren't exact. Therefore, I'm giving him credit for the good stuff, but I stake a claim on any mistakes made.

The talk uses Ruby as the implementation language for the example internal DSLs, but I think the four types mentioned above can be generalized. At the least, they are possible in ColdFusion - but with Java, I'm not so sure. In particular, I think you'd need mixin ability for the Top-level methods style of DSL.

Next, Obie catalogs some of the features of Ruby you see used quite often in crafting DSLs: Symbols (because they have less noise than strings), Blocks (for delayed evaluation of code), Modules (for cleaner separation of code), Splats (for parameter arrays), the different types of eval (for dynamic evaluation), and define_method and alias_method.

Finally, he winds up the presentation with a bit about BNL. I liked this part because I found Jay Fields' blog and his series of articles about BNL.

As always, comments, questions, and thoughts are appreciated. Flames - not so much.

And it worked. No fuss or anything.

Is anyone else doing this? What sorts of object methods have you added? What would you find useful if you had the stomach to do this?

• Added the automatic include of views/layouts/application_header.cfm and views/layouts/application_footer.cfm even when using view CFC (formerly, this only happened if you instructed it to and using .cfm templates for views). Also added auto-include of controller-specific templates simply by placing controllerName_header.cfm and controllerName_footer.cfm in views/layouts.
  
  
• Brought the existing unit tests up to date (amazingly, with all the neglect there was only a couple of very minor problems, mostly due to changing paths).
  
  
• Migrated request processing out of individual applications and into Dispatcher.cfc. This is a major improvement in the DRY arena, and allows me to make changes to the ways requests are processed without having to make them to every application that uses cfrails (Thanks to Dan Lancelot for prodding me to do this a couple of months back).
  
  
• Added some minor Spry integration for form validation - this only works with required text fields at the moment.
  
  
• Added the ability to use primary keys that aren't named "id" (although, still no composite PK support)
  
  
• Changed behavior of a particular file to not catch errors (it was getting hard to debug)
  
  
• Added an onchange attribute for form elements (can edit onchange per column without writing out the entire form)
  
  
• cfrails now automatically creates blank controller, model, and view cfcs if you request one where it doesn't exist. This just makes it easier to develop, and may end up being a setting you turn on and off.
  
  
• Added validations where you can specify a function to call and a message to display upon failure for validating forms (as opposed to simply using the built-in autovalidation based on DBMS metadata).
  
  
• Added support for bigint, tinyint, and smalldatetime data types.
  
  
• Added the ability to call a function when outputting a column in a list (though complete closures are not yet fully integrated)

So, there was a lot done as you can tell. And, MySQL support is coming soon, I promise!

Update: Oh yeah, I forgot what might be the most important part: New, more complete docs on cfrails are available (thanks to cfcdoc).

There has been some suggestion about packaging this into CFCs (and I like that idea). That will happen down the line (when someone has time to sort through and categorize the tags), but I see it as being an option to use instead of simply including the entire file. So basically, I'm thinking related functions/tags will be in separate libraries, and then the respective CFCs will include them, and a monster CFC might aggregate all the individual ones (not sure about the monster CFC, but I think a monster file would be good for include purposes if you choose that route).

I want to stress that I only checked that it compiles. I've got some non-CF work to do over the next couple of days, so if anyone has CF work to do and wants to use it, we'd appreciate any comments on what works, what doesn't, and what works but not how you might expect. Again, feel free to tell us these things using the issue tracker at RIAForge or the scriptaGulous Google group. I'll commit to responding and fixing things relatively quickly (most likely within the same day).

There are no files yet, but here's how I'd like to see it go:

1. The code for the CF8 version will be easy since all tags can accept attributeCollection, we just send named parameters to the function and send arguments through attributeCollection
2. I'm tentatively thinking all methods should be prefixed with "cf." Thus, <cfabort> becomes cfabort(), and so on. This has two potential benefits: It makes memorization easy, since we already think in those terms for tags, and any potential naming conflicts with actual CF functions are gone (as far as I can think from the top of my head).
3. It's going to take a while to do every tag. I don't want this to be "Sam's project" (though will eventually complete it even if no one contributes), but more of a community project. So if you've got time to do a tag with all the possible combination of attributes (so this will work in previous versions), I'm sure everyone would appreciate the effort. I'll try to commit to doing a tag every other day or so.
4. I'm thinking it might be nice to put these into packages of related functions (based on the CF documentation), but I don't know how useful that would be.
5. If you already know of something like this, please let me know and I'll gladly drop this one (or if you want to call it ScriptaGulous and host it on RIAForge perhaps we can get them to transfer project ownership).

So, what do you think? Your input and/or participation is/are most welcome!

We were looking at 3 different computers and trying to figure out why one used to work, one stopped working long ago, and one worked just fine. We were lucky enough to be doing a fresh install of Windows on one of the computers, and on the clean install FTP worked fine. After installing CF Studio 4.5 (he's old school) the FTP stopped working. Problem found.

To fix it, we just needed this technote from Adobe (which was hard to find because the searches were yielding links to House of Fusion --> Allaire.com --> Macromedia.com --> Page no longer exists on Adobe.com). Anyway, that's what you'll likely need to do to get your FTP working again if you're a CF developer and it stopped working.

The first item where I see a distinction that needs to be made is on, "what do we mean when we are talking about XML?" I see two types - XML the paradigm where you use tags to describe data, and the XML you write - as in, the concrete tags you put into a file (like, "see that XML?"). We're talking about XML you've written, not the abstract notion of XML.

The second idea: what is code? What is data? Sean Corfield offers what I would consider to be a concice, and mostly correct definition: "Code executes, non-code (data) does not execute." To make it correct (rather than partially so), he adds that (especially in Lisp) code can be data, but data is not code. You see this code-as-data any time you are using closures or passing code around as data. But taking it a bit further - your source code is always just data to be passed to a compiler or interpreter, which figures out what the code means, and does what it has been told to do.

So is XML code? Certainly we see it can be: ColdFusion, MXML, and others are languages where your source code is written (largely) in XML. But what about the broader issue of having a programmatic configuration file versus a "data-only" config file?

Is the data in the config file executable? It depends on the purpose behind the data. In the case of data like


I think (assuming there is nothing strange going on) that it is clearly data. Without knowing anything about the back end, it seems like we're just creating a data structure. But In the case of DI (and many others uses for config files), I see it as giving a command to the DI framework to configure a new bean. In essence, as Peter notes, we've just substituted one concrete syntax for another.

In the case of XML, we're writing (or using) a parser to send data to an intepreter we've written that figures out what "real" commands to run based on what the programmer wrote in the configuration file. We've just created a higher level language than we had before - it is doing the same thing any other non-machine code language does (and you might even argue about the machine code comment). In the configuration case, often it is a DSL (in the DI case specifically, used to describe which objects depend on which other objects and load them for us).

While we don't often have control structures, there is nothing stopping us from implementing them, and as Peter also notes, just because a language is not Turing complete), doesn't mean it is not a programming language. In the end, I see it as code.

Both approaches are known to have their benefits and drawbacks, and choosing one over the other is largely a matter of personal taste, size and scope of problem, and problem/solution domain. For me, in the worlds of JIT compiling and interpreted langages, the programmatic way of doing things tends to win out - especially with large configurations because I prefer to have the power of control structures to help me do my job (without having to implement them myself). On the other hand, going the hard-coded XML route is especially popular in the compiled world, if not for any other reason than you can change configurations without recompiling your application.

I don't make a distinction between the two on terms of XML is data, while programming (or using an in-language DSL) in your general-purpose language is code. To me, they are both code, and if either is done incorrectly it will blow-up your program.

Finally, I'm not sure what value we gain from seeing that code is data (and in many cases config data is code), other than perhaps a new way of looking at problems which might lead us to find better solutions. But that isn't provided by the distinction itself, just the fact that we saw it.

Comments, thoughts, questions, and requests for clarifications are welcome and appreciated.

Is it good to be such a hybrid? I would think it keeps the staticians and dynamos both happy, but I'm not sure if it can (or even should) be quantified as good/bad. Do we want it to continue down the path that leads to having complete static options available (for instance, you can type your variables if you want to, and it will provide benefits of static languages, but leaving them off keeps it dynamic)?
What if you could write Java code as: name = "Johnny"; name=3; (and switch back to having it statically typed in the same class)? What would it look like to do operations on a dyanmically declared variable when a static one is expected? I suppose this has more to do with strong vs. weak typing, really, but would the mere existence of allowing dynamic typing mean the compiler can no longer check any method call for correct argument types? Would the fact that you can declare variables without a type make statiphiles cringe (because they may intend to use the typing and leave it off, and then the compiler won't tell them)?

Does allowing any typing to be done dynamically mean all checks must be performed at runtime? If so, what is the point to adding static features into CF (to make it throw an error perhaps 2 lines prior to where it would have thrown it - at the beginning of the method rather than where the argument is used)?

Does it even make sense to classify a language as a static/dynamic hybrid? (And can I ask any more questions?)

My interest lies in the second complaint though - that writing it to a live file (rather than dynamically figuring it out - which I've been calling "synthesis," as opposed to generation) means you can't get the benefits of upgrades to the scaffolding engine (and, it is not following DRY!). But, if you just use the default scaffolding, what happens if it doesn't come out just right (assuming you've even passed the notUgly test in the first place)? Well, thats a big part of what I'm trying to solve with cfrails, by using a DSL that provides tweakability to the scaffolding, without requiring you to write to a file (though, if the tweaks won't work, you are always welcome to go to a file with HTML and the like). The interesting part for me about the RoR plugins above, therefore, is that it appears (I haven't checked them out yet) that at least Hobo contains a DSL, called DRYML, to help along those lines.

I'll be having a closer look at those when free time becomes a bit less scarce. What do you think? Is it the holy grail, or can there be very useful "scaffolding"?

Most of the code below is tangential to the example, but as a whole, it illustrates the point I was trying to make, and shows a practical aspect of closures (using Sean Corfield's Closures for CFMX). In particular, there are a couple of methods of interest. Rather than passing a closure directly into the method, as one might normally do, I created a function, onOutputOf(), that puts the closures into a data structure to be checked by the method. This is only because it made more sense in my original application to do it that way, since the programmers are not typically calling the list() function themselves.

Next, there is a method to find a closure for a column if it exists. Rather than have the list function checking the closure storage data structure itself, I opted to use getClosureToBeRunOnColumn() to keep the methods as cohesive as I could see.

Of course, the list() method itself shows the use I found for this. And finally, callClosure() shows how I'm using Closures for CFMX, and the next to last line shows an example of passing in some code as a string.

If you have any questions (or comments on how I can improve this!) please feel free to let me know. I enjoy the feedback =). I'm done rambling now. Here's the code:

Being that I've been burned a lot in the past by not following the DRY and OAOO principles, I'm quite fanatacal about following them nowadays. So when I needed the list() function to behave a bit differently for this one type of object, I didn't want to replicate the entire list function -- I just wanted to add a bit of logic in the middle. That's where closures enter the picture.

I decided I'd create a function, onOutputOf(column="column that special stuff needs to be done to", run="function, or code"). So, instead of outputting the value of the cell, the list now knows to run the code provided in the run parameter.

And now, some really complex code doesn't need to be replicated elsewhere. Isn't that sweet?

Then, of course there are the generic getters and setters that Peter Bell is often talking about. You know, where you have some implementation like (excuse the pseudocode -- I'm just lazy right now =), but it shows the idea):


This is easy enough to implement without resorting to file I/O, and it has the side benefit of allowing you to check if a getAttribute() method already exists, and call it if so. And where this morning starts getting wierd is that I randomly came across this post from David Harris where he and Peter are discussing this strategy in the comments. What a coincidence.

But what I really wanted is something you see in Ruby (and other languages too): the attr_accessor (or reader and writer) in Ruby. You can do something like this:


See how easy that is?! That's what I really want to be able to do in ColdFusion. And the best part is, it really would only (as far as I can tell) require a couple of changes to the language. The first one being something like if (cowtype= is defined as a method on the object I'm working with) call it when doing an assignment; and the second one is being able to define a method like so:


In any case, the morning got more coincidental when I ran into this post on InfoQ about Adding Properties to Ruby Metaprogramatically, which I'd recommend reading if you're wanting to metaprogram with Ruby (or if you are just interested in that sort of thing).

It's now time for me to crawl back into my hole and write about the wonderful world of The History of Partial-Order Planners. (The good news on that is I'm getting close to the fun part - actually programming one).

So method= is going on my wishlist for CF9, and in the mean time I'll probably end up going the generic getter/setter route. What are your thoughts? What would you go with?

I can honestly say I've never had to build an application that needed to be portable between databases, aside from between different versions by the same vendor (although there was once where it might have been nice, since I used the apparently impossible to find in hosting combination of Microsoft SQL Server and Java). My guess is that unless you're building software which is meant to be deployed on different databases (such as a framework, library, or software you plan to sell over and over again which you won't be hosting), you probably haven't needed to do so either... Or at least, not often enough to let it impact your coding.

So today, I started my quest to turn cfrails into a DB-independent framework (as of today, March 30, 2007 it works only with Microsoft SQL Server). Since one of my goals is to limit the amount of configuration the programmer has to provide to get up and running, simply telling cfrails your database metadata (instead of having it look in the database itself) was out of the question (XML or otherwise). I've looked at using CF's getMetaData(query_result) function in the past, but that doesn't have near the amount of data that would be useful for me.

I figured my best bet would be to drop down into Java and use its DatabaseMetaData interface. This way, I could let Java take care of the differences between DB implementations for me. Sure enough, looking over the docs, it had everything I needed. I went ahead and whipped up some code to use it and test it before trying to put it into Coldfusion. If you're interested, here's most of what I needed (only using the jdbc/odbc bridge). If not, feel free to skip this code:


(if you need more info, see the docs I linked above -- this is just a small sampling of the metadata available for columns in a table)

Sweet! I didn't really need ORDINAL_POSITION since it already ordered the results by it, and COLUMN_DEF (the column's default value) wasn't all that important (well, I could figure out a way to get that later, anyway). But, when I went to put it into ColdFusion, I realized - I have no clue how I might get the information I need to connect- forget about the fact that I'm not sure the average developer (CF or otherwise) will know what a catalog or schema is, or which one they need to access if they do. Who in the world isn't going to have to go through some work to figure out what driver they need and what the connection URL is?

So, I needed a way to let Java know these things through CF. I thought I could get at least some of that information from the CFAdminAPI (or whatever the proper case and spelling is), but I didn't really like that option. I couldn't think of any other option that would allow me to use Java to get the metadata, so I decided I'd have a look at the codebases for Transfer ORM, Reactor, and DataMgr and see how those smart guys did it. I only browsed, but I didn't find anything useful for me trying to use Java to do the trick. So, it was back to writing a DB-specific implementation for each database I wanted to support.

Luckily for me, even though my design doesn't approach the modularity and complexity of Reactor or Transfer (yet... I'm a big fan of YAGNI and haven't had any reason to break out of the simple design I currently have and Transfer is downright confusing for me to quickly browse it and figure out what's going on), it is quite flexible enough for me to do the different implementations and integrate them with ease. To do so, I only need to refactor the current routine that gets the MSSQL Server metadata, and provide a way to figure out which database type it is looking in (each of these only has to happen once). After that, all that remains is to write a query to get the metadata and convert its output (for each implementation) to the interface the rest of my code expects. Not hard by any means, but it would have been cool for maintenance purposes and greater overall simplicity if I could have let Java handle it.

So now that you've gotten this far in this boring post with no conclusion, have you used Java to get the DB metadata in CF (or even outside of CF)? Have you found a way to do it without making the programmer specify the connection URL and driver?

If you set those files up and navigate to start.cfm, you'll see that "the session.id was 0." is output in places you wouldn't expect.

I understand the cause, but I would have expected CF to behave as it does in 7, since (I'm guessing that) appending cgi.query_string to your URL would be fairly common practice. And, as a side note, I'm fairly sure I've run into this before - only I had just forgotten (it seeeeemed familiar). That's why I wrote it down this time (and to help anyone who may also have had the problem).

I'm a big fan of choice, and of learning, so I tend to think that having more choice (to learn from as well) is better, but if choices are substantially similar, one should certainly strongly consider contributing to the similar project. Of course, a question reveals itself here: "how can I be sure what I'm doing is substantially different?" One cannot take the time to review every framework out there, and probably asking for just a couple is too much. In my case, I didn't review all the frameworks I knew about, but I did read up on them, at least. (As a side note, I don't know when Model Glue started generating code, can anyone confirm when for sure - has it always been there?)

I feel like cfrails is "substantially" different than anything I've seen (even CF on Wheels, which I also plan to contribute to), but I always worry if what I'm doing may at some time be considered to be stepping on toes. I don't want to be perceived that way, for sure. Further, I don't think I'm solving any problem that hasn't been solved before, but I do think I'm doing it in a different way - and I think there is merit in that, even if, in doing so, I am duplicating some of the effort of others (for instance, cfrails has its own ORM, but I felt that changing what I knew about the existing ones might make them more specific than they were intended to be). With all that being said, I think generating code, while certainly being nothing new in computer science or even ColdFusion, is relatively new to being done publicly or in Open Source in ColdFusion. So, even though there are a lot to choose from, right now, it is a good thing, in my opinion, as we are all able to learn from each other, and may discover better ways of doing things.

In the end, as far as the generators go, it seems most of them are quite different in how they tackle the problem, and some of them even in the problems they solve. I think it would be great if those behind the various frameworks would write about how they came about (or, if you don't mind, leave a link here if you've already written it) and what problems they are intended to solve, and how they solve those problems. That might be a great learning exercise for all of us in the community (I plan to write that up for cfrails after the next couple of weeks -- midterms, you know? =) ).

So, now we have a table, let's call it drink. It has columns id (int, pk, identity), brandname (nvarchar 50), label (nvarchar 50), bottle_date (datetime), and description (ntext). The code currently looks like:


In this case, assuming you have the corresponding controller and view set up, cfrails will show label as "Label" and bottle_date as "Bottle Date" to the user. description will be capitalized as well. Label will be shown as an <input type="text"/> and description as a <textarea>.

But while brandname would show up as an input text field too, it will be displayed as "Brandname." Now, if you had called the column brand_name or brandName, the space would be put in automatically. That's easy enough to do with new tables, but if you're working on an old one where you had used the convention numBottles, for instance, it would show up as "Num Bottles," which is not something you want displayed to the user.

Another issue shows up with bottle_date. Since it is a datetime column, cfrails will display it with a select for each of year, month, day, hour, minute (when I get around to implementing that). But in this case, you might only care about the month and the year.

How do you solve those two problems? That's where some of the configurations / extended metadata comes in.

One of the goals I have in cfrails is to keep related data together. I'm not fond of the thought that I might need to have to open an extra file to configure my class. So, if I want the view to show the name of brandname as "Brand Name," I should be able to do that within drink_view.cfc (actually, you can hand-code your views in templates if the customization options don't fit the bill, but I'm not getting into that here either).

What I wanted to be able to do was have my files look like this:


(As a side note, you could use the underscore version of those functions as well. Also, the month/YearDate has not yet been implemented as of Feb. 20, 2007, and is subject to change, but you get the idea.)

That certainly seems easy enough. But, since the controller has to pass the model into the view through the view's init() method, that configuration code at the top will be executed before the view has any knowledge of a field called "brandname."

It was a long time coming, but we've finally reached the point where I needed a stack (well, the order didn't really matter, so what I implemented wasn't really a stack, but it served the same purpose).

Instead of running the actual function when it is called, I checked if the component was fully instantiated. If not, I "stack" each function that needs to be run, along with the supplied parameters. Then, in the init() method, the stacks are "popped" (it's actually done in a queue fashion, but it feels odd saying that).

Thoughts?

I also noticed that I needed to mix views on occasion (for instance, a Person_view might need to display a list of checks written, which would be in another component). For this reason, I added getModel(modelName) and getView(viewName) to the mix. This is a great abstraction to remove the long and repetitive createObject calls. The downside is that apparently, CF6.1 and CF7.0 look components up differently, so these don't work (depending on your directory structure, I think) in the older version of CF.

The most significant addition, however, is that there is now some ability for relationships among tables to be added into your models without much effort: via a call to the autoload_relationship() method in your models. This was a bigger pain than I thought it would be, but well worth the effort.

Overall, I'm starting to get excited about the progress being made on this. I've been using it to extend an application we had written some time ago, and the results are good. For comparison, I took one of the "old" modules that was written without cfrails, of similar size to one I am working on now. The one written using cfrails is a bit smaller in scope and less complex, so I added in the other 2 "modules" I've been working on using it as well. Whereas the "old" one was over 500 lines, the 3 parts (two are quite small) using cfrails total to about 65, including white space. I spend more time marveling at the thing than I do programming (just kidding, of course =) ). Of course, this isn't any scientific comparison, especially since I'm sure I've grown as a programmer since writing the other one. But, it is still pretty remarkable. The hardest part is going to be memorizing the function calls. Well, not that it's hard, but it will take time looking at the (almost non-existent) docs, which may slow you down.

So, what's ahead? First I need to reconcile differences between CF7 and 6.1. After that, I need to add in abstractions to make "select" form inputs, as well as groups of radio buttons and checkboxes. There are some other abstractions I'm thinking of putting in, but I'm not sure if they add any value or just clutter the interface, so I'm leaving them out for now. Other than that, there are some more options I'd like to put in, like concatenating columns of a (HTML) table, to make the list more customizable. Finally, a few more major things are integration with an Ajax framework, integration with more DBMSs (right now, only SQL Server 2000 and 2005 are supported), and I'm toying with the idea of automatically loading referenced tables when there are columns of the type tablename_id. The biggest thing is re-writing the documentation. The only docs I have right now are based on version 0.1.0. Clearly, a lot has changed since then.

Those are the things off the top of my head. Even though the version number is only 0.2.1, I'm starting to feel awfully close to version 1. The main thing is that I want the interfaces (of the code variety) to be intuitive and easy to use, so I'm getting practice in using it. I may try to write a different application using it before I go to version 1, to make sure that it remains as easy to use in that one as it is in the application I'm using it for now.

What if I want to load some data from the database, but I don't know which data -- only the function name will tell me that? Then, you have to either write out to a file or alias that function to a function that throws an error, examines the line, and figures out what the function was called (hat tip, Per Djurner -- I still love that solution =) ). Even that gets iffy if there are two on one line. Let's take it even further - in the previous case you already know the function name. But why can't I just let the user type what they want for a function, and I figure out what its supposed to do? I'd love to be able to do that.

The point here is that I'm not the only one running into roadblocks when it comes to doing dynamic stuff. Smart people who've been using ColdFusion for many years are having to run workarounds. And not only that, newcomers as well!

So three things I'd like to see (which are all somewhat related) in ColdFusion:

1. Evalute a string of code
2. A method_missing handler, to figure out what to do if a user calls a non-existent method on a class.
3. A way to dynamically write functions (which would probably be about the same as item 1)

And, if there is some capability which exists and I have said it doesn't, please let me know!

Update: I suppose if I really wanted to, I could write a CF interpreter in CF (or drop down into Java). Anyone want to join me on that endeavor? =)

And so we have something to test with, I've made this simple test.cfc:


And finally, here we see how to create a pseudo-object of test.cfc:


Now, it was fun to play around with this stuff, but I realize it's probably useless (I mean, I didn't do any benchmark testing to see if it was faster than object creation, since I had to read and write to the file system). Further, I'm not at all happy with the design. My main concern is that the semantics are exposed: a user of this code needs to know they have to call the methods in the order shown in test_pseudo_object.cfm to have it work. That could be fixed however, if I required that the name of one of the variables (and its scope) be passed into the CFC via the init() method. Then, it would return the struct, all in one go. In fact, if I were to ever start using this, I would reimplement it in that fashion, and it wouldn't be very hard.

Anyway, what do you think of this useless ...

I think part of it may be due to bad habits and not having the ability to notice a spot where I need another class. But, I have no evidence of this, since I haven't been able to notice it =). I do have evidence of the other two options however: CF being so much more high-level than Java is fairly self-evident: Where I'll often use a façade to interact with a database in Java, in Coldfusion it would be pretty useless - since there is no need to instantiate a connection, prepare a statement, prepare a record set, and then close/destroy them all when you don't need them any more. The other, about the performance hits, I know sometimes stops me from creating more classes. For instance, I recently used a mixin to achieve some code reuse with a utility method that didn't belong to any of the classes that used it. Clearly it broke the "is-a" relationship rule, but I didn't want to incur the cost of instantiating an object just to use a single, simple method (the application was already seeming to perform slow).

Of course, this is not a zero-sum game. It is certainly due in part to all three. I'd like to think it is mostly due to the expressiveness of Coldfusion, but I'll always have to wonder about the bad habits.

In Ruby, I've developed a couple of "significant" applications, but those didn't approach the size of the one's I've done in Coldfusion or Java. It's hard to say for sure, but I'd guess that I'm writing much fewer classes in Ruby than I would in Java. Again, this is mostly due to Ruby's expressiveness.

In any case, have you noticed yourself writing fewer classes in CF than in other languages? Have you thought about it? Come to any conclusions?

But surely you can't provide every combination! That's where you'd want to provide a filter argument. Here it comes again: another but! You don't want to simply provide filter="where column1=#form.column1#" - you want to parameterize the query for the developer. Now, one way to do that would be to parse the filter argument and reconstruct it using your cfqueryparams in all the right places. But, an easier way would be to something like this:



Note: I would probably move the "where" into the abstraction, and just let the programmer provide the clause, but for illustration purposes, I left it in there.

For illustration purposes, my function only does this:


But it would be easy enough to modify it to do something useful. Right now, if you run it, you see all the arguments are available in the arguments structure. The first one comes under the key "filter", and the rest come under numeric keys, in the order they were passed (of course, looping over it won't show you that order, but the number of the key represents the position in which it was passed. Now, you can parameterize the arguments, first checking more restrictive and moving to less restrictive parameters (for instance, you'd want to check isDate(), isNumeric(), and fall back on string).

You could name the arguments too, but I haven't found that useful for this example, since they will have their names as keys in the arguments structure, and I know of no good way to figure out the order in which they were passed.

Finally, I haven't yet implemented this in cfrails, so I don't know how feasible basing parameterization on the CF type is, but I'll try to remember to post it when I do implement it. In any case, I'd be willing to bet there are other uses for this - it's just that this is the most important one on my mind recently.

Update: It did occur to me as I was writing this post that you could accomplish the same thing by using an array as a defined argument, with the added benefit of keeping your interface well defined. In all honesty, I would prefer to do it that way if you could define an array inline like: [9, "blockhead", "charlie brown"]. But the way you define arrays in Coldfusion really makes for too much code in what I'd like to accomplish with this, so in this case I'd provide a hint in the function to let a developer know it expects you to pass those arguments. This way, I get cleaner syntax and a decent solution to the interface problem.

In a nutshell, the goal is to have such highly customizable "scaffolding" that there really is no scaffolding - all the code is synthesized - or generated "on the fly." Of course, this only gets you so far. For the stuff that really makes your application unique, you'll still have to code that. But you can compose views from others and such, so it's not like related tables have to stay unrelated, but I do want to stress that right now there is no relationship stuff implemented in the ORM.

I've skipped a few "mini-versions" from 0.1.3 to 0.2.0 because there were so many changes that I haven't documented one-by-one. That's just sloppiness on my part. Basically, I started by following Ruby on Rails' example, and taking my own experience about what I find myself doing over and over again. That part is done, except that the ORM needs to be able to auto-load and lazy-load relationships at the programmer's whim. In any case, once I got enough functionality to start using it on my project, I've been developing them in parallel. The problem is, I've fallen back on poor practices, so the code isn't as nice as it could be.

In particular, there aren't any new automated tests after the first couple of releases, which isn't as bad as it might otherwise be, since a lot of the core code was tested in them. But on that note, I haven't run the existing tests in a while, so they may be broken. Further, since I've been thinking in Ruby and coding in Coldfusion, you'll see a mix of camelCase and under_score notations. My original goal was to provide both for all the public methods, and I still plan to do that (because, since I can't rely on case from all databases for the column names -- or so I think -- I use the under_score notation to tell where to put spaces when displaying the column names). But right now, there is a mix. Finally, the DSL needs a lot more thought put behind it - Right now it is a mix-and-match of variables.setting variables and set_something() methods. Right now it is really ugly, but when I take the time to get some updated documentation up and actually package it as a zip, I should have it cleaned up. In fact, I shouldn't have released this yet, but I was just starting to feel I needed to do something, since so much had be done on it and I hadn't put anything out in quite some time. Besides that, I'm quite excited to be using it - it's been a pain to build, but it's already saved me more time than had I not done anything like it.

In the end, I guess what I'm trying to say is: 1) Don't look at it to learn from. There may be some good points, but there are bad points too, and 2) Don't rely too heavily on the interfaces. While I don't anticipate changing them (only adding to them, and not forcing you to set variables.property), this is still less than version 1, so I reserve the right to change the interfaces until then. =)

Other than that, I would love to hear any feedback if you happen to be using it, or need help because the documentation is out of date, or if you tried to use it but couldn't get it to work. You can contact me here. You can find cfrails at http://cfrails.riaforge.org/ .

His each() method is only slightly different from mine, but significantly so:


He mentions that I "[name] the method and then repeatedly [bind] the value variable. That doesn't actually do what he thinks." He's right. I was using it incorrectly, and I thought that it was sort of odd to do it like that. In particular, I didn't like having to name the closure. This syntax is much cleaner, and easier to understand. And the best part follows: I had mentioned that, "from my understanding of closures, you should be able to modify 'outer' variables within them. Thus, to be a 'true' closure, outputting beenhere above should show true" in reference to this code:


Sean shows that in fact, had I used bind() as it was intended, outputting beenhere would have shown true, as expected. Basically, you can use bind(outer=variables) on the closure to get access to it. Then, in the code for the closure, you could use outer.beenhere = true and of course, the value is changed as you would expect. That is close to what I was thinking of when I said my idea would muck up the syntax, but I was thinking of passing it when creating the object. I actually like this a little better, as it is much cleaner than what I envisioned, and it is easier to see what's going on. It would be nice of course, if there was a way to do it without the programmer worrying about it, but I don't think that is possible with CF, since it uses the the same name for (from what I can tell) are two different scopes.

Anyway, thanks again to Sean for cleaning up my mess.

Luckily for us, Sean Corfield made a Closures for CF package available a couple of months ago (as far as I can tell). A couple of days ago, I gave it a whirl.

The first thing I have to say is that I had to move the CFCs into the same directory as my closure_test.cfm file because it wasn't able to find the temporary templates it was generating. Sean let me know it was because of a pre CF 7.0 bug in expandPath(), which explains why I had trouble with that function in cfrails. Anyway, I've posted a "fix" for 6.0/6.1 to the comments on his blog, which basically just replaces expandPath(".") when it is creating a temp file to use getDirectoryFromPath(getCurrentTemplatePath()).

After that, it did take a while to understand what was going on, but once I figured it out, it was pretty cool. So here's my first class of using closures, and what I did in Coldfusion to implement it: really "encapsulating" something.

Encapsulation is a word you see thrown around quite often. But you don't achieve it by putting getters and setters everywhere - particularly if you are passing back complex data structures from a get() method (without first making duplicates, of course). The idea behind it, as far as I've been able to understand, is to hide implementation details from code which doesn't need to know it. Not only does this make the client code need to be less complex, it certainly lowers your coupling as well. However, if you have a Container (for instance), how can you encapsulate it? If you return an array representation, you've just let the client code know too much. If you don't let the client code know too much, how will it ever iterate over the contents of your Container?

Certainly an Iterator comes to mind. You could provide that if you wanted to, complete with next(), previous() and anything else you might need. But you could also use a closure to allow you to encapsulate iteration over the Container, while still allowing you the freedom to do what you wanted within the loop. Here's my CF code:


And here's the resulting code that uses Container.each(), and passes a closure to it:


Now, from my understanding of closures, you should be able to modify "outer" variables within them. Thus, to be a "true" closure, outputting beenhere above should show true. This would be relatively easy to do by sending the appropriate scopes to the closure, but the only ways I can think of to do that would muck up the syntax. Other than that, it is not code that makes me comfortable, so to speak. The closure_test needs to know that it is expected to use value as the name of the variable. There may very well be a simple way around this that doesn't screw up the syntax, but I have yet to spend enough research in figuring it out. In fact, in Ruby you'd use |value| do_something_with(value), so it's probably as simple as that, or providing an extra parameter (as I saw in one of Sean's examples). Perhaps I will when I run across a need for it (and I'm sure I will be using this in the future).

And now for comparison purposes, I've included the Ruby version that does the same thing:


This isn't normally how you'd code in Ruby, but I tried to keep the look approximately the same. In particular, the each() method is already implemented for arrays in Ruby, so there would certainly be no need to rewrite it.

As you can see, the syntax is a bit cleaner in Ruby than in the CF version. It's so easy to do, one doesn't need to think about using closures - you just do it when it comes naturally. Even with all that, I think Sean reached his goal of proving himself wrong (he said "I was having a discussion about closures with someone recently and I said they wouldn't make sense for ColdFusion because the syntax would be too ugly. On Friday evening, I decided to try to prove myself wrong"). I think it might be possible to improve it a little, but I don't think it would ever become as "pretty" as Ruby's style (unless, of course, it was built into the language). Because of that, and the effort involved in creating them, I don't think it will ever become a natural solution we turn to in CF (although, if I started using them enough, it would eventually become a natural solution). In any case, he certainly did an awesome job, made the syntax much cleaner than I would have thought possible, and gave us a new tool to solve problems with.

In closing, I'd like to talk a bit about the "other" use for closures I've found, what I called a generalized Template Method pattern above. Really, this is just a general case of the way encapsulation was furthered with the each method above. Basically, you are defining the skeleton of an algorithm, and allowing someone else to define its behavior. I mentioned that it's not really the template method pattern, since we aren't confining it to subclasses, and we aren't actually following the pattern - we're just implementing its intent, in a more general way. So basically, you might think of it as a delayed evaluate for letting some other code customize the behavior of your code that expects a closure. I don't know if I've made any sense, but I sure hope so (and if not, let me know what I might clarify!).

Finally I'd like to thank Sean for providing this tool to us, and ask everyone, "what uses have you found for closures, if any?"

Update: Thanks to Brian Rinaldi, Sean Corfield found this post and addressed the problems I encountered. It turns out I was just doing it wrong, so to see it done right, visit his blog. I also responded to a few of the issues raised here.

After I posted a function which I thought solves the problem, Teddy Payne mentioned "this looks academic," as in he couldn't find a use for it. I have to agree with Teddy, in that I can't find a use for it either. But, when some one has a tough problem that I can't see immediately how to find an answer to, I like to find an answer. That compulsion gets stronger as the number of "this can't be done" replies increases. =) (I said). For Per, the problem is more than academic (and this doesn't quite solve it -- I'll investigate that tomorrow). We were discussing ways to dynamically decide what a function should do based on its name.

I started out searching through all the Java stuff to find a solution. When I didn't find one, I thought, why not just search through each scope and compare if the memory address is the same? Well, that didn't work well, but the function below did. I got the idea to insert a UUID key into the CFC and compare the variables in the scope to this from looking through Paul Kenney's cfcUnit (brilliant Paul!).

In any case, if you ever have the need, here's how you can find all the references to an object, but you need to provide the scope to search. This was done because I didn't take the time to figure out if there was a way to reference the "global" variables scope from within a CFC. Now, of course you may want to make it return a list of all the names, or to search all the scopes available, but you get the idea from this.

Since these applications are already well established, I didn't bother creating any layouts - so the page titles aren't exactly representative of each page, but it helped in that I could just wrap cfrails with the current template, and I was done.

Except, for one last issue: clicking on links back to the "old" part of the application meant that cfrails was looking for actions called "thispage.cfm," and didn't know how to handle it. So, in the newly (re)named cfrails.cfm, I just needed a simple check (after the action was extracted from the URL):


It's not something anyone couldn't have figured out, for sure. But, I was just amazed at how easy it was, because this was never a design consideration. I was building it for use on a completely fresh application.

Now, I've known this for quite some time - it's still the same box we were running CF 4.5 on. I think it may have had a couple of upgrades since then, but it's certainly not more than 528 MB RAM with a 1 GHz processor. In any case, I was doing some work this morning in a non-cfrails utilizing project, and it is equally slow (whereas a few months ago, when I was developing heavily on it, it was "normal"). That means it's likely cfrails performance isn't as bad as I thought it was, though I won't stick my neck out and say it performs well.

The further good news is that since I'm working in CF again, I'm wanting to use cfrails for the new work - and that means I'll be adding features to it a lot more often than I was in the last 3 months. Which means that sooner rather than later, it will become useful for someone besides me!

Anyway, we've got a new development box that's been in the works for a little while, with all the latest stuff. And since the current one is forcing me to wait longer for pages to load than the time I spend coding (literally), I'm taking a break from coding until we get it up - which will hopefully be soon.

Finally, if you're one of the few people who have downloaded and used cfrails, I'd like to hear from you if you've had performance issues. So let me know through the contact form until I get some comments up and running.

First, I want to start off with my position that "there are no best practices, only better practices" (to quote my professor Venkat). And to paraphrase Peter, a lot of people who know a lot more than I do use XML configurations. On the other hand, a lot of people who know a lot more than me don't use them.

Second, I'd like to point out that I found it mildly funny (not in a bad way) in reading some of the reference documents that some of us consider XML to be human readable. I don't think you'd find that statement anywhere except the Coldfusion community.

Finally, now that I've got the mile-long introduction out of the way, let's continue.

In Hal Helms' article, he is touting the benefits of configuration files in general. He makes a convincing argument, and has motivated me to be a better job of extracting things I never thought would should be configurable, unless I was building something that needed to be internationalized. He compared the INI file to those of the XML variety, and mentions that "some configuration information is more complex and is a poor fit for the simple nature of the INI file. In such cases, XML can be the ideal solution." I think it would be hard to have a problem with that statement, especially given the context of the article.

Joe Rinehart also only touches on the topic - he says that XML "provides the easiest way for us to partition off our configuration into logical 'chunks,'" and doesn't go into detail beyond that (for explaining why XML is a Good Thing.

But, Peter goes into a lot more detail about the benefits of XML. In particular, he mentions (Peter, if you feel I've reproduced too much of your article here, let me know and I'll trim it down)

Validation
As long as you have a DTD/Schema for your XML, you can validate XML against the DTD/Schema to confirm that it meets the specificed requirements. This can be extremely useful in pre-checking your data before getting a ColdFusion error because your configuration data was not well formed. Better yet, you don’t need to write any of the validation code – just the DTD/schema.

Manipulation
With XSLT you can transform XML files from one structure to another and you can even query the data using XPath. There are lots of great tools for working with XML files and again, they’re just there. You don’t need to write them.

Editing
There are plenty of good free plug ins for Eclipse and stand alone editors (XMLBuddy seems to be a popular Eclipse plug in). People are mostly used to editing in XML and there are good tools with highlighting and that will validate against a DTD/Schema in editor.

Consistency
As in Steve Krug's "Don't Make me think", framework design choices should bear in mind that many people spend most of their time using something other than your framework. I may not use MG often, but apart from learning the vocabulary of the DSLs, there is no real learning curve in terms of how to create an XML config file for Reactor, CS or MG. Same for Mach-II and anything else that uses XML configuration files. It is usually better to do things the way people expect so they have less to learn.

Like Peter, I find XML to have a low signal to noise ratio (in "Benefits of XML Config Files...", though he said "high," if he'll allow me to put words in his mouth, I think he meant low).

I should also put a bit of a disclaimer here: I've spent the last three months working on a Java project, so I'm sick of XML configuration. And in general, I've only rarely used XML configuration files with Colfusion, and even then in fairly distant past. In that regard, let me profess some ignorance, especially regarding DTDs. Lately, I've been preferring programmatic configs (written directly in CFML/script). However, I've also not been needing much in the way of configuration, so that may help blind me to the benefits of XML. In any case, let me get into some reasons as to why, comparing that with the perceived benefits of XML.

Validation: As opposed to validating my XML file with a DTD/Schema, I validate my code with unit tests. These can similarly be provided to other programmers to validate their configurations. Now, I don't know how in depth validation can go, but it seems to me I'll be writing some validation in CF even if I use XML, and if it wasn't well formed, or did not meet my expectations, the unit tests would catch that. Another thing frequently mentioned is that it would be better to catch any errors in the spot the error occurred. I agree with this, but if you are validating your programmatic config file (with unit tests or otherwise), it is easy enough to throw an exception in that one spot, rather than letting the error propagate through your code.

Manipulation: I don't suppose I can use a converter to change my programmatic config file, but I'm not sure I see a reason to either. If I chose to, I could query the data with SQL using a query of query, as opposed to XPath.

Editing: I feel it is as simple to edit CFML as it is XML, so I don't see a benefit here, except the highlighting he mentions. But, if we're writing CFML, it is easy enough to check for syntax errors, and if you're using a decent editor, it will also highlight syntax errors for you. I realize it won't highlight "syntactic" errors by missing out on something you need in the config, but I don't value that highly, since I cannot remember a time I wrote a config file from scratch, unless I was creating the application (and in that case, the XML highlighting is unlikely to help you).

Consistency: As he mentions, there is little to learn in how to configure different apps when they use similar XML. But, assuming the person configuring the XML file is also the programmer, the part you have to learn is precisely the part you need to learn in a programmatic configuration file! You don't need to relearn how to do <cfset name = value>, you need to learn what valid names and values are. In the case that the person configuring the application is not a programmer, I'd almost certainly have provided a GUI frontend to the config file, or likely stored it in a database in the first place.

Finally, Peter mentioned that although just a few variables don't need XML, you might need it when configuring 200 beans for ColdSpring or LightWire (both dependecy injection frameworks). I have a hard time seeing that something which is weighty for a small project could be anything less than unwieldy for a larger one. Especially in this example, if I was using something that required an XML configuration file, I would undoubtedly store the information for all those beans in a database, and write a script that would write my XML file for me.

Overall, I like what Martin Fowler said the best: "I often think that people are over-eager to define configuration files. Often a programming language makes a straightforward and powerful configuration mechanism... My advice here is to always provide a way to do all configuration easily with a programmatic interface, and then treat a separate configuration file as an optional feature."

So having said all that, if and when I find a good case for using an XML configuration file, I'll gladly do so, and so should you. Nor should we discount XML - as Peter said and I agree, a lot of really smart people use it. But at the moment, I don't see a need for the excess clutter in my life - CFML provides enough of that for me =).

Update: Sean Corfield wrote about this behavior back in 2004. It's quite a bit annoying, since CF says it is a struct, and treats it like one in just about every way you can imagine, except for this. I'm getting the same behavior using the duplicate() function as well. Guess I'll have to copy it manually.

Update 2: Actually, I said in the last post I was getting the same behavior using duplicate(), but I was wrong. I wasn't doing the exact same thing, but I was doing something I would still expect to work. Here it is:


Still strange.

Update 3: I didn't think about it until this morning (November 1, 2006), but this is a gross violation of LSP (Read Uncle Bob's PDF on LSP). Now, I know the underlying Java classes are different, as Sean Corfield pointed out in the post I linked to above, but to me, if you return a true value from isStruct(form), then that tells me it ought to behave in every way like a structure.

In Java/JSP, no, I would almost never write the HTML in a class. I may have some small bits in a UI class from time to time, but never more than a single tag or perhaps two. But, it is not for any other reason than writing HTML like the following is completely unreadable and impossible to follow, and more importantly easy to screw up:


Can you imagine doing an entire page like that? Or even just a table with more than a couple of columns? It's unwieldy.

With CFC's however, you can still keep the HTML in its readable form. In some cases, I prefer to use CFCs to accomplish the goal of reuse, while in others, I prefer to use includes. I find myself preferring CFCs in particular, in cases where there may be slight customizations to a basic display, based on individual client preferences. I can inherit (or aggregate, if it makes more sense) from the base display (cut up into logical chunks) and reuse those parts which make sense, and only rewrite the custom bits. It lets me keep duplicate code to the absolute minimum - something I'm very fond of these days. The inheritance in particular has the extra benefit of allowing me to keep client code unchanged, as long as I've followed Liskov Substitution Principle.