I think the solution is to start under-promising and over-delivering, as opposed to how most of us do it now: giving lowball estimates because we think that's what they want to hear. But why lie to them?

If you're using iterative and incremental development, then if you've over-promised one iteration, you are supposed to dial down your estimates for what you can accomplish in subsequent iterations, until you finally get good at estimating. And estimates should include what it takes to do it right.

That's the party-line answer to the question. In short: it's never OK to write sloppy code, and you should take precautions against ever putting yourself in a situation where those viscous forces pull you in that direction.

The party-line answer is the best answer, but it doesn't fully address the question, and I'm not always interested in party-line answers anyway. The viscosity (when it's easier to do the wrong thing that the right thing) is the force behind the bodge. I don't like it, but I recognize that there are going to be times you run into it and can't resist.

In those cases where you've already painted yourself into a corner, what then? That's the interesting question here. How do you know the best places to hack crapcode together and ignore those things that may take a little longer in the short run, but whose value shows up in the long run?

The easy answer is the obvious one: cut corners in the code that is least likely to need to change or be touched again. That's because (assuming your hack works) if we don't have to look at the code again, who really cares that it was a nasty hack? The question whose answer is not so easy or obvious is "what does such a place in the code look like?"

By the definition above, it would be the lower levels of your code. But if you do that, and inject a bug, then many other parts of your application would be affected. So maybe that's not the right place to do it. Instead, it would be better to do it in the higher levels, on which very little (if any) other code depends. That way, you limit the effects of it. More importantly, if there are no outgoing dependencies on it, it is easier to change than if other code were highly dependent on it. [1]

Maybe the crapcode can be isolated: if a class is already aweful, can you derive a new class from it and make any new additions with higher quality? If a class is of high quality and you need to hack something together, can you make a child class and put the hack there? [2]

Uncle Bob recently discussed when unit and acceptance testing may safely be done away with. He put those numbers around 10 and a few thousand lines of code, respectively.

In the end, there is no easy answer that I can find where I would definitively say, "that's the place for a bodging." But I suspect there are some patterns we can look for, and I tried to identify a couple of those above.

Do you have any candidates you'd like to share?

Notes:
[1] A passing thought for which I have no answers: The problem with even identifying those places is that by hacking together solutions, you are more likely to inject defects into the code, which makes it more likely you'll need to touch it again.

[2] I use inheritance here because the new classes should be able to be used without violating LSP. However, you may very well be able to make those changes by favoring composition. If you can, I'd advocate doing so.

1. Know the fundamentals
   It was no surprise that Neal led off with the DRY principle. Being one from the school of thought that "code reuse" meant "copy-and-pastable with few changes," I feel this is the most important principle to follow when coding.
   
   Neal uses it in the original sense: Refuse to repeat yourself not just in your code, but all throughout the system. To quote (If I remember correctly, The Pragmatic Programmer): "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system." I can't think of anything that wouldn't be improved by following that advice.
   
   Continuous integration and version control are important fundamentals as well. Continuous integration means you know the code base at least compiles, and you can have it run your unit tests as well. The codebase is verified as always working (at least to the specs as you wrote them - you might have gotten them wrong, of course).
   
   Using version control means you don't have to be afraid of deleting that commented out code that you'll probably never use again but you keep around anyway "just in case." That keeps your code clean.
   
   Finally, Neal suggests we should use static analysis tools such as FindBugs and PMD that inspect our code, look for bad patterns, and suggest fixes and places to trim. Those two are for Java. Do you know of similar applications for other platforms? (Leave a comment please!)
   
   
2. Analyze code odors and remove them if they stink
   Inheritance is a smell. It doesn't always have to smell bad, but at least you should take the time to determine if what you really need is composition.
   
   Likewise, the existence of helper or utility classes often indicates a "poor design." As Neal notes, "if you've gotten the abstraction correct, why do you need 'helpers'?" Try to put them in "intelligent domain object[s]" instead.
   
   Static methods are an indication you are thinking procedurally, and they are "virtually impossible to test." Seeing the procedural mindset is easy, I think, but why are they hard to test?
   
   Finally, avoid singletons. I know Singleton is everyone's favorite design pattern because it's so easy to conceptualize, but as Neal mentions:
   • They're really just a poor excuse for global variables.
   • They violate SRP (link is to a PDF file) by mixing business logic with policing you from using too many
   • They make themselves hard to test
   • It's very hard to build a real singleton, as you have to ensure you are using a unified class loader.
   • More that I won't get into here...
   
   
3. Kill Sacred Cows
   Neal started off saying that "sacred cows make the tastiest hamburger" and told the story of the angry monkeys:
   
   A few monkeys were put in a room with a stepladder and a banana that could only be reached if the monkeys used the stepladder. However, when a monkey would climb upon the stepladder, all the monkeys would be doused with water. Then new monkeys were introduced and when they tried to use the stepladder, the other monkeys would get angry and beat him up. Eventually, the room contained none who had been doused with water, but the monkeys who had been beat up for using the stepladder were still refusing to allow new monkeys to get the banana.
   
   That is the light in which you may often view sacred cows in software development. In particular, Neal points out that
   • StickingToCamelCaseForSentenceLongTestNamesIsRidiculous so_it_is_ok_to_use_underscores in languages_that_standardized_on_camel_case for your test names.
   • Using getters and setters does not equate to encapsulation. One of my favorite sayings, and a reference to why they are evil is appropriate.
   • Avoidance of multiple returns in a method is antiquated because it comes from a time when we weren't using tiny, cohesive methods. I still like to avoid them, but will use them when it makes the code more readable.
   • Polluting interface names with "I", as in ISomeInterface should be avoided. It is contrary to what interfaces are about. Instead, you should decorate the concrete class name.
   
   
4. Your objects should be Good Citizens. Never let them exist in an invalid state.
   
   
5. Use Test Driven Development
   TDD provides "explicit dependency management" because you must think about dependencies as you code. It provides instant feedback and encourages you to implement the simplest thing that works.
   
   You can take it further by analyzing your dependencies with JDepend.
   
   And don't forget about YAGNI. Quoting Neal,

   Speculative development saves time if
   • You have nothing better to work on right now
   • You guarantee that it won't take longer to fix later

   Is that ever going to be true?
   
   
6. Use reflection when you can (and when it makes sense)
   It's not slow, and it can be extremely useful!
   
   
7. Colour Your World
   Nick Drew, a ThoughtWorker in the UK came up with a system of coloring the code you write by classifying it as to who will use the feature: only a specific business customer, only a particular market, many markets, or unchangeable infrastructure. Based on the color and amount of it, you can decide what value or cost your code has.
   
   It's a very interesting system, so I recommend seeing a more detailed overview in Neal's handout (PDF). You can find it on pages 21-22.
   
   
8. Use a DSL style of coding
   It lets you "[Build] better abstraction layers, using language instead of trees" while "utilizing" current building blocks. As he pointed out, "every complicated human endeavor has its own DSL. [If you can] make the code closer to the DSL of the business [then] it is better abstracted."
   
   I'll add that it's easier to spot logical errors as well.
   
   A funny quote from Neal: "It's almost as if in Java we're talking to a severely retarded person." Regarding that, he recommended looking at EasyMock as a good example of fluent interfaces, and that having setters return void in a waste. Instead, we should return this so that calls can be chained together (and if you are using fluent-interface type names, you could construct sentences in your DSL that way).
   
   Neal also noted a distinction between an API and DSL: API has an explicit context that must be repeated with each call. However, a DSL uses an implicit context.
   
   
9. SLAP tells us to keep all lines of code in a method at the same level of abstraction. Steve McConnell's Code Complete 2 tells us about this as well, but I don't recall if it had the clever acronym.
   
   
10. And finally, Think about Antiobjects.
    Quoting Neal, "Antiobjects are the inverse of what we perceive to be the computational objects." So instead of solving the really hard "foreground" problem, have a look at the "background" problem (the inverse) to see if it is easier to solve.
    
    As an example, he used PacMan: Rather than constructing a solution for the "shortest route around the maze," the game has a "notion of a PacMan 'scent'" for each tile." That way, the ghosts follow the strongest scent.

As with my write-ups on Scott Davis's Groovy: Greasing the Wheels of Java, and Stuart Halloway's JavaScript for Ajax Programmers, Neal gets all the credit for the good stuff here. I'm just reporting from my notes, his mindmap, and my memory, so any mistakes you find are probably mine. If they aren't, they probably should have been.

So which strategies have you used? Will you start using any of the above?

Of course not -- some fields should map to a foreign key which should show up as a select form element, which lists the human readable descriptions of those fields (as opposed to their database identifiers). Further, some fields should be hidden, and some fields should be compositions of others. Values for some fields should be in a range of legitimate values, and so the list of possible customizations goes on. In those special cases (which occur frequently enough), you just need to provide a means to override the default behaviors (through extra, user-defined metadata, a DSL, or some other tactic or a combination of them).

Then, in the really rare edge cases, you can always drop out of the framework and code it yourself (or have it generate a template for you to modify). Even for search forms (which I think are more often in need of customization than simple CRUD stuff), you can provide the basics and allow the programmer to customize as needed (and drop out of the framework all together if need be).

But is there a similar strategy for generating reports? My initial thought is that there isn't. Reports just seem to need too much customization to have any useful automagic generation. But, I'm trying to look past that initial impression, because I don't have as many years of experience in hand-writing the same crap code over and over again in reporting as I do in writing CRUD operations (now, I have the same crap code for CRUD in one place and it just figures out what to do, but at least I don't ever need to write it again =)).

In any case, I just don't see it. But, I can see using metadata/DSL, where you can define the tables and columns you want to report on, and generally it will still be much faster for developing reports (can you tell I'm trying to work on this at the moment?). Taking the idea further, what about bringing convention over configuration into this realm? Once you get out of the world of 8.3 filenames and realize you can name variables (or columns) pretty much anything you want, it is easy to see the benefit of having names that describe what you want the user to see. For instance, instead of orderGTot, just go ahead and name the column order_grand_total or orderGrandTotal. Who cares about the extra characters? It's not like you have to type them that many times, since the framework (presumably) takes care of most of it for you, and you don't have to define what the user should see in more than one place.

Of course, this is also just metadata. But, instead of having what amounts to the same metadata in different places, you've just got it in one - your column name. So if we take this idea to reporting, what if we just tag certain columns with something like _reportable? I haven't figured out how we might generate reports on that amount of information alone, but I see it as a useful start.

When your classes average between zero and twenty lines of code (which amounts to just customizing interfaces and performing validations, along with perhaps a couple of calculations), it's time to stop worrying about DAOs, DTOs, PPOs, and Cheerios. Skip to the IPO and let the frameworks worry about the rest. I want one to do my reporting for me, because that seems to be one of the last major hurdles that I see (at least, as far as getting rid of repetition goes).

What do you think?

My interest lies in the second complaint though - that writing it to a live file (rather than dynamically figuring it out - which I've been calling "synthesis," as opposed to generation) means you can't get the benefits of upgrades to the scaffolding engine (and, it is not following DRY!). But, if you just use the default scaffolding, what happens if it doesn't come out just right (assuming you've even passed the notUgly test in the first place)? Well, thats a big part of what I'm trying to solve with cfrails, by using a DSL that provides tweakability to the scaffolding, without requiring you to write to a file (though, if the tweaks won't work, you are always welcome to go to a file with HTML and the like). The interesting part for me about the RoR plugins above, therefore, is that it appears (I haven't checked them out yet) that at least Hobo contains a DSL, called DRYML, to help along those lines.

I'll be having a closer look at those when free time becomes a bit less scarce. What do you think? Is it the holy grail, or can there be very useful "scaffolding"?

Being that I've been burned a lot in the past by not following the DRY and OAOO principles, I'm quite fanatacal about following them nowadays. So when I needed the list() function to behave a bit differently for this one type of object, I didn't want to replicate the entire list function -- I just wanted to add a bit of logic in the middle. That's where closures enter the picture.

I decided I'd create a function, onOutputOf(column="column that special stuff needs to be done to", run="function, or code"). So, instead of outputting the value of the cell, the list now knows to run the code provided in the run parameter.

And now, some really complex code doesn't need to be replicated elsewhere. Isn't that sweet?

Initially, that struck me as an idea worth exploring, since those are two of my most-used, favorite, and life-saving principles (I mean, try violating them over and over for years and see if you don't feel like dying!). In fact, I use them a lot in determining when to create a new method (I use other heuristics as well, such as readability, which is often the most important reason I create a function ... more here about some reasons for creating methods). But, I thought about it some and came to this conclusion I posted to the pragprog group (which I changed a bit for here to make more sense):

In essence, I'm not comfortable with either of those definitions (DRY or YAGNI) as lines for determining when you are soft or hard coding. Suppose the following (generalizing to your pleasure -- since I'm having trouble writing it, I'm sure others will have trouble understanding it, as no one would ever (I think) write an application this way)

1. it always costs the same to ship each item: 5.00 (units of currency)
2. the company only needs to worry about one tax rate: 5%
3. the product costs 100 dollars each, and there is only one product.
4. customers can buy a variable quantity of this item (called "quantity" below)
5. there is a handling charge of 3.14 (units of currency) for each item.

Then your total_cost = (100+5+3.14)*(1.0+5/100)*quantity.

We see 100 twice here. Do we pull it out? What do we name it? oneHundred? That is the only one that makes sense for both cases. Or we could change it to to get rid of calculating the percentage: total_cost = (100+5+3.14)*(1.05)*quantity.

I'm not even confident I've got the right formula - and I'm the one making this up. What happens down the line if the state changes the sales tax? Am I sure I know which number to manipulate? Here, it may look clear, but it may not be the case in other situations. And then when I need to do a circleOperation with crappyApproximationOfPi = 3.14 (the product is cylindrical, and we need to know about how much ribbon we need to wrap the package when someone selects the gift wrapping option), who gets the naming rights?

If there is a line that should be drawn (and I'm not convinced there is), and there exists a place to put it, I don't think the DRY or YAGNI principles have anything to do with its placement. Readability is the key for me, and I don't know to what degree that can be quantified.

One in particular I dislike is when people have uncommented literals like 0x0032aBlahBlahBlah in the code. How (and why) on Earth would you expect anyone to know what that referred to? Will you remember the next time you look at it? I wouldn't, but my brain may just be a sieve.

Your thoughts on where to draw the line? (Or not draw it?... Or tell me I'm wrong?)

But sometimes, I think I can go further. And sometimes I do, while others I don't. There is a common thread between most of these times I feel I could go further: if I could name the block of code. And the commonality between most of the times I don't extract that block into a method: the code is not especially complex, and I am too lazy to do it "right now."

So, I wonder, would it be a fairly valid rule to say "if you can name a block of code, put it in a method"? Of course, you can't simply say "if you can't name it, don't put it in a method," since it might be hard to think of a good name (or, it may just be that the code is not cohesive). But I think the other way might go a long way to reducing complexity and making the code more easily understood.

What do you think? Do you follow any guidelines that you've noticed (either intentionally or not)?

I think part of it may be due to bad habits and not having the ability to notice a spot where I need another class. But, I have no evidence of this, since I haven't been able to notice it =). I do have evidence of the other two options however: CF being so much more high-level than Java is fairly self-evident: Where I'll often use a façade to interact with a database in Java, in Coldfusion it would be pretty useless - since there is no need to instantiate a connection, prepare a statement, prepare a record set, and then close/destroy them all when you don't need them any more. The other, about the performance hits, I know sometimes stops me from creating more classes. For instance, I recently used a mixin to achieve some code reuse with a utility method that didn't belong to any of the classes that used it. Clearly it broke the "is-a" relationship rule, but I didn't want to incur the cost of instantiating an object just to use a single, simple method (the application was already seeming to perform slow).

Of course, this is not a zero-sum game. It is certainly due in part to all three. I'd like to think it is mostly due to the expressiveness of Coldfusion, but I'll always have to wonder about the bad habits.

In Ruby, I've developed a couple of "significant" applications, but those didn't approach the size of the one's I've done in Coldfusion or Java. It's hard to say for sure, but I'd guess that I'm writing much fewer classes in Ruby than I would in Java. Again, this is mostly due to Ruby's expressiveness.

In any case, have you noticed yourself writing fewer classes in CF than in other languages? Have you thought about it? Come to any conclusions?

But surely you can't provide every combination! That's where you'd want to provide a filter argument. Here it comes again: another but! You don't want to simply provide filter="where column1=#form.column1#" - you want to parameterize the query for the developer. Now, one way to do that would be to parse the filter argument and reconstruct it using your cfqueryparams in all the right places. But, an easier way would be to something like this:



Note: I would probably move the "where" into the abstraction, and just let the programmer provide the clause, but for illustration purposes, I left it in there.

For illustration purposes, my function only does this:


But it would be easy enough to modify it to do something useful. Right now, if you run it, you see all the arguments are available in the arguments structure. The first one comes under the key "filter", and the rest come under numeric keys, in the order they were passed (of course, looping over it won't show you that order, but the number of the key represents the position in which it was passed. Now, you can parameterize the arguments, first checking more restrictive and moving to less restrictive parameters (for instance, you'd want to check isDate(), isNumeric(), and fall back on string).

You could name the arguments too, but I haven't found that useful for this example, since they will have their names as keys in the arguments structure, and I know of no good way to figure out the order in which they were passed.

Finally, I haven't yet implemented this in cfrails, so I don't know how feasible basing parameterization on the CF type is, but I'll try to remember to post it when I do implement it. In any case, I'd be willing to bet there are other uses for this - it's just that this is the most important one on my mind recently.

Update: It did occur to me as I was writing this post that you could accomplish the same thing by using an array as a defined argument, with the added benefit of keeping your interface well defined. In all honesty, I would prefer to do it that way if you could define an array inline like: [9, "blockhead", "charlie brown"]. But the way you define arrays in Coldfusion really makes for too much code in what I'd like to accomplish with this, so in this case I'd provide a hint in the function to let a developer know it expects you to pass those arguments. This way, I get cleaner syntax and a decent solution to the interface problem.

In a nutshell, the goal is to have such highly customizable "scaffolding" that there really is no scaffolding - all the code is synthesized - or generated "on the fly." Of course, this only gets you so far. For the stuff that really makes your application unique, you'll still have to code that. But you can compose views from others and such, so it's not like related tables have to stay unrelated, but I do want to stress that right now there is no relationship stuff implemented in the ORM.

I've skipped a few "mini-versions" from 0.1.3 to 0.2.0 because there were so many changes that I haven't documented one-by-one. That's just sloppiness on my part. Basically, I started by following Ruby on Rails' example, and taking my own experience about what I find myself doing over and over again. That part is done, except that the ORM needs to be able to auto-load and lazy-load relationships at the programmer's whim. In any case, once I got enough functionality to start using it on my project, I've been developing them in parallel. The problem is, I've fallen back on poor practices, so the code isn't as nice as it could be.

In particular, there aren't any new automated tests after the first couple of releases, which isn't as bad as it might otherwise be, since a lot of the core code was tested in them. But on that note, I haven't run the existing tests in a while, so they may be broken. Further, since I've been thinking in Ruby and coding in Coldfusion, you'll see a mix of camelCase and under_score notations. My original goal was to provide both for all the public methods, and I still plan to do that (because, since I can't rely on case from all databases for the column names -- or so I think -- I use the under_score notation to tell where to put spaces when displaying the column names). But right now, there is a mix. Finally, the DSL needs a lot more thought put behind it - Right now it is a mix-and-match of variables.setting variables and set_something() methods. Right now it is really ugly, but when I take the time to get some updated documentation up and actually package it as a zip, I should have it cleaned up. In fact, I shouldn't have released this yet, but I was just starting to feel I needed to do something, since so much had be done on it and I hadn't put anything out in quite some time. Besides that, I'm quite excited to be using it - it's been a pain to build, but it's already saved me more time than had I not done anything like it.

In the end, I guess what I'm trying to say is: 1) Don't look at it to learn from. There may be some good points, but there are bad points too, and 2) Don't rely too heavily on the interfaces. While I don't anticipate changing them (only adding to them, and not forcing you to set variables.property), this is still less than version 1, so I reserve the right to change the interfaces until then. =)

Other than that, I would love to hear any feedback if you happen to be using it, or need help because the documentation is out of date, or if you tried to use it but couldn't get it to work. You can contact me here. You can find cfrails at http://cfrails.riaforge.org/ .

In Java/JSP, no, I would almost never write the HTML in a class. I may have some small bits in a UI class from time to time, but never more than a single tag or perhaps two. But, it is not for any other reason than writing HTML like the following is completely unreadable and impossible to follow, and more importantly easy to screw up:


Can you imagine doing an entire page like that? Or even just a table with more than a couple of columns? It's unwieldy.

With CFC's however, you can still keep the HTML in its readable form. In some cases, I prefer to use CFCs to accomplish the goal of reuse, while in others, I prefer to use includes. I find myself preferring CFCs in particular, in cases where there may be slight customizations to a basic display, based on individual client preferences. I can inherit (or aggregate, if it makes more sense) from the base display (cut up into logical chunks) and reuse those parts which make sense, and only rewrite the custom bits. It lets me keep duplicate code to the absolute minimum - something I'm very fond of these days. The inheritance in particular has the extra benefit of allowing me to keep client code unchanged, as long as I've followed Liskov Substitution Principle.