Before I started this trek, I made a commitment to leave after an hour if I found my attention drifting toward something else. I never started checking email, reading blogs, or obsessively reloading twitter to see my tweeps latest tweets, so I thought I was in the clear.

However, even though I felt like I was focused, the fact that I had been sitting at the computer for so long contributed to poor decision making. The first of these was to keep searching Google even though every search was coming up useless. I always followed the path of least resistance - even if it wasn't going to get me to the goal quicker than an alternative path. If it was less challenging, it was for me.



After a while, I ran out of mentally casual paths and resigned myself to tracing through the source code (it is open source, after all, and this is one of the benefits everyone claims but so few practice). It was what I knew I should have been doing as I started out, and I had wasted several hours trying to tip-toe around it for the sake of my poor, tired brain.



Now that I was sure I had the right data type being returned, I needed to narrow down where the problem was occuring. I knew SQLServerAdapter was using DBI to connect to the database, so I figured I'd use a quick irb session to test DBI itself. The test came back negative - DBI was getting the correct data. I also ran a quick test in the Rails app, reaching through ActiveRecord::Base to use the connection manually. That worked, as expected.



I had thought, and now confirmed, that the best place to look would be SQLServerAdapter. If it were a Rails problem, certainly someone would have run into it by now. So it made sense the problem would be in the interface between Rails and Microsoft. Why? Because if Rails is a Ghetto, Rails with Microsoft products is a fucking concentration camp.

Excuse the profanity. I don't often use it here, so you know I must mean it when I do.

As I began to browse the source, I was first drawn to this innocent looking code from rails-sqlserver on github:


But it wasn't obvious how it was being used elsewhere. I even tried using the reverse operations in my objects - to no avail. And after searching in the source file, it certainly wasn't being called inside of SQLServerAdapter. So I went on a quest for the answer inside /opt/local/lib/ruby/gems/1.8/gems/activerecord-2.1.1/ .

For quite some time I went back and forth inserting and removing debugging code between Active Record and SQLServerAdapter. select(sql, name=nil) is a protected method defined in the abstract connection adapter in Active Record. SQLServerAdapter implements it privately, and it was both getting and returning the correct data.

After ActiveRecord calls object.instance_variable_set("@attributes", record) when instantiating our object, object.attributes[binary].size becomes less than record["binary"].size. That was the problem. I thought for sure instance_variable_set was a monkeypatched method on Object, and that all I needed to do was issue a monkeyfix and all would be well.

Only I was wrong. It's there by default in Ruby, and Rails wasn't monkeypatching it (that I can tell).

All the sudden things started looking bleak. By this time I knew how I could fix it as a hack. I even had a nice little monkeypatch for my objects that I could issue and have it feel less hacky to be used. I had given up.

But for some reason I picked it back up after an hour and found that ActiveRecord was actually calling that string to binary method in SQL Server Adapter. It allows them to register calls that should happen before defining the read and write methods on the object. Excellent!

I opened up SQLServerAdapater, there it was: a different binary_to_string method that totally explained it. The pair in this version were encoding and decoding the data to/from base 64. That would work fine, if my data was going through the encoding part. But it wasn't - it was coming straight from Sharepoint.



There's a comment in the code about some arbitrary 7k limit on data size from SQL Server being the reason for encoding as base 64 before inserting the data. I don't know about inserting, but retrieving works fine without it. If I could think of a way to distinguish, I'd submit a patch for the patchwork. Ideally, I'd rather find a way around the restriction, if it actually exists.

The original code I was looking at was on github. It (not surprisingly) differed from the code in use on my machine. Another side effect of the 16 hour monitor stare.

It's called the 8 hour burn for a reason. The only things burning after 16 hours are your brain from all the stimulants and your wife, wondering WTF you're really doing because there's no way you're working from before she wakes up until after she goes to bed.

What's the point?

There's two, if you ask me:

1. You have the source code. Look through it. You have no other choice when no one has had your problem, and you might benefit by doing so even if someone already has.
   
   
2. Even when you think you're focused working late, and resolve to leave when you lose the focus, you're still going to make stupid decisions that you won't notice until the morning. I turned a 5 hour journey into a 12 hour marathon. Sleep, FTW.

Oh, and Rails on Windows is a concentration camp.

My first job primarily used Cold Fusion. When I joined AOL Time Warner I gave up Cold Fusion for PHP. When I joined IAG I gave up PHP for C#. And so on. As you can see, I've never been too tied to a language. I've always been most interested in learning and growing. I love jobs that help me improve my skills.
...
I prefer jobs that allow me to learn new things. Think of it as job security -- I shouldn't ever be out-of-date when it comes to technology experience. Think of it as an investment -- everything I learn creates a broader range of experience that I can leverage for future projects or jobs. Think of it as experimenting -- by trying many different solutions I may find ways to combine them and innovate.

Programming in C for the first time seems to be a chore. There are many details about your Code you need to keep track of. And C doesn't even lend you much of a hand when programming. There are no iterators, memory management is done manually, and handling strings correctly is akin to walking through a minefield with a hood over your head. While on fire. And being chased by a pack of wild dogs.

But then the question still stays: "what should one learn next?". Algorithms, data structures, complexity, math... Learn the core abstracts, ideas, and skills that are language independent, and that transfer from one syntax to another. Learn the ability to learn. When a new opportunity with new technology comes along, you should be able to get over the learning curve fairly quickly.
...
So lets drop this obsession with learning to say "hello world" (or some more complicated version of essentially the same) in every programming language one can name. Lets also drop the idea of finding "one language to rule them all; and retire". Once we put the Science back in Computer Science, it wouldn't matter which language you'll end up using.

Ninja credit

Outlook: not so good.

Goal: Start regular practice sessions. The year started off on the right foot, and ended up better than it was in the middle. In the middle was a disaster. Practice was not the operative word here - I got plenty of "on the job" practice. I wanted to focus on the regular part, outside of work.

Goal: Devote more time to open source software. Here again, the year started off in the right direction. I submitted a couple of patches to JRuby (I don't know if they were used or not), but quickly found too much work to continue devoting entire days to established open source projects.

Goal: Do something not useful (and something useful) in Erlang. I installed Erlang and got a simple program working. Nothing useful, nothing I could consider a success by the standards I hold.

Goal: (Practice in order to) speak at a conference (through user groups). Complete and utter FAIL. I kept putting "write a presentation to give at XUG" off until "next month when I have more time." Instead of giving a presentation at a user group, I managed to miss every meeting of every group I'm interested in. How does that happen?



Goal: Attend a conference (or multiple): Check. I was able to make it to Lone Star Ruby Conference this year, and it was well worth it.

Goal: Increase earnings. This one had a few subgoals. Releasing a game with my game dev group did not happen, though we did place in the top 20 of the Imagine Cup Game Development Competition. We also had a couple of meetings towards developing another game, but we let it fizzle out (or else they just didn't invite me back).




The other subgoals wouldn't be classified as successes either, except for getting another regular source of income. That was a major win, which made the entire goal a win. In fact, I might go so far as to say I pwned that goal.



Overall, the year looks like a failure if I examine it by the professional goals I set here. It doesn't feel that way though. Still, I think when I sit down to make my goals for next year, I'll need to set review dates and time lines for them to keep them closer to achievable.

How did you do in achieving your goals for 2008?

Anselm Bradford took the time to look at PureMVC and learn its underlying architecture.

Marcos Placoná figured out how to program his Sony PSP, and submitted one project for checking time zones, and another for translating common French/English phrases.





Increasing Participation
I got more submissions this year than last year, but it was still small compared to the number of readers who saw the post. There were around 20% fewer people who saw this post as opposed to last year's, but RSS subscribership has gone up around 500%. (Chad Fowler linked to last year's post, which accounts for the difference.)

Last year the ratio of submissions to people who saw the post was about 1/1000, while this year it was 5.7/1000.

What can be done to improve that number? It'd be nice to see 50 in 1000, or perhaps even 100. This year I gave a few suggestions, and judging by the entries, it looks like they were taken to heart.

But I'm not sure if we're at peak interest, or if it can be improved again next year. So here's the question for you: if you saw the post and didn't submit an entry, why not? No time? Prize too weak? Just not interested in the first place? Something else?

Feel free to use the contact form to email me privately if you'd rather not give your reasons in public.

Thanks for any feedback, and major thanks and congratulations to all the contestants! I look forward to seeing your projects again next year!

nice_code, stupid_submitter - in which TheDailyWTF jumps the shark by ridiculing perfectly good code.

Let's forgive the misuse of the wornout phrase and get to whether or not looking at the code should result in utterance of WTFs.

It goes something like this:

1. Setup, in which we use globally shared memory
2. Negate the number if it is negative, for no reason I can think of, and set a reminder flag.
3. Set buffer to point to the final spot. Move it back one, dereference it, and insert the null string terminator there.
4. Move the buffer backwards. Dereference the pointer and set the value there to the character '0'. Add the remainder of the number divided by ten to that, since '1' is 1 away from '0', and so forth.
5. Divide the number by 10 so we can get to the next digit.
6. If we've done the loop 3 times, move the buffer pointer back one location and insert a comma.
7. Repeat starting at step 4 until the number is 0.
8. Cleanup - if the front character is a comma, remove it by moving the buffer pointer forward.
9. Cleanup - Move the buffer pointer backwards and insert our '-' if the negative flag has been set.

I felt like that required too much brain power to follow for doing something so simple. So I decided to make my own version. I thought I'd try the following:

1. Copy the number to a string.
2. Starting at the end, copy each character to another string, inserting a comma every 3rd time.

A thousand times simpler than the convoluted mess of nice_num. Here's that attempt, in C:


I think it's clearer - but not by much. Certainly the variable names are better because you don't first have to first understand what's going on to know what they are for. I think moving the pointer arithmetic back into the more-familiar array notation helps understandability. And removing the trick of knowing that the ASCII codes for '1' is 1 more than for '0' ... for '9' is 9 more than 0 means less thinking.



On the negative side, the commadify code is slower than nice_num, most of which is caused by using malloc instead of having preallocated memory. Removing those two instances and using preallocated memory shaves a couple of tenths of a second off of the one-million-runs loop. But you have to compensate with more code that keeps track of the start position.

So what's the verdict? I don't think we're reacting to the WTFs I mentioned above when we see the nice_num code. I think we're reacting to C itself. We're so used to very high level languages that common tricks and things you'd know as a C programmer are the WTFs to us.

This kind of stuff isn't outside the realm of what a strong programmer should know. It isn't even close to the border. The truth is our low-level skills are out of practice and we should probably get some.

What do you think?

Code with main program and comparison with nice_num is available at my github repository, miscellany/commadify.

Update: Thanks to Dave Kirby's comment, I've fixed memory leaks and updated the code in this post and at the repository. Link to repo originally was to the specific commit - so I've changed that to link to master instead.