My Secret Life as a Spaghetti Coder
home | about | contact | privacy statement
This post might be better titled, "How (and how not) to help yourself when Google doesn't have the answer: A whirlwind tour through Rails' source" if only I wasn't too lazy to change the max length of the database field for titles to my blog entries.

Google sometimes seems as if it has the sum of all human knowledge within the confines of its search index. It might even be the case that it does. Even if you prefer to think that's true, there may come a time when humanity does not yet have the knowledge you are seeking.

How often is that going to happen? Surely someone has run up against the problems I'm going to have, right? That hasn't been the case for me the last couple of months.

I may be the only developer writing Rails apps on MacOSX to be deployed to the world on Windows where SQL Server 2008 is the backend to a Sharepoint install used by internal staff to drive the data. I'm not so presumptious to think I'm a beautiful and unique snowflake, but I wasn't finding any answers.

Before I started this trek, I made a commitment to leave after an hour if I found my attention drifting toward something else. I never started checking email, reading blogs, or obsessively reloading twitter to see my tweeps latest tweets, so I thought I was in the clear.

However, even though I felt like I was focused, the fact that I had been sitting at the computer for so long contributed to poor decision making. The first of these was to keep searching Google even though every search was coming up useless. I always followed the path of least resistance - even if it wasn't going to get me to the goal quicker than an alternative path. If it was less challenging, it was for me.

What is a binary column's data type in Ruby?

After a while, I ran out of mentally casual paths and resigned myself to tracing through the source code (it is open source, after all, and this is one of the benefits everyone claims but so few practice). It was what I knew I should have been doing as I started out, and I had wasted several hours trying to tip-toe around it for the sake of my poor, tired brain.

Indeed, the binary column from Active Record does resolve to a Ruby String.

Now that I was sure I had the right data type being returned, I needed to narrow down where the problem was occuring. I knew SQLServerAdapter was using DBI to connect to the database, so I figured I'd use a quick irb session to test DBI itself. The test came back negative - DBI was getting the correct data. I also ran a quick test in the Rails app, reaching through ActiveRecord::Base to use the connection manually. That worked, as expected.

Using DBI I got the correct data, checking SQLServerAdapter because Rails sucks on Windows

I had thought, and now confirmed, that the best place to look would be SQLServerAdapter. If it were a Rails problem, certainly someone would have run into it by now. So it made sense the problem would be in the interface between Rails and Microsoft. Why? Because if Rails is a Ghetto, Rails with Microsoft products is a fucking concentration camp.

Excuse the profanity. I don't often use it here, so you know I must mean it when I do.

As I began to browse the source, I was first drawn to this innocent looking code from rails-sqlserver on github:

class << self def string_to_binary(value) "0x#{value.unpack("H*")[0]}" end def binary_to_string(value) value =~ /[^[:xdigit:]]/ ? value : [value].pack('H*') end end

But it wasn't obvious how it was being used elsewhere. I even tried using the reverse operations in my objects - to no avail. And after searching in the source file, it certainly wasn't being called inside of SQLServerAdapter. So I went on a quest for the answer inside /opt/local/lib/ruby/gems/1.8/gems/activerecord-2.1.1/ .

For quite some time I went back and forth inserting and removing debugging code between Active Record and SQLServerAdapter. select(sql, name=nil) is a protected method defined in the abstract connection adapter in Active Record. SQLServerAdapter implements it privately, and it was both getting and returning the correct data.

After ActiveRecord calls object.instance_variable_set("@attributes", record) when instantiating our object, object.attributes[binary].size becomes less than record["binary"].size. That was the problem. I thought for sure instance_variable_set was a monkeypatched method on Object, and that all I needed to do was issue a monkeyfix and all would be well.

Only I was wrong. It's there by default in Ruby, and Rails wasn't monkeypatching it (that I can tell).

All the sudden things started looking bleak. By this time I knew how I could fix it as a hack. I even had a nice little monkeypatch for my objects that I could issue and have it feel less hacky to be used. I had given up.

But for some reason I picked it back up after an hour and found that ActiveRecord was actually calling that string to binary method in SQL Server Adapter. It allows them to register calls that should happen before defining the read and write methods on the object. Excellent!

I opened up SQLServerAdapater, there it was: a different binary_to_string method that totally explained it. The pair in this version were encoding and decoding the data to/from base 64. That would work fine, if my data was going through the encoding part. But it wasn't - it was coming straight from Sharepoint.

And so the tour ends.

There's a comment in the code about some arbitrary 7k limit on data size from SQL Server being the reason for encoding as base 64 before inserting the data. I don't know about inserting, but retrieving works fine without it. If I could think of a way to distinguish, I'd submit a patch for the patchwork. Ideally, I'd rather find a way around the restriction, if it actually exists.

The original code I was looking at was on github. It (not surprisingly) differed from the code in use on my machine. Another side effect of the 16 hour monitor stare.

It's called the 8 hour burn for a reason. The only things burning after 16 hours are your brain from all the stimulants and your wife, wondering WTF you're really doing because there's no way you're working from before she wakes up until after she goes to bed.

What's the point?

There's two, if you ask me:
  1. You have the source code. Look through it. You have no other choice when no one has had your problem, and you might benefit by doing so even if someone already has.

  2. Even when you think you're focused working late, and resolve to leave when you lose the focus, you're still going to make stupid decisions that you won't notice until the morning. I turned a 5 hour journey into a 12 hour marathon. Sleep, FTW.
Oh, and Rails on Windows is a concentration camp.

Hey! Why don't you make your life easier and subscribe to the full post or short blurb RSS feed? I'm so confident you'll love my smelly pasta plate wisdom that I'm offering a no-strings-attached, lifetime money back guarantee!

Leave a comment

There are no comments for this entry yet.

Leave a comment

Leave this field empty
Your Name
Email (not displayed, more info?)


Subcribe to this comment thread
Remember my details

Picture of me

.NET (19)
AI/Machine Learning (14)
Answers To 100 Interview Questions (10)
Bioinformatics (2)
Business (1)
C and Cplusplus (6)
cfrails (22)
ColdFusion (78)
Customer Relations (15)
Databases (3)
DRY (18)
DSLs (11)
Future Tech (5)
Games (5)
Groovy/Grails (8)
Hardware (1)
IDEs (9)
Java (38)
JavaScript (4)
Linux (2)
Lisp (1)
Mac OS (4)
Management (15)
MediaServerX (1)
Miscellany (76)
OOAD (37)
Productivity (11)
Programming (168)
Programming Quotables (9)
Rails (31)
Ruby (67)
Save Your Job (58)
scriptaGulous (4)
Software Development Process (23)
TDD (41)
TDDing xorblog (6)
Tools (5)
Web Development (8)
Windows (1)
With (1)
YAGNI (10)

Agile Manifesto & Principles
Principles Of OOD
Ruby on Rails

RSS 2.0: Full Post | Short Blurb
Subscribe by email:

Delivered by FeedBurner