Category Archives: code

Be the Google

Would you like 500mb of web hosting, plus Python, plus Django, plus a lot of Google database and application goodies? Would you like it for free? Then, my friend, what you need is Google App Engine. No mention yet of what the pricing is like after you hit your 5 million hits per month. But trust me, if you’re at 5 million hits per month, you don’t care.

Sorry Amazon S3, but you can basically suck it.

Update: um, yeah, it’s wait-listed. Though if you have Google Apps, you’re probably in.

Return a Record for Each Date Between Two Dates in SQL Server >= 2005

Blogging this so I don’t forget it…

It used to require some fairly ugly, resource intensive hacks (cursors, temp tables, etc.) to emit an inclusive list between two data points when the source data might not include an entry for every point (for example, a calendar, where not every day contains an event). In SQL Server 2005 and above, this is trivially easy, with a Common Table Expression (CTE) and a Recursive Query. To emit one record for every date between 1/1/2008 and 1/31/2008, you do this:


WITH datecte(anydate) AS (SELECT CAST('1/1/2008' AS datetime) AS anydate
UNION ALL
SELECT anydate + 1 AS anydate
FROM datecte AS datecte_1
WHERE (anydate < CAST('2/1/2008' AS datetime) - 1))
SELECT anydate
FROM datecte AS datecte_2

If you need more than 100 days (the recursion limit is 100), add this to the end:

OPTION (MAXRECURSION 1000)

The fact that they stop recursion short at 100 by default would seem to indicate that this is an expensive procedure, but even if you’re just using this to produce a dummy table with all the dates for several years, it’s a nice shortcut.

I just tried the following query, which emits a record for every day between 1/1/2000 and 12/31/2020:


WITH datecte(anydate) AS (SELECT CAST('1/1/2000' AS datetime) AS anydate
UNION ALL
SELECT anydate + 1 AS anydate
FROM datecte AS datecte_1
WHERE (anydate < CAST('1/1/2021' AS datetime) - 1))
SELECT anydate
FROM datecte AS datecte_2
OPTION (MAXRECURSION 10000)

On my P4-641+ the script emits 7671 records in 0 (that’s zero) seconds and “spikes” the processor to all of 3%. Granted this is not a complex query, but at least we know the recursion (if it really is recursion internally, which I doubt) isn’t expensive by itself.

G-Archiver and the Risks of Random Downloading

This is a pretty amazing story about a free utility with a malicious back-end twist.

This is so bad that I assumed it was a hoax. However, I downloaded the program, installed it (on a virtual machine), decompiled it, and verified that it is, in fact, “phoning home” with your gmail user name and password. Yikes.

The manufacturer’s page has been updated to indicate that this “was in no way intentional,” but does it really matter?

Dynamically Adding Option Elements to Select Objects… The Real Story

I don’t normally blog about pure code subjects, so this is going to be way too technical/boring for the general readership of this blog. However, I wasn’t able to find the definitive answer to this question after searching around with Google, so I figured I’d contribute a little. This is also a wiki-style reminder to myself the next time I need to do this.

Mostly you’ll be interested in this post if you’re trying to programmatically add options to a select list using JavaScript. Of all the ways to do this, there are a few that work and many that don’t. The catch seems to be that IE is picky about how and when the data “inside” the option tag (the visible text in the option list) is set. According to the documentation there should be any number of ways to set this–.text, .innerHTML, .innerText. Of those, the only one that seems to be broadly compatible is .text. .innerText is IE-only, so it’s right out. .innerHTML is valid across platforms, but it messes with the object model in IE6 and IE7 (and IE8 beta 1, now that I look) in such a way that if you use it, you have to add the option element to the select element before setting the .innerHTML property. See below for a breakdown of the methods, plus test code. Blogger might break this, so be sure to look at the plain test page (“view source” is going to be a lot friendlier on that page as well).

In any case, here’s the take-away when adding options to select elements:

  • Always use the .text property of the newly-created option object to set the visible text for the option.
  • As a precaution, add the new option object to the select object’s options collection before setting other properties of the option.

create option, set .value, set .text, add to options list (generally compatible)


function createSetTextAddOption(selectEl,val,displayText) {
var o = document.createElement("OPTION");
o.value = val;
o.text = displayText;
selectEl.options.add(o);
return false;
}


create option, set .value, set .innerHTML, add to options list (fails IE6, IE7, IE8-‘invalid argument’)


function createSetInnerHTMLAddOption(selectEl,val,displayText) {
var o = document.createElement("OPTION");
o.value = val;
o.innerHTML = displayText;
selectEl.options.add(o);
return false;
}


create option, add option to list, set .value, set .innerHTML (generally compatible)


function createAddOptionSetInnerHTML(selectEl,val,displayText) {
var o = document.createElement("OPTION");
o.value = val;
selectEl.options.add(o);
o.innerHTML = displayText;
return false;
}


create option, add option to list, set .value, set .text (generally compatible)


function createAddOptionSetText(selectEl,val,displayText) {
var o = document.createElement("OPTION");
o.value = val;
selectEl.options.add(o);
o.text = displayText;
return false;
}


Attempt to add new options/selects inline (option 2 and select 2 fail in IE, either silently or with the “invalid argument” error)

According to the Microsoft documentation, this shouldn’t work at all. Here’s what they say about the option.add() method:

This method can be used to add elements only after the page loads.

If the method is applied inline, a run-time error occurs.

And yet, it mostly does work, aside from the innerHTML limitation…


// generally compatible inline adds
createSetTextAddOption(document.getElementById('sel5'),'1','added value 1');
createAddOptionSetInnerHTML(document.getElementById('sel5'),'3','added value 3');
createAddOptionSetText(document.getElementById('sel5'),'4','added value 4');

addNewSelect(document.getElementById('sel5'),'1','added value 1',createSetTextAddOption);
addNewSelect(document.getElementById('sel5'),'1','added value 3',createAddOptionSetInnerHTML);
addNewSelect(document.getElementById('sel5'),'1','added value 4',createAddOptionSetText);

// these two fail in IE6, IE7, IE8
createSetInnerHTMLAddOption(document.getElementById('sel5'),'2','added value 2');
addNewSelect(document.getElementById('sel5'),'1','added value 2',createSetInnerHTMLAddOption);

Again, I encourage you to view the plain test page to avoid any Blogger-induced weirdness.

When You’re an End Node, It Doesn’t Pay to Ask Why

I think what he’s saying here is that where you look for answers as a developer is heavily influenced by the domain in which you’re operating. Yes, you need to consider “best practices” (groan), and sometimes it’s a good idea to “think outside the box” (wretch), but most of the time you really need to concentrate on what is possible and efficient and makes sense in the current context. This is why when you want to learn about a technology you can read a book, but when you actually have to implement it you end up sorting through a lot of discussion groups and blog posts, and especially blog comments–the ultimate end nodes of the infocloud.

Just a neat blog I stumbled on in an otherwise anxious, code-heavy week of integrating things that were never meant to work together.

Cheap Router as Wireless Bridge

I recently had a desperate need for a wireless bridge device. The need has passed, but I finally figured out a way to do it without spending $60+ for a dedicated (one port) bridge. The main goal here is to provide a physical Ethernet jack somewhere out on your wireless network for a device or devices that can’t connect to wireless directly. I was able to get this working just now using a $25 refurbished Netgear WGT624v3 from Fry’s. I followed (and interpreted, because it’s not as step-by-step as it could be) these instructions. Supplemented with information from this thread.

What’s really amazing about this is that you end up using a shell session on the router, without having to hack the firmware (though you are exploiting a disabled interface and a NetGear diagnostic tool that turns it back on). It’s pretty strange.

In any case, I can now put four physical Ethernet ports anywhere within range of my wireless network. The bridge is effectively dumb and invisible–DHCP, DNS, etc. all come from the access point.

Not directly needed, but here’s some interesting background on hacking NetGear equipment.

The best part about this? I found the original thread, and the fact that this was all possible, using my Sprint phone while standing in Fry’s staring at the blank brown box of the refurbished WGT624 wondering “WTF is this?” (iPhone? We don’t need no stinking iPhone.)

Potential limitations that may reduce the usefulness of this. I don’t know if these are actual limitations, but I haven’t tested beyond my own setup.

  • Tested only bridging to NetGear access point (potential issues with other brands?)
  • Tested only 64-bit WEP encryption (some of the comments mentioned problems with WPA)
  • Tested only with published SSID at the AP
  • Possible wireless saturation/interference–when I tried this with the bridge a few inches from my Thinkpad, the internal Centrino wireless could no longer connect, and I’ve read that some BIOS versions of this router produce illegally-strong radio signals

It Strikes Me that Most People Don’t Do Anything

I’ve been reading a lot lately about how to manage email and how between facebook and twitter you can do 90% of your job. And I’ve come to believe that most people, as the central function of their job, do nothing other than chat rather trivially with other people. Because I simply cannot do my job in 140 characters or less. I can’t do five sentence emails. Most of the work emails I get reveal to me that the people I work with haven’t the slightest clue what’s going on at any given moment on any given project. The occasional brain dump is required. I still document my code, if you can believe that.

In fact, the main service I provide, in a very detailed usually concise fashion, is the interpretation of vague and incorrect assumptions into working, usable processes and applications. Tell me how that’s going to get done over twitter?

I’m sorry to say it, but in the real world we write the specification as we go. We morph the process on a daily basis. And unfortunately that requires a little more documentation and accountability than you get from MySpace and YouTube. In the real world we work with SVN and SQL Server and FTP and Terminal Services and FogBugz and wikis and virtualization and yes, oddly enough, email. Occasionally we pass around a Word document. Shocking, huh?

The Great Date Debate

I was all set this morning to do a long rant on why we should be using this date format:

2007/05/22 (or 2007-05-22 in situations where the “/” is inappropriate)

Looks a little weird, I know, but no matter what culture you’re from, there’s no doubt what date that refers to. I didn’t even have to tell you that it’s YYYY/MM/DD–because what else could it be? My personal reason for adopting this format over 10 years ago was that it sorts well in plain-text data–like in the file system on a computer. It continues to sort well even when followed by a similarly-standardized time (like 16:22:03).

Anyway, the reason I don’t have to do a long(er) rant is that a fast Google Search reveals a lot of other people thinking about this:

And perhaps the most solid recommendation in the modern era, and what got me thinking about this today: Google uses it. It’s right there at the bottom of the Gmail advanced search options.

And if Google does it, it’s got to be good, right?

Amazon EC2: Virtual Hardware as a Service

I’m not sure how I missed this, or how long it’s been available, but as the next logical step after their “storage as a service” S3 solution, Amazon has come out with with the Elastic Compute Cloud (EC2). Essentially this combines server virtualization (which if you’ve worked with me, or heard me talk about work, you know I’m all about lately) with the massive server farms at Amazon via an ever-expanding web service system. Basically you build or choose virtual machine images and run them on an arbitrary number of virtual servers at Amazon. It’s “dedicated server” hardware co-location without the hardware. The idea is you can create an entire “data center” by interconnecting these images–for example, running several web servers, against a couple of database servers. And if you need to double the size of your data center, it’s a batch copy to invoke more servers–just pay for the “instance hours” you use. This is heady, brain-baking stuff. They even have a pre-built Window 2003 Server image running under Fedora Core 6 via Qemu (itself a virtualization environment). Hey, wasn’t I just talking about turtles all the way down?

So, how much does all this cost? The short answer is, you pay for the flexibility–it’s more than root access co-lo for a single-server setup. The long answer is, to run a web server, it’s about $100/month (1 CPU @$73 + 160GB storage @$24), plus bandwidth (where you can really get killed). Compare this to, say, the $100/month root server plan over at 1and1, which comes with two terabytes of transfer (maybe… many ISPs will throttle or boot you if it even looks like you’ll approach the max on your plan). All things being equal, that transfer would cost you an extra $300 at EC2.

A Compiler, Debugger and Software CPU in 50K

This is an amazing JavaScript virtual machine for 6502 assembly language. In one JavaScript file (under 50K) they managed to include a compiler, debugger, software CPU, and graphics sub-system. I use virtualization software every day, but not since reading The Diamond Age have I seen such a compelling demonstration of the fact that any computer can act like any other computer through software alone. This is exactly the kind of turtles-all-the-way-down metaphor that gets us things like The Matrix and the domino computer. It’s tempting to think of the logic gates in the CPU as the bottom rung of the computing process, but in essence those silicon pathways are there to reverse-virtualize (realize) the logical rules of computing. The CPU is in fact an interface to those rules, and by extension to the brains of the engineers and mathematicians who designed the rules. And what are rules but software? The only place this breaks down is inside the human brain–we don’t know the rules that govern in there–so while you can emulate a computer with your brain, we can’t yet emulate our brains on a computer. But outwardly our participation is governed by rules so effectively we’re part of the machine.