Arbitrary Thoughts

Firefox and Version Numbering

2012-01-18T12:20:00.000-05:00

I wrote this post about a year ago, left it in my drafts folder and forgot about it. I'm not sure why. It's quite good. The timestamp in my drafts folder says: 2/20/11.

Mozilla, the makers of the Firefox web browser, want to release four versions per year. I find it funny that most of the discussion about this in the arstechnica comments is about what the version numbers should be. As if this is the most important aspect of the decision.

Software development revolves around releases. A release is a software version that is available for purchase or download (ie: released to the public). Software manufacturers will then add new features to this old software and release a new version later. A software version number tells you which release of the software you are using.

By convention, the first release is called 1.0. The next major release is called 2.0. If there's been only bug fixes the version would be something like 1.0.1. If they're fixing bugs and introducing a few minor features the release might be 1.1. If they're introducing lots of minor features they might skip a few numbers and go straight to 1.5. The result of all this is you can use the version number to make an educated guess as to how much has changed in the new version.

All this version numbering is based on the assumption that you actually have major releases and minor releases. Some software is developed continuously and released on a fixed schedule. Every four months or so there might be a new release which contains whatever is finished at the time. Some releases might have only have bug fixes but others may have major new features. If you try and fit this into a typical version number scheme it becomes difficult to decide which release get a new major version number and which require a minor version number change.

Google's chrome Web browser is a good example of software development on a schedule. They have effectively abandoned major and minor version numbers. Google increments the major version number by one for (almost) every release irrespective of what new stuff it contains. We used a similar approach when we were developing Myster. Releasing on a schedule is a natural fit for the development team but can confuse those who expect a more traditional numbering scheme. It's confusing because the version number seems to lie; there's not always major new features despite the major version number change!

Mozilla has said they will be putting Firefox on a scheduled release system with four releases per year. Up until now Firefox has been using the more traditional release schedule and version numbering scheme. The new release schedule basically mandates that the major version number is incremented by one. Simply increasing the major version number by one with each release is so much easier than having quarterly arguments as to what the version number should be based on what happens to be in that particular release.

In these days of automatic updates, version numbers are practically irrelevant. What matters is whether you're up to date or not. I would like Mozilla to downplay Firefox's version number if they are releasing on a schedule. If they focus the conversation on the capabilities of the software and less on what it's version number is most other concerns will fall into the background as people adapt to the new reality.

Me from the present says: I'm using Firefox 9 and feelin' fine.

Amdahl's Law and Critical Sections

2012-01-08T13:20:00.000-05:00

Many moons ago I wrote a blog post on critical sections and efficiency. The blog post was a discussion of Amdahl's law. Aater Suleman over at future chips pointed out that Amdahl's law doesn't model critical sections. I used the wrong mathematical model! Oops.

Well, the good news is that my main point about critical sections hurting performance when you scale up to multiple processors is still valid... but all my graphs are wrong. *heavy sigh*. Since posting those graphs, I've been trying to find a moment to write a follow-up posting. What really drove home the importance of this was I noticed that just about every concurrency related session I attended at Java One this year mentioned Amdahl's law in relation to critical sections. Doh! It looks like I am in good company about miss-applying the law.

Amdahl's law applies to systems that can be split up into multiple tasks. Each one of these tasks is dependent on the completion of the previous task. Some of these tasks are 100% parallelizable others are not.
If we took this system and ran it on a machine with 3 processors, the amount of time it would take would be the amount of time for part 1 plus the amount of time for part 2 divided by the three processors:

In this example, on a single processor system, part 2 is three times longer to compute than part 1. So if we had three processors part 2 will run 3 times faster. Part 1 would stay the same speed because it can't be parallelized. The net result is we spend half of our time in part 1 and half of our time in part 2. It also means that overall, with 3 CPUs, this task will take half the time it would if we ran it on one CPU.

A real world example of this sort of situation would be painting a house. The first step would be going to get the paint from the store. It doesn't matter how many people you have it always takes the same amount of time to get the paint. The second step would be painting the house. Painting the house is something that can be split up amongst many people and so is parallelizable. The key thing here is you still need to wait until you have the paint before painting can begin and driving out to get the paint doesn't benefit from multiple people and so is not parallelizable.

Critical sections, on the other hand, are parts of a program that can only have one thread executing them at a time. A real world example would be having your friends help you move. Having multiple people can speed up moving house by every person carrying a different box out to the moving van. The thing is, two or more people can't fit through the front door at the same time. The door acts like a critical section; only allowing one thread (or person) to use it at a time.

This task contains a tiny piece that can't be done by multiple processes at the same time. In this example that same task will be executed 3 times on three different processors:

In the diagram the tasks on CPU2 and 3 started a bit late for some unimportant reason. However, when the process running on CPU 2 gets to the critical section it doesn't need to wait because CPU 1 is already finished with the critical section. Unfortunately, the processor running on CPU 3 does have to wait for CPU 2 to finish using the critical section. That's bad luck for process running on CPU3.

Critical sections are different from the situation described by Amdalh's law because the system might run completely in parallel; two processes that wait for each other is an unlucky occurrence. This means that, for a single run, we have no idea how much time the program will take because it depends on how often two or more threads both hit the same critical section.

While searching for Amdahl's law and critical sections I found a paper that provided a mathematical model for critical sections. Stijn Eyerman, one of the co-authors of that paper, created an web page that generates the kind of graphs (factor speedup vs number of cores for critical sections) I give below. The only downside is that the page is completely unfathomable unless you've read the paper. The paper is no picnic either. So what do the graphs look like when I use the right mathematical model?

Below is a graph of the factor speedup vs number of cores. This is is for critical sections:

Notice how the impact of the critical section is small up until the point when it suddenly starts to drastically limited the amount of speedup. Conceptually speaking this is the point where there's almost always a thread in the critical section... Or as the paper puts it the execution time "is determined by the serialized execution of all the contending critical sections".

For comparison, this is the graph for Amdalh's law:

In contrast to critical sections, the system starts to show inefficiencies much sooner but degrades much more gradually.

Here's the same data as above but in terms of % efficiency as the number of cores increase. First critical sections:

Now for Amdahl's law:

Here's what the graphs looks like for up to 100 cores. First critical sections:

Now for Amdahl's law:

Here's a direct comparison between Amdalh's law and critical sections at 95% parallelizable up to 100%. The red line represents critical sections.

(If you're using the Modeling critical sections in Amdahl's law web page the parameters I'm using are sequential code: 0 and Contention Probability: 1 )

Well, I hope this has been educational.

Critical Sections and Efficiency

2011-06-22T22:23:00.037-04:00

Oops, I made an error when I wrote this post. The mathematical formula I'm using to make all them pretty graphs doesn't actually apply to critical sections.

Here's a followup posting with the correct graphs for critical sections!

It only applies to code that must run in parallel then must run sequentially. One step must finish completely before the other can begin. In our moving example below it would be like step 1 would be moving the furniture into the van (highly parallelizable), the second part would be driving the single van to the new location and step 3 would be moving the furniture into the new house. Steps 1 and 3 are parallel and step 2 is sequential (only one van and one driver. Everyone has nothing is just riding along in the van or their own cars or something. Maybe they teleported.) So in conclusion I'm trying to understand the math for critical sections and I'll write something on that later.

Modern computers tend to have more than one processor core. Each one of those processor cores is like an individual computer brain. This has led to something of a crisis in computing circles because figuring out how to make use of those two cores is hard. A big chunk of the problem is splitting up a computing task into pieces that can be done independently. For example, moving house is easy to parallelize because every person who shows up to help can grab a different box. Walking through doors is harder to paralyze because if all your moving buddies walked through the same door at the same time they're going to bump into each other. That would be funny be not very productive.

In the computer world, whenever you parallelize a task you end up with a bit of that task that must be done sequentially (one at a time). If people are helping you move they can each move a box but they can't go through the door at the same time. In the computing world things like doors are called a critical sections. A critical section is a piece of a task that cannot be done in parallel.

Ever since working on a project to improving the performance of InteleViewer's image decompression on multiple processor machines I've been wondering how critical sections affect performance. InteleViewer will often open a stack of something like 500 images and each image will need to be decompressed before it can be displayed. This is an easy task to split up since you can just give each core a different image to compress. The thing is, there's a very tiny amount of task co-ordination that needs to be done in a critical section. When we ran the code on a two core computer we notices that the cores were only being used at 98% efficiency.

First of all, that's an incredible loss in efficiency given that the critical section was doing practically nothing and secondly, how much will the CPUs be idle on computers with something like 12 or 16 cores? In our moving day analogy it's roughly analogous to wondering how many much time will be wasted while people wait for each other to walk through the door if a large number of people are helping you move.

Luckily I recently stumbled across Amdahl's law. Amdahls's law isn't a new courtroom drama series premiering this fall on Friday nights at 8pm (9 central). It's a mathematical model that deals with the efficiency of parallel task running the presence of critical sections. I took the formula and produced a graph to see what was going on.

This chart has the number of cores along the X axis and the factor speedup along the Y axis. The lines represent how much of the task can run in parallel. The blue line represents the ideal of a task where 100% of it can run in parallel. The orange line represents a task where 99% of it can run in parallel and the light blue line at the bottom represents a task where only 50% of it can run in parallel.

(To go back to our moving analogy, the orange line is a house with a door and the light blue line is a house with a door and a very long, narrow hallway that you have to pass through first. The dark blue line is a house with no door.. which would be kinda weird in real life but perfectly safe in a world populated only by math.)

(Click to enlarge, higher is faster)

Notice how the light blue line at the bottom never quite reaches the 2X speed up even when you put up to 16 cores on the problem.

I found it surprising how inefficient things get even when 90% of the task is able to run in parallel. At around six cores there's almost no point in adding more cores. They don't really speed up things that much.

I also have a graph of the % efficiency. This grap htells you how busy each processor core would be if you gave it a some task and some number of cores.

(Click to enlarge)

Here's what the graph looks like up to 100 processors:

(Click to enlarge)

Notice how even a task that runs in parallel 99% of the time only makes use of half the available computing power at 100 CPUs. We don't have computers with 100 cores yet but with the number of cores on a processor doubling every 18 months or so we could have that in less than ten years. The main problem is that the InteleViewer's image decompression example represents close to a best case scenario and even there we have code that's not going to scale up to that many cores. Making programs run efficiently on multiple processor cores is hard. It's all just a tad depressing.

The people who make processors have noticed this problem too. In the latest batch of chips they are opting to not add more cores but instead to put a video processor (GPU) and other task specific hardware on the processor die... but that's another blog post.

The InteleViewer example actually does have a happy ending. We decided to switch from using critical sections to using a magical technique called compare-and-swap. It's not always possible or wise to use this technique but for InteleViewer it worked great. We ended up making our 1% overhead disappear. Well, as far as we can measure on a 12 core (6 + 2X hyper threading) anyway.

There's no equivalent analogy I think of to compare-and-swap in the moving example. The closest I can get is if all your moving buddies decided to run through the door regardless of who else was trying to get through. So long as collisions don't have a large time penalty and so long as they are infrequent enough it might be faster. I don't recommend trying it for doorways but for InteleViewer it worked great.

So, in conclusion, buy InteleViewer.

We're done here.

Why Threading is Hard (link)

2011-05-30T13:14:00.005-04:00

As someone who spends quite alot of time explaining and debugging multithreaded applications it's nice to see someone else take a go at it for a change.

Future Chips (Mmm chips) has an articles on what makes parallel programming hard. It goes over why not all programs get a speedup with more cores and why it's hard to write and debug multithreaded (MT) programs.

They have also posted a follow up article on writing and optimizing parallel programs; a complete example. Which is nice...

My Top Three User Interface Books

2011-02-10T19:34:00.006-05:00

As far as I know there is still no course or university program to become a user interface designer. Part of me thinks this is a shame because I think there's a lot of things you can teach someone who wants to design user interfaces for computer programs or websites. Another part of me is happy because I can claim to know a thing or two about user interfaces without the bother of getting another degree.

I've been doing computer programming for a while now (that's an Imperial "while" you'll note. Multiply by 1.6 to get the more typical metric "while".) And just about all of that programming has involved some sort of GUI work on a desktop application. In my opinion there are many great books that can help you become better at this sort of thing. However, if you want to learn as much as possible in as little time as possible then I recommend three books.

The first book tells you how to know what to build. This is surprisingly difficult because you often aren't building a program for yourself you're building it for someone else and you don't know what they need. If you're a programmer this is even harder because programmers think in a very peculiar way with respect to computers. If you're a developer and you're building a user interface what you end up with an interface that doesn't do what it needs to and does it in a very strange and round about way.

The Inmates are R unning the Asylum is a book by Alan Cooper. This is a book for programmers who want to write user interfaces. In the book he explains how developers think differently than your average man on the street. Specifically he points out how certain practices that work well if you're developing software get in the way if you're trying to write a user interface. For example, one of the easiest traps to fall into is to write a user interface that can deal with every single possible edge case that the user might ever have. Trying to deal with every single case is essential if you're writing computer software.In fact, if you don't do this your computer software will break. When writing user interfaces it's counter productive. Trying to do everything means you're not doing anything well. One warning, don't let your pointy haired manager get hold of this book or you might never be asked to program a UI again. This book is is written for developers and there's a lot of hyperbolic developer bashing. As a developer, it gets your attention, but may get miss-understood by others. Developers can code good interfaces.

If you're not a developer you should read About Face. About Face is also by Alan Cooper and contains the same material as the inmates are running the asylum but also contained additional chapters and give more detail on constructing processes in an organization to formalize interface development. This is useful if you have to convince the organization or even the developers on your team that's something you're suggesting make sense. Believe me you'll need to do this at some point. Probably before you actually get any real work done.

In both these books, Alan Cooper explains several methods you can use in order to figure out exactly what your software is supposed to do. Newcomers to the field often think that the way you figure out what your software should do is you ask the user. In practice this doesn't work very well unless the person you're asking is a user interface designer. Users are extremely bad at expressing what they want. For it to have any hope of working you'll need a tight feedback loop and lots of prototyping/trial and error. The tight feedback loop is almost never possible in practice and even when it is the results are still... average. To do better to do better you'll need to read the book.

Book 2 is GUI Bloopers. Actually, this book is a stand-in for simply knowin g user interfac e guidelines and standards for the platforms and GUI widgets you be using. It's funny because Alan Cooper downplays the importance of sticking to interface guidelines and standards. In my opinion you absolutely have to know all the standards and interface guidelines for your platform. There are no excuses. You don't have to be totally pedantic about it but you really need to know the rules before you break them. It's important to know this stuff if you're a developer because UI designers don't always design to this level of detail.

Most people have been exposed to computer interfaces before and have an intuitive grasp of how buttons panels menu bars and the like should all work. GUI Bloopers is about the things that user interface designers often get wrong when they try and transfer this intuitive knowledge into an explicit design. The book is also extremely fun to read. It brings up examples of bloopers in real-life and often well-known applications with often funny results. Sometimes I read it as a form of therapy.

The end result of having an application that obeys platform conventions is that it feels very familiar and natural. The user can start using the application right from the start without feeling disoriented or having to think about what he's doing. In the past there were many people who had never used a computer before. These days most people interact with a computer every day. If you build an application in such a way that it works very similarly to other applications you will have drastically reduced the learning time and people will have a very warm fuzzy feeling of familiarity with your application right from the start. On the other hand if your application works in a completely weird and unexpected way it will make the user feel as if they're starting over from scratch again. It will be disorientating and confusing and that's not a a good first impression to make. Let's not be bush-league.

If you've mastered the other two books it's time to move on to the advanced stuff. The last book is all about flow. Jef Raskin's book The Humane Interface is an extremely detailed and often formal look at flow. Flow is the feeling you get when you're concentrating on what you're doing rather than how you're doing it. When you're in flow the computer disappears. Your mind melds with a computer and everything you want to do just suddenly and effortlessly just happens. You're still using a computer but it has become an extension of your being. In the same way that a painter's brush becomes an extension of the painter or a hockey players stick becomes an extension of the hockey player. It's where an idea or desire effortlessly flows into a series of actions to accomplish it. Jef Raskin's book is all about writing computer interfaces that encourage this feeling. I'll give you an example of what I mean: Dialog boxes suck for flow.

So those are my three books in roughly the order of importance. I think I'll end my post here. I'm sure you got a lot of reading to do.

My Pet Computer Mice

2011-02-07T21:49:00.009-05:00

I always try to spare no expense when buying monitors, keyboards and mice but in the last few years it's been difficult to figure out which mouse to buy.

In the days of ball mice I tended to get a high-end Logitech mouse. I must admit that Logitech is my favorite brand-name for mice. I've had so many bad experiences with other mice that I just tend to ignore any other brand. That said, Microsoft occasionally makes a good mouse. The one major problem with Microsoft is they seem to have no common sense. Every time I use a Microsoft mouse I wonder what's gone wrong with the mouse wheel. Microsoft mice often don't have a clicky mouse wheel. Their mouse wheels are extremely smooth as if the mouse wheel was an analog device. There are many situations out there where you need that precise clicky control of the mouse wheel and for Microsoft to ignore this just makes me suspect their sanity.

On the other hand, Logitech does silly things with the mouse wheel as well. I have a Logitech MX Revolution mouse. Its mouse wheel is a mechanical marvel. It can either be a clicky style mouse wheel or smooth scrolling mouse wheel depending on how you configure it in the software. I'm not sure how exactly they do this but it's really cool. There's even a special mode where the mouse wheel will be clicky until spun fast enough where it'll become a free wheel and turn with no resistance whatsoever. As soon as the wheel stops spinning it becomes a clicky wheel again. This is extremely addictive behavior. You essentially think nothing of scrolling rapidly through documents and in my opinion I think every radiologist who uses InteleViewer should have one of these things. I've seen how much radiologist use the mouse wheel when they use our product. This mouse wheel would be a godsend.

The one major problem with the Logitech MX Revolution mouse wheel is it surprisingly hard to click. On most mice it is easy to click the middle mouse button (which is the mouse wheel itself) but on the MX the mouse wheel takes an unexpectedly large amount of force to click. Add to this a mouse wheel that rotates easily (even in clicky mode) and more often than not you'll accidentally scroll when you click.

The Logitech MX Revolution mouse is also quite heavy. This is because it's a wireless mouse with it's own battery, air supply and guest washroom. So that shouldn't be too surprising. I don't like heavy mice and almost took it back because of its sheer weight. On the other hand the weird shape of the MX Revolution mouse isn't just there to make it look like a spaceship. It's incredibly comfortable. Holding onto one is like putting your hand in glove. It's so very, very nice.

The Logitech MX Revolution 's wireless reception isn't so nice, though. I'm in a apartment block where there's quite a lot of radio traffic. The wireless reception of the mouse is so bad that I had to get a USB extension cord to bring the wireless receiver within 30 cm of the mouse itself. Only then could I get decent performance. With the transmitter any further away the mouse tended to pause while being used. This is annoying while doing ordinary office tasks and completely infuriating when playing StarCraft II.

The Logitech MX Revolution doesn't work very well for games either. I'm not sure what Logitech did. I figure it's either the latency added by the wireless link or some sort of smoothing system in the mouse itself. The pointer just doesn't feel connected to the mouse. It sort of, lazily wallows around the screen in some sort of pot-like daze. It's quite a shame too because it tracks quite well otherwise. I couldn't get the mouse to skip or jump when moved quickly like some of the cheaper optical mice.

Speaking of cheaper optical mice I also have a Logitech Optical Mouse. This one doesn't have a spcial name. It's essentially the cheapest optical mouse that Logitech makes. It's a fantastic mouse! It's light, has a good mouse wheel and a great shape. What's more, the cursor feels like it's connected to the mouse unlike the Logitech MX Revolution. The only thing bad about this mouse is if you move it really fast the cursor gets confused and shoots off in the opposite direction. This makes it completely useless for games. I like this mouse so much and it's so cheap that I buy many of them and give them to my friends. Many of them have terrible mice that are so bad that I just take pity on them. If you want me to fix your computer you should at least get a decent mouse for me to use.

Between the lazy, disconnected performance of the Logitech MX Revolution and the Logitech optical mouse's unsuitability for games I was left with a hole in my life. I needed a mouse for StarCraft II! I think I might have found it in the steelseries XAI Laser Mouse.

I just got this mouse recently and its been consistently impressing me. The cursor feels connected to the mouse to an extent I don't think I've ever experienced. It's also light and has a shape very similar to the Logitech optical mouse. It is a bit bigger than the Logitech but it's also more comfortable. I must admit I am a complete addict of this mouse shape and while the MX Revolution is capable of giving me an orgasm every time I grab a hold of it I still keep going back to the traditional shape. Though at $99 I'm not going to give it away to friends for free. It's also not wireless. Which I like but might be a dealbreaker to some people.

Steelseries, the manufacturer, seem to take their mice extremely seriously. They don't ship any software in the box. You have to go and download it. I suspect this is because they don't want their clients to have out-of-date software although at $99 I expect to see some obsolete software in the box. Give me a CD you cheap bastards!

To be fair, the mouse will work fine out of the box. Steelseries actually advertises it as a driverless mouse. That is, a mouse that doesn't need drivers rather than a mouse that doesn't actually ship with the drivers. When you download the "driver" software, however, things start to get interesting.

The software is pretty well done on the whole although I noticed a repaint error or two. The software contains many delightful settings as well as an extremely informative manual attached to the side of the window. This is a good thing as I had no idea what any of the settings did until I read the manual. A full review of all the different settings is way beyond what I'm prepared to do but thankfully there are others to fill that gap. My only complaint, if it is a complaint, is that the software was done using anesthetic that a hard-core gamer would love. Black and orange! Yes! We could probably do that for InteleViewer. The radiologists would love it. Peow peow.

This mouse has made me very happy. It was made to the exacting standards of gamers but also works incredibly well for office work. Most importantly, it actually tracks properly! Hooray! I think I found a new favorite.

Using my psychic powers I can guess that someone's about to ask me if I've ever used any of Logitech's gaming mice. To be fair, I've never had the courage. They look massive! I'm not sure I would like using a mouse that big. To be sure I grabed a large potato and dragged it across the mouse pad a few times clicking on its eyes. It made a mess as I'm sure the gaming mouse would do. Want to clean up the mess? What better way than to use a mouse that looks like a bar of soap. That's my philosophy.

Dragon test run

2010-10-10T15:43:00.002-04:00

HelloI am testing Dragon NaturallySpeaking with the new microphone. So far it looks like it might be working okay. Although it's too soon to tell. I'm speaking particularly quickly this time. I hope that Dragon NaturallySpeaking will be able to keep up with me. There have been two errors so far. I hope this isn't a trend.

Hello (again). I'd like to talk to you today about spam. Spam is some kind of luncheon meat. I'm not entirely sure what it's composed of. I've been told it's composed of spiced ham and meat although this is only a rumor. I've only managed to taste it once and I haven't been impressed. It's not that I thought it tasted bad it's that I didn't think it tasted much like anything. I have always liked ingredients with strong taste. I like things like onions and strong cheddar and spiced turkey breast and hot pickles with maybe a gherkin as a side dish and some sour balls for dessert. I guess I like the feel of a nuclear explosion in my mouth.

While we're not on the topic I'd like to change the subject to which ingredients go best in sandwiches. My personal fav is old cheddar, mayonnaise, cucumber and tomato. If I can be bothered I sometimes chop up some onions and throw them in there as well as sometimes some spicy hot pickles. I used to make the same sort of sandwich but add in some lettuce. I've slowly been coming to the conclusion that lettuce in sandwiches is just a waste of space. It doesn't really offer much in terms of flavor and while it's true that it does create a pleasent crunchy texture you can get much the same effect with cucumbers.

One thing I can't understand is why anyone would put cheese and meat in the same sandwich. In my opinion a sandwich should based on one or the other. Tthey are both superstar ingredients. Both cheese and meat deserve their own show; they deserve to have other ingredients bask in their glory. To put both in the same sandwich is doing both a disservice. First of all they don't go particularly well together. Both meat and cheese have similar textures and basic tastes. Unless you're meat is salami and cheese is Swiss then I can see that. The worst combination is ham and mild cheddar. If you do that you might as well just shoot yourself or use spam.

The worst damage I've ever made is one that use plain white bread with butter and mild cheddar. That's absolutely completely unpleasant. I'm pretty sure that sandwiches classified as a torture device under the Geneva Convention.

Anyway, this concludes my test of Dragon NaturallySpeaking with a new microphone.

New USB microphone: Plantronics 655

2010-10-10T14:46:00.012-04:00

Wait a moment, I never told you about which microphone I got.

So when you get Dragon NaturallySpeaking you get a free microphone in the box. I really like this microphone as it's free and I only had to pay $100 for Dragon to get it. As a rule, free stuff you get with other stuff you have to pay for is good. The best place to make use of this philosophy is at a convention where you get a whole bag of free stuff for going to the thing you had to pay for which you can then throw at people you went with and sometimes they even give it back which is all good but I digress.

After buying Dragon NaturallySpeaking I started to use Skype quite a bit. I used my Dragon NaturallySpeaking headset with Skype. If I could digress once again I'd like to point out that anyone who's not using a headset of some sort when talking with Skype is inflicting terrible pain on the other person. Whereas the person with the headset is producing beautiful, CD-quality audio without any background noise but I was not using the headset is creating an echo chamber full of background chatter, claiming pots and pans any inevitable crying baby noises.

Okay, now what was I talking about again? Oh yes, my old headset with intent to use that as a lead-in to my new headset. Well, that kind of went off the rails. Man, you'd think I'd be able to fix all this by editing it after I dictated it. Too bad Dragon is dictating this in "pen mode" and I don't want to leave any whiteout on my screen. Oh well.

So anyway, Skype got me addicted to headsets and I started to use it more and more often. Then Starcraft II came along and my life would never be the same again. You see, I play Starcraft II with a friend over Skype. This is great but the Dragon headset really isn't up to the task. For one thing it only has a speaker for one ear. For another it seems to be meant for someone with a really skinny head. After a few hours of playing it becomes extremely uncomfortable. So I went out and got myself a good, new, stereo headset.

What I got was a sort of middle middle-of-the-road Plantronics 655 USB headset. My intention was to use this headset for games and for dictating to Dragon NaturallySpeaking. Also keep in mind that my friend Guillaume had written a very nice article explaining how to purchase a headset for use with Dragon. As a result I also got a stereo, noise reducing, USB headset.

So far, it's worked quite nicely for both Starcraft II and Dragon. Having the full stereo headset makes a huge change for Starcraft II. I can hear a pin drop and can drop a siege tank on the poor SCV that dropped it. I can also dictate to Dragon NaturallySpeaking very nicely too. The headset also works nicely when listening to music and has two little buttons on the side of the earpiece that can control the sound volume.

The microphone is not bad. The recording is buzz free although it does pick up more background noise then I would like. It doesn't seem to pick up more than my old headset but I guess I was hoping that the noise cancellation feature would do miracles.

When used as a pair of headphone headphone, one thing that's so good it's almost creepy is that there's no buzzing. Usually a headset plugged into the stereo mini jack on the computer will create a tiny background hum or hiss all the time. This is typically because the audio card on the machine isn't perfectly isolated from all the electrical noise coming from inside the computer. Because this is a USB headset, however, there is none of that. It sounds as if the headphones are not plugged in if the computer is not playing any sound.

Now, if you're looking for super amazing sound quality then you might want to look elsewhere. I say "might" because I really don't know if they're any good or not. I don't have any fancy equipment at my disposal to test with and I don't have a golden ear either. All I know is pretty much matches what I've come to expect from full ear headphones.

The headphones also don't isolate you from noises in the room very much at all. It's got to be the least sound isolation I've had from a full-ear headset. I'm not sure this is a good thing or a bad thing. It would probably be a good thing if you're working in an office and somebody wanted to interrupt you. It's probably a bad thing if you're working in an office and you don't want to be interrupted. At least I'm aware of how loud I'm talking into the microphone.

The last minor quibble I have is that my standard speakers don't work with the headset plugged in. This means I'm forever unplugging the USB headset in order to switch to my regular speakers and back. My original headset used a typical stereo mini jack and I have two of those on my machine so I could keep the headset constantly plugged in and still play audio through the speakers. I just wish there was an easy way to flip the audio without unplugging the headset or going into the windows sound control panel and switching the default sound output device. Maybe some sort of keyboard shortcut? Hum, I'll have to look into that.

(OMG it works with skype!)

Links ahoy

2010-09-24T13:08:00.003-04:00

Yeah, I haven't said anything in a while. The reason: Starcraft II. It's highly addictive. So, while I'm playing my way through the campaign here are a few web pages I've found interesting:

Jeef Atwood's posted on the latest developments when it comes to SSDs. Apparently, they've gotten cheaper and faster. Ah but who's the fastest?
http://www.codinghorror.com/blog/2010/09/revisiting-solid-state-hard-drives.html

xkcd's worried that the windmills will kill us all.
http://xkcd.com/556/

.. and my good buddy Guillaume's posted his take on how to get the most out of Dragon Naturally speaking. You don't have dragon? You should. :-)
http://blog.gmarceau.qc.ca/2010/09/dictate-to-your-computer-like-pro.html

Also... did I mention starcraft II?

Dialog Boxes

2010-09-07T20:45:00.000-04:00

Dialogs suck. They rudely jump in your face, block you from doing anything else, can be cryptic and are often just plain pointless. When a dialog comes up users (that includes myself) want to make it go away as fast as possible. It is the computer equivalent to wack-a-mole. As a result nobody is really interested in what the dialog is trying to say.

When a dialog pops up in front of a typical user they will start looking for the fastest way of making the dialog go away. The easiest thing to do is look at the buttons first. If the buttons are labeled "Save", "Don't Save" and "Cancel" then you really don't need to read the rest. It's a save confirmation dialog so press save if you need to do that, "don't save" if you don't and cancel if you were really trying to do something else. Labeling dialog buttons with what they do is excellent practice.

(This isn't the standard windows buttons order but otherwise a fine example)

Even if you don't buy the speed argument, putting the actions on the buttons themselves can help the user decode the rest of the dialog text. Example: A dialog with the text "The pliny has splonged the wirly. A momi is required." with "Get Moni Now" and "Get Moni Later" buttons is probably asking whether we'd like to do something now or a delay it until a little later. This isn't the best dialog because it's not clear what the consequences of the actions are but it's nice to see we can continue our work without getting a moni right away. Hopefully the pliny can limp along with a splonged wirly until then.

If the dialog has two buttons "Yes" and "No" then the user has to read the dialog text to find out what to do next. You can make this even worse by asking the user a question in the dialog text that doesn't require a yes/no answer. "Do you want to save or discard?" with "Yes" and "No" buttons, for example. That is not a yes or no question! You forfeit the rest of your questions! Again, labeling the buttons with the actions the user can take can avoid this class of error.

(Consider which options to put in a dialog carefully. 3 buttons is plenty. Don't be WinSCP.)

If the dialog text is incomprehensible the user starts to cast about looking for clues. This includes reading your dialog's title. Most of the time a dialog boxes' title is not that informative so the next thing the user tries to do is close the dialog with the close box. The idea here is the the close box essentially maps to cancel and so "canceling" whatever the dialog box is talking about is usually a safe action. Knowing this don't go and disable the close box action. That's like removing a panic button and replacing it with a profiterole.

So there you have it: a user will tend to look at the buttons first, then read the text then start looking for clues in the dialog title. They read them backwards I tells ya!

Undefined C programs and Game Loops

2010-08-27T09:59:00.001-04:00

If you're in the habit of using C (and who isn't), you'll want to read this. It's a fantastic description of all sorts of way you can write programs with undefined behaviours in C. It also goes over the horrible and unintuitive things that can happen when you do.
http://blog.regehr.org/archives/213

I've also been looking at tweaking the game loop inside space smilies and found an excellent article on how to write game loops. If you've ever want to write a game, read this first!
http://dev.koonsolo.com/7/dewitters-gameloop/

My Clock

2010-08-21T15:28:00.002-04:00

One day I woke up and was surprised to see I had time traveled to the year 2108. Then I realized it was only the 21rst of August and went back to sleep much relieved.

Scroll-wheel follows mouse

2010-08-19T21:20:00.003-04:00

One of the few things I like about using the Gnome based system at work is that it scrolls the widget your mouse cursor is pointing it. It doesn't matter if the thing in question has the focus or not. It doesn't matter if it's in another window that doesn't have the focus it just scrolls whatever is under the mouse cursor. It's really nice.

Windows 7 doesn't do this. I miss this behavior.In Windows 7 you have to click on whatever you want to scroll to make sure it has the focus. Sometimes you don't want to click because it's hard to find something "safe" to click on. That is, something that won't screw up what you're doing. Using the window explorer/file manager is a good example. The way I use it, there's a pane on the left hand side that shows me the list of folders and where I am in the file hierarchy. Clicking just about anywhere in that pane changes the selected folder and moves you around in the hierarchy. If I want to scroll this pane I have trouble finding a place where I can give it focus while not getting myself lost in a sea of folders. Having the mouse cursor determine which scroll pane is scrolled, rather than the focus solves this problem nicely.

I've managed to find a utility that enables this behaviour. It's called WizMouse. Give WizMouse and the scroll-wheel follows mouse a try. In a little while you'll wonder how you lived with it any other way.

Doodling

2010-08-14T16:08:00.002-04:00

You know how it goes. Someone is talking about.. something.. Your mind starts to wander and you start to doodle in your notebook.

This one made me laugh so I thought I'd share it.

History of the SSD

2010-08-07T10:13:00.002-04:00

As some of you may know I'm a big SSD fan. For those who are worried about the negative reputation that SSDs have gotten you should check out an article pointed out to me by our chief architect.

History of the SSD courtesy of AnandTech:

http://www.anandtech.com/show/2738/1

Light Text on a Mixed Background

2010-08-05T21:03:00.004-04:00

Let's say you wake up one morning and find you're the guy responsible for deciding what colour the subtitle text should be. After overcoming the initial shock of having slept through a major career change, you'd come realize what a difficult problem this is. If you make the text white it'll show up fine on a dark background as you would find in a horror movie or Tim Burton film. On the other hand it will leave the audience puzzled at on a white background like the snowy battle of Hoth. If you give in to the star wars fans and make the text black the Tim Burton fans will grab their pitchforks and head up to your spooky castle during a midnight thunderstorm. Things get even more embarrassing if half the screen is white and half the screen is black. The audience will be able to read half the sentence and no doubt fill in the rest with whatever their dirty minds come up with. This, of course, will lead to many viewer complaints about how smutty modern films have become. What are you to do?

I've run into this problem a couple of times during my programming career. In my case it was adding text overlays to images to show additional image meta data... while fighting the Sith with Jonny Depp! It's actually surprisingly easy to get text to show up nicely on almost any background. All you need to do is have lightly coloured text with a shadow effect.

The simplest way to do this is to simply paint a copy of the text in question one pixel down and one to the right in black. For added readability add another copy one pixel to the right.

Here's some sample code:

g.setColor(Color.BLACK);
g.drawString(TEXT, 10 + 1, 10 + 1); //second shadow
g.drawString(TEXT, 10 + 1, 10); //first shadow

g.setColor(Color.WHITE);
g.drawString(TEXT, 10, 10);

Nice, large, translucent drop shadows can also increase the readability of text. In my case, though, I only want the shadow effect to increase the readability of the text. The simplicity and subtlety of the shadow effect above is a feature.

I've included a real time java demo of just how amazing this effect can be.

Shadowed text demo!

(Requires java. Open the file with the "java web start" application)

Source code is here!

For those without java this is what the incredibly light pink text looks like on a pure white background:

(The application isn't this blurry when you run it :-) )

Google makes use of a subtle and more fancy shadow effects if you select a background for you Google homepage. You can use any photo for the background. In this image I've selected a plain, white background:

(The Google homepage slightly cropped.)

Booting XP with the new motherboard

2010-08-01T18:25:00.003-04:00

You may remember how I recently got a new motherboard.. and then I changed the CPU. I thought I'd get my motherboard to boot Windows XP.

When I upgraded to Windows 7 from Windows XP I installed Windows 7 on a brand-new solid state drive (remember when I got one of those?). This left me with the original installation of Windows XP on the existing hard drive. This is a sweet arrangement as it meant I could boot up with Windows XP if I discovered of applications that can't run in Windows 7. It's also useful to have your old Windows XP around if you want to look up some setting or bookmark or something that won't transfer over to Windows 7.

When I changed the motherboard I could no longer boot into Windows XP. I'm not entirely sure why that happened. My suspicion, at the time, was the drive numbers changed around. Windows XP keeps track of which drive number it's booted from in the boot.ini file. I went into the boot any file and tried changing the numbers to my best guess as to what what they should be but for some reason it wasn't working. I'm still not sure why it didn't work, actually. I think it may have had something to do with me screwing up the file format a little. I thought the '#' symbol was used to denote comments when in reality it's the ';'. Oh dear.. a little too much Linux shell scripting.

In any case, it wasn't working so I tried to use the utility EasyBCD to re-create the Windows XP boot option. This failed as well. I'm not sure why this failed either. I'm not sure what EasyBCD was trying to do but my booting arrangement is a bit unusual in that my old Windows XP drive contains the boot loader which then boots Windows 7 on my new solid-state drive. I think EasyBCD was a bit confused by this. When I would launch Easy BCD, it couldn't find the BCD file, for instance. I think it was setting everything up thinking the boot loader was being run from the solid-state drive. In any case, it I was left pretty much where I started except that now I kept getting messages about NTDETECT failing.

I noticed that EasyBCD had messed with files that Windows XP uses to boot. I tried to repair the Windows XP installation by running a few commands off the Windows XP repair CD. Many of the Windows XP commands were throwing errors and clearly had no idea what Windows 7 and easy BCD had done to the installation.

(The BIOS bone's connected to the MBR record. The MBR's connected to the PBR. The PBR points the BCD file.. but not on my system 'cause it's b0rked. yeah!)

I tried rebooting and was surprised to find I couldn't even boot Windows 7 now. After much swearing I used the Windows 7 repair CD to try and repair the installation. The Windows 7 installation CD had no idea what I'd done. I had no idea what I'd done. Things were going from bad to worse. I knew that the Windows 7 repair CD (or at very least the Windows 7 installation itself) would have the tools I needed to repair the Windows 7 installation. Theoretically, all I would need to do is generate a new BCD file that would point to the already installed Windows 7 installation. The thing was, I didn't know what the damn commands were that did that.

I tried to use EasyBCD but that didn't work as it needed the full Windows 7 to run; the repair CD didn't cut it. I couldn't even run the command line versions of EasyBCD. After a further sampling of four letter expletives, I started up the old Dell netbook and poked around the Internet looking for a solution. It turns out that, in the Windows\System32 directory, you can find two commands that will allow you to manipulate the BCD file. "bcdboot.exe" allows you to build a new BCD file and "bcdedit.exe" allows you to edit said file to add additional boot options. Sweet.

To make a long story short I used bcdboot.exe to make a new BCD file on the solid-state drive. I then switched the solid-state drive to my primary boot drive in the BIOS. With that I could boot into Window 7. yay! I then added the Windows XP installation as an option with EasyBCD. I now had a choice between the two operating systems when I booted. This all went so well that I also took a stab at repairing the BCD file on the Windows XP installation just in case my solid-state drive (containing the all important BCD file and Windows 7 installation) died. If the solid-state drive died I could boot directly to Windows XP on my other hard drive. I haven't actually tested it yet. I think it might boot Windows 7 but I don't hold out much hope that it's going to actually boot Windows XP. To do that I would need to use that BCD file to boot into Windows 7 and then used it two commands to add a second entry for Windows XP. This is precisely what I couldn't figure out how to do the first time. Oh well, maybe I'll try it again.. A bit later...

Now that Windows XP is booting all I need to do is install the drivers and get ACHI working again. Weeeeeeeeeeeeeeeeeeeeee......

IM IN UR BOXEN MESIN WIF UR BOOT DRIV

CPU heat sink woes

2010-07-31T18:16:00.012-04:00

Okay so here's the story of what happened to my CPU after I installed the new motherboard.

When you install a new motherboard you need to take the CPU off the old motherboard and put the CPU onto the new motherboard. This is a bit involved but not as difficult as dealing with the heatsink. The standard heatsink for the socket 775 series CPU is absolutely silly. Essentially, it's held on with some pushpins. It's a bit difficult to describe exactly how this works but basically you have two plastic flaps being held open with a sort of wedge. The wedge is on the head of this pin. The wedge is flattened so that as you twist the pin can wedge open the two plastic flaps or not as you wish.

I didn't know any of this when I started and the manual wasn't exactly helpful. After a while I managed to get the heatsink off but I would definitely suggest reading a handful of tutorials on the Internet on how to do it before you start. The way I did it was wrong. I actually twisted some of the pins in the wrong direction. They eventually turned but they weren't too happy about it. If you do it the right way it's much easier.

So anyway, I successfully got the heatsink off and transfered the CPU over to the new motherboard. There were no bent pins on the motherboard when I bought it. The guy at the computer shop was really eager to show me this. "You see" he said. "No bent pins". Mmm kay. I'm guessing that some motherboards have bent pins or some people bring motherboards back in with bent pins because they put too much pressure on the CPU when they installed it or something. I can definitely see how that could happen but if you're carefully following the instructions (which is actually well explained in the motherboard user manual at least in my case) you shouldn't screw anything up.

Once the CPU was on the new motherboard all that remained was to replace the heatsink. At this point I felt like a bit of a fool as I had completely forgotten about the thermal paste. You see, when you move a CPU like this you typically have to reapply the thermal paste (sometimes called thermal grease). In my case I didn't have any thermal paste. Nor did I have the isoproterenol you need in order to clean off the old thermal paste. Cunningly, I took advantage of both these situations and simply reused the existing paste. This was okay because it would still work, just not very well. Even if it didn't work Intel CPUs have little thermometers in them which stop you from damaging the CPU if they get too hot. It also stops them from bursting into flames if the heat sink falls off which is handy.

Intel CPUs (and modern AMD CPUs) do this by under clocking themselves until they cool down. So a 3Ghz chip might run at 1 Ghz or even lower if it gets too hot. The general rule is the higher the Mhz/Ghz the more heat is produced so lower Ghz means less heat. When this feature was introduced people accused Intel of crippling their CPUs because they would under clocked themselves when after you've installed them.. umm... wrongly... I still don't fully understand that attitude. I am quite thankful for this feature because I much prefer a slow CPU to a burning CPU.

When I got my system running I checked to see what the temperature on the CPU was. The temperature was about 75°C which is not that hot. Under load it went up to 80°C. These numbers didn't seem that high to me but I knew that different CPUs have different operating temperatures. Some Pentium 4 chips used to get very hot indeed. This being a Core 2 Duo (E6400) and not a Pentium 4 it might have a lower operating temperature. After some research it turned out that it was only supposed to run at 61°C maximum. Dang.

So I went down to the computer store and picked up some thermal grease and went to the drugstore and picked up some extremely pure isopropanol. I then took off my heat sink (with minimal swearing this time), used some cotton swabs and isopropanol to take off the old thermal paste and replaced it with the new thermal grease in the usual way. I then booted up my machine to check the temperature.

At idle the temperature was 65°C and under full load the temperature was 80°C. This annoyed me. I hadn't even finished overclocking the CPU. It was a 2.1 GHz CPU that I wanted to run at 3.2 GHz. I already knew that the chip could run at that speed just fine. My guess was that the thermal grease I installed was crap. The other possibility was that I had installed the heat sink slightly wrong... I made the decision that I wasn't going to reinstall the heat sink again as I was no longer on speaking terms with it. I wanted to run my system overclocked so I figured that I should get something better then the stock cooler anyway. With any luck it would be quieter and come with its own (better) thermal paste too.

I went down to the local computer shop and got an Arctic Freezer 7 ver 2. I did this because it had some good reviews on the web. It was also in stock at the local computer store. At only 35$ dollars it was a good deal too. Yeah, it included the thermal paste.

Installation was shockingly easy although I must confess that I had to read the instructions (Well, look at the instruction pictures really. There were no words.). I was also worried about over tightening the screws that held the heat sink onto the black, plastic mounting bracket. The 775 socket is made to take a lot of pressure but I didn't want to overdo it. I had visions of going back to the computer store. I'd have to get another replacement motherboard because I'd bent the pins! I couldn't even claim they were bent to begin with. :-)

In the end I installed the cooler, booted up and checked the temperature. Idle temperature was now 40°C. Temperature under full load was 60°C. Woohoo! What made it even nicer was that the fan wasn't even going at full speed like the old heat sink fan was. This meant I had some headroom to work with. It also meant I wasn't going to go deaf from the fan noise. Going deaf sucks. Especially if you don't want to.

I was very happy with all this and so tried overclocking it to the full 3.2 GHz. Idle temperature was still 40°C temperature and under full load it was still 60°C. I was very, very happy. I did notice something weird though. The the CPU seemed to be operating only at 2.4 GHz when the CPU was idle but when the CPU is under full load it was operating at 3.2 GHz. I'm guessing this has something to do with saving energy. If the CPU is not doing anything then there's no point in running it at the full 3.2 GHz. It just wastes energy. I'm still investigating exactly which feature that I inadvertently installed that is causing this to happen. When I find out I'll be sure to tell mention it.

Now all I had to do was get my second hard drive (with Windows XP) booting again.

Why the U2211-H panel?

2010-07-30T13:57:00.002-04:00

Why did I get this relatively small 21.5 inch panel (note that I paid 219$ CAD not the 319 it is normally) instead of, say this much larger, comparably priced panel.

Doesn't go into portrait mode.. Doesn't have a display port.. is too big 4 me (24 inches yipes! 3 of those won't fit on my desk) and it's a TN panel..

from
http://www.pchardwarehelp.com/guides/lcd-panel-types.php

"TN (Twisted Nematic) panels are the most widely used panel type in the manufacture of LCD monitors. TN panels are cheap and offer excellent response times, making them perfect for fast paced gaming. The response times of current TN panels range from 2ms to 5ms. However, color reproduction, viewing angles and contrast ratios of TN panels are the worst of any current LCD panel technology. Unlike most 8-bit IPS/VA based panels, TN is only 6-bit and unable to display the full 16.7 million colors available in 24-bit true color. They can mimick the 16.7 million colors of 8-bit panels using a technique called dithering, but the results are unimpressive. TN panels have become popular with the average computer user because they are very inexpensive and currently dominate the LCD display market in availability."

The "ultra" panels from Dell are the latest generation of ISP panels:

"S-IPS/H-IPS (In Plane Switching) panels are generally considered the best overall LCD technology for image quality, color accuracy and viewing angles, but this comes at a price. They are well suited for graphics design and other applications which require accurate and consistent color reproduction. S-IPS panels offer the best viewing angles of any current LCD technology, with wide viewing angles up to 178 degress. The response time of S-IPS is adequate, ranging from 6ms to 16ms with current panels. This is only slightly slower than TN panels. However, gamers should take this into consideration. Fast paced games may suffer from motion blur or ghosting with S-IPS panels that have a response time higher than 8ms."

Wikipedia also has a good article on the topic of panel types:
https://secure.wikimedia.org/wikipedia/en/wiki/TFT_LCD

Dell monitor for cheap U2211H - 100$ off

2010-07-28T12:44:00.002-04:00

Well, in the end I couldn't find a good 4:3 aspect ratio monitor at a price that made sense. Instead I got 3 U2211H monitors. Dell is selling them at a 100$ CAD discount each so that's not bad. They are full 1920x1080 monitors with tilt and swivel bases so you can use them in portrait mode. I'm not sure how long the 100$ deal lasts. Check it out:

<http://accessories.dell.com/sna/products/Displays/productdetail.aspx?c=ca&l=en&cs=cadhs1&sku=320-9271>

Price with discount is $219.00 CAD

My broken motherboard

2010-07-23T21:49:00.010-04:00

I hate bad capacitors.

I got back from vacation, sat down at my computer, hit the "on" switch, all the fans would spin up but then nothing. The screens wouldn't turn on and I couldn't even get access to the BIOS. It was dead!

Don't you hate it when that happens? It could be almost anything. Before leaving on vacation I had added brand-new RAM. I figured it was worth tinkering with RAM to see if that was the problem. No, removing new RAM didn't help.

The next suspect was my power supply. I had already had a computer die in a similar fashion due to a funky power supply. What can happen is the power supply still provides power but doesn't provide it at the right voltage or at a steady voltage. If you stick a voltage meter onto the power supply you may get the right voltage displayed but as soon as you start to draw any amps you can get sudden voltage drops which cause the computer to crash. So I went out to my friendly, neighborhood computer store and got a new power supply. When I got home I hooked everything up turn it on and it still didn't work.

If it wasn't the power supply, the next logical thing would be the motherboard. This filled me with dread. While I often brag to my friends and colleagues that I had built my computers from scratch, in truth the man from the computer store had built it. All I had done was chosen the components. Changing the motherboard wasn't something I looked forward to. If I was going to do this, I might as well make damn sure that it was the motherboard that had died and not something else.

If you take a modern computer and remove everything from the motherboard you should still get a beep when you turn on. Well, you should still get a beep if you hadn't been stupid enough to remove the PC speaker when you'd removed everything else. I had arranged things so that this was impossible as I had never bought a PC speaker for the new PC in the first place. I had an old computer, however, which had a PC speaker. This was convenient. I plugged in the PC speaker (which is quite an adventure in old-school, giant pin array connectors) and turned on the computer to see if I could get it to beep. The PC didn't beep. This made me very unhappy and I decided to go onto the Internet to see if anyone had similar symptoms.

I found a note in one of the forums that someone had a similar issue with their machine and had left it off overnight and turned it on in the morning and it booted up. I also remembered that I had a similar issue with my Macintosh G4 computer and the problem had turned out to be a flat battery. I tried removing and re-setting the battery but that didn't help. So I left it for a few hours while I went out to collect some lunch and go talk to the man at the computer store about motherboard warranty replacements.

After lunch I was quite surprised when I turned on the machine and it beeped at me. After connecting a few more components and restarting the machine it beeped at me again although it sounded distinctly sick. Instead of a short, sharp beep it was just a long, droning tone. This isn't how a computer is supposed to sound when you start it up. I decided to leave it until the morning.

The next day I suspected that I would probably only get one good boot out of the machine, assuming I would do anything at all, so I decided to hook up as many components as I needed to use it and then try to boot. To my shock it actually worked and I managed to get into the BIOS and browser around BIOS menus. It then promptly crashed and all subsequent attempts to reboot it failed with the symptoms I had originally. I couldn't even get it to beep anymore with nothing connected. This was definitely a motherboard issue. A working motherboard should beep when you turn it on!

My old motherboard was a P5B Deluxe. It is super awesome when it's not broken and I wasn't looking forward to trying to find a replacement. Luckily, Asus was making a motherboard that is practically a drop-in replacement except that instead of using three and a half year old technology (older actually) it used state-of-the-art two year old technology. Well actually, this isn't fair. The motherboard uses components that are brand-new and the board was only released last fall. It just uses a design that's based on a chipset that's about two years old. It also works with just about every type of 775 socket CPU and DDR two RAM which is good because that's what I have.

The motherboard has the rather ridiculous name of P5Q Turbo Pro. There's something very interesting in the motherboard's description on the Asus website that might not jump out at you. Here's the list of features for motherboard:

ASUS 8-Phase Power Design
TurboV / Turbo Key
ASUS EPU
ASUS Drive Xpert
100% All High-quality Conductive Polymer Capacitors! (VRM 5000hrs lifespan @105°C, 500,000hrs @65°C)

The important line there is the last one. "100% all high-quality conductive polymer capacitors!". Now, if you want to sell a motherboard to somebody you need to know what the important selling points are. Given that people's attention span is limited you can really only give the top five features of the motherboard before people lose interest. It's very telling that Asus has, as one of its five most important features, that it has "high-quality conductive polymer capacitors". The reason it's very important that this board has high-quality capacitors is because the board I have doesn't. That's the reason it broke. Probably....

It's been called the capacitor plague. Essentially, some Chinese company stole a secret formula for making capacitors from IBM. Unfortunately the formula wasn't finished. If you actually built a capacitor using this formula they would tend to fall apart after about three years. Since computer manufacturers generally don't care where they get capacitors from, or perhaps I should say they *didn't* use the care where they get capacitors from, just about every manufacturer had at least one batch of these bad capacitors in their products. This means that their products died after about three years. Add to this the fact that the standard industry warranty is three years and you have a big mess. Ask Dell.

Did I mention I hate bad capacitors that? Yeah, I'm pretty sure I have.

So anyway, my three and a half year old motherboard (out of warranty) was dead and I needed to get a new one. In the end I bought the Asus P5Q PRO Turbo. It works! My computer is back. I'm quite happy with it. It's essentially my old motherboard but better in lots of little ways. For example, it's more power efficient and faster. It's also not as expensive as the original motherboard. It turned out to be about 50% more expensive than the power supply. I didn't even need to reinstall Windows. How cool is that?

So that's the story of my motherboard. Maybe later I'll tell you the story of what happened to my CPU later that week.

Canadian unemployment rate

2010-06-21T13:00:00.002-04:00

Annoyingly, the old Canadian economy site is gone. I was use it to keep track of what was happening with the unemployment rate. The responsibility appears to now fall to stats-can.

You can quickly find out what the unemployment rate is by going here:

http://www.statcan.gc.ca/subjects-sujets/labour-travail/lfs-epa/lfs-epa-eng.htm

You can even find out what the historical unemployment rate was and it's conveniently presented as a little graph.

I've also blogged about it a few times.

Facebook Privacy Roundup

2010-05-24T14:12:00.003-04:00

If you're trying to follow what this whole facebook-privacy here's what's been going down.

The biggest story is that Facebook has been loosening the default privacy settings for all of the information it contains. Here's a nice visual history:

http://mattmckeon.com/facebook-privacy/

If you're worried about your privacy settings you can use this tool:

http://www.reclaimprivacy.org/

Or you can do it all manually. Facebook settings can be confusing to navigate so you can use these links to jump to all the different privacy panels in Facebook without having to hunt for them one at a time.

A second, unrelated issue is that Facebook is sharing who you are when you click on one of the advertisements that appear in Facebook.

I'm not too worried about advertisers, though because I'm more worried about the amount of information third party Facebook applications get. Even if you trust facebook with your data, do you trust every single facebook application maker?

Bizarrely, Facebook has threatened to sue a researcher that crawled through all the publicly available data on facebook as part of his research. Oh sure, you make the information public but they get annoyed when someone collects it. The researcher has made a map of how people connect to one another to see if there are groups of highly connected people in the US. It turns out there are. It's quite fascinating. It's a good thing he made his crawling public or else facebook would never have known they needed to sue him.

The CEO of Facebook has made some worrying statements about how he thinks that privacy is no longer a social norm. My blogging buddy Guillaume has detected this attitude in Facebook's options to turn off their instant personalization feature.

Then there's the instant personalization feature itself which "personalizes" other web sites with Facebook information. I'm not sure how much it's leaking because I haven't worked through all the security implications yet but it's creeping me out. :-)

Oh, Guillaume's just pointed out this link to me:
Facebook runs afoul of privacy watchdog again
Apparently Facebook's got into legal trouble in Canada over its behaviour... And the fun continues..

Three Monitors are Better Than One

2010-05-24T13:29:00.006-04:00

Well, that'll teach me to procrastinate. Jeff Atwood has recently posted a blog posting that pretty much echoes everything I'm about to say. If you need more convincing then go check it out.

I'd like to talk to you today about computer monitors. The computer monitor is one of the most important things to consider when buying a computer. Most parts on a computer make the computer faster or able to load bigger programs but a computer screen actually goes beyond that. A computer screen actually dictates how you interact with your computer. If you have a very large screen you can see more information at the same time so there's less scrolling. If you've ever had a large, wide screen and work with spreadsheets you know how positively awesome this can be. It can also allow you to track multiple things at the same time. As a programmer I often have to have my editor, the program I'm working on and the log file that program all open at one time. Big screens let you get more work done.

With modern PCs and their modern graphics cards you can usually attach at least two monitors to your machine. Two monitors really are double the fun. What tends to happen is one of these screens becomes the main screen and the other becomes the auxiliary screen. Two screen can often be even better than 1 large screen because the two screens natural segment your workspace. You put your e-mail, tool palettes, log files, desktop widgets, Skype and all that sort of thing on a second monitor and keep the first one for working with. Two, monitors works great but you know what works better than two monitors? Three monitors.

Three monitors have recently become a real possibility thanks to the new series of ATI cards which all out you to attach three monitors. Here's the list that I know have three monitor output.

* Radeon 5670 (~$120)
* Radeon 5770 (~$180)
* Radeon 5830 (~$230)
* Radeon 5850 (~$310)
* Radeon 5870 (~$410)

I personally have the Radeon 5770. It has a respectable 3-D performance while drawing only moderate amounts of power and at a good price too. If you're only going for three monitors though you can use the cheapest since the cheapest card in a modern series will handily beat whatever graphics card you have when it comes to 3-D performance unless you're one of these guys who upgrades their video card every year.

Currently I'm using three old 19 inch CRT screens with my card. The Radeon 5770 has two DVI outputs and one display port output. It also has an HDMI output but in order to use three screens you need to use the display port output. Since my old CRTs are all VGA I needed to get a few DVI converters. It was surprisingly easy because every video card I card I every motherboard I bought as come with a DVI to VGA converter. The display port is more of a problem. Display port seems to be more complicated than DVI and so converting between its VGA or DVI requires a pretty fancy converter. Even with one of these fancy converters it's not guaranteed to work. ATI has published a list of dongles that it knows work.

http://support.amd.com/us/eyefinity/Pages/eyefinity-dongles.aspx

I didn't know about that list at the time and bought an adapter by StarTech. It works in the sense that it will display a picture on third screen but it doesn't work in the sense that the picture will wobble about once a minute and every once in a while I'll have to unplug and re-plug adapter to get the video card to recognize it. It also won't drop into the special eyefinity gaming mode most of the time.

The CRTs are starting to get a bit old. No, scratch that the CRTs are very old and in desperate need of being replaced. I've been trying to find a monitor with a display port that I could buy three of. Annoyingly every manufacturer seems to be selling the exact same monitor in the exact same configurations. They are all 20 to 24 inch and come in wide screen format of 1980X1080 and 1600X900. The only exception is that there's a 24 inch screen with a resolution of 1980X1200.

I've never tried three widescreen displays. I would expect that it's probably a bit too wide. Three normal aspect ratio screens arranged horizontally work quite well because they fill in your peripheral vision. With three wide screens I suspect it would just be overkill. The only alternative is to get an older screen that doesn't have a display port on it and then get an expensive, hard to find adapter. This is what Jeff Atwood did. If I was to do this I already know which one I would get. It would probably be an UltraSharp 2007FP 20-inch. This is the kind of monitor I use at work where I have two of them. I have found them to be quite good. The high-resolution, small physical size and ability to set them up in portrait mode make them ideal for programming.

I think that most manufacturers have stopped designing new 4:3 aspect ratio monitors. They work very nicely in three monitor configurations but they also work very nicely in portrait mode too. As someone who works on long documents all the time viewing them in absolutely fantastic. Widescreen monitors used in portrait mode are, I suspect, a bit ridiculous.

Oh well, I'm going to actually go out and buy three monitors some point soon. One of my CRTs has just broken. If I leave it on too long the image would grow really large and turn purple. I've been keeping it turned off because at this point I'm scared it will catch fire or explode or something. Another monitor makes a buzzing noise whenever I display certain webpages. I'll be sure to tell you what I eventually bought and how it's worked out for me. Until then see you. Bye.

Why you should get a high end PC - RAM

2010-04-15T20:58:00.009-04:00

This is part 2. Part 1 is here.

RAM is good. The more RAM the better.

RAM is so important that most of the time we just call it memory. There's lots of different types of memory attached to modern computers. There's hard disk, flash, buffer and cache memories to choose from. The thing is, when you just say “memory” and don't bother to qualify it, you mean RAM. That's how important it is. It doesn't even need the word “the” in front. It's implicit. That's how friggin' important it is.

The main benefits of having more memory is that allows you to use applications that manipulate huge chunks of data all at once.. Programs like this include games, photo editing applications, video editing applications and medical imaging viewers.

Generally you don't need to make the decision to buy more RAM. The applications you're using will do it for you. Essentially, one day you'll see a new, shiny application available for download that you just have to have. You'll download the application, try to run it and and suddenly realize you need more memory.

Windows 7 is the first consumer 64-bit OS from Microsoft that you'd actually want to run and unlike windows XP, it's not limited to 2 (ish) gigs of memory. Expect software developers to take advantage of the new memory situation by making their apps take up more memory (or should I say, "make their apps do something cool". No. No, I shouldn't.)

Curiously, modern operating systems don't just give up or complain when they run out of memory. Instead they will simply cram as much into memory as possible and stick the rest in the pretender-to-the-throne-memory – hard disk. In this case, you know you've run out of memory when your hard drive starts going bonkers and your PC slows down to a crawl. If you're not saving something or loading something your hard drive shouldn't be doing anything. If it is, you may need more memory.

Having more RAM than you need for the applications you run can also speed up the computer. If you have more RAM available than your computer is actually using, the machine will allocate the extra RAM to a disk cache. This is good because it means that even when you're loading something, it won't read it from the hard disk it will just use the copy already in memory. As a result, it loads incredibly fast. In my first article about storage I mentioned that before my solid state drive I used to get more RAM than I really needed. The disk cache is why. If you're too cheap to get an SSD at least max out your system memory. A run of the mill 7200RPM consumer grade hard drive will see huge gains. With a laptop's slow 4200RPM hard disk it's even more important.

(Cute girls know your laptop needs more RAM)

Note that caches work by keeping a copy of the data in memory after it's been read the first time from a hard disk. PCs often read the same things off the hard drive repeatedly due to the way programmers build their applications. The net result of this is that having a disk cache can make a surprising number of things faster.

One last thing, make sure you get your RAM from a reputable brand. Cheap, no name memory may seem like a good deal but there are quite a few bad RAM chips out there. Bad RAM can be a huge pain to diagnose since it just shows up as random crashing - which could be symptoms of almost any problem. A few good brands are Corsair, Kensington and OCZ. There are others..

So, in conclusion: RAM is good. The more RAM the better.

.. and, might I add: get more.

Next up: monitors. Why only having one is so old school.

Part 1

Why you should get a high end PC - storage

2010-04-11T20:44:00.007-04:00

I've noticed a recent article that said that Apple has a 90% share in the computer is over a thousand dollars category. I think this is rather sad.

I don't think it's sad that Apple is doing well, this is actually quite nice. What I find sad is that people aren't buying PCs over a thousand dollars. There are some excellent reasons why you'd want a PCs over a thousand dollars.

Let's start with the hard drive. Most PCs come with a relatively small hard drive. This is silly because coming with a small hard drive means you're actually paying a lot per megabyte. It also means your PCs that run very slowly because smaller hard drives have a smaller information density which means the drive has to spin faster to achieve the same data transfer rates. Every generation there's an optimum price point for megabytes per dollar. It's not that expensive either. Last time I check this optimal point was for hard drives costing about 100$. Getting a hard drive outside this range is just throwing money away in my opinion because you always need more hard drive space.

Hard drives are otherwise my mortal enemy. Hard drives have gotten tremendously large but they haven't gotten that much faster. While the transfer rate of the drive tends to be proportional to the size of the drive the access time is proportional to the rotation rate, the speed the head can move and physical size platter size of the drive. These factors haven't really changed. The rotational speed has only gone from 4200RPM to 7200RPM but the drive size has gone from 20 MB to being 1 TB in size. That's 1 000 000 MBs! Every time the system has to retrieve a byte from the hard disk it has to wait an eternity. I hate the stupid things! I've even gone so far as to add much more RAM to my system than usual to have a huge disk cache so my machine doesn't need to access the hard disk. Have you considered a solid state drive recently?

I've recently bought an OCZ Vertex solid-state drive for use with Windows 7 on my machine and it is completely awesome. To put the speed difference in perspective consider this: good hard drives have access times of around 18 milliseconds. Good solid-state drives have an access time of about an 0.18 of a millisecond. That's a hundred times faster. Additionally, they can have transfer rates of over 200 MB a second. Hard drives have a transfer rate of 50 MB a second. In practice the performance is extremely noticeable. I'll say it again, they are super awesome.

They only downside to a solid state drive is that it doesn't hold a great deal of data for the price. My OCZ Vertex cost around 400$ and only holds 120 GB. This isn't as much of a problem as you might think though because you just have to put all your data – music, video other data, on a standard hard drive and use your SSD for windows, the swap file and most of the applications. Your system will still fly and you can store all the data you want. I recommend getting a drive slightly bigger then you think you'll need. Getting too much space is a bit embarrassing, getting too little space is a time consuming disaster.

If you've bought a netbook with a solid-state drive and haven't really been impressed by the performance we're not really talking about the same things here.

(The difference in speed amongst SSDs is huge. It's worth reading up on why that is and which drives are really worth buying.)

The technology has come along way since those early drives. The software has also been improved. Windows XP will fight with a solid-state drive but Windows 7 includes optimizations to maximize the performance you get out of the drive. This means you don't have problems like, for example, the performance of the solid-state drive degrade over time. Modern drives also don't freeze the entire computer whenever you write lots of pieces of data to them at the same time. Early and current netbooks sometimes use out of date drive controllers that still have this problem.

Another thing that useful on the higher end PC is more memory. But I'll talk about that next time.

Part 2

Space Smilies - a technological demonstration in one lvl

2010-04-08T20:52:00.008-04:00

In 1999 I was first introduced to Java. I quite liked it. It was the first language that I'd come across in which it was possible to write your own draw routines while simultaneously being comprehensible. Sure, I already knew C and to a lesser extent C++ but to me it had always seemed like doing anything in those languages was incredibly painful. In Java it was relatively easy. The best thing about Java, though, was its runtime and draw routines were fast enough to write a small video game in. That March break I decided to write a video game. I called it Space Smilies after the placeholder smiley face icons I was using at the time.

Ever since then I've occasionally revisited code to see if I can figure out, using my newly acquired X years of experience, why somethings never quite worked right. I also spent some time to try and take advantage of the new abilities of the latest Java virtual machine (not to mention the processor power of the newest machines). I've just completed my latest batch of fixes and enhancements and am ready to release them to the world. Let me give you a rundown of what's been enhanced recently.

First off, I've corrected all those dang threading bugs. While I understood what threads were I wrote the original application I didn't really understand all the different ways in which you can screw up using threads. Since then I've learned a great deal about threads and, in fact, have become quite familiar with them. As a result of this, the newest version of Space Smilies draws the back buffer to the component on the event thread. It used to draw back buffer to the component on the thread I was using for the game engine. This doesn't really work well because AWT and swing aren't thread safe.

I'm also using a sort of triple buffering. The game engine draws to a back buffer. This back buffer is then given to the event thread on a sort of back buffer queue. The event thread then takes the latest back buffer and draws it to the component. After drawing to the component the back buffer is put into a free pool. When the game engine needs a new back buffer for the next frame, it looks in the free pool to see if one is available. The upshot of all this is that the final blitting is done on a different thread than the game engine which is doing the compositing.

Additionally, I'm using volatile buffers wherever possible to let the video card do most of the blitting and compositing. For whatever reason I can't get bit masked volatile buffers to be accelerated. This means that the bulk of the compositing is done on the CPU although the star-field (which is the biggest blob of pixels by far) is composited all on the video card.

(This is all on windows, your millage may vary on other operating systems)

Finally, I got the resolution of the game up from postage stamp sized to 1000 x 700. The old game used to remain fixed at that resolution but the new game will scale itself to take up as much area on the screen as it can.

I'm quite interested to know whether it works for everyone. You'll need Java 6 though. With any luck you actually have that but don't know it. Try it out either way.

GO PLAY THE GAME NOW!

Is it a bug or a feature?

2010-03-22T20:59:00.011-04:00

If you do development for long enough large enough team you will find yourself asking this question. It matters because you need to make decisions about what gets fixed in the current release and what only gets fixed in the next release. In other words, what should be part of version 1.0.1 and what should in version 2.0. Additionally, there is also a school of thought which says that bugs should always be fixed before new features are added. Which is great, but this assumes you actually know what a bug is.

So what is a bug and how does it differ from a feature?

What is a bug depends on who you talk to. To a developer, a bug is a mistake. To tell a developer that there is a bug in their code is to tell the developer that he be screwed up. This can lead to all sorts of unfortunate confusion between end users, QA and the developer. In reality no one (except possibly the developer) actually cares whether or not it's a particular individual's a mistake. What people care about is whether or not the program is working in a way that makes sense. For example, if an application gets out of large knife and stabs the user in the eye when they select "Copy" from the "Edit" menu it doesn't really matter if the developer intentionally programmed it to do that. What matters is the behavior is undesirable. Well, I presume the behavior is undesirable. You might actually be trying to create an application that stabs the user in the eye whenever they select Copy from the Edit menu. I don't presume to know what crazy project you're actually working on.

QA, for instance, will tend to think of any stupid behavior as a bug. For example, let's say that QA files a bug that the font sizes to o small to be readable. The developer takes a look at what font size should be, double checks to see that he's using the right size and then simply claim that it's not a bug. Presumably he then goes on to stab QA in the eye.

Sale and marketing are even more out there. They have no idea how anything actually works and as a result can consider six man years worth of work to be a bug. For example, if the email client you're building doesn't do SMTP autentication then that's a bug. It's a bug because it doesn't work with this particular SMTP server. I mean come on! What were you thinking?

At Intelerad, we've gotten fed up with discussions on whether something is a bug or a feature and started thinking about everything as change requests. Once you're looking at everything as a bunch of change requests you can start focusing on the things that actually matter about those change requests and use that criteria to make decisions.

What are we trying to encapsulate when we ask if something is a bug or feature? It may matter because you made a policy that says that bugs can get fixed on existing branches or releases would features will always be fixed along the trunk. Okay, but why did you make the decision that only bugs get fixed along batches? Did you make the assumption that bugs would simply be things that were easy and quick to fix? If that's the case then why don't you say that only change requests that are easy and quick to fix get fixed along branches. Did you make the assumption that bugs would be severe; that they would stop the user from being able to use some feature of the application? If that's the case then why don't you say that only change requests that affect critical functionality will get fixed along branches. Did you make the assumption that feature work would be too destabilizing along the stable branch? If that's the case then why don't you say that only change requests that are not destabilizing will get fixed along branches. Don't say that he will only fix bugs be precise about which aspects of the change requests will allow you to fix an issue along an existing branch.

Here are the things we typically consider:

How bad is the problem?
Would the fix destabilize the old version?
How long does it take to fix?

The first two questions are the most important. The last question is more of a sanity check. If the answer to the last question is anything more than about a day, then you're probably lying to yourself about it not destabilizing anything. It should also go on the trunk because it represents significant development.

I'm making the assumption that you've already discovered that you should be working along the trunk by default. Ideally you shouldn't be back porting anything. You should be releasing major versions often enough that your clients (or users) can wait until the next major version. If they can't wait until the next version you're not releasing often enough. That's pretty much a golden rule. The length of your release cycles should be matched with the expectations and demands of your client base.

The speed size and dependability of a reddit traffic spike

2009-06-11T20:26:00.008-04:00

Humm.. It looks like my blog posting on progress bars has taken off on reddit. I actually posted the article to reddit after a friend's blog posting had hit the big time on that site as well. although it didn't work.. yeah..

He had just written a blog posting comparing the speed, size and dependability of almost all programming languages using a novel graphing technique. He is a computer languages guy and ever since reading tufte he's been into using graphs to represent data in new and enlightening ways. I thought the blog posting was impressive. Apparently somebody else did too and posted it to reddit. From there it took on a life of its own and found its way to multiple sites including the venerable Slashdot.

I've always loved how that happens. I used to write a peer-to-peer application called Myster. When introducing Myster to the world, we didn't do much advertising. The only thing we did was to post it to one or two Macintosh oriented websites. One of which was the venerable MacInTouch. Oh wow. I woke up the morning that it was posted to a set of crashed computers. Apparently my p2p network wasn't as scalable as I thought.

After frantically fixing the accidental thread bomb (and about a dozen scalability issues for which the application's new found fame turned out to be very useful) I started to track where Myster was being mentioned. The answer, it turned out, was everywhere. It was showing up on news sites I'd never heard of. It was in application repositories I'd never added it to. Myster had been available for about a year and had never really gotten anywhere. Then, one morning we posted it to the right place and it explodes.

I quickly realized that in order to have any control over how my application was going to be presented I needed to move quickly or else somebody would do it for me. This, being an important task, was delegated to a neighbor of mine who turned out to be very good at it. :-) He is now a bike courier. Completely wasted talent if you ask me. He did the website for Myster as well is all the graphics. Just look at the logo. It's gorgeous! I mean argh! irk! egh! Anyway, enough of that.

The speed that news travels on the Internet is impressive. It's scary how fast you can go from "Oh, someone has posted my article on reddit " to "Oh crap, how much is all this bandwidth going to cost me?".

My friend had gotten his page to the financial-panic level of Internet popularity and I was curious to know if I could do the same with my article on progress bars. So, I dutifully signed up for a brand, spanking new account on reddit and tried submitting my article. It didn't get much love.

Actually, it didn't get any love. It got exactly zero votes. This was not the reception I was hoping for but I figured it was probably just the content. I mean who would want to read a 5000 ish word article on the intricacies of progress bars anyway?

My friend said I was completely wrong.

In his opinion my reddit submission title completely sucked. He was sure he could do better. No, that doesn't accurately reflect his scorn for my reddit abilities. He was sure he could do better with one hand tied behind his back and a family of monkeys beating his head with old IBM clickety keyboards. But first he had to wait a few days because reddit doesn't like duplicate submissions...

Well, today he took a stab at it and last time I checked it was top of the heap.. so I guess he's right. I have my doubts as to whether the monkeys were using real, genuine, old-school IBM clickety keyboards but I must admit it's still impressive.

Just for the record, I'm also still glad I delegated Myster's PR fanfare to that neighbor-friend of mine. Perhaps he should start a blog? I'd read it and I promise not to submit it to reddit. :-)

How to work with progress bars, part 2

2009-05-31T11:43:00.022-04:00

On the last episode of "As Andrew Rambles About Progress Bars" (or whatever I'm calling this series) I raised the following questions:

What if, you have a task for which it is completely impossible to gauge how long the task will take?

What if, you have a task for which you have a rough estimate for how long it will take but, you have no way to increment the progress bar because the library/process/whatever doesn't provide any feedback. For example, maybe you're running a third-party process and there's no way for you to get any feedback about what that process is doing. Now would you do?

What if there's a bomb on a bus. Once the bus goes 50 miles an hour, the bomb is armed. If it drops below 50, it blows up. What do you do? What do you do?

I have answers to these questions. Well, all questions except the last one. But first: task estimation.

In the comments of my last article, a friend of mine pointed out that I never actually mentioned how you come up with the numbers you pass to compositeProgressBar.buildSubProgressBar(). Typically, all you need to do is run your program a few times and time how long each sub task takes. You can then figure out how long the task is spending in each one of the subtasks. You then use that to figure out what to send to compositeProgressBar.buildSubProgressBar().

For example:


sub task | time in seconds | value to send to compositeProgressBar.buildSubProgressBar()
1        | 10              | 10 / 45 = 0.222
2        | 20              | 20 / 45 = 0.444
3        | 15              | 15 / 45 = 0.333

Total time: 45 seconds

And there you go. I would suggest doing more than one timing with many sets of data.

Sometimes the ratios don't stay fixed and instead vary depending on what sort of machine you're running on or what set of data you have.

The programmers I've talked to are always concerned about giving inaccurate information in a progress bar. They've all experienced the frustration of an inaccurate progress bar. It's important to not fall into the trap of thinking that the progress bar needs to be super accurate. At the same time, you need to be careful you don't make the classic error of having a progress bar that jumps to 90% done right away and stays there forever. It can be a delicate balancing act but in my experience, programmers often underestimate the value and accuracy of the information they have.

Consider the following: as a programmer, you almost always know approximately how long the task will take to complete to complete. Put yourself in the user's shoes. The user has absolutely no clue whatsoever how long the task will take to complete. As far as the user is concerned it can take a second, an hour, a day or a millions years. If you know the task will take, at most, one minute, then you already know far more than your user. Communicate this very valuable information to them.

Before we continue remember that there is no solution to determining how long a task will take in the general case. To solve that you would need to solve the the halting problem. That's not going to happen. Instead I can show you a few tricks I've learned about displaying a progress bar in circumstances where you wouldn't think it would be possible.

First question: what is the probability that the task will finish at any given time?

If the probability graph looks like this:

That's good. It's an easy candidate for progress bar because the progress bar tends to finish in one range of values of time.

If you have a task made up of subtasks, it's also makes things easy if the time taken by each sub task stays in a fixed ratio. That is, if one sub task takes 1 second then you know that the next sub task will take approximately 1.5 seconds and the sub task after that will always be a little shorter. In this case, let's say half a second.

If each of the subtasks doesn't stay in a fixed ratio than that's a bummer. When estimating the time a task will take, it's good to consider whether it's likely your task is network bound, disk bound, CPU bound or other. This is important because if you have a large task that consists of, say, a network bound task and a CPU bound task then it's going to be difficult to give an accurate (at least with respect to time) progress bar. On one computer, you may have a very fast CPU with a very slow network connection and on another computer the situation may be reversed. This will cause the progress bar to zip through one section of the meta- task and then crawl through the other section.

The situation above are fairly rare. Most tasks tend to use some combination of the resources of a machine and as a result the differences in the speed of various components tend to cancel out so that the ratios between the different subtasks tend to remain constant (to some reasonable approximation).

If you have run your progress bar a few times and are not happy with its accuracy you have two choices. 1) Run a calibration loop or otherwise figure out how to compensate for these things on that particular machine or 2) live with the fact your progress bar isn't going to be super accurate.

There's something of a black art to writing calibration loops. You have to be careful of things like caches and you have to make sure that you add the right fudge factors to make the progress bar behave correctly. It's surprisingly time-consuming to do this right and most times, in my experience anyway, it's not worth the effort. If you find yourself having to do one of these then it is going to take you a while.

A more common problem is that a program might have no way of getting any feedback as to the progress of one of its subtasks. This can happen if you have to hand off a task to a library.

If you actually do know, approximately, how long that task will take to complete in real time then you still have a chance. You may know that it will take approximately 2 seconds for every item that it's processing. Or you may be able to guess at how long the task will take given the time it took to complete some previous processing step. For example, if it took 5 seconds to complete the first step, it should only take about 10 seconds to complete the second step.

In these situations I like to use a technique that I call Zeno's progress bar. It's a way of providing a progress bar when you have no idea what the real progress is. It's terrifyingly convincing.

Before I continue I would just like to say that I am serious when I say it's terrifyingly convincing. I've actually been duped by my own Zeno's progress bar at least once.

While using the product I work on (Inteleviewer) one day, I noticed that it was providing a progress bar. However, I was under the impression that this was impossible because the task it was using a library that had no way of providing any feedback. I was curious to see if someone on the team had fixed the library or had found some other clever trick to get the library to provide feedback. Someone had found a trick. It was me! It was my own Zeno's progress bar! I had made this error knowing that there were these sorts of progress bars lurking in the code. It was then I knew I had to share this idea with the world.

I came up with the idea based on watching how people would provide their time estimates for tasks that they weren't being informed on the progress of but had some idea of how long it should take.

An example would be an online merchant that knows that something is supposed to ship within two weeks but since it's being shipped by the manufacturer of the product, they really have no good idea of the order's real status. They will tell you it will ship in two weeks and if it doesn't then they will tell you it will ship next week and if it doesn't then they will tell you will ship in the next few days. It still hasn't shipped they will always keep repeating that it will ship in the next few days. Only at this point you start to realize that it may never ship at all.

If you think about it you can probably come up with other examples of this sort.

The technique works really well when your estimate is off slightly. It works even better when your estimate is dead on. This is what it looks like in a progress bar:

Let's say you have a task that could take 10 seconds. You start off with the progress bar moving at a speed that is consistent with the task taking 10 seconds. At the 5 second mark you slow the speed of the progress bar by half. So that by the 10 second mark the progress bar will only be at 75% complete. If all goes well, and your estimate of 10 seconds is correct, then the progress bar will just jump to 100% at this point. If it doesn't and your task is going to take longer then you slow the speed of the progress bar by half again. By the 15 second mark the progress bar will now be at 87.5% complete. This goes on until your task completes.

Obviously, you should try to avoid giving overly optimistic estimates. Being completely off results of one of those annoying 99% complete progress bar that we all love to hate. Being off by only a factor of two, however, actually still gives a half decent effect.

Okay, so how do you code this puppy?

Well, the first step is to create an object to update the progress bar without any help from the task itself. You see, usually the task would tell the progress bar to update from its own thread. In this case it won't so we'll have to reproduce the effect using a timer. It looks like this:


/**
* Makes a progress bar that can show progress if all you know is approximately
* how long the task will take.
*/
public class MagicProgressBar {
  private static double BASE = 0.5;

  private final Progress  progress;
  private final int       estimatedTime;
  private final Timer     timer;

  private long            startTime;

  public MagicProgressBar(Progress progressBar, int estimatedTimeOfTask) {
      this.progress = progressBar;
      this.estimatedTime = estimatedTimeOfTask;
      timer = new Timer(30, new ActionListener() {
          public void actionPerformed(ActionEvent e) {
              //progress.advance( magic??? );
          }
      });
  }

  public void start() {
      startTime = System.currentTimeMillis();
      timer.start();
  }

  public void stop() {
      timer.stop();
  }
}

You would use it like this:


private static void doMagic(Progress masterProgress) {
  MagicProgressBar magicBar = new MagicProgressBar(masterProgress, TIME_ESTIMATE);

  magicBar.start();
  doStuff(); // doesn't provide progress
  magicBar.stop();

  masterProgress.advance(1);
}

Where "doStuff()" is the task were waiting to complete. Note that TIME_ESTIMATE is not usually a constant. In many cases it would be calculated by using a weighting number, like we sent to compositeProgressBar.buildSubProgressBar(), and the time it's taken to do some earlier step in the task.

Well, we still need to add the implementation of the timer in the MagicProgressBar class. Let's do that now.


timer = new Timer(30, new ActionListener() {
  public void actionPerformed(ActionEvent e) {
      long timeSinceStart = System.currentTimeMillis() - startTime;
      int unitLengthInMillis = estimatedTime / 2;
      int units =  (int) (timeSinceStart / unitLengthInMillis);
   
      double percent = ( timeSinceStart % unitLengthInMillis ) / (double) unitLengthInMillis;
      progress.advance(findSum(units) + percent * Math.pow(BASE, units + 1) );
  }
});

private static double findSum(int n) {
  return (Math.pow(BASE, n + 1) - BASE) / (BASE - 1);
}

"findSum()" is calculating how far complete the progress bar should be given at a given time unit index. In our example, where we expect the task to take 10 seconds, unit index 1 would be the first 5 seconds, unit 2 would be the next 5 seconds, unit 3 would be the 5 seconds after that etc... a unit's length is equal to one half of the estimated time of the task.

The rest of the code is either trying to figure out which time unit we're at or how far through the current time unit we are.

Let's see what the object looks like when we put it all together:


/**
* Makes a progress bar that can show progress if all you know is approximately
* how long the task will take.
*/
public class MagicProgressBar {
  private static double BASE = 0.5;

  private final Progress  progress;
  private final int       estimatedTime;
  private final Timer     timer;

  private long            startTime;

  public MagicProgressBar(Progress progressBar, int estimatedTimeOfTask) {
      this.progress = progressBar;
      this.estimatedTime = estimatedTimeOfTask;
      timer = new Timer(30, new ActionListener() {
          public void actionPerformed(ActionEvent e) {
              long timeSinceStart = System.currentTimeMillis() - startTime;
              int unitLengthInMillis = estimatedTime / 2;
              int units =  (int) (timeSinceStart / unitLengthInMillis);
           
              double percent = ( timeSinceStart % unitLengthInMillis ) / (double) unitLengthInMillis;
              progress.advance(findSum(units) + percent * Math.pow(BASE, units + 1) );
          }
      });
  }

  public void start() {
      startTime = System.currentTimeMillis();
      timer.start();
  }

  public void stop() {
      timer.stop();
  }

  private static double findSu5m(int n) {
      return (Math.pow(BASE, n + 1) - BASE) / (BASE - 1);
  }
}

Now you have the power of Zeno's progress bar. Use it wisely, young grasshopper. Zeno's progress bar has no mind. Isn't it wiser to seek proper progress reporting then to desire the swift completion of the programming exercise? Oops, I'm channeling David Carradine again. All I'm trying to say is don't use it for the hell of it. That's all.

Okay, on to the next part which is all about indeterminate progress bars.

This is what an indeterminate progress bar looks like:

The one of the left is used on the Macintosh. The one on the right is used by Windows.

Indeterminate progress bars are used to signal that we have absolutely no idea how long a task is going to take. Both the Macintosh and Windows version of this type of progress bar use animation to signal that, yes, there really is something happening. The computer has not fallen asleep. It has not started daydreaming. It hasn't quit and joined a hippie commune where it's not asked to do tasks that are difficult to estimate tasks all day.

If you really, really don't know how long the task is going to take then this is the progress bar to use.

The decision to use an indeterminate progress bar is a hard one. Most of the time you'll want to avoid using it. Users don't like being in the dark about how long their computer might be occupied. Going back to my online merchant example above, this would be the equivalent of an indeterminate progress bar:

Customer: How long is it going to take the ship?
Merchant: I don't know.
Customer: Is it going to take about two weeks? Or two days? Or two years?
Merchant: I don't know.
Customer: Are you saying it may take two years for me to get my package?
Merchant: It could. I don't think that's very likely though. Well, I don't know, it could take two years. Yes. Maybe.

Despite the fact that indeterminate progress bars are annoyingly, I can think of two common uses for them..

The first use is for the relatively short-lived progress bar that shows up because the computer is taking a little longer than it should to do something and the programmer thought that he should provide some obvious feedback that the computer hasn't crashed. These indeterminate progress bars are often short-lived. Most the time the programmers didn't even think it would be shown at all. I haven't seen one of these types of progress bars in a while. In this case the indeterminate progress bar means "hang on a second".

Another common use for the indeterminate progress bar when it's used at the beginning of a long task when the computer is still calculating how long it will take for it to do the task. For example, if you're running a calibration loop at the beginning of your process, you might show an indeterminate progress. Another example would be the OS might show an indeterminate progress when it's calculating the number of files involved in a file copy operation. It's also often used at the beginning of a task when trying to connect to a remote server. Connecting to a remote server is one of these things that will either be instantaneous or take a while due to the server being down or a network error.

In these examples you go from an indeterminate progress bar to a regular progress bar. This sort of transition is okay. You're essentially saying to the user "I have no idea how long this is going to take" because you're trying to figure out how long the task is going to take. So long as this step is relatively short it's ok.

Going from a regular progress bar to an indeterminate progress bar is less ok. It's like saying you know how long a task is going to take then saying you have no idea then claiming you know again.

I have seen this behavior when it's used to mean "hang on a second, I've got an unexpected delay". A task might display this when it tries to connect to a server that's taking a while to respond. I've also seen it during a compression task that would occasionally start an optimization step. If the optimization step was taking too long the progress bar would become indeterminate for a while.

Progress bars that go from a determinant progress to an indeterminate and back are very rare. It's probably not a pattern you'll ever need to use.

Well, that's all I have to say. If you want to read more about progress bars (and who doesn't.. pfff..) then I'd suggest visiting Jeff Atwood's blog post on the subject. It's a great starting off point. He also goes into how to make your task look like it ran faster without actually making the task run any faster.

You can also check out the miscellaneous human interface guidelines documents by Microsoft and Apple. Perhaps you want to expand your horizons into other kinds of progress indicators.

Oh, and here's the source code for all the examples I've given.

Until next time bye.

Cross Canada Bike Trip

2009-05-28T11:22:00.000-04:00

Well, my parents have officially left for their cross canada bike tour. The tour is to raise money for the JDRF. They are making a blog of their trip. They will posting as they find wi-fi spots. I wish them luck.

Canadian Unemployment Rate

2009-05-18T13:44:00.003-04:00

Hello everyone. I thought I'd give an update to what's been happening to the unemployment rate since my last post.

The unemployment rate for this month is 8%. That's pretty high but not completely ridiculous. The last time the unemployment rate was at 8% was January 2002. The highest unemployment rate on my graph is 8.2% during April 1999.

It's nice to know that the unemployment rate is not increasing anymore. If you look at the graph you can see how the rise in the unemployment rate is unprecedented over the last ten years or so.

It's a pity I don't have the unemployment rate graph going back longer than 1999. It would be nice to see how this latest recession compares historically to others.

CRTC feedback

2009-04-20T20:59:00.003-04:00

Recently a local Internet service provider called TekSavvy e-mailed its clients about the fact that Bell was asking the CRTC to allow a to put a cap of 60 GB per month on its wholesale DSL.

Bell sells DSL services to Internet service providers. Essentially, it sells the link between your house and Internet service provider. The Internet service provider is responsible for linking you to the rest of the Internet. Bell is already throttling everyone claiming that it's trying to control congestion on the network. I am mighty suspicious of this claim.

This is what I sent to the CRTC:

It's important to foster competition between ISP in order to make sure that consumers can have a wide selection of billing plans / possibilities available so they can make the best decision for them. Bell should not be allowed to set monthly caps per user. The decision to do this should be up to the internet service provider. Mandating this destroys all sorts of potential pricing structures.

Pricing for wholesale DSL rate should be based on the scarcity of the resource in question. Capping bandwidth per user is an attempt to deal with bandwidth issues. If it's the number of megs per second causing the slowdown then it makes sense to bill wholesale DSL service based on that. It doesn't make sense to bill on that *and* some other thing, however.

If we are to have DSL service whole selling then it must be considered to be part of the spec that the DSL network be data neutral (no preferential throttling speedup or slowdown based on the data flying over the network; pretend it's encrypted) and user neutral (rate doesn't change based on which account I use or how much any account has used). This configuration sends the right supply/demand signals back to the ISP.

Consider that I don't use Sympatico or any other DSL ISP because they violate network neutrality by throttling certain specific protocols. If Sympatico continued to throttle certain protocols but other DSL ISPs did not I would have an alternative ISP to go to. This despite the fact that that other, none throttling ISP may have a badwidth cap in place at, say 30 or 40 gigs per month. The third party ISP in this example chose an different approach to dealing with their bandwidth issue.... The argument works for many other potential plans. Currently I'm on a plan with that limits maximum bandwidth speed to a relatively modest level but also provides a large download cap for a relatively high price. This sort of trade off is possible with the simple system mentioned above.

I would've kept going but there was a 2000 character limite and so I had to stop there. Here there was some of the stuff that was cut out.

Consider this:

By putting all of its throttling on the *DSL* half of the link it *guarantees* that ISPs won't be able to find innovative solutions to the bandwidth crunch. Given that there are many independent ISPs and only one Bell ISP (Sympatico) it's likely that if an innovative (and counter intuitive) approach to managing the bandwidth crunch were to be developed, it would be developed by a third party DSL ISP. This would essentially mean that sympatico would be vulnerable.

What bandwidth shortage?

How to work with progress bars, part 1

2009-04-20T20:55:00.005-04:00

Have you ever seen a progress bar that goes from empty to full then starts over at the beginning again? I really hate those.

I had always assumed that progress bars were pretty uncontroversial. It wasn't until I worked at Intelerad for about a year that I found out that many people don't actually understand what progress bars are for, how they should work or how to write code to implement them.

When you present a progress bar to the user you're telling them:

1) That the computer is doing something that'll take a while.
2) That the computer hasn't frozen or otherwise become unresponsive.
3) Approximately how long the computer will be busy.

It's important to remember that a progress bar is not to display the progress some arbitrary chunk of code. If you think of a progress bar showing the progress through some function or chunk of work, you're likely to show multiple progress bars in a row where each filling of the progress bars will represent the progress through a particular task. Don't do this.

What the user actually wants to know is how long the computer will remain busy doing its thing. Alternatively, how long it will be until the computer returns a result. When you've filled up a progress bar and restarted it, you're messing with the user's head.

(Don't mess with the user's head)

A common worry when creating code that has a progress bar is that if you show progress bar for the entire time the computer is busy, and not for individual subtasks, then that code will not be reusable. It won't be reusable because you'll have to hard code the values you're setting the progressbar to in that function. As a result you won't be able to use that code with, say, another progressbar because the values that you need to set the progress bar to in that context will be different. 80% done for this progress bar might be only 40% done in a different context.

It's actually very easy to create little subroutines that only worry about the progress though their portion of the task and then wrap those progress meters into meta-progress meters to show the progress through any larger task. Here's how you do it.

Progress bar-fu:
(n.: the ancient Japanese art of making progress bars that don't jerk the end-user around)

Think of the progress through the overall task as being the sum of the progress through each individual task. Looking at the problem this way it should become apparent that each sub task can look at its own progress as going from 0 to 100% with this value being a smaller proportion of the progress of the overall task.

Let's look at an example. Let's say we have an overall task composed of three subtasks. The first sub task is 40% of the overall task. The second sub task is 50% of the overall task. The last task is 10% of the overall task. Drawing this out your great little chart like the following:

Here we can see that the first subtask's progress, as a value that goes from 0% to 100%, is equal to the overall task progress going from 0% to 40%. All we need to do to convert the sub task progress to the progress through the overall task is to multiply it by .0.4 (or 40%).

This strategy composes nicely.

Let's say that our first sub task is composed of two subtasks. The first sub sub task goes from 0 to 30%.

We can calculate the value of the subtasks progress by multiplying the sub-subtask progress by 0.3. We can then see the sub sub task's contribution to the overall task by multiplying it by 0.3 then 0.4 (or 0.4 * 0.3 = 0.12). So when the sub sub task gets to 100% we will have completed 12% of the overall task.

Here' how you do this in code. I'm going to use java because lots of people use it, understand it and it's what I know.

First you need an interface like this:


public interface Progress {
   /**
    * @param progress number between 0 an 1 that signifies the progress through a task.
    */
   void advance( double progress );
}

You pass an object of this type to any function you want to track progress for. Like this:


    /**
     * Tells a progress bar to go from 0 to 1 (complete) in steps
     * 
     * @param progress
     *            to set
     * @param steps
     *            to take.. More takes longer.
     */
    private static void pretendToDoSomething(Progress progress, int steps) {
        progress.advance(0);
        for (int i = 0; i < steps; i++) {
            sleepAWhile();
            progress.advance((double) i / steps);
        }
        progress.advance(1);
    }

Oh course with real code you'd be something useful, but you get the idea.

Next you'll want an object to represent the overall progress. Here's what the interface would look like:


/**
 * Allows for a progress bar made up of multiple sub progress bars.
 */
public class CompositeProgressBar {
    public CompositeProgressBar(Progress progressBar) {
    }

    /**
     * Builds a new sub progress bar. A sub progress bar is a progress bar whose
     * full length maps to the param subProgressBarSize of the
     * {@link CompositeProgressBar#masterProgress}.
     * 
     * THis method will also advance the previous sub progress bar to 100%.
     * 
     * @param subProgressBarSize
     * @return a {@link Progress} that represents param
     */
    public Progress buildSubProgressBar(final double subProgressBarSize) {
    }
}

The idea is you'd pass the Progress object representing the overall progress and you'd call buildSubProgressBar to, well, build the sub-progress Progress objects.

Here's a typical example:


    /**
     * Demonstrates how the progress bar can be used recursively.
     * 
     * @param progress
     *            - could be any progress - goes from 0 to 1
     * @param label
     *            for text output
     */
    private static void moreSubTasks(Progress progress, Sayable sayable) {
        CompositeProgressBar compositeProgressBar = new CompositeProgressBar(progress);
        sayable.say("part 1");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 100);

        sayable.say("part 2");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 102);

        sayable.say("part 3");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.6), 103);
    }

Here's the implementation of CompositeProgressBar.pretendToDoSomething(Progress progress, int steps);


    /**
     * Builds a new sub progress bar. A sub progress bar is a progress bar whose
     * full length maps to the param subProgressBarSize of the
     * {@link CompositeProgressBar#masterProgress}.
     * 
     * THis method will also advance the previous sub progress bar to 100%.
     * 
     * @param subProgressBarSize
     * @return a {@link Progress} that represents param
     */
    public Progress buildSubProgressBar(final double subProgressBarSize) {
        progresSoFar += currentSubProgressBarSize;
        currentSubProgressBarSize = subProgressBarSize;
        return new Progress() {
            public void advance(double progress) {
                if (progress < 0 || progress > 1)
                    throw new IllegalAccessError("\"progress\" should be less "
                            + "between 0 and 1 but was: " + progress);
                masterProgress.advance(subProgressBarSize * progress + progresSoFar);
            }
        };
    }

For context here' what the entire object looks like:


/**
 * Allows for a progress bar made up of multiple sub progress bars.
 */
public class CompositeProgressBar {
    /** The master progress bar we're splitting into sub progress bars. */
    private final Progress masterProgress;

    /**
     * Amount of progress we've gone though so far, not counting the amount is
     * the latest sub progress bar
     */
    private double progresSoFar = 0;

    /**
     * The length of the current sub progress bar.
     */
    private double currentSubProgressBarSize = 0;

    public CompositeProgressBar(Progress progressBar) {
        this.masterProgress = progressBar;
    }

    /**
     * Builds a new sub progress bar. A sub progress bar is a progress bar whose
     * full length maps to the param subProgressBarSize of the
     * {@link CompositeProgressBar#masterProgress}.
     * 
     * THis method will also advance the previous sub progress bar to 100%.
     * 
     * @param subProgressBarSize
     * @return a {@link Progress} that represents param
     */
    public Progress buildSubProgressBar(final double subProgressBarSize) {
        progresSoFar += currentSubProgressBarSize;
        currentSubProgressBarSize = subProgressBarSize;
        return new Progress() {
            public void advance(double progress) {
                if (progress < 0 || progress > 1)
                    throw new IllegalAccessError("\"progress\" should be less "
                            + "between 0 and 1 but was: " + progress);
                masterProgress.advance(subProgressBarSize * progress + progresSoFar);
            }
        };
    }
}

.. and, as I've just mentioned, you use the object like this:


    /**
     * Demonstrates how the progress bar can be used recursively.
     * 
     * @param progress
     *            - could be any progress - goes from 0 to 1
     * @param label
     *            for text output
     */
    private static void moreSubTasks(Progress progress, Sayable sayable) {
        CompositeProgressBar compositeProgressBar = new CompositeProgressBar(progress);
        sayable.say("part 1");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 100);

        sayable.say("part 2");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 102);

        sayable.say("part 3");
        pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.6), 103);
    }

All we have to do now is look up this composite progress bar to some sort of GUI component. To do this we have to create a progress object that wraps a JProgress instance. Here's how you this:


    private static Progress convertToProgress(final JProgressBar progressBar) {
        return new Progress() {
            public void advance(double progress) {
                progressBar.setValue((int) (PROGRESS_MAX * progress));
                progressBar.setString((int) (100 * progress) + "%");
                progressBar.repaint();
            }
        };
    }

You can then place this JProgress into a JFrame and send the Progress object to the CompositeProgressBar constructor and you're all set.

Congratulations. You are now masters of the first level of progress bar Fu. You can write a progress bar that accurately reflects the progress of the overall task, even when the overall task is made out of little, tiny pluggable pieces of code. What is more, those little tiny pluggable pieces of code can now be reused in different contexts, with different progress bars. This is truly a great day for the user.

The remaining question is, how do you get to level 2 of progress bar Fu? Ah, that is a good question young grasshopper.

What if, you have a task for which it is completely impossible to gauge how long the task will take?

What if, you have a task for which you have a rough estimate for how long it will take but, you have no way to increment the progress bar because your code is blocked doing something else. For example, it may be doing some I/O in a different thread. Alternatively, maybe running a third-party process and there's no way for you to get any feedback about what that process is doing. Now would you do?

For the answers to those questions you'll have to click here to go to part II

Part II

New desktop

2009-04-09T09:45:00.002-04:00

This is the sort of thing I was dreaming about when I was a kid waiting for the release of Apple's "Copland" OS.

http://arstechnica.com/software/news/2009/04/hands-on-bumptop-may-be-the-desktop-revamp-you-waited-for.ars

Three dimensional desktop. Awesome new spatial OS or yet another gimmick?

CRTC hearings on internet throttling and network neutrality

2009-02-24T19:46:00.003-05:00

The CRTC is holding hearings on internet throttling. The union voice is running a handy submission form where you can post your messages. Normally I'd write a letter but I'm just about out of time.

Here's what I posted:

I submit that the CRTC should stop Internet Service Providers from discriminatory traffic-shaping practices.

Given the near monopoly enjoyed by the high speed internet service providers, they should not be imposing any traffic shaping rules on data flowing over their networks.

The lack of vigorous market competition amongst last mile internet service providers means that Bell has effective control over which applications can be used.

Bit torrent is not a content provider, it is a content delivery system. It allows a single entity to effectively host a huge amount of data without having to directly server all that content directly to each user.

This allows for the hosting of huge amount of data by a single individual to be done very cheaply without having to involve a third party "free" distributer like youtube, flicker or download hosting service.

Bit torrent is an example of a disruptive technology that can change the balance of power between publishers and content providers. It allows anyone to host content directly for no cost as extremely high speed. It is important that this technology not be throttled.

Additionally other technologies are waiting in the wings. Things like Voice over ip and radio / video streaming (such as internet radio and things like the BBC world service) are both high bandwidth application that could end up competing with services that Bell provides (cable-TV like services, video on demand, phone services). It would also be easy to fabricate reasons why these too should be throttled. This is to say nothing about any new service or capability not yet developed.

Many of the largest network providers have also branched out into providing other types of content and services on their networks. These services can be looked at as being proprietary versions of capabilities that are provided or could be provided by providers on the internet. Bell and Videotron can be seen as being in competition with services on the internet. As a result they should not be allowed to throttle or
packet shape any one particular service or protocol.

If consumers could get away from Bell's throttling by leaving Bell's sympatico ISP and joining another third party DSL ISP (like TekSavvy, for example) then it would not be a problem. The third party ISPs could decide their own strategy to deal with an over-loaded network. They would almost certainly compete with one another to find innovative way of solving their congestion issues.

Since no one has yet been successful in splitting the responsibility between the companies that run the wiring and the companies that use the wires to send and receive data over those wires, ISPs must remain neutral and should be allowed to squelch new protocols and services as they appear on the internet.

As Canada moves deeper into recession, we must create new opportunities for innovative new companies to create new products and services not protect incumbent network providers attempts to control which new services and products are provided to consumers and on what terms.

The CRTC can do its part by enacting and enforcing policies that help build an open, fast, and accessible Internet in Canada.

Blogging with an email address

2009-02-15T14:52:00.002-05:00

Neat, I just found out that you can add a blog posting by emailing an address. This is built into blogger.

This would meant that if you wanted to add a blog posting all you would need to do is send an to blogger and it would convert it into a blog posting. I wonder if images would get added too?

Historical Canadian Unemployment Rate

2009-02-08T21:02:00.004-05:00

There's been a lot of talk recently about Canada's unemployment rate. It's now 7.2%. I've been following this for a while and I think it's a good idea to put this jump in the unemployment rate in perspective.

The graph below was generated using the statistics Canada data available on the Canadian economy website.

The Canadian economy website is incredibly useful. It gives you all the important economic trends including historical data. Trying to find all this stuff on the statistics Canada website is incredibly difficult. I'm glad they put it all in one place for you.

(Click on the image to make it larger)

Notice the overall trend in the unemployment rate. I can remember the unemployment rate being something like 10%. I'm very glad it's no longer that high. I'm also very glad that I happen to be in an industry where the unemployment rate is much, much less than even 6%. Those were bad times.

It looks like the an unemployment rate hasn't been this high since November 2004.

Emotions and Ekman

2009-01-04T20:09:00.003-05:00

Paul Ekman is featured in the book "Blink". He works on how the face produces emotions, what faces correspond to which emotion and which facial expressions are learned vs inate. Here's an interview I found with him.
http://www.youtube.com/watch?v=IA8nYZg4VnI

After that you can cleanse your pallet with this:
http://www.youtube.com/watch?v=xpcUxwpOQ_A

NaturallySpeaking 10 test drive

2009-01-02T16:44:00.004-05:00

Hello.

I suppose I should actually do a blog posting about Dragon NaturallySpeaking. I've been using Dragon NaturallySpeaking, now, for about six months. The version I'm using is version 10. I got at the moment it came out. In fact I actually bought it before Amazon Canada managed to get any stock of it.

I have to say that the speech to text engine is really quite impressive. And I can easily get two hundred words per minute. I suspect a lot of the reason for this is because I speak with a fairly common northeastern American accent. A friend of mine who has a sort of Québecois French Canadian accent and he had a lot of trouble getting Dragon NaturallySpeaking to work for him. I think he had to take pronunciation courses in order to get it to work at all.

Installation went fairly smoothly. I don't really like their installer, though. It doesn't have a progress indicator on it. Well, that's not completely true. It does have a progress indicator on it but the progress indicator doesn't actually show progress. It starts at the beginning and then fills through to the halfway and then goes through to 100% and then starts over again. I didn't count how many times it did this but it was enough to be annoying. The first time the progress indicator came up I was using it to gauge approximately how long it would take to do the install. I could just hear it laughing at me when it started back at zero again. Ha ha. You thought I was done.

When you use Dragon for the first time it does sound checks and make sure that the microphone you're using is of good enough quality. I use the included Dragon NaturallySpeaking microphone and was rather surprised to find out that it failed the quality test. This turned out to be because I had a second microphone plugged into the back panel of my computer. It was using the crappy microphone that was included in the computer instead of a Dragon microphone. Doh!

My dad was curious to try it as well. Since he has a different accent from mine I was curious to know if it would work as well. At first he tried use Dragon on 64-bit Vista. It didn't work. Apparently it doesn't support 64-bit Vista. It won't even install. booo.

My dad then tried it on his desktop computer. While it installed fine, we couldn't get past the voice quality test. Unlike me he didn't have another microphone plugged in. I spent some time trying to figure out what was wrong. I could record just fine in a third-party audio application. The microphone sounded great! Unfortunately Dragon and just wouldn't work with it. I still don't understand why.

In desperation we tried it on his other laptop computer. This time it worked flawlessly. I'm still annoyed that it didn't work on his 64-bit Vista laptop or his desktop computer. I don't understand why didn't work on his desktop machine. It's most perplexing. All of this means that Dragon only successfully installed on half the computers I tried it on. That's kind of depressing. I would make sure you can get your money back just in case it doesn't work.

My dad reports that it works great on his older laptop computer. It recognized what he was saying almost perfectly. At least this means it works with a hybrid British and American accent (which is what my dad has) without problems.

Integration with random applications doesn't work as well as with the included DragonPad application (which looks like a modified version of WordPad). I'm actually okay with this. It's not a problem for me to dictate all my text and then copy and paste it where I need it. It would be nice if DragonPad would transparently save as you go, though. I've actually had Dragon crash for no apparent reason on my machine and I lost a bit of work.

That brings us to stability. The software seems kind of flaky, in my opinion. I have a fairly high standards when it comes to quality of applications. Most of the time it's not unstable enough to become a bother. The most common error I get is that every once in a while it pops up with some kind of dialogue about how it can't run the speech analyzer or something. I have no idea what the error means. If I close the application, restart it then run the speech analyzer manually it works just fine.

You do find yourself fiddling around with a microphone and other settings in order to get it to do recognize your speech better/faster. I've been having problems with Dragon inserting the word "him" every once in a while when I'm not saying anything. You may have seen this on one of my past posts. Go ahead, laugh. I don't mind. I fixed this problem by tweaking the sound driver I was using and also changing the "speed versus accuracy" slider setting in Dragon NaturallySpeaking's preferences.

The package I got was the most basic one. It didn't have things like integration with Microsoft Word. As a result I can't tell you how well it works with Microsoft Word. It works tolerably well with most applications and text entry boxes, however. Sometimes it doesn't insert a space when it should. Some other times it doesn't capitalize a letter like you would expect. Sometimes it just bugs out in a way that's so weird I can describe it. These problems don't exist to the same extent in DragonPad. I would expect that integration with word would be similar.

Dragon NaturallySpeaking does tend to use a lot of RAM. It looks like its using about 256 megs. My machine has 2 gigs of RAM. I didn't notice any problems I could trace back to lack of RAM. I'm surprised to find out that a program that uses as much ram as it does isn't 64-bit yet. On the other hand maybe I shouldn't talk since the program I work on, InteleViewer, is even more ram hungry but isn't 64-bit ready yet either.. at least we *run* on a 64-bit OS, though.

My processor is a 3.2 Ghz Core 2 Duo monster machine. Dragon runs fast on this machine as you would exepect. One thing I did notice, however, was that it wasn't threaded. If you're running on a multiprocessor machine Dragon will only use one of the processors to do it's thing. That means if you're trying to figure out if your machine is fast enough, don't count the number of processors you have. It really doesn't matter. Dragon recommends a 1Ghz CPU with SSE2 support and at least 1 Gig of ram.

My dad's old laptop uses a 1 Ghz laptop processor and, reportedly, works fine.

Dragon NaturallySpeaking definitely improves with training. Going through all the included training texts will improve accuracy. Dragon can also go through your old e-mails and documents to learn your writing style, which help quite a bit too.

The biggest problem I have with Dragon NaturallySpeaking is that I often speak too quickly. Dragon seems to be limited to about two hundred words a second. Faster than that and it tends to lose words. I tend to speak considerably faster than the average person and slowing doing to only 200 words a minute is quite an exercise in self control. Dictating at this speed give me two realizations. The first is "Wow, I speak really fast". The second is "200 words per second is insanely fast compared to typing!". Writing these blogs posts now takes a fraction of the time.

So there you have it. Apart from the odd quirk or two it actually works very well. I'm a happy camper.

By the way, this post, and the last few posts too, were dictated using Dragon.. so any mistakes you find our it's felt. :-) <= real dragon error there

Market caculus

2009-01-01T20:13:00.007-05:00

I am very surprised to see that Microsoft is still having trouble selling Vista. I would've expected, by this point, it would have been the uncontested operating system.

Every time that Microsoft has released an operating system there has been some resistance to it. Others, like Windows ME were a flop. Operating systems, like windows 98 SE and Windows XP were, eventually, a great success.

What does it take to have a successful operating system? Why did some operating systems do better than others? What I want to concentrate specifically on is how a company can release a follow-up and have it fail to catch on.

Let's look at word processors for moment. Word processors are like operating systems when it comes to whether or not their follow-ups will be successful. If I buy version 1 of a word processor I make a natural assumption that version 2 will be similar, better, more stable, have more features, and most importantly will be backwards compatible with all the files that I wrote in the previous version of the word processor. It will, in essence, be a better version of the product that I already own.

Let's say I buy version 2 of the word processor I really like. Version 2, however, does not work at all like version 1. It's, in fact, completely different. It doesn't have the bugs of the old version but then again it has brand-new ones. It has new features that the previous one doesn't but then again it's missing features that the previous one had. Finally, it doesn't read the files created by the old version. In a situation like this, I would argue that this is in fact a brand new product.

What is the difference between a version 2 of an existing product and a competitor (completely different product)?

Is it the name? Is it that I know I'm getting something with at least as many features as the previous version? Is it that I know will work with my old files? Is it that I know it will work in a way that I like and am used to?

If any one of these things is missing will it be a true version 2? How many of these things does it need to miss before calling it version 2 becomes a lie?

Instead of looking at version 1 and version 2 of the product why don't we just think of each version as a completely new product. By looking at it this way, version 2 of a product is actually in competition with version 1.

Version 2 has a lot going for it in its competition with the existing version of itself. For one thing it's called version 2. This naturally implies that it is like the previous version in every way that matters only better. While we could be a bit pedantic and assert that this is not necessarily the case, this is exactly the impression we give when we name a new version of a product version 2.

What does version 2 have to its advantage when in competition with other word processors in the marketplace? Well, if you didn't like version 1 because it was missing a certain feature then why not try version 2? It might have the things you need.

When someone is upgrading from version 1 to version 2 things that they are interested in is

1 -- I assume it still works the way I like.
2 -- Have they finally fixed that bug that annoys me?
3 -- I assume it is compatible with everything else that I'm running.
4 -- I assume it has all the features I need.
5 -- Does it have any new, useful features that I want or can make use of?

When someone is looking to switch to a competitor the things that they are interested in are

1 -- Does it do everything I need it to? Does it have all the features I want?
2 -- Will allow me to use my existing files?
3 -- Is it robust enough to be usable?

(Note: someone buying their first version of the work process or does something very similar to the person looking to switch from a competitor. The main difference is the competitor is something that is not a word processor. Another possibility is that this is the first time they have done anything that would require a word processor. In this case what they can do is ask people's advice and ask only question one.)

The barrier to go from version 1 to version 2 of an existing product is almost always lower than to switch to a competitor. The reason for this is because upgrading an existing version tends to be a drop-in replacement. So long as you're version 2 is a drop-in replacement of version 1 and doesn't do anything stupid like take away existing feature or make things that used to works just fine in the previous version now not work due to bugs, people will always upgrade.

If the barrier from version 1 to version 2 is high then the mask used by consumers to decide whether or not to upgrade becomes progressively more similar to the math they use in order to decide whether or not to switch to a competing product. If version 1 and version 2 are different in ways that matter to the consumer than the decision to upgrade or not becomes indiscernible from a decision to switch to a competitor.

The opposite is also true. If a competitor comes out with a product that is a drop-in replacement to an existing product but with more features, fewer bugs, fully compatible and a streamlined workflow that consumer is more likely to see a competing product in terms of an upgrade to their existing product. If you have such a product, all you need to do is wait for your competitor annoying their user base enough and you are practically guaranteed that your market share will increase (assuming you are known as an competitor).

Over the lifetime of a product the calculus of whether or not to invest developer time in a new feature or refinement of a good and existing feature slowly changes. When a market is new the overwhelming drive should be to add new features and to try to grab as much market share as possible by adding the features required by each segment of the marketplace. As the product matures it will have enough features for every segment of the marketplace. When that happens the competition becomes more about refinement to those features and about the features themselves.

This switch from competing on features to competing on the refinement of those features can occur very rapidly. As soon as a segment of the market (a niche, if you will) is satisfied with a given feature set they will tend to start to switch their purchasing decisions to be based more on refinements over features. It's not always obvious what the niche is our because nieces are dependent on feature sets rather than any other kind of categorization.

I think the way to play this game is to map out the niches available to a product is in terms of feature sets used (structured in terms of workflows used). Using this map you can then get a handle on which niches are the easiest to target in terms of how easy it is to implement the new features required to have a product for that niche. You then compared this to how many users (and how much money) you can make in that niche.

Once the market is saturated you need to shift to targeting your competitors by making the transition to and from your competitors as painless as possible.

Notice I said to and from your competitors and not simply from your competitors. The reason I say this is because it's very easy to switch to and from your competitors and there is no risk for your clients to switch to. It is difficult to go back then you need to be significantly better for your competitors before your clients consider going on with you. The reason is because it is far riskier for your clients. If you're competing in the marketplace we are far, far superior then you don't need to worry about going back to your competitors. If you're working in a marketplace where the differentiation between your competitors and yourself is small then you need to worry about the fact your client is taking a risk. If you're product was drastically better they would go with you but since it's not a fear of switching will win out over the potential benefits.

(I could go on on this topic for long time. If you're interested in this sort of thing I would recommend reading "Crossing the Chasm", and "Inside the Tornado" by Geoffrey A. Moore.)

How does all this play out with Vista and XP? Well, let me ask you this: what is the new feature in Vista that will make you upgrade from XP? Let me also ask you this: what's the downside to upgrading to Vista? Will your drivers work? Will your programs all work? Will your hardware, scanners, printers etc... all work? Is Vista more robust than XP? Is Vista faster than XP? Does Vista to work like XP? (I actually know the answer that question! It's no. Sure, it's similar but the answer is still no.) Is there any compatibilities with Vista and XP? Like CD-ROM formats? Like network problems?

And there you have it. People aren't buying Vista because it creates problems and it doesn't offer anything compelling enough to justify fighting through the hassle.

I think Microsoft is too concerned with coming out with something different. If they just came out with something slightly better than XP they be able to sell it. If they reduce the memory, increase the speed, improve the experience of setting up a network (!!!) And they probably have enough to endear themselves to the consumer space. If you do that, though you can't break backwards compatibility. The thing is, I don't think Microsoft has the ability to come up with something slightly more compelling than XP. The company is undoubtedly geared towards large features and sculpting its software based on user feedback. In order to go beyond that they would need to pull user feedback and go do some real usability testing. Anyway, maybe I'll elaborate on that later.

How will this Vista upgrade disaster interact with the 2 gig / 64 bit barrier? Will the need to address more memory be Vista killer feature? yuck.

(Funny enough, years ago when Apple transitioned from Mac OS nine to MacOS X I switch to Windows. I've never regretted that decision.)

(Yay, no wikipedia links in this one!)

The 99.99999% done problem

2008-12-31T13:27:00.000-05:00

I don't know what exactly causes it but some things are hard to finish.

Consider programming projects. Have you ever been on a project where... Actually, maybe I should stop right there. Have you ever been on a programming project before? Oh.. Well, in that case I'm going to pick another analogy.

Have you ever written a story or book or something? Well, as you probably have observed, getting the first draft completed, while nice, doesn't mean you're finished. Depending on how much you care about the final version, you'll go through many other drafts. The text will be written and rewritten each time getting better. It may take more time to go from the first draft to the final copy than it took to get the first draft in the first place.

From the point of view of someone who has never written, this may seem weird. How can it take more time to go from the first draft to the final version than it took to get the first draft? Well, for professional writing it can. It depends on how good you want the end product to be.

This sort of thing happens on software projects too. In fact this sort of thing almost always happens on software projects. The bigger the project the more likely, in fact. What happens is that there is a big difference between having all of features in a project working then having all the features in a project working to the point where you can sell it as a product.

Here are the usual things that need work before you can release it to the general public:

1) Error Handling.

Error handling is one of those things that doesn't add any obvious features, that no one wants to plan or think about, but that people yell about when it's missing.

When you're working on the code, you'll notice that some function or other can fail. Any network call, any disk IO function can fail, for instance. What do you do if there's an error? The most common thing to do is just to put up a dialog box that says "I can't do that because something bad happened." but this is rarely the right thing to do. The right thing to do is to try and recover.

For example, if you talk to the central server and there's an error, can you try another server? Can you simply reconnect to the server and restart the communication? This sort of thing is hard to plan out and implement but the users love it. It means that the software can automatically recover from errors without the user having to do anything.

2) "Minor" UI tweaking.

When finishing up an application "minor" UI tweaking often gets dropped by the wayside. The reason for this is once you have a bare-bones basic UI to access a feature everything else just adds time to the schedule. What adds to this, is that either nobody cares about the UI or everybody gets to add their two cents to the UI. In both cases the UI will end up sucking.

3) Bugs!

The difference between the number of bugs you find if you have one, ten, one hundred, one thousand and one million users is impressive. We seen this happen with our product at Intelerad. The more places we deploy our product to, the more bugs we get back. What's impressive, is that this product has been on the market and in use for over five years. The bugs we get back are ones that were introduced five years ago. These bugs have lurked in the code for over five years. The reason we are only getting them now is because the larger user base the more people there are using it in different ways.

People don't report bugs as a rule. When they do report bugs it's because either they're the kind of person that reports bugs or because the bug is blocking their work. The more people use your product the more likely that someone will find data but is blocking their work. Also, the more likely that you'll stumble upon someone who will just happen to report a bug. People who report bugs for the fun of it are extremely rare. I would say there much less than one in a thousand. As a result, you are still gaining many of these people as the total number of people using your product goes past a hundred thousand.

A you could release when there were only a thousand people using it can be more buggy than if there are one million people using it.

A product that is usable for demoing purposes can have many more very hard to find bugs that I product that has to be released to the general public.

4) Scalability:

It is far easier to build a web site/ service for use by a tiny number of people than for a large number of people.

Intelerad know how hard scalability is; it's one of our specialties.

Jeff Atwood does too.

.. so I'm not going to dwell on it.

If you want to cut corners, not making your site/application scalable is a great way of doing it.. unless you need it then you're doomed.

Mmm cookies

2008-12-11T20:17:00.003-05:00

Okay, so it's finally all gone. I have finally got rid of all my cookies. I am very happy as I thought I would never give them all away. You see, it all started on Saturday when I decided that I'd make a triple batch of chocolate chip cookies. I was making chocolate chip cookies for the company charity auction. Every year Intelerad, the company I work for, set up an auction site on the company LAN and employees can put things on the auction site and sell them to other employees. The idea is all proceeds go to charity. It's quite a lot of fun, although it can be quite distracting at times. I don't have any scientific data but I'm pretty sure most employees spend their time simply watching auctions and trying to snipe one another on various goods.

I don't like to buy things in the charity auction so much as sell things. I think this is partly because I don't like to buy things period. At any rate, what I usually do is cook up a couple of batches of chocolate chip cookies and sell those on the auction. I can't figure out why exactly, but everyone seems to like them almost to the point of farce. They sometimes act as if my chocolate chip cookies or some kind of drug. It's quite flattering, but also really silly. The recipe I use the one you find on the back of the Nestlé chocolate chip package. Many people have tried to equal my cookies but apparently all have failed. I'm not entirely sure how.

The weekend before the auction I decided to make a triple batch of cookies. I like making a large number of batches at once because it takes almost the same amount of time to make a triple batch or quadruple batch of cookies as it does to make a single batch of cookies. Making larger batches is better because eventually you end up with so many cookies you can start to make little houses out of them or put the, in long lines, all over your furniture (what? Only I do that?). After you finish playing with them you can then give them away and make lots of friends.

In the end, I couldn't make any cookies over the weekend and so I made them late on Monday night. On Tuesday, I took in the cookies I made the previous night and started putting them up on the auction site. I amused myself way too much putting cute, little, nonsensical descriptions of the cookies on the website. I would sell them in units of two or three cookies each. This works well however, I was putting in way too much work coming up with cute descriptions that I got tired and stopped. I then realized that all my auctions have it in time approximately 7 minutes from each other. This guaranteed that I would be bothered almost precisely every second minute when somebody coming to collect their cookies and chat. While fun, it made doing code reviews much more difficult. On Monday I got through about two thirds of the first half of the three batches of cookies. (For extra points, what fraction of the total number of cookies was that?)

On Wednesday I somehow tickled a bug that stopped me from making more than one auction at a time. Every time I created a new auction all it did was modify the last auction it created. I started selling cookies in larger batches feeling a slight twinge of panic that I might never be able to get rid of them all. On the plus side, I got slightly more done that day. I say slightly more because Wednesday was chilly day and I got involved helping others get everything ready. Chilly day is a great day. It's a almost spontaneous outpouring of food. I love spontaneous out pouring of things.

Today, Thursday, I managed to figure out how I tickled the bug and started to make multiple auctions again. This time I omitted the cute descriptions and placed the end of each auction at least fifteen minutes apart and often up till thirty minutes apart. This makes creating multiple auctions very quick and it only took me a fraction of the time it did on Tuesday. What is more, the end of the auctions were enough apart that they weren't a perpetual distraction. I managed to complete the work I wanted to do that week during that day and so it was a very productive. (I say this in case there are any Intelerad employees reading this)

Thursday also marks the last of the cookies. I am now officially out of cookies. I hope the cookie junkies don't come after me now.

AHCI and freezes during startup while detecting SATA devices

2008-10-18T16:16:00.004-04:00

You know how sometimes you're dealing with a computer problem and you find yourself trying a bunch of things, almost randomly, in a desperate attempt to try and get it to work? There's a name for that. If you're doing that while writing a program it's called voodoo chicken coding. If you're doing it while trying to debug some sort of operating system problem then I suppose it's voodoo chicken troubleshooting. it's called voodoo chicken coding (or troubleshooting) because you don't actually know what the problem is that so it is trying a bunch of random things to get it to work. It's the equivalent of mumbling things to the gods while waving some kind of voodoo chicken over the computer in a vain attempt to get to it to work.

This is not a good strategy. Even if you do manage to solve the problem this way it will probably reoccur again since whatever you did to fix the problem probably just fixed the problem by accident. (I say this coming from a background of trying to debug heavily threaded software).

What is not often stated, however, is that knowing how a system works can sometimes be of no help either. I recently had a problem on my PC where the computer would freeze early in the startup process while the BIOS was still scanning for IDE devices. I ran into this issue I was trying to switch my SATA bus over to AHCI. AHCI stands for Advanced Host Controller Interface. It's a newer protocol for talking to serial ATA devices that offers more features than the normal parallel ATA protocol such as hot swapping and native command queuing. I wanted to enable this on my internal hard drive for two reasons: 1) I thought I already had enabled it at some point in the past. 2) I wanted to enable native command queuing because it sounds cool. 3) I need to enable it in order to run the Mac OS which I've been trying, unsuccessfully, to get running on my computer since I bought it about a year and a half ago. At least part of the reason I haven't managed to get it running is because I haven't managed to get my computer running with AHCI.

Anyway, I turned on AHCI in my BIOS I started getting this freezing problem when the BIOS was scanning the SATA bus looking for new devices. I was a bit confused because as far as I know there's absolutely no way that anything I had done to the hard drive, in terms of formatting or partitioning or installing software, could cause this problem. Scanning for new devices shouldn't be reading anything on the hard drive. That's just weird. Well, it turned out that this was indeed the problem.

After trying everything I could think of I decided simply to wipe the drive in a voodoo chicken debugging attempt to try and get the system to recognize the hard drive without freezing. Amazingly, after doing a low-level formatted the drive and rebooting it worked fine. I still don't understand why this is. Why the heck is it reading off the drive during the device detection routine? Well, I don't know. Anyone want to explain this to me then feel free. Anyway, I'm just happy it's working now. Native command queuing is indeed cool!

So, I'm not sure what the moral of this story is. I think it is that once you've eliminated everything as being impossible the only thing that's left is the impossible, which is impossible. This in turn means you have absolutely no clue what you're doing and might as well start trying a whole bunch of random stuff that shouldn't work.

Now if you'll excuse me, I have some chicken stew to eat. yummy!

Sidenote: Switching from parallel ATA to AHCI requires installing drivers under Windows. Unfortunately installing drivers under Windows requires AHCI to be enabled. Enabling AHCI renders Windows unbootable unless it has the drivers for the AHCI controller installed. This is a very fun situation as it means you can't install AHCI drivers until AHCI is enabled and you can't load Windows if AHCI is enabled. I managed to get around this problem, on my machine by using my two SATA controllers to enable AHCI on the second controller, install the drivers in Windows for this controller, then manually change the hard drive over this this second controller. I then rebooted Windows under the second controller and turned on AHCI under the primary controller. I then moved my hard drive back to the primary controller and start up my machine again. And that effective all these shenanigans with the have AHCI enabled on both controllers and a half the drivers for these two AHCI controllers installed and enabled. There are apparently ways of installing the drivers by booting up from a CD-ROM or other startup disk and then inserting a floppy drive with those drivers at some point. I didn't bother reading up on how to do that as the above methods seemed far simpler to me.

Computer issues

2008-10-09T20:59:00.002-04:00

Arggh! This is driving me crazy! Is there anyway I can actually get any work done without being thwarted by stupid, little computer issues.

Yesterday is was trying to setup an FTP server on my WindowXP machine. This is pretty much a single click operation on MacOS X (not including the usual NAT shenanigans) on windows it took forever.

The user manual was wrong.
The IIs FTP component needed to be installed separately.
There's a hidden button for setting the file sharing permissions into "simple" (read useless) mode. For some reason, the document doesn't mention that certain menu items in the MMC sharing controls don't show up in simple mode. grrr..
The sharing controls are hideous and there's at least one hideous violation of standard UI widget behavior that blew my mind. For the curious it's a checkbox on a settings dialog that doesn't represent a setting but an action to do when you apply the settings. Someone apparently figured that since they were writing code that did actions (like saving settings) when the user clicked ok, that the dialog represented the actions to take when hitting ok instead of the state of the preferences that are mirrored when the user pressed ok.. grrr.

Today it's open office. I wanted to finally start fixing up one of my long text documents - adding things like style information and a table of contents. I opened up the file and spent 3 hours or so adding style information and a table of contents. Just a few moments ago I reopened the document to start adding content again. Humm, all the style information had gone. Did I open the wrong file.? nope, all the textual information changes were still there, just the style information. I looked at the file type: txt!

You have got to be kidding me.

Apparently, OpenOffice was fine with me adding the text style information but had no intention of actually saving said information.. Or even pointing out that I was adding text style information to a text document. Which makes no sense!

Last time I used microsoft word (version 5.0 mac) it didn't do this. The behavior was to stubbornly insist that if you wanted to save to something other than word's default format you manually go through the save-as process each and every time. It insisted on warning you that you might loose formating information each and every time. In the end one tended to give up, write the thing in word's native file format and save-as something else before sending.

I believe that the modern version of word will actually warn you that the specific things inside your current document can't be saved to whatever you've chosen. I might be wrong on this, though. I haven't been able to get past the modern version of word's interface. Where are the keyboard equivalents listed???

Photoshop won't silently save a file with layers or other non-savable info to a png (ie: it won't loose your layers silently). I can't think of any program that would have allowed me to just waste time like this in all the years of using a computer. This is a first: a completely novel way of destroying my data.

Thankyou Open Office. Thankyou, you stupid, useless application. May the idiot that responsible for destroying my data suffer some sort of miss-fortune... like loosing his data while using Open Office in the same way.

US healthcare documentairy

2008-10-01T20:59:00.002-04:00

I just finished watching the Frontline documentary. It talked about the US healthcare situation. Basically, what the documentary was about was how healthcare is done in five different countries. It contrasted this against how healthcare is provided in the US.

Healthcare, in US, is very inefficient. The health expenditure per capita in the US is basically twice what it is in most other countries. That's really quite spectacular.

I don't live in the US, however I get the impression that the health care debate is incredibly politicized there. It looks like this is making it difficult for them to make any progress with their healthcare system. That's a shame.

I live in Canada. Our healthcare system has had it's share of problems recently. From what I gather, most of these problems have stemmed from underfunding. This underfunding was in turn caused by a need to service a large national debt. Now that the debt situation is under control funding is increasing again. It looks like we're doing pretty well.

Canada wasn't one of the five countries in the frontline documentary although Wikipedia apparently has an article comparing the Canadian and US health care services for those interested.

Think of it, with all the money Americans would save they could buy a couple more maglev trains. You can never really have enough maglev trains.

I like maglev trains... and efficient healthcare ... and Wikipedia. :-)

700 billion dollars.

2008-09-29T21:04:00.002-04:00

Wow, 700 billion dollar bailout.

A few weeks.. or was it months?.. or maybe it was yesterday.. I can't tell time.. Anyways, at some point in the past I ranted about the lost opportunity that was the Iraq war and how they could have used the money on several other projects. That amount was 500 billions dollars. Now they want to spend 700 billion.

For comparison, canada's national debt is now 467 billion dollars. Aiii!

To be fair they say that they'll get the money back.. Well, most of it.. probably.

It's only ~2000$ per man, women and child in the US. Canada's national debt is ~15000$ per person which is still pretty bad given that Canada's population is about 33 million vs the US' 300 million or so.

Congress, at this writing, is balking at this amount. Honestly, I don't blame them. This is a ridiculously large amount of money and as much as I can appreciate the occasional need to prevent contagion, this is on a greater scale then.. well.. anything I've ever hear of. Honestly, I'd give this a miss too without some really convincing evidence that they know exactly what their doing.

I wonder if Colin Powell would consider giving another speech.

Communication between engineers and management.

2008-09-10T22:16:00.003-04:00

Human beings work on two levels. The first is the emotional level. This system is very good at making very quick decisions based on the data but doesn't think very deeply. The second level is the rational level. This is the level that can do mathematics and understand software design. Psychologists think of these two levels as being different systems in the brain. They call the first level system one and the second level system two. Given that there's absolutely no way of understanding a very complex piece of software, like an operating system, if someone's trying to explain to you what's special about the newest version of Windows or Linux or MacOS X, there may be a little technical data transmitted however the bulk of the information will be directed directly at system one, the emotional system.

Very often explaining the positive and negative aspects of the complex design doesn't involve trying to make people understand the whole system, even when the system itself is really only understandable as a whole. It's possible to transmit one's excitement about the system or the elegance of the system through one's own excitement and a few choice examples. You need to transmit your excitement because otherwise your words come off as disingenuous. You need to use a few examples because this is something the brain can understand. In antiquity we weren't always able to prove things mathematically so what we did instead was we used anecdotal evidence. Examples are like anecdotes. They don't have the advantage of being associated with a different person, but if you can make your example personal and that's almost as good.

When it comes to engineering, one of the greatest dangers and engineer faces is misunderstanding a system. To simply not understand a system is not as much a problem because you typically know that you don't understand the system and seek out knowledge, advice and otherwise treat the system with respect one would expect to give to a potentiality dangerous blackbox. When an engineer misunderstands the system he feels free to tinker with it, to change it and then put it into production. Many bugs in software actually exit because an engineer changed the system, either data feature or fix a bug, but didn't understand how that existing system worked. As a result of their tinkering they introduced a subtle problem. As a result, software engineers, in fact all engineers, tend to become professedly more paranoid about misunderstanding concepts as they get older.

This sort of paranoia about misunderstanding a system does not exist in the general population. In fact, it may not even exist in engineers with respect to non-technical matters. When people not in technical roles interact, in such ways that can influence the design and manufacture of complex engineering system, there will almost certainly misunderstand the system.

Working on a complex piece of software requires holding a lot of state in your head. It acquires understanding in detail the software system. In a typical day a software engineer will make many decisions that will affect how long it takes to build a piece of software, how robust that piece of software will be and whether or not a feature gets implemented. He will make these decisions either explicitly or implicitly based on the design he chooses to implement. While requirements suggest design design suggests requirements also. A good engineer will optimize the time it takes to write the software, the quality of the software, and the number of features in the software. Frustratingly for managers, the only person who actually has enough information to be able to make the trade-off is effectively is a software engineer. Frustratingly for software engineers the only person with enough information to be able to understand whether the system should be optimized for speed of development, quality or features is the manager.

From the manager's perspective, it is impossible for manager and to know everything they need to know in order to make a design decision that will influence the schedule, quality and capability of the software their team is building. While I think it would be possible for managers to be able to do this with certain high level features. However, in practice a good designer will be able to understand the whole system in its entirety and for any given set of features, schedule and quality objectives will be able to optimize the design, in its entirety, to the optimal. A common manager mistake is to try and exert control over the software team by withholding important prioritization information.

From the software engineers perspective, it's impossible to know exactly which of features, schedule or quality is most important given the current political climate. This includes pressure from clients, budget pressures and maintenance duties. A common software engineer mistake is to build the wrong thing to a ridiculously high standard.

Communication between management and software engineers is tricky. From the software engineers point of view, he can't give the whole picture because it would simply take too long. In fact, if you were to give the whole picture to the manager to manager would know the same amount as a software engineer about the system. Nevertheless, software engineer doesn't need to give an accurate picture of how the system works. All we need to do is give some idea as to the emotional landscape of the solution space. Essentially, any combination and quality features will result in a schedule with some risk parameter. Negotiating a combination of quality, features and schedule is the process of understanding the solution landscape and then picking a solution with an agreeable combination of factors and a tolerable risk.

A manager can help speed this process along by attempting to communicate the political climate, as much as it relates to the priory of features, the satisfaction with current quality and schedule pressure to the engineer. By doing this the manager gives the engineer context as to what sort of environment the software is being built in. This process is very similar to going to visit an on-site client to find out what sort of workplace pressures the client is under and what sort of environment the software is expected to run under. While a manager can drag and engineer round with him to get yelled at by executives the manager can try and communicate as much as possible the current priorities.

Managers need to be over trust are software engineers to make the right design decisions. This is a matter of professional trust and competency but also a question of having the right information.

Software engineers need to try and poll their managers and under other members of the organization for what the priorities are what the priorities are likely to become and how satisfied everyone is with the current state of affairs.

Learning is fun

2008-09-01T17:38:00.003-04:00

Well, you learn something everyday. This is especially true if you're working with computers. Today I learned the following:

1- windows file sharing permissions are weird.
2- Entourage X (mac outlook) will irreversibly corrupt its database if you push it over 2 gigs.
3- Sylpheed doesn't know how to import an mbox file if said mbox file uses mac style line ending.
4- Sylpheed crashes most spectacularly if you try to import a 1.6 gig mbox file with mac line endings.
5- Practically no text editors will work with 1.6 gig files.
6- Knowing how to program in Java and having a development environment ready to go has its advantages.

Tune in next week when I learn that beating oneself over the head with a pan is a good stand-in for using a computer.

RE: Bill C-61, the Canadian copyright bill.

2008-08-24T09:13:00.000-04:00

Dear Jim Prentice,

I've been following recent developments regarding the new C-61 copyright bill. There are many things I don't like about it. Here are three.

1) It becomes illegal to copy DVDs for backups or for playing on another device.

I have recently started to move my DVDs onto a separate hard disk so that I can play them from my computer without going through the bother of finding the physical disk first. Essentially I have made a sort of crude movie jukebox. I find this to be a great way of watching movies.

I also make temporary copies of DVDs to my laptop for use on long flights or bus rides. Playing from the digital copies doesn't take up as much battery lifespan as playing them from disk. Also I don't need to carry around the DVD drive, not to mention the disk itself. This is especially useful since my laptop machine doesn't have a DVD drive.

I am concerned for people making copies of DVDs for use in their iPod movie player devices. While I don't do it I don't think this should be made illegal. I can see a time in the near future where it will be possible to put every movie I own onto one of these devices. I would like to see this doesn't become illegal.

I am also concerned about the parents who want to make backup copies of their children's DVDs because their children tend to destroy them. I think this is a reasonable, fair use.

2) The anti circumvention clauses.

All free and open source DVD players on linux are, to my knowledge, based on the DeCSS. This code was backward engineered to allow DVD playback on Linux. This would be made illegal.

From my understanding, the development of this code would be illegal. I'm not even sure that the use of this code is legal, therefore I'm not sure whether there's any way of legally playing DVDs on linux. I think this is a bad thing.

Putting linux aside for a moment, the breaking of CSS has opened up the possibility for me to make copies of DVD for the uses I mentioned above. In a very real way I owe these new capabilities to the breaking of the encryption. It looks like bill C-61 makes format shifting in general illegal and breaking DRM to do so doubly illegal.

I believe that DRM and encryption are examples of how digital technology can be used to create new business models. Digital technology, and the use of encryption, can allow the content producer to control how their content is consumed and paid for. Historically, this has been defeated by other who break the encryption and backward engineer their formats. Anti-circumvention legislation removes the ability of third parties to do this and tilts the balance of power in favour of content producers.

With DRM, piracy is a red herring. DRM certainly doesn't help stop piracy since all you need is one non-DRM copy to begin to circulate for all piracy to be possible. It is, however, a great way of getting people to pay extra for the ability to VCR programs for later viewing... Or to pay to re-buy tracks they actually own but need to buy again because their tracks all use a DRM for a type of player that doesn't exist any more.

Finally, I would like to mentions the Sony root kit incident. Sony's CD copy protection DRM was obnoxious and invasive. It used a root-kit style attack more common of trojan horse (computer virus) cracking attempts. It's buggy modifications to Windows has caused me personally to spend time fixing machines broken by its buggy implementation (known commonly as the Sony rootkit fiasco). In my opinion this sort of drive by virus-like behaviour from software should be illegal and not any attempt to circumvent it!

How to fix your desktop application

2008-08-12T21:47:00.004-04:00

Ok, a quicky..

Google's calender, spredsheet applications, mail have started to displace desktop applications. Why? IMHO they suck. They try to be desktop applications but are nasty, buggy, pale imitations. They do have a few things that desktop applications can learn from.

1) No dang installation step. I've always hated installing applications. Is anyone here over 30? Can anyone remember installing applications on Macs circa 1992? The correct answer to that question, with a few exceptions, was no. You just dragged the application from the floppy to where you wanted on your hard disk. The only reason you didn't run the dang program directly from the floppy was it ran slow. 'm using windows XP and everything little thing has an installer. Step one to making desktop applications suck less, get rid of installers. Let's get a standard where I can run desktop applications from the web and cache them locally, please.

2) No load time. Web applications don't need to load. To be honest I'm still not sure why desktop applicati0ons have a load time... and I've been writing them for years! While writing Myster I tried to reduce the amount of time it took to load. In the end I managed to get it down to some reasonable fraction of what it took to load the java virtual machine but really it should have been even shorter.

Just what the heck is happening during a program launch anyway? The answer is the machine reads a block of computer instructions from the HD and starts executing them. This is actually quite fast.. even on windows. The trouble comes when these initial instructions start loading libraries and building tables and constants and loading the code that loads the preferences and reading from the preferences and loading parsing them then loading all the icon resources then displaying those. The list goes on and on. In the end, desktop applications take a long time to load.

If your desktop application takes longer to load than my perception of instantaneous, then you should be making it faster. If your application feels the need to present a splash screen it's taking too long to load. If your app takes longer to load than a typical web page then it's too slow.

3) Web pages can be accessed from anywhere. I'm not entirely sure why I can't access my home documents or application setting from another location. Part of this problem is that applications require an installer and I don't want to go through this heavy install process in order to access my information from another PC. The other part is because I have to find someplace where I can store my documents or setting in order to access them.

Web sites don't have this problem. In one of the weirdest examples or this I have ever seen, my web browser of choice has an option to store its settings remotely. The idea is that when I use my web browser on a different machine, the settings I usually use follow me there. Hurray! Now the three machine I use daily will be in sync. The thing is, in order to use this feature I have type enter a server to connect to to store my settings. Are you kidding me? I got this browser from a website.. A web site that appears to have no problem handling a bazillion downloads of said web browser every month, not to mention other page hits etc.. But it won't allow me to store my settings anywhere on its servers. Is this desktop application group think?

4) Platform compatible. The three machines I use daily are all on different platforms. I have a mac, a PC and a Linux box. I can view the same web sites on all. Desktop apps? Yeah, there are some ports but I would have expected that we'd have cross platform code by now. Java has been around for some time now and does it fine.

gumble grumble grumble..

So at work I'm currently working on a brand new desktop application product. I want to give demo/beta applications to people. I don't want to keep sending out installers to everyone what can I do?

Well, The application is in java so I use Java web start.

With java web start you go to a web page, click on a button or link, the application is then downloaded to your machine (if it's not cached there already) and run. The whole process is a bit quirky in practice since you have to click on a box acknowledging that you're downloading an application by someone named whatever.. but it works. Want to run it offline? Yep, you can do that too. It's like having a desktop application available from a web page. This application, also has roaming user preferences as well so if you go to a different machine, the preferences can follow you around via your login. Basically, it nails 3 out of the four things above. The startup time isn't the best.. I mean this is java, but it's still fast than open office, for example, so it's not bad...

yeah.. I can see this stuff happening.

RSI is a pain in the wrist

2008-06-29T13:57:00.002-04:00

Repetitive strain injury is a serious risk for all coders and for many office workers too. Many of my colleges at my workplace display symptoms of RSI problems and it worries me.

I once went through a period of about 2 months where I couldn't move at all. I could barely move my wrist, opening doors was difficult etc.. all the classic RSI symptoms. On the advice of my family I went to see a physiotherapist and over 2 months I managed to get use of my fingers back. In a very real way I'm still recovering now. While, I can use the computer all week for the normal amount, I still sometimes get pains in my wrists.. this is 3 years later.

RSI is bad new and represents one of those few times in life where panic and anxiety is a reasonable emotional reaction. Untreated RSI can destroy careers and lives. It can cripple for life. A friend of mine had to abandon his Ph. D in computer science and pretty much change his entire career plan. This same friend has written a short article about what he's learned battling RSI.

One big tip is stop typing! Don't type through the pain!

Being unable to type for 2 months is much less of an issue then being unable to type for the rest of your life.

Tomato Plants on the Balcony

2008-05-31T11:01:00.003-04:00

Many years ago, when I lived with my parents, I would plant a garden in our backyard. I'd grow things like beans and tomatoes. I quite liked doing it because it was relaxing.

When I moved into the city I wanted to grow tomatoes in pots on the balcony. My first year wasn't very successful. My second year wasn't either. I figured my balcony wasn't getting enough sun or something.

I switched apartments a few years ago and this new apartment had a balcony that was in the sun for most of the day. The tomato plants still didn't do well. In fact, they didn't grow at all. All they did was turn purple.

Eventually I figured out what was wrong. It turns out that I was using the wrong soil.

For whatever reason, "black earth" has, in my family, always been a by-word for rich soil. So, when I went looking for soil to plant my tomato plants in I went with the big bag labeled black earth. Unfortunately, the black earth I got didn't help the tomato plants grow. I'm still not clear on why, although I've been told that tomatoes like something more organic. Great. Tomatoes are hippies.

Anyway, the second time I planted my tomatoes on the sunny balcony of my new apartment, I used a mixture of soils. I actually suspected that my they-aren't-getting-enough-sun hypothesis was bogus, so I planted 6 tomatoes plants n a mixture of different soils. The soil that did best was regular potting soil made for house plants. I can't remember the name of the soil but it was something ridiculous like "Mr. Magic's incredible miracle soil".

(for the experienced tomato growers: No, I didn't have compost in my selection of soils.. I'm getting to that :-) )

They did fine in the miracle soil (by fine I mean they actually grew. This had not happened before). Then I went to visit my parents. My parents still grow tomatoes and their tomatoes were twice the size of mine. I was annoyed but curious. Apparently, there was still room for improvement.

A little while later I noticed my plants had stopped growing. I became very frustrated but hypothesized that they might need more fertilizer. I had this jar of flowing houseplant fertilizer and I decided that since my tomato plants weren't producing anything that I might as well amuse myself by testing this hypothesis by trying to over fertilize them.

I started by giving them the weekly recommended dose every day. Their response was to turn an incredibly deep green then start growing by about an inch per day. It was really impressive.

I went to ask my local plant person about this and they said that tomatoes are always hungry. They really like nutrients and completely sap the soil of any. The house plant stuff I used was nicely balanced but not rich enough in the stuff the tomato plants wanted. I should plant them in pure compost. This made me relived as I half expected them to say they needed blood! Fresh blood and plenty of it! I dodged a bullet there.

So this year I have planted them in a mostly compost mix. They are bright green but and a bit tiny but I'm betting it's because they are cold. After bringing in a plant for a week and seeing its size doubled my hypothesis seems to be confirmed. Well, June is coming so that shouldn't be much of a problem anymore (June always likes the heat turned up to full. She's a funny girl is June.).

Growing tomato plants on your balcony is fun and results in a comical number of tomatoes at the end of the summer (if you've done it right).

The things you need are a balcony that gets plenty of direct sun from about 10am to 2pm. You need pots that are about 17 to 22 L (10 inch pots, deeper is better). You need pure compose soil (maybe mixed with something - opinions on the internet differ. I'm mixing with various amount of regular house plant soil in ratios of two thirds compost to seven eights compost the rest regular house plant soil).

Cherry tomato plants seem to be more resilient and less finicky.

Anti-squirrel gun installations may be needed depending on where you live.

Since my parents have a large compost pile where they compost just about everything I went down there and picked up some real, organic compost. You can also buy compost at the store. I've been told sea shell compost is quite good since tomatoes like the calcium. There are other composts as well. If you buy your soil at a real plant place you can ask the guy. If you're visiting a downtown botanist make sure you make it clear you're not using tomato plants as a euphemism for pot plants. Man, the miss-understandings I've had.. wow... As long as you use some type of organic compost soil you're probably ok. I'm still looking for soil tips to try out next year. Post pictures of your plants and soil tips in the forums :-)... I'd love to see them.

Tomato plants are fun to grow and fun for kids too since once they start growing you can actually see the amount they grow per day. This is growth on a kid-friendly time scale.

Oh and tomatoes taste so much stronger straight off the plant. Don't put them in the fridge because it destroys the taste.

Ok, enough about tomatoes. See you next time.

Worst website nomination

2008-05-12T12:37:00.004-04:00

So there I was, looking for some information on JavaFX, the new.. ummm.. something language from Sun which is great for.. well.. I'm not sure..

Anyway, I was looking for informaiton on JavaFX when I came across the JavaFX website:

http://javafx.com/

My jaw dropped to the floor. They really worked hard on it. It's so sad. The site is a total nightmare.

JavaFX.com is the kind of site that gives rise to inside-the-box thinking.

Here are the problems I have with it:

1) The "next page" icon things are inscrutable.
2) The amount of information per "page" is tiny.
2b) The content is laughably superficial as well.
3) The "floating" thing.. hereafter called the WTF windoid is so tiny as to make is useless.
4) The information presentation is noisy, filled with decoration compared with the amount of information displayed.
5) The minimized WTF windoids are hard to read.
6) The current WTF windoid hides other minimized WTF windoids behind it.
7) The transition effects don't work smoothly on my machine; everything just jumps around.
8) My font size settings break its assumptions about font sizes, so text flies off the end of the WTF windoids "titlebars".
9) There's not "throw a brick at the author" button.
10) Ugly as sin.. And black with white or sharp contrasting colors is my favorite color scheme too! - That's how my website is done. :-)

A "unix" UI rant

2008-04-20T19:50:00.002-04:00

I think I'm finally getting the hand of the Unix/Linux mentality.

For years, now, I've maintained that trying to use Linux was about as much fun as a visit to the dentist. I think I'm finally starting to get it, though.

You see, Linux, in general, has a terrible UI. Not only that but it has been incredibly slow to improve. I fully expected, back in 1999, that'd this problem would be completely solved by now. With the massive rush of ex-apple programmers and general interest it shouldn't be too much effort to fix up Linux to be usable by people unwilling to dedicate several years of their life to learning it. yeah...

So anyway, today I managed to install a piece of software that was obviously programmed by some unix-type. I'm so proud of myself. It only took me two tries. The second try I approached it with the idea that 1) the programmer expects me to know about how everything and 2) put no thought into user interface.

Fun facts:

1) You needed to tinker with a config file that came with its own syntax.
2) The client and server concepts were backwards (*my machine* was the server and I had to ssh into the other machine and tell it connect to *me* so I could use it).
3) Security by insanity - do I know your login name? Yes? Ok, I'll talk to you. I mean, at least you could pretend it was a password; that's what it acts like.
4) The configuration was such that you needed to add two configuration entries to do something that really only needs 1 (for the most common use case). If you only added one configuration line your mouse cursor became trapped on the other screen. oh no!

The thing that really gets me though, is that once it was configured it actually works. I mean, there's nothing wrong with it! No stupid, bugs or miss-features it does what it was supposed to do. It's really a shame that this great software is hiding behind such a terrible interface.

Want to try it yourself? (the software or configuration odyssey. I'm not picky) The software is called synergy. It's purpose is to allow you to use a second computer, with a second screen as if it were really just an extra screen for your main computer.

You see, I run a PC most of the time, but I also have a mac that I want to use. I don't want to keep switching keyboards and mice, though. I'd rather just be able to mouse over to the mac screen and use it like that.

Those familiar with VNC can think of it as setting up VNC session on a monitor hooked up to a different computer. Synergy works better than that because isn't of using VNC to display the remote screen on your computer. You just use the physical monitor attached to other computer's and send your mouse and keyboard command are sent to it.

This works great if you have the two computers sitting on the floor next to you.

Now if you'll excuse me I have to flick the mouse over to the Macs screen because the screen saver has come on again..

Ok, so it has one (known) bug:
"The Mac OS X port is incomplete. It does not synchronize the screen saver, only text clipboard data works (i.e. HTML and bitmap data do not work), the cursor won't hide when not on the screen, and there may be problems with mouse wheel acceleration. Other problems should be filed as bugs."

OpenPro Computer - A mac clone!

2008-04-15T12:58:00.005-04:00

Apparently Psystar is making macintosh clones. The OpenPro Computer model looks familiar. Oh yes, the case is the Antec P182. Essentially that same as my home PC's P180. Good choice.

"UI is the root of all evil" or UI is hard

2008-04-14T11:30:00.002-04:00

A friend pointed this one out to me. It's an article about the dangers of not considering the GUI when doing development.

http://blogs.pathf.com/agileajax/2008/04/the-user-interf.html

In the context of my previous blog posting, think of it as what could happen if the UI design is late and your design process is not agile. :-)

(planning is good)

Cooper vs extreme programming

2008-04-13T11:51:00.006-04:00

I was at last year's SD Best Practices conference at Boston. One thing I remember was the two times I heard programmers complain about Alan Cooper's views on UI design.

The complaints were both by programmers directed at the aspects of Interaction Design which are incompatible with extreme programming.

First off, I've never been a big fan of extreme programming.

I believe that a great number of programmers have a code-first-think-about-it-later approach to programming. I believe this mentality is a big barrier to creating large, scalable, long-lived software. I find it disturbing that someone would actually advocate a process that actually *encourages* more of this.

I met 3 programmers at SD Best Practices that took extreme programming to mean absolutely no design. As if thinking ahead was banned outright. Talking to them brought to mind scenes from 1984 where instead of being banned from remembering, you were banned from thinking ahead. I don't buy it. To plan is to think ahead is to avert disaster. Imagine trying to build a building or plan a trip or save for retirement without thinking ahead! The extremist extreme programming mentality is that the future so uncertain that all planning is futile. This is only a very coarse approximation of what extreme programming actually advocates but what extreme programming actually advocates is complete overkill in most projects anyway.

At any rate, the Interaction Design (as explained by Copper) conflicts with XP (extreme programming as explained by Kent Beck) in at least two ways.

1. XP advocates user involvement in the design process. You ask the user, you give them a prototype, they tell you what's wrong, repeat until convergence.

User Interaction design says "don't ask the user!". The rational is that users don't know what they want! UI design is hard and should be done by a trained, professional Interaction Designer.

2. XP advocates short iterations and an iterative methodology that takes into account learned mistakes as the project progresses..

User Interaction mandates that the Interaction Designer do the design more or less in one shot, up front.

These two methodologies are not directly comparable. Not only are they trying to solve different parts of the software engineering problem but they are coming from a different set of assumption about what sort of environment they are operating in.

Interaction design is all about user interface design. It does mention how to go about developing software except where software development intersects user interface design. Doing design up front is a good idea with user interfaces because 90% of GUI design is optimizing for the most common cases. You either know what the common cases are or you have to go out into the field and observe the users at work.

The net result of this process is usually a design that is highly optimized to how people work. As with any highly optimized system, GUIs can have a large amount of cross cutting features; that is features that are interdependent. As a result of this, a small change somewhere could change the nature of the GUI completely. Adding one extra click or a pause or adding a step can have a huge impact if that case is common. As a result, user interfaces should be thought of as one holistic piece.

Consider this example: Radiologists look at X-rays and diagnose problems like broken feet and arrows piercing the head. Their workflow (simplified) is 1) load image 2) dictate image 3) next image. I've watched them at work and they work fast. They average about 2 minutes per case. If we had a workflow that added 1 extra click somewhere to that workflow we would have added somewhere around 7 * 60 / 2 =~ 200 clicks. Multiply this by the number of radiologist at work and that tiny, little design decision has single handedly caused hundred of cases of repetitive strain injury! You can do a similar time base calculation. if your extra click requires the user to think about what to do next.

This may seem trivial to some but we've actually had certain minimum workflow standard written into our contracts with some of our clients because they are sick of vendors pushing an extra click on them for no reason.

I've always viewed XP's desire to bring the user into the design process and have them practically design the UI as a first order approximation on how to design a GUI. XP's assumption about having continuous access to a user (or group of users) is very a-typical, in my experience. Also, users are very bad at vocalizing what they want of need:

http://andrewtrumper.blogspot.com/2008/02/why-ui-design-is-import-in-apis-and.html
http://headrush.typepad.com/creating_passionate_users/2005/09/listening_to_us.html
http://www.joelonsoftware.com/articles/fog0000000356.html

etc..

My favorite users-don't-know-what-they-want story comes from when I was developing a piece of image-burning software. The idea was that this software would be for burning X-ray (or other modalities) series to CD. We knew what sort of users we wanted to target. We thought we knew what they wanted in the software, since we had this big feature list. The only question remaining was what should the "GUI look like?". I looked at the feature list and the target user and I couldn't match them up in my head. I just couldn't believe that the users we were targeting really wanted this big complicated GUI. I decided to ask for an on-site visit. Before the on site visit I'd prepared a list of questions that I wanted to answer. These questions were about whether a certain features were desired or not and about the relative frequencies of various occurrences.

I was curious about what they'd tell me if I just asked them my questions point blank. Did you need this feature? yes! My test users wanted every feature, expect that ones they didn't understand. Did this edge case happen frequently? no! How often? Once every 6 months of so.

I then observed them for three hours. During the three hours each one of the edge conditions happened at least once. They also never needed the extra features. It turned out that the software they were using had the features I was asking about. However, even in cases where they should have logically used these features, they didn't because it wasn't worth the effort to configure them! AGH! I came away from this with a better, streamline GUI and the notion that users (at least these users) had no clue what they wanted.

The second major point of disagreement between ID (Interaction Design) and XP (extreme pornography.. I mean programming) is the use of iterations.

As I've mentioned ID isn't about how to build the thing; it's about the GUI. Nothing is stopping you from using iterations to build the big-bang-GUI. In fact, in my practical experience, you have to start building before the GUI is completely ready because it takes so-freaking-long to design the GUI. Yes, by all mean do iterations. Not doing iterations is silly. You might want to stretch out your iterations longer than the ones suggested by XP, though. XP mandates uber tiny iterations (1 week). With such iterations you're actually under-valuing planning.

Planning is good! Over planning is no so good. Being forced to over-plan then stick with that plan is extremely bad. In my experience, excellent planners (or planner *teams*, rather) can plan two months in advance with a good degree of accuracy. Good planners are about a month and ordinary, mere mortals can plan a week or two if they practice at it. All these estimates depend on the context, of course. Your planning horizon can vary tremendously depending on the problem, the degree of familiarity and the skill of the planner.

My favorite planning story is the time we planned out our "roaming preferences" feature. Essentially we managed to plan out about three months of work and have that plan executed with little or no change within 5% of the estimated number of hours. Hurrah! A hole in one! Then, in the next iteration, we tried to plan out 1 month of work and both the major aspects I was responsible for were completely wrong.. Luckily both things that we got wrong were wrong in opposite directions. One took 4 times longer than I though and the other tool about 1/10 the estimated time. The reason for this was that we had to make a modification that changed where the bulk of the work went so it's not really a co-incidence. We actually knew this was high risk going into it since we were modifying old code we didn't know very well.

Remember to include the risk component in your time estimates and proposed designs!

So, iterations are a way of mitigating against the risk of planning errors. They do this by scheduling points where the design and progress of a project can be re-evaluated and adjusted. The downside to short iterations is that it can artificially limit you to a fixed design horizon. Limiting your design horizon can introduce costly refactoring or code scaring. For a real-life analogy of the dangers of under planning, consider the joys of mountain biking with your eyes closed. Ouch!

Using super short iterations on a project with no risk is silly. Using super long iterations on a project with high risk is silly.

None of this has to do with GUI design. Even if your GUI design is low risk, other aspects of implementing that design might be high risk. The length of iterations for a project is a programmer thing. Interaction designers do their GUI thing and programmers mitigate against their potential errors and their own by having short iterations. The two do not conflict because they are tackling different aspects of the problem. The only major change for XP people is that the GUI design is done by someone else and becomes lower risk. (In practice, politics and errors affect this somewhat)

I like to say that even if you're not doing iterations the best way of actually these implementing these big designs is by making use of iterations.

In my mind, ID (Intelligent Design .. I mean Interaction Design) and XP do not conflict. ID is a way of creating a better GUI. XP is a set of techniques for mitigating against the troubles brought by changes by embracing them.

bye.

Lucid dreaming

2008-03-12T18:52:00.003-04:00

Recently, at a party, I got into a conversation about lucid dreaming. A lucid dream is a dream in which you know you're dreaming.

I first became fascinated by this subject when I started to wonder why you always know that you're not dreaming when you aren't but why you don't when you are. Then I actually experienced a dream where I realized that I was dreaming and that really spooked me.

Why aren't all dreams lucid?

I didn't feel like I was dreaming when I was dreaming. Maybe I'm doing something equivalent to dreaming all the time - spending my life in a daze and not really knowing it. I decided that I was going to work on making all my dreams lucid dreams.

Talking to the guy at the party I was reminded that many people don't remember their dreams at all. Other have never had a lucid dream. Others don't believe it's possible.

Yeah, lucid dreaming is possible. It's not as fun as you might think but it's possible.

After practicing for a few years I managed to have lucid dreams about one every other night.

Many people seem to think that if they knew they were dreaming they'd be able to do all sorts of stuff they've always wanted to do. In practice, my experience is the dream already has an agenda and if you don't play along you wake up. That's not to say you can't do some cool things in dreams, it's just a lucid dream isn't your own personal holodeck.

Dreams are manifestations of your expectations.

Let's say you built a computer. This computer makes up models of the world. It tries to find patterns and guess what's going to happen. This computer is very good at it and after some amount of training develops a whole bunch of models of how things work. It knows when lunch is. It knows you like coffee at the start of the day. It knows if you let go of something in mid-air it falls. It's got a fairly good view of the world.

Now you want to take it to the next level. You want to make this computer interact with the world very quickly; without having to think about what's going to happen. You essentially want it to figure out that if I drop something fragile it should attempt to catch it! Fast! There's no time to think about what's going to happen when you let go then figure out it's going to fall then figure out that it's fragile then figure out that it will break and that you will get mad and yell at it. It needs to know the rule that: If fragile item falls you (the computer) catch it.

How do you do that?

Well, as I said, the computer knows about how the world works. So if you bring up its memory of the lab and then ask it: "What sort of things could happen in the lab?" it will know: the same few people tend to walk in, in the morning. Lots of these people have coffee. Coffee is always in cups. The sun shines through the window in the morning. People occasional brush past things and push things onto the floor. etc.. blah blah.

Really, you don't need to teach the computer about the world, you can simply run it's models of the world against itself and it will figure out (eventually) most of what it needs to know. So long as you take more or less random branches out of the possible scenarios you'll end up producing experiences that have never happened. This is useful to train against.

The human brain is a bit like this. It makes models of the world and builds up expectations. Dreams just run these expectations against one another... presumably to help you react better if this situation should come up in real life.

Actually, there's another important factor: emotion.

If dreams were only about expectations you would have really boring dreams. Most of the stuff we expect to happen is fairly dull. The thing with dreams is they often choose which expectation wins based on emotional response.

Here's an example. Let's say you were in high school and you're standing in the cafeteria. Consider these events:
- Someone you're friends with waves at you from across the crowd.
- Someone drops their tray.
- There's a good seat available for you a few feet away.

Not much emotional response. How about these:
- You buy some lunch but discover that your pocket has a hole in it and all the money is gone!
- All the seats are somehow taken.
- A teacher walks up an casually mentions you just got an A on a test you were worried about.
- Terrorists!

Dreams go for the big emotional responses.

If you become lucid you can't suddenly start trying to make things happen that are just "neat". "neat" doesn't play well in dreams. "Dangerous" is fine. "Angry" is fine. "Really totally awsome" is fine.. "neat" isn't.

.. and don't try things just to find out what will happen. It's a dream! Trying stuff in a dream just to see what will happen is like having the following conversation with yourself:

"So.. what will happen if I do this?"
"I dunno. What do you think?"
"No, I'm asking you."
"Well, what do you think the answer should be?"
"Actually, I'm just curious about what you think should happen."
"Well what do you think should happen?"

If you do this enough, you dream will go "ah, screw this" and wake you up.

In a dream your job is to react to things not make stuff happen. You can make stuff happen but you have to go along with the flow of the dream. Try to make it a reaction to something.

Oh no a tiger! Good thing I'm in a dream since that means I can fly away! (model of how thing works goes "yeah ok".. so you fly.. Good thing your instinctual brain never took physics).

That's about the long and short of it.

In the end I started having weird semi-lucid dreams. In this semi-lucid dreams I know on some level that I'm in a dream because I know that all the things I used to do in lucid dreams will work.. I also know I can just escape by waking up. Well, unless I have a really bad nightmare in which case I an usually so focused on the moment I don't get that strange "this place follows dream logic" feeling. What I think happened is my behavior adapted to included the fact that, in dreams, the rules are different.

Sometimes I think my brain has built its own parallel universe. I have a "dream montreal" that doesn't look like real montreal but is more-or-less consistent every night. I have a map of my home town that includes things which are completely wrong but very consistent between dreams. They aren't spatially correct but they are emotionally correct. That is, the distance between two buildings is a "short" or "long" walk. That the main street contains an "annoying" amount of traffic. That buildings in the new art of town are "taller" and "newer" than the old part. That some buildings are "tall" while others are "scary tall". It's hard to describe. It's not montreal. It's obviously not montreal.. but it feels like it.

.. oh and all the elevators are dangerous and unreliable. I know that if the elevator doesn't work it's because I'm dreaming :-)..

So how do you get into a lucid dreaming state?
Read this link:
http://www.lucidity.com/LucidDreamingFAQ2.html#techniques

The most important thing is to remember that you can never tell you're dreaming! Especially if you've woken up from a dream! Your body doesn't like waking you up for no reason so it will tend to fake waking you up. When waking up from lucid dreaming always make sure you're not still dreaming.

.. checking for dream signs requires excellent self control.. Actually I think the ability to extract yourself from the moment is the primary benefits to pursuing lucid dreams.

So read all that stuff and you will have a lucid dream tonight. Seriously. (Go flying in reaction to something. It's scary fun.).. good luck having a second lucid dream but the first one usually comes easy. :-)

good book:
The Head Trip - It deals with various states of consciousness including lucid dreaming. No LSD or drugs here. All these states are natural ones.

If you like lucid dreaming you might like to experience the watch. Another fun altered state of consciousness easily accessible by messing with your sleep patterns. :-)

Why UI design applies to APIs and frameworks

2008-02-09T16:56:00.000-05:00

That blasted book! (see previous post).. Now it's got me thinking about API and framework design..

Ok.. So I'm reading a chapter about David Heinemeier of 37signals. He's the dude who wrote ruby on rails. In it he says the following:

"You need to innovate on behalf of your customers, but they don't often know what they want. And it's the same thing for programmers. If you went around and asked them what they wanted in a framework, you wouldn't get a good product out of that. You need to be able to source input from alot of sources, and then have your vision of what it's going to be and then drive that."

(emphasis mine)

There's this idea floating around that the reason a software sucks is because the people make it don't pay listen to the customer. All the layers of bureaucracy between the programmer and the user causes a disconnect. If we could just talk to the customer directly we'd be able to ask them what they want directly and just program that.

This is partly true but there's a problem: users don't know what they want.

I have a feeling that many who have not actually done a software project with real users see this as an arrogant statement. It's not.

Users are not programmers. They don't know interaction design. They probably don't have very good aesthetic tastes. If you let your users build your specification for you, you'll end up hearing the dreaded "This is what I asked for but not what I want". This is why extreme programming, which advocates the above, has a short iteration time.

On the subject of framework design.. Every once in a while someone will post a bit about how painful it is to do something or other in Java vs Python. Most of the time it's something trivial that's not really a Java-as-a-languge issue but Java-as-an-API issue.

Java's API can be irritating at times. My favorite example of this is reading a file.

Let's say you were in Python and wanted to read a file as text. Here's what your code would look like ->


   with open("hello.txt") as f:
     print f.read()

Now for the Java version. Settle in because this is a bit long.

First off let's do this as a function that returns a String


    public static String readFile(String fileNameToRead) {
        return null; //for now.
    }

Ok, so I didn't bother doing this for python. Beer with me here.. Mmm beer..

It's nice to get a quick poke about how verbose declaring a function is :-) .

Now We have a function let's create a file object:


    public static String readFile(String fileNameToRead) {
        File file = new File(fileNameToRead);
        return null; //for now.
    }

Unfortunately, the File object in Java doesn't have any method to read it's contents. To actually read a file we have to create an FileInputStream. Let's do that.


    public static String readFile(String fileNameToRead) throws FileNotFoundException {
        File file = new File(fileNameToRead);
        FileInputStream in = new FileInputStream(file);
        return null; // for now.
    }

The problem with InputStreams in general is that 1) You can't get a String out of them and 2) if you ask it to fill and array of bytes it actually won't fill the array of bytes. It might fill the array to completion but it probably won't. It's your responsibility to loop over the "read" method in FileInputStream an accumulate all the bytes you want.

To heck with that! DataInputStream can read a full byte array so let's use that!

ByteInputStream can wrap ANY InputStream and add functionality to it. This means you can use it to read bytes from a network connection, file on disk or anything else. It's very cute.


    public static String readFile(String fileNameToRead) throws FileNotFoundException {
        File file = new File(fileNameToRead);
        FileInputStream in = new FileInputStream(file);
        DataInputStream dataIn = new DataInputStream(in);
        return null; // for now.
    }

Notice that we had to add the "throws FileNotFoundException" to the end of the method there. This is because we're doing IO which could fail. If it fails it throw an exception. Java won't let you compile until you've told it how you want the exception handled. Most languages don't do this and just let the exception bubble up and kill the program. We're just going to tell java to throw it up to the caller and let someone else take care of it.

Ok, now we have a DataOutputStream but we can't read a String using it. It actually has a method called "readUTF()" which returns a String but it doesn't do what we want because readUTF() expects the data in the file to be in specific format. That format is only written by the corresponding method from DataOutStream.writeUTF()..

Ok, lets create the byte buffer!


    public static String readFile(String fileNameToRead) throws FileNotFoundException {
        File file = new File(fileNameToRead);
        FileInputStream in = new FileInputStream(file);
        DataInputStream dataIn = new DataInputStream(in);
        
        byte[] bytes = new byte[(int) file.length()];
        
        return null; // for now.
    }

Now we need to read in the bytes.. which is actually fairly easy now that we have our DataInputStream..


    public static String readFile(String fileNameToRead) throws IOException {
        File file = new File(fileNameToRead);
        FileInputStream in = new FileInputStream(file);
        DataInputStream dataIn = new DataInputStream(in);
        
        byte[] bytes = new byte[(int) file.length()];
        dataIn.readFully(bytes);
        
        
        return null; // for now.
    }

Notice the exption changed. This is because DataInputStream.readFully() throws its own exceptions. Since both exceptions are the same kind (IOException) we're just going to tell java that all IOExceptions should be dealt with by the caller.

Yay we have the file's bytes! Now bytes aren't a String so we still need to convert to a String...


    public static String readFile(String fileNameToRead) throws IOException {
        File file = new File(fileNameToRead);
        FileInputStream in = new FileInputStream(file);
        DataInputStream dataIn = new DataInputStream(in);
        
        byte[] bytes = new byte[(int) file.length()];
        dataIn.readFully(bytes);
        return new String(bytes);
    }

But wait, there's more! We still have to close the file! In the Python code this is done by the "with" statement.. We have to do this by doing this ->


    public static String readFile(String fileNameToRead) throws IOException {
        DataInputStream dataIn = null;
        try {
            File file = new File(fileNameToRead);
            FileInputStream in = new FileInputStream(file);
            dataIn = new DataInputStream(in);

            byte[] bytes = new byte[(int) file.length()];
            dataIn.readFully(bytes);
            return new String(bytes);
        } finally {
            try {
                dataIn.close();
            } catch (Exception ignore) {
            }
        }
    }

There we go. Now if anything goes wrong the file will be closed.

Wow... That's longer than the python version...

The thing is.. If this function existed in the API you could call it to print out the contents of a file by doing this ->


  System.out.println(readFile("hello"));

This line does the same thing as the python version. They could have put this function in the core API. They didn't.

I'm betting that what python has to do to provide that functionality is very close to what we did here.

Java-as-an-API doesn't have a very good file-reading / string-manipulation toolbox. This is an API issue. It can still do everything but it's a huge pain. Python rocks for this.

On the other hand Java-as-an-API has a massive framework for making GUIs. Python-as-an-API doesn't.. You need to use a third party library.

Just one more thing before I sum all this up: Java both as a language and as a toolkit was the totally wrong tool for doing applets.

People wanted to make banner and cute animated graphics and such.. Look at what people are suing flash for today and that's what people wanted to do with java applets.

Java applets should have had an API for doing animation. It should have had an animation studio attached to it.

... oh yeah and it had to have drop dead easy deployment..

... oh yeah and it shouldn't have cause the web browser to freeze up solid for 30 second whenever it was used...

.. I could go on but this is all well known.

Sun didn't do their research. Don't be like Sun.

Designing frameworks is hard. You need to balance a host of things from flexibility to speed to ease of use to power and it just goes on. If you don't know who your building for and what they need you're doomed to failure. They can't tell you either because they havn't thought about the problem very deeply.

The process you need to use to build a good API is the same process as the one you need to use to build a good interface.

You need to know your UI basics either way.

ok, I'm done.

Skating in the park with flounders

2008-02-09T15:05:00.001-05:00

Wee, I just got back from playing hockey in lafontain park in Montreal.. It's about 0 degrees outside today so it's perfect weather for skating.. Not cold enough that you start to worry about the consequences of breathing.. not warm enough to melt the ice. Excellent.

heh heh.. While trying to stop someone from scoring I tripped and fell into the net.. I scored myself.. Luckily I wasn't hurt. I realized that it was about to happen so I put my hands out in front of me just as I fell. Those nets are heavy and you can seriously hurt yourself it you bang your head against it. We don't play with equipment either so you got to be careful.

I'm still trying to finish reading "Founders at Work" which is a great book that interviews the founders of major tech companies about what it was like to start up a company. These guys are crazy. They work crazy hours on practically no sleep.

anyways... I'm having trouble finishing the book because after a few pages I want to throw the book aside and go found a startup company. It's just silly.. I'll be sat there reading then I'll stop and wander around the apartment thinking about what sort of thing I'd like to invent and how I'd market it etc.. Eventually I'd calm down and sit back down with the book where the cycle would begin again.. It makes reading the book very slow going.

Anyway, I'm going to make some hot chocolate, sit down by the fire for an hour or two and read.. about 5 or 6 page.. thereabouts.. yeah.. bye.

War is not cheap.

2008-01-29T17:32:00.000-05:00

Currently the total cost of the war in Iraq is about 500 billion dollars. I've often wondered what this war is worth in terms of opportunity costs.

http://www.nationalpriorities.org/costofwar_home

2.5 magnetic levitation /vacuum trains from London to New York. These puppies can get you from London to New York in 1 hour. They fly through tubes under the ocean that are almost at a total vacuum at 4000 mph on magnetic tracks. Much faster than flying.

http://www.klanrur.co.yu/forum/index.php?showtopic=45349

Pfff, in fact all of the US could be setup with coast to coast maglev links

http://en.wikipedia.org/wiki/Maglev_train#United_States

21 Manhattan projects

http://en.wikipedia.org/wiki/Manhattan_Project

1 600 $ tax rebate for every human alive in the US. That would stimulate the economy.

http://www.census.gov/main/www/popclock.html

35 Superconducting Super Colliders.. Physics would love you!

http://en.wikipedia.org/wiki/Superconducting_Super_Collider

34 ITER nuclear fusion reactors.. Wow.. that would be cool.

[This link is missing because X-Windows copy and paste is unreliable and I don't want to go find the stupid page again]

1 man on Mars. awww... looks like Bush wanted a war instead...

http://query.nytimes.com/gst/fullpage.html?res=950DE3D6163FF932A15752C1A96F948260

4 International Space Stations

http://en.wikipedia.org/wiki/International_Space_Station

340 Montreal Olympic Stadiums! (wow, that beast really was expensive)

http://en.wikipedia.org/wiki/Olympic_Stadium_(Montreal)

$3,749 per iraqi

http://zfacts.com/p/447.html

agh it goes on.. Carbon credits, electric car research, laptops for developing countries, UN back-dues...!

What's your favorite, expensive megaproject?

beep

2008-01-13T13:01:00.000-05:00

Hello, I can't come to the phone right now. If you'd leave me a message I'd get back to you as soon as possible.

Hi there. Umm.. It's Andrew. You know, I hate answering machines. They defeat the purpose of calling someone. I mean I might as well email or leave a message on any one of the billions of message-board style communication mediums. Frankly those are better because I'm not put in the unusual situation of having to come up with something without a backspace key.. Also, I've heard what I sound like on tape. I wouldn't wish that voice on anyone. This has been a long message. I'm surprised I haven't been cut off. Some of those cheap answering machines have so little space on them that you can barely get a message out. Have you noticed you always get cut off at a point that makes it sound like you're insane or insulting:

"Hi there, well, I've been thinking and I'm come to the conclusion that you're just as much an asshole as.." beep!

"Hi there, It's me again, I was about to say just as much an asshole as I've been so I really shouldn't be mad.. Ok, well if you.." beep!

agh!

Try calling back.. nope.. tape full. ha! You'll have to learn to talk in sound bites.

So, you known what. I'm not going to tell you why I've called. I'm going to leave you my number and get you on the phone so I can talk to you. I've got more to ask you from you than a question. I want more from you than a reply. So call me back. My number is

The fuzz point.

2008-01-10T17:05:00.000-05:00

When trying to predict something, it is often the case that you need progressively more processing power as you increase the precision of your predictions. There's a point, however, at which increasing the amount of effort you put into creating the perfect prediction runs into the hard truth that either your model or your initial reading of the system's initial state limit your prediction's accuracy. Past this point, there's no reason to invest more effort because the amount of precision you'll have in your answer is more than the margin of error for that answer. This is the fuzz point. It is the point of diminishing returns.

It can show up in interesting places. My favorite place is the classic, intractable argument over aesthetics. If you just work at a little more it would look better. This isn't always the case. Consider your ability to predict what people find aesthetically pleasing. Consider any data you have on the topic and how much error there's likely to be. Consider the amount of time you've spent arguing about whether the arrow should be green or blue. You've past the fuzz point.

Aesthetics aren't unique. A special case of the fuzz point shows up when prioritizing. Bug fixes and feature in software.

How accurately can you predict how long something is going to take to fix?
How accurate can you be in predicting how important a feature is to implement?
How long are you going to argue about it?

The motto is, past the fuzz point, flipping a coin is actually cheaper in the long run.

The fuzz point is very small for small bug fixes. So small, in fact, that you get multiple different sorts of penalties.

In many shops all bug reports must be prioritized before it's decided whether or not they are worth doing. For bug fixes < 4 hours weird things start to happen.

The cost of figuring out if the bug is severe becomes more important.
The cost of tracking down the cause of the bug tends to be much more important.
The cost of the bureaucracy of fixing the problem becomes very important.
The cost of merely context switching away from to bug for enough time for it to be prioritizes becomes important.
The difficulty in measuring the relative importance of all these things increases.

Small bug fixing is fuzz point land. If a bug takes a short amount of time, there's no point in prioritizing it. The amount of time you've spent just trying to figure out the true severity and the cause dominate. If the fix is quick, don't prioritize, do it now, on the main branch and the deal with the risk portion of the bug fix separately. (Essentially review the severity and risk of each bug and fix and decide if they must be back ported to the old branch for a bug fix.. Also decide if it's worth running it by QA. The answer is almost certainly yes.).

If you do this, however, you will notice that you're development will stop. This isn't good. The way of getting around this is to allocate a fixed amount of resources to the task and prioritize bug fixing in its entirety with the adding of new features.

If you must prioritize fixes then poll the list of bugs looking for important ones. Don't force everything to be run through the bureaucracy before anyone can get a time budget for it.) .

If you implement this make sure you clearly say how long to spend in the various stages of bug tracking before giving up (how much time trying to ascertain the severity vs how much time investigating for each level of severity vs how much time trying to implement the fix). This is a heuristic but it works fairly well, because bug fixes show up in timesheets so you can see violations.

Must be going. I've been spending too long on this issue.

HAPPY NEW YEAR!

2008-01-01T01:56:00.000-05:00

I would like to wish all my friends a happy new year.. but I can't. I can't send them an email because that would be SPAM.. and I can't send them each an individual email because that would be an easily automated job and it's against computer programmer ethics to not be lazy in situations like this. I can't say HAPPY NEW YEAR! in my blog because only two people read it and I really want to tell everyone. I can't put "Andrew is wishing everyone a happy new!" in my facebook status because no one actually reads those things anyways. They exist only for self amusement and occasionally as an existential venting mechanism as in "Andrew is a spoon". Hummm.. I can't think of any way of doing it so I'll stick it in all these places and send personal notes to a few people arbitrarily. It's not perfect but it's the only way to avoid annoying myself.

Andrew is a fork.

Scary

2008-01-01T01:35:00.001-05:00

Dear Steven King,

BOO!

Sincerely,
AT

Scalability of software product teams

2007-12-30T21:34:00.000-05:00

I'm curious that with all the software development practices being flung around these days whether nay though has been given to how to scale up software development.,

I've often heard it said that small teams are better than large teams but what happens when you have a large application or perhaps a large suite of applications that are all supposed to work together. How do you scale up development in such a way that you don't end up programming resources.

Fred Brooks says that a large programming teams won't as well as a small programming team. The thing is, in some sense, every day programmer work on huge projects even if all they do is write a small, 100 line python program. That python script will use python (a large project) which relies on an operating system (a large project) made up of large components etc... We are always leveraging someone else's code on someone else's project.

writing library of framework code is much harder than writing application code. The trick is how much more effort does it take? ..especially if the library is only being used in house? How big should development teams get before the project is split into two groups or three? If we say that teams should be of about 5 people, how many teams should we have before we should create a team responsible for tracking down and eliminating duplicate code - by turning it into an inter-team library?

I haven't seem anything that attempts to tackle this.

There's another question.

Java's API and python's API both have really nice documentation. I would expect that any teams that try an work together would need at least this amount of documentation. Do they need anything else? I mean python's API and Java's documentation are all I use.

If joint design is being done, that is an interface is being negotiated, which is the best way of doing this? Sure, if you on have 2 teams negotiating a handful of interfaces this probably isn't a problem but what if you have 3? 4? 5? 10? 50? If a project is made up of many teams is it a good idea to have a meta-team responsible for maintaining the large project's design integrity?

I don't know. I haven't seen anything written about this. It's important, though. Many software projects are huge and right now most best practices are optimized for smaller teams and small software projects. Imagine trying to to native agile program with 50 teams of 5. You're project would be a mess.

..now what?

Sleeeppppp zzzz

2007-12-21T22:36:00.000-05:00

I've been reading a book about people who started up companies. These guys are crazy. The hours they work are insane.

There's a story about someone who would sleep for 4 hours every 2 days. Another guy would work for 4 days solid until he just fell asleep. That's pretty gosh darn bonkers.

I am unusual in that I appear to need a couple more hours of sleep than the average person. I usually like between 9 and 10 hours of sleep. Usually it's 10 hours and it can be more if I'm learning something (taking a course, going to a conference for example).

The "average" amount of sleep per night is 8 hours. There are some people who like more some people that like less. Apparently the typical range is between 6 and 10 hours.

I'm not sure how that works, I start to go a little crazy if I have only 8 hours sleep.

I've heard that attention deficit disorder might have a sleep deprivation angle to it. Given that I doubt many people are getting 8 hours of sleep, many people, if not just about everyone, may be walking around having slept too little. That's not good.

Humm.. better not become a member of that group. I'm going to sleep now.

Programmer's theater group

2007-12-20T23:48:00.000-05:00

I belong to a amateur theater group. Every year we put on two large shows. These shows take month of preparation. Not only do the actors need to learn their lines and go on stage but there's a huge amount of behind the scenes work to do as well.

There's things like:
- Choosing the script
- Directing
- Renting the hall for the play
- Renting the space for the rehearsals
- Ticket selling
- Costumes
- Designing and building the set
- Props
- The staff running front-of-house during the productions
- Advertising
- Ticket selling

All in all even with a cast of about 15 people, we end up using more than 30. All of this is done on a volunteer basis. All done in people's spare time.

It takes allot of time. As an actor, I spend 5 hours a week for the first two months at group rehearsals then 9 hours for the last month culminating in about 20 hours during the week of the production. Generally, people also tend to spend hours on their own learning their lines as well.

The more I think about it the more I'm amazed at how all this works. I'm curious if anyone is trying to do this sort of thing with software project.

The idea would be to get about 5 or so programmers who are interested doing a project and aren't particularly picky about what the project is. Then brainstorming on ideas until there's one that stands out then coding it, setting people up as project leads, designers, etc... As well as grabbing others for things like building the website and doing all the work with registering the finished projects with websites and sending out press releases.

With 5 people spending about 5 hours a week or more on the project I don't see why we can't have something interesting in 6 months or so.

The idea would be to set it up as an agile style process. Meet once a week for a SCRUM type status. Have a "director" or lead to decide the high level direction and focus of the project. .. and guarantee some hours of availability for code paring.

The goal would be to set some fixed time span (this is important) and a goal and try to ship a workable solution to the goal, preferably as an OS project. The timespan would probably be about 5 to 6 months in total with about 4 months of coding/design time.. the other 2 months would be just deciding which problem to tackle by way of looking at problems/possible programs/possible new features that can be written in such a short time. The idea here being that if you're going to spend the next few months tackling a problem you might as well think hard about which problem to tackle.

Given what I've seen with the theater group, some hours of inter-team interaction together would be needed for social reasons and as a good motivator. This is why I would say that it's important for team members to set aside some time at which all team members would be working on the codebase at the same time. If everyone has a laptop it could all be at the same location too. It's always great coding in an environment where you can bounce ideas off each other... nt to mention things like peer review too.

The project would proceed (Look Phil! Two "e"s!) in phases:

- Meet once a week for a while and each week present a possible project as a problem or need to fulfill and a goal for the project. After some time recap all the projects and vote on which one it the best. The project must have a team lead/"director" and must have enough programmers who want to work on it. New features to existing projects are allowed.

- Build a high level design and mock up for how the program should be built. This is mostly up to the director to organize. They should pick someone to help design the project. During this phase, any part of the project that may not be feasible should be investigated up until the point where everyone is convinced it will work. Brain storming sessions should be held twice a week on invitation of the project lead.

- Director choose who should work on which section of the problem and works with those people on explaining what the behavior should be. Teams are made. High level design of these module is done and code is started. Two sessions per week ~ 2.5 hours a week.

- After two months or so we start to attempt to going all the pieces together and run the project looking for bugs and behaviors that are not desirable. Schedule adds a 4 hour session to the existing 2.5 hour sessions.

- During the last week, the schedule accelerates to bug fixing every day of the week and work is wrapped up. Over the week-end the program is compile and uploaded or if the project is a website it goes live and links submitted to search engines etc..

So currently I'm curious to try this out. I've currently got two potential prospects. We've been brain storming potential ideas. I'm curious to know if we can get something out of it.

Anyhow.. food for thought. You want to audition for the next project? You know how to reach me. :-)

The dangers of reducing coupling.

2007-12-19T18:05:00.000-05:00

This is a continuation of part 1

Once programmers discover the joys of sectioning off code and reduced coupling there's a tendency to go a bit too far. (This is all from a java programmer perspective)

One of the common anti-patterns people fall into is to try and manage their dependencies by tracking which classes/packages know about which other classes/packages. The idea is that a class should only know about certain other classes. As a result it should be possible to group related classes into a package and then use a tool to automatically generate a map of dependencies between packages. This map can then be used to figure out where classes are making dependencies and destroy stupid ones.

There's a few problems with this:

1) It gives a false sense of security because two classes can depend on one another without having a compile time dependencies. The most common problem I've seen is some sort of complex event-driven system that ends up getting tied in knots because the code was written using this event driven system partly as an attempt to compile time dependencies with the goal of retaining the positive aspects avoiding compile time dependencies.
2) Compile time dependencies are not run time dependencies. Even if you consciously try and avoid falling into the trap of turning your compile time dependencies into run time dependencies, you will still fail because there's often no way of expressing the run time behavior of a system with a static, compile time dependency map. In the worse case, the attempt to do so will put limits on what sorts of patterns you can use while coding in order to try and keep the static dependency graph matching with the actual real graph.

Essentially these two reasons boil down to 1) it won't work in practice and 2) even if you could make it work in practice it still wouldn't be a good idea.

Keeping your inter-package dependencies clear and clean is absolutely a good idea. It is not, however, a panacea. It can't solve world hungry and it won't bring anyone back from the dead.

The thing with managing compile time dependencies is it's works up to a point. That point is the point at which the compilers understand how your program is put together.

What you should be trying to do is manage the coupling of your code. A dependency is a hint that there's some sort of coupling. It could be high or low but the hint is there's a coupling. If the compiler doesn't show a link that doesn't mean there's no coupling, it just means there's no compile time coupling. To be more accurate it means there's not compile time dependency. A compile time dependency can be thought of a form of coupling.

Ok, so let's say we're a developer and we've seen the light and now we know that coupling is bad. Is there any other ways to screw this up? Yep. Trying to reduce coupling to zero.

Reducing coupling between components to 0 can't work. I've seen people try to do this by removing as many constraints as possible.. For example, removing compile times checks via using something like a Map or events or something. Don't do this. There are languages out there that don't have compilers. Ask programmers in those languages if they experience problems with things being too coupled.

Another favorite is to keep chopping up code way past the point of sanity. The way to replicate this at home is to take a reasonably well written program and try to make every line of it into a framework. After a few hours you'll wind up with a dense soup of spaghetti.

This happens because you can't remove coupling. If you try to remove coupling by splitting things into incredibly tiny pieces you end up with a program with more tightly coupled components and more coupling related issues and more complexity than ever.

Before I cojntinue I'd like to introduce the idea of cohesion. Cohesion represents the idea that some things in life are inheritantly* coupled together tightly. A real world example of this is a table leg. Every molecule in that table leg can be viewed as a separate entity. However, it makes sense to manipulate the table leg as a whole so we don't consider the fact it's made up of molecules which are made up of atoms. Sure it's a lie but it's a convenient lie that makes the world easier to understand.

* however the heck you spell that.

The good programmers recognize a cohesive object when they see one.

Using this approach of building sections of code with high internal cohesion and low external coupling you can build some fairly amazing things.

It works in ANY language.

Oh, you can even use the trick recursively. Build an object out of some cohesive properties then use a collection of relatively coupled object to build a meta-object and so on. As such you can build mind bindingly complex things.

Everytime you use Swing or SOAP or RSS to write an application you're building a sort of meta-object that includes libraries with high internal cohesion. TCP/IP, XML, Swing object like JTable (shudder)..

Ok, so how do we do this? Well, the easy way is to do test driven development. I'm not sure why but test driven development seems to help programmers make programs that have objects with higher internal cohesion. I have my theories:

1) It actually forces programmers to think first - to design*.
2) Writing unit tests is easier if you have objects with simple interfaces that aren't dependent on a myriad of things.
3) Writing objects with simple interfaces also means you want to clump related functionality into one object so you don't drive yourself insane writing tests for millions of little, tiny objects.

* Design is the "D" word. Don't say it at any agile software conferences or you'll spend the next half an hour explaining yourself.

I suspect that taken together these things can be responsible for the worst hyperbole I heard while at the SD2007 best practices conference. I quote (more or less):

"Test driven development is the silver bullet Fred Brooks say didn't exist."

sigh...

Low coupling, high cohesion. It's the mantra of good software.

Further reading:

http://en.wikipedia.org/wiki/Test-driven_development
http://en.wikipedia.org/wiki/Coupling_%28computer_science%29
http://en.wikipedia.org/wiki/Fred_Brooks

Why coupling is bad.

2007-12-18T12:57:00.000-05:00

There's a large difference between a 1000 line program and a 100000 line program. Most of it has to do with making an application's architecture scalable.

Uh-oh, someone asked me to introduce a new feature into my established codebase!

When programming there's twos sorts of new features. There's the kind of feature I like to call a "vertical" new feature and there's a "horizontal" new feature.

A vertical feature is one that doesn't really interact much with other code. If you picture the code for the feature as being coded as a stack of layers that only interact with one another you end up with a vertical stack. When this stack is added to the established codebase it doesn't make it much more complicated. I mean there's the complexity in the feature itself but that complexity is nicely contained it's own stack. It's almost as if it's another program that just happens to be compiled with the established codebase. If you were to make a change somewhere in the code of the existing codebase you wouldn't need to worry about breaking the new feature because there's practically no chance that what you're doing will affect that feature. Vertical features are self contained and are therefore lightly coupled with the established code.

Horizontal features are those that affect a large cross-section of the application. The best example of a horizontal feature in InteleViewer would be key images.

InteleViewer is a application that shows medical images like CTs or magnetic imaging or X-rays etc.. The thing is, key images are images that aren't really images. They are references to images. From the Viewer's perspective, it gets a command that says download image X. The Viewer goes "OK", downloads it and then is surprised to find out that image X is actually a reference to three other images Y, Z and Q. The Viewer then has to go out and transfer them.

Now this all sounds perfectly simple. In your head you can imaging that all you'd need to do is have the KeyImage code transparently get the image it's referring to and download it. Well, yeah.. except for that fact that KeyImages came relatively late in the history of the Viewer and the Viewer was making lots of assumptions about the nature of an images to implement other existing features. Here are a few issues:

- The KeyImage might refer to an image that's already loaded and these things are huge so we don't want to load them twice. We have to add some code to make sure we're not loading the same thing twice in the loading code itself instead of in the code calling the loader.
- We have multiple different protocols which we can use to download images. Some of them have their own constraints as to what's possible to do vis-a-vis downloading images. We have to be aware of this and deal with each source individually or try and build the key images code out of abstracted operations that already exist.
- If we had any protocols that we wrote that assumed we were only sending images we need to re-write them a bit.
- Since key images, on their own, are a file, but don't contain any image data, you can't blindly send the files themselves to image manipulation routines.
- We cache all images on disk but key images don't have any image data so we need to be aware of this in the cache code too.
- key images have filtering operations that apply to the underlying images, so these filtering operations have to be combinable.

..the list goes on. key images have introduced constraints all over the code and the more constraints you have the more likely the next feature you add will need to know about key images. Key images adds constraints across the application's loading and caching systems and therefore makes any code in those systems more complex and subtle than before.

Here's a question for you? Can we abstract away the annoyances of key images by clever use of layers and factories and such?

Well, every programmer should try and some of the things I mentioned can be hidden by interfaces and abstractions. The simple fact that adding KeyImages was possible, was because we'd worked hard trying to hide the complexity of the loading system. The thing is, there's a fundamental limit to what you can do with abstractions.

Consider this: You're abstractions are shaped by what is possible to do with thing(s) you're abstracting. It's fairly easy to come up with a case that makes abstraction impossible. Here's an example:

You want create a program that downloads a movie file from an existing website and then plays it. The system must allow for the movie to start playing as it is coming in. Unfortunately, one of these movie formats puts some important information at the end of the file. It's not possible to actually play the movie until it's arrived and the server you're talking to doesn't allow you to asks for specific bytes before others. Net result: you're doomed. No abstraction can save you because it's not possible to provide an implementation that will do what you want.

This exact same thing can happen in more subtle ways with horizontal features. If you have a feature that adds on a requirement that's a contradiction of an existing requirement there's no abstraction you can do that will fix it.

The first rule here is to make horizontal features into vertical features whenever humanly possible. If you don't you're application will get old before its time. I would go further and say "no" to horizontal features or even look for little used horizontal features to remove from your app. Functionality not being used? Every piece of functionality has some horizontal component. If it's not used it should be removed.

What we're doing here is actually reducing the feature's coupling with other components. Being paranoid about coupling is a very powerful idea. When programmers discover it they jump for joy and then go onto make applications that are much larger and more complicated then ever before.

Then they run into the next wall.. More about that later.

Part 2 - When reducing coupling goes bad.

CSS etc..

2007-12-17T16:15:00.000-05:00

So I've updated the look of the blog recently. I'm having trouble getting it to look decent.

Basically this whole re-design was prompted by the fact that the previous template didn't grow horizontally with the size of the web browser. On my large monitors with large text it looked completely silly; there was this skinny column of text centered right in the middle of the page. sigh..

Well, what I did was I took a blogger template that actually did change its size depending on the horizontal width of the web browser window and tweak the heck out of it.

Here's the original:

Yeah.. I changed it a bit. I consider it an improvement based just on the fact that there's no freaking orange in it. :-) Orange makes me an sad panda. If I remeber correctly the tangerine iMac was the least popular so it looks like I'm not alone.

One of the things I was unhappy with is there's no way of dividing up the space between two widgets in such a way that one widget takes up a fixed amount of space in pixels and another takes up whatever is left over. Not sure why that is since I seem to remember doing something like that in the old do-everything-with-tables days.. Although I might be confusing HTML table layout with GridBagLayout.

When doing this site redesign I ran into the links-that-don't-look-like-links problem. Essentially the only way you can tell the what is a link on a web page and what isn't is by either mousing over it and noticing your cursor changes or by looking at it and seeing it's a link because it's a different color or because it's underlined. The thing is, the default blogger template don't do this consistently. Links come in at least 3 different colors. Some are underlined and some aren't. In ye olden days this wasn't a problem. Site couldn't over-ride the link color. But now they can and what's worse is CSS actually allows you do have a different linking style for every piece of text and widget on the screen. It's often happened to me that I'll be mousing around a webpage suddenly notice that a piece of text is actually a link. This happened to me on coding horror. If you mouse over a blog posting title it's actually a LINK! It's not underlined and it's not the same color as any other links on the page.

So when I was doing my blog I wanted all the links to be underlined and the same color.. but it looked like crap so I though ok, I'll at least have them all underlined.. I still haven't dug deep enough to figure out how to convert all the sidebar links to be underlined but I'm getting there.

At any rate I still feel fairly bad about having my blog post titles as links but having them white instead of the blue.

I like blue links. It was the default color for all hyperlinks for years.. and red was the default color for previously visited links.. argh.. that reminds me! I have to figure out why previously visited links don't change color.. they should be a shade of red but it's not working.. grumble grumble.... Ok, time to do that.

Song...

2007-11-20T00:18:00.001-05:00

So, for about three weeks I've had this damn song stuck in my head. I heard it on the radio and remembered the words to the chorus. I figured in this day and age I'd be able to find it again using google or one of those fancy searches we have these days. Nope.

Ok.. so now I'm thinking that I heard the words wrong.. as sometimes happens.. I'm going through every possible thing it could be and still No dice. Then, just as I'm about to leave to go to rehearsals, I hear it on the radio again. I quickly look it on on the radio station's recently played list (thank the internet for that one) and bingo.

The track is called "BIG CHANGE". Presumably it's in all caps because you need to yell it. !!!BIG CHANGE!!!. apparently it's coming. Who knew?

"Goating" is evil

2007-11-18T17:32:00.000-05:00

Goddam it. First thedailywtf and now Jeff. Nothing says "I am purile" more than goating.

For the uninitiated goating is the pulling of some prank on someone who has left their computer unattended without locking it. I used to see things like this when someone would forget to log-off their station when I was in university. If you did that in a public computer lab anyone could walk up and use your terminal logged in as you.

This was actually becoming something of a problem since, at least initially, there weren't that many terminal available. If someone forgot to log-off, the screen saver would quickly kick in and then you couldn't log off someone until the full timeout which was something fairly large. When the labs were really busy it we often asked an admin to kill their ghost session.

For a while there was much pranking but it became clear that many people took this sort of thing extremely personally and so I, and a few others, started to log off people who left there terminal unattended before anyone else could get at them occasionally sending an email off to them mentioning that they forgot to logout and this was a bad idea etc.. This played out well because one time, I don't know what I was thinking, but I neglected to log out and was saved by one of the people I had helped earlier.

I am against pranking for multiple reasons.

The first is it's a big waste of time. Some of these pranks are elaborate and take time to undo. That's wasted time. I dislike spending 45 seconds at a traffic like I dislike people spending 5 minutes screwing with my setting and another 5 me tracking them down and fixing them.

The second is it can alienate those with different cultures, backgrounds or mindsets. If you're doing this sort of thing with a close knit group of friends that's fine. They've all agreed to it. Dragging arbitrary people into it can hurt feelings and breed distrust and bad blood amongst the team.

The third is it's absurdly unprofessional.

There's also this notion that Jeff mentioned that this sort of prank is pulled for someone's good. I'd believe that if those doing the pranks didn't have so much fun doing them.

Many places don't need this sort of security policy anyway. It's just creating stress and conflict where it's not needed.

RE: http://www.codinghorror.com/blog/archives/000997.html

Exponents, ram, 64-bits...

2007-11-11T21:46:00.001-05:00

Ok, I really should be doing something else but I'm going to take a moment to give my two cents on this topic:

http://www.codinghorror.com/blog/archives/000994.html

In it Jeff Atwood ponders the move to 64-bits. Apparently he's a bit surprised at the speed at which the 32-bit addressing limit is becoming a problem. Actually I'm surprised it hasn't become a problem sooner.

I seem to remember, back in the old days, when I first started computing, machines came with 4 megs o ram. This was a mac, BTW. Macs were used quite allot for Photoshop and Photoshop demanded stupid amounts of RAM.. I mean you could have 32 or 64 megs of RAM and it still wouldn't be enough.. OMG!

The reason I mention this is because what an average PC user is running is very tiny compared to what people who actually use their machines for RAM intensive things use.

Currently I work on an image viewer program called InteleViewer. The thing is our image viewer likes RAM. I mean, we're viewing large images and we have to remain responsive while downloading the JPEG versions of several thousands images and decompressing them and witting them to disk... oh and did I mention we're using Java.

Yeah, yeah.. Believe me it is possible. What's crazy is we can do this with as little as 200 megs of RAM. It's not super speedy but it's usable.

At any rate, the doctors using our product like speed and will purchase anything to make it faster. Naturally, if they buy a workstation, one of the cheapest upgrades they can do is to buy more RAM. The more RAM then less hard drive swapping our viewer needs to do. Oh!. and if you want to view the stack of images in 3-D you better have enough RAM to fit the whole thing in RAM because we can't do much in the way of swapping when you can be looking at the stack of pixels from any angle.

RAM good. InteleViewer want more RAM!

We started to bump up against this RAM limit thing a while ago.. about the same time Apple was shifting to 64-bits. I can't remember what state we're in when it comes to the java virtual machine but the situation with window vis-a-vis 64 bits is shockingly ridiculous.

A 64-bit OS should not treated as some sort of pro feature. During a transition you want to minimize the pain. 64-bit OSes is a problem, a bug not a *&%$ feature upgrade.

Microsoft's attempt to split their 64-bit OS from their 32-bit OS has probably made the transition more difficult because with fewer 64-bit OSes floating around there's less incentive to make better, more mature drivers.

I'm an extremely gunshy when it comes upgrading so my tendency is to wait quite a long time because getting the new OS. I've only recently upgraded to XP from Windows 2000. I'm a bit cheesed that I'm not going to be able to run yesterday's OS with lots of memory because the transition only became really feasible recently.

At any rat, I'm going to stick with windows XP and 32-bit drivers for as long as I can. Not only does the old stuff tend to be more stable but it also runs faster and uses less of my system resources, which make everything else run faster too. From experience I've found that my idea of "speedy" and "stable" is a bit higher then the average bleeding edger...

Frustrations, theater etc..

2007-11-11T21:32:00.000-05:00

Between rehearsing for the new play I'm in, work and the usual madness of everyday life I've not really had an opportunity to compose anything of note for this space.

That said, I do have a large article (and I mean embarrassingly large article) which goes into how to document code in an effective way. I wrote it because I kept having deja-vu when it came to code comments. The things I wanted to know weren't there and what was commented was completely obvious. I was also getting the vibe from many on the team that code commenting was completely pointless, which was quite obviously bogus since Sun has managed to provide some very decent java doc comments for their code. What I have done is kept track of which things I want to know when I approach new code and put them all into a large document in a check-list sort of form.

The document became rather large when I realized that many of the items in the list only really made sense if you knew how to write good code. As a result I needed to explain how to write the piece of code in a format that doesn't require code comments before getting into what to comment. In fact, many of the items in the list can be thought of as warnings of the form: "Watch out! This code is not written how you think!".

In the end the guide is as much about how to code well as it is about how to document well. It's quite a nice read.

Anyways, that will be coming up as soon as I can figure out how to translate the behemoth into blog form.

Ok, gotta get back to learning my lines. See ya.

P180 vs PowerMac G4 MDD

2007-10-13T21:44:00.000-04:00

For some reason two people I know recently decided to go out and buy a new PC. They decided to get a desktop PC and to have it custom built for them. When you have a PC custom built you need to specify every part like which CPU, motherboard, Video card etc.. It's a whole lot of fun and one can spend days and weeks mulling over benchmarks and spec sheets to.. well.. just to understand Asus' catalog of motherboards. In any case it's alot of fun and at least one of my favorite bloggers has made the argument that it good for programmers (at least) to build their own PC so that they have a deeper understanding of their tools. This is totally ridiculous of course. If it were true I'd feel some need to make use of my VHDL skill and build up my own CPU or plunge back into the world of assembly of C every 5 years... The reason you want to build your own PC (or have it built for you) is because it's fun dammit!.. A very geeky kind of fun but fun non the less.

(I can't bring up custom computer building without mentioning arstechnica's computer buying guide. Every time I've ever made a custom PC I've done the week of so of research and ended up with exactly what arstechnica recommends. Well, disturbingly close anyway.)

At any rate, the point I was getting to was the question of which case to buy came up in conversation. These days it seems impossible to talk about computer cases without mentioning the mighty P180 case from Antec (or the new, updated P182). The Antec P180/P182 is widely thought of as one of the best if not the best case on the market. One of the most important about this case is the designers made quite allot of effort to make it --QUIET--! I own one. It's quite nice. The doors and sides of the thing are made of some sort of sound dampening material. The drives and fans are mounted on vibration dampening mounts.. It's well thought out.

That said, it doesn't *look* very exceptional. The case comes in black and silver. I bought the silver version and I think it looks like a mini fridge with the front door shut. Needing a name for it on the DNS, I called the it "the fridge". Funny eh? no? well never mind then!

anyways.. So the P180 I have is good and all but I can't help but compare it to the PowerMacs G4 MDD case. I mean they sit side by side on the floor next to me so it's hard to resist the urge.

When I first brought the P180 into the house I was impressed at its height. This is a full tower case. Really it's in a whole different class than the MDD since the MDD (MDD stands for mirror door drive just in case you were wondering) is a half height tower or Mini tower as some people call them. The youngens I think call them that. Damn things keep playing in my yard. Get out of my yard you rascals. Wait, I don't have a yard.. never mind.

The PowerMacs G4 MDD is freaking fantastic. To open it you just use the handle on the side and the whole side door comes open like this:

Notice the lack of cables? The cables are all built into the case. They took the time of threading all the big fat IDE cables through the case. I love it. The case has space for 4 hard disks and two optical drives. Putting in new HDs is incredibly easy and you don't have to worry about the big phat IDE cables getting in the way.. did I mention that?

Upgrading the RAM is easy, you just open the door and the slots are laying there in plain view uncluttered by cabling on the open door. Just plunk in RAM and your good to go.

The computer even runs with the door open. It doesn't like it but it runs.

The experience with the P180 much like any other case in this respect. First there's the screws:

Yep, I took a picture of the screws. They can be undone without a screwdriver but, meh, I didn't need to unscrew anything with the mac. It has a handle with a metal latch.

Then you have to slip the big door off like this:

.. just like every other PC case.. booo.

oh and be very, very careful when you do this. The first time I did this I was having trouble getting the door off because the monkey that made my machine put the side panel on such that some of the panel's slotting hook thingies (technical term) were on properly and some of them weren't. The panel was really on there tight.

So what I do is I grabs the front of the case and pulls the side panel with all my might... you know.. the way you do when the door is stuck. BAM! The damn thing exploded in my hand.. bits of the door hinge when flying all over the room. "%^*&(^!" I yells.

(The picture it a view of the front of the case from the top. You can see the optical drive in the right of the image.)

The P180 has a big front door on it. The idea is the big front door covers up the drives and air intakes and such so the sound is muffled. This door is on a double hinge that lets you fold the door out of the way.. The first picture on the blog shows you what the computer looks like with the door folded to its side. Unfortunately the double hinge is a bit weak. If you put pressure on it it breaks into many pieces. (5 actually).

So.. I super glued all the pieces back together.. That didn't work very well.

and sent an email to antec that basically said

"Um.. I broke the door, can you send me a new one of those plastic door hinge bits."

They sent a new door so I'm happy again but I leave you with the warning: "The door hinge on the P180/P182 is weak. Don't apply any pressure on it or you'll break the hinge into 5 pieces!".. That said it looks like Antec will email you a new one free of charge if you or a former friend breaks the door .. presumably within some sort of warranty period that I should now about but haven't bothered to read. :-)

Anyways.. inside the case looks like this:

.. well... it looks like that if you happen to have a computer inside it.

The P180 puts the hard drives in the little removable compartments so mounting them is easier. It's no MDD but it will do..

See the circle thing that looks like a keychain ring? If you undo the screw underneath and if you unhook it and pull the whole thing comes out. Very nice.

Dear Antec,

Please get an old PowerMac G4 MDD and make a case that has the sound dampening of the P180 but the convenient access of the MDD and I will declare it to be the ultimate case.

Sincerely,
me

Yeah yeah we know.

2007-10-02T11:53:00.000-04:00

If you have an opportunity to get feedback never dismiss it.

Getting feedback from customers is never straight forward. The most important thing, I believe, is that you only get a fraction of the feedback you think you're getting.

When it comes to quantity of user feedback I generally operate on a rule which I discovered while writing Myster. The rule is, you won't get feedback unless your app doesn't run.

This is an exaggeration and simplifications, but it's there to underline a point. Customers don't exist to provide you with feedback so when they feel compelled to give you feedback it usually because they've found something that makes them yell. It can be because you app doesn't work at all, it could be that you've taken away something someone had based their work flow on, and it could be because your app is missing some capability that's vital for its continued existence in that firm.

Now, just because someone isn't yelling at you, doesn't mean that you're product isn't bad. Clients/users don't give feedback if the problem is not large enough to trigger a strong enough emotional reaction to bother complaining about. If your customers are used to dancing bearware then they aren't going to report things that aren't really, really broken. If you're customers are used to something really refined and you suddenly make a big mistake and disable or remove something they really liked then you'll get feedback. In one case your program really sucks so you'd expect to get lots of feedback on where it's broken.. In the other case your program fulfills their needs even with the change and you're still getting angry feedback. This happens because feedback is triggered based on the emotional reaction of events.

There exist some companies that use only customer feedback to guide their development efforts. These companies are usually fairly successful with this strategy however this represents a minimum level of competence. As someone who wants to present the best possible piece of software you need to be more proactive. Solicit feedback wherever possible. Find time to just sit and watch your client working on your software. There are really big gains possible for software that really fits the client needs. If you're software doesn't convert your user base into a heard of zealots there's still room for improvements. :-)

Before I leave this topic i want to point out something. The more "advanced" your users the more likely they are to feel confident to give you feedback. For example, if you're product is aimed at the technical community you can expect to get alot more feedback per user than if you're aiming at the consumer space. Also, users that try out your product early tend to be of the more adventurous type and will also tend to give you lots of feedback as well. Be aware that as your product matures your user base will naturally tend to shift to more casual users (and the user base will get used to your product's quirks) so feedback starts to dissipate later in that product's cycle.

ok see ya.

The Linux loop

2007-09-28T12:50:00.000-04:00

I'm using X but something does work.

Use Y

I am using Y but something different doesn't work.

Use Z

I am using Z but something else doesn't work.

Use X

etc...

Data and type safety.

2007-09-26T09:52:00.000-04:00

What's the difference between :

String path = ...

and

File path = ...

?

For that matter what's the difference between:

int imageId = ...

and

ImageId imageId = ...

?

What is the point of creating an class for an object that contains one piece of data? Well...

In languages with a compiler enforced typing system, like java, it's good practice to leverage the type system wherever possible. Creating types for things like paths or ids can prove very useful. By using the type system the compiler will help you catch more errors earlier. This make the code easier to write and maintain.

When should we create a class for a type? If you look at InteleViewer's code base, we don't always substitute implicit types (of the form "int id") with explicit types (of the form "ImageId id"). The main reason is that sometimes it doesn't give us anything and the class can add a layer of bureaucracy that just serves to make the code more complex.

There's really a set of cases that determine how important/useful it is to turn an implicit type into an explicit type.

Cases:
- You have a 1 variable primitive type in which all values possible for the primitive value are allowed values for the implicit type but you don't do anything that actually needs to rely on the type.

You generally don't need to worry about this case unless your app is getting confusing enough that you want to use the compiler to check your code.

- You have a 1 variable primitive type in which all values possible for the primitive value are allowed values for the implicit type and you are often doing operations on it (formating, funky math etc..)

If you're doing operations on it (especially operations that take an object of the same type as a parameter. Like addition, for example.) then *always* make a class for that type. Dates are fairly good examples of this. They have a mapping to longs but we generally want to manipulate them in respect to each other. File objects are another good example (I think every value String more or less makes sense as a path).

- You have a 1 variable primitive type but there are some values that are valid for the primitive type but not valid for the implicit type.

I generally make a class for this if the value escapes from the object/code module it's in. The reason is that things can get dangerous across module boundaries.

You generally want to defend against badly formated implicit types across functional boundaries so that errors are caught early. Having an explicit type guarantees that the data you're getting as a param is well formed.

- You have a 2 (or more) variable implicit type.

Always make a class for this case.

Oh, one last thing.. Always make your data types immutable if at all possible. See ya.

Amazon vs Amazon. A catfight.. to the death!

2007-09-23T16:56:00.000-04:00

The dollar (US) to dollar (CN) parity has been quite an interesting development recently. Unfortunately it means that Canadian resellers (like amazon.ca) are in danger of being over-run by more aggressive competitors like amazon.com. Consider these prices differences:

For

The Age of Turbulence, Alan Greenspan's latest literary masterpiece, is (at this writing) 26.46 CDN and only 20.99 USD

The story is much the same with DVDs too. Take, for example the BBC's fascinating documentary series on the life and times of other species throughout the universe, Doctor Who. The third series is 87.49 CDN at amazon.ca and only 69.99 USD at amazon.com.

With the Canadian dollar and US dollar parity how are Canadian resellers like amazon.ca going to compete with US resellers like amazon.com? While some might argue that this is only a short term inbalence and that the laws of economics will take care of things, I suspect otherwise! I would suggest massive, arbitrary government intervention to help Canadian resellers maintain their high prices.

P.S.: I am not a crackpot.

Stupid Linux/X-Windows tricks

2007-09-11T11:20:00.000-04:00

Linux doesn't include a GUI component however most Linux distributions will ship with some form of X-Windows. X-Windows has a broken copy and paste implementation that has been the bane of my existence for the last 3 years.

I've just run into an example of how this brain damage makes my life unpleasant. I bring this up here because every time I mention that copy and paste is horribly broken on linux people deny it. Try this on your linux box.

I've opened about 20 files in gedit (a very simple editor that works as a normal window/mac program does.). I've written something in the first tab, copied it, closed the tab and tried to paste it into the next tab. It doesn't work. It doesn't work because you've closed the window that contains the thing you've selected. No other operating system does it this way. Other operating systems, at least conceptually, store whatever it was you copied to a system clipboard. You can then quit the application and your data is safe.

The current linux situation means that I have to copy and past my text into some other window or application so that my clipboard contents don't go away when I close the tab that used to contain the information. This is totally absurd. You want to know why I'm not an Linux fanboy? It's because of stuff like this.