blog.leidegren.se

SOLID software engineering principles

2012-02-05T13:53:00.001+01:00

Back in 2009, the StackOverflow podcast #39, Joel Spolsky started openly bashing the SOLID principles and TDD (Test Driven Development) and claims they were put together by someone who doesn't really write a lot of code. This made me upset enough to check out all the facts and I listened through the Hanselminutes podcast in question. What I heard was quite different from what I think Joel heard. Joel went on about the adverse effect of writing unit tests, that it's probably time better spent doing something else and the absurdity of turning everything into an interface.

I think this story is really interesting, because Joel has this idea that it's crap and a nuisance. But then invites Robert Martin (Uncle Bob) to the show, after he published an open letter to Joel and Jeff. This became the topic of podcast #41. And Joel starts the podcast by apologizing.

Experienced software engineers realize when they are wrong.

But it's important to point out that Joel is also right, in that he creates a situation where you have a programmer that's at 99% code coverage and he's now presented with three choices:

Get to 100% code coverage
Deliver a key feature to a key customer
Dramatically improve usability

Only one of the above choices is wrong and it's obvious why.

At this point I'm 50 minutes in to the follow up podcast and I'm quite confident that they won't be addressing the core issues. Joel been going on and on about GUI testing and how unfruitful that is. However, he's using examples in which he appears to be trying to test layout quirks and not essential interaction logic.

It feels as if the whole discussion is geared towards using SOLID and TDD where it's not appropriate and it's unfortunate because they effectively fail to highlight the importance of SOLID software engineering principles.

The Pragmatic Programmer is a great read, but I think the key takeaway for any such book, is that as a programmer you should be pragmatic. Use the right tools for the job.

To answer the question, why SOLID and TDD? You only need take a look at the real world and make a simple observation.

Things change (and time to market matters).

What SOLID is all about is understanding the underpinnings of SOLID software engineering principles. Why modular code better responds to change and how you avoid friction while adhering to principal object-oriented design practices. It's a lot of fancy wording to say that you can design code to not be tightly coupled and still deliver on schedule, all while quickly adapting to change in the real world.

You're not a SOLID zombie just because you care about system wide interactions.

Now, I often forget that people don't typically share the same history and I sometimes argue my point without realizing that my audience probably don't have a clue where I'm coming from. To ensure that I do not make that mistake here, I'd like to end with why I value SOLID principles.

It started when I was trying to get into the habit of writing tests. I slowly realized that I couldn't test my code because the implementation was in most cases tightly coupled. There was simply no way for me to pull out the important parts and isolate them in a controlled manner, and run meaningful tests.

To enable the above scenario I had to learn new principles and through practice and hardship I eventually got to the SOLID principles.

The point of doing SOLID is that you're testable by design. But to accept SOLID you don't have to accept the entirety of TDD you simply have to acknowledge the value in being enable to do automatic testing, something which you cannot do if you don't design for it.

Testing is a way to drive SOLID designs, which is a way to approach the dynamic business climate.

Looking for a job? We're looking for a web developer!

2011-11-15T14:57:00.001+01:00

Do you like Web and Microsoft .NET related technologies?
Do you like jQuery and MVVM?
Do you live in Stockholm, or are you willing to relocate?
Do you speak Swedish?

If you answered yes on these questions, there might be a job waiting for you. Snow Software is looking for someone to head up our web development. We need someone who understands the web as well as standard and emerging technologies, such as jQuery and HTML5. We also need someone with a keen eye for detail, someone who's comfortable with making aesthetically pleasing design decisions and deliver upon those.

We work with a wide range of Microsoft technologies and it's good if you're already comfortable working with Microsoft technologies.

If you have experience of writing testable web applications, that's a plus!

If you're interested, drop me an e-mail for more information, at john.leidegren@snowsoftware.com.

Initialization, JavaScript and HTML/CSS friendly behaviors

2011-05-09T21:11:00.007+02:00

Way back (early 2009) I started a Stackoverflow question about this very same topic. I didn't find the answer I was looking for but recently I've managed to put some pieces of the puzzle together.

Normally, I'd rely on the HTML id="" property to be a known value and get at it using getElementById or $("#id"). I've never liked this approach but what are you supposed to do? This is how 99.99999% percent of the people I know do it and they think it's fine. Fine!? It's a freaking atrocity. I know it's very subtle but if you think about what your doing, your effectively duplicating knowledge everywhere, filling up your global namespace with a bunch of identifiers and having to go through hoops just to do parametrized initialization.

I think we can do better.

Oh, and If you haven't already gone there, that Stackoverflow question contains a bit of code I wrote as an example and answer to my own question. On how to go about this in an unobtrusive way.

It sort of began with the realization that jQuery addClass, hasClass and removeClass can be used to toggle a Boolean value. The state of this Boolean is reflected in the presence of a CSS class. Without thinking about it, I had stumbled on a system for handing visual states. When these CSS classes represents meaningful visual state information you can use them to style your JavaScript (or augment) using established CSS rules. These things fit well together.

All my client-side validation does is that it toggles an error CSS class at some place and I have a planned out HTML fragment with additional classes that let's me apply a style to the validation state without meddling with the original script.

I started using this more and more and it's truly an ingenious marriage between two technologies but I was still bugged that I couldn't do the same with client-side behavior, or so I thought.

By simply rethinking the way we've (or the way I've done it so many times in the past). I started thinking about CSS classes as more than visual state. What if I reserved a CSS class for a specific behavior?

By reserving certain classes for use with specific HTML fragments I can create very rich client-side behavior without violating the DRY principle, encapsulation or relying on other ugly hacks. Say good bye to printing ("<%=...") strings in JavaScript and hurray for that!

What you do is that you pick a CSS class to identify the HTML fragment and then use HTML5 data to attach properties (think of 'em as parameters). From within a document ready function, using jQuery, you query the document for all the CSS classes and do your initialization here. You have a reference to the DOM node, data availability (parametrization) and complete encapsulation (no publicly visible interface, except for the data members).

The Stackoverflow question contains a simple sample where I use this trick to add regexp validation using CSS classes and HTML5 data only (assuming that the script is already written). I can reuse this code in many different places without having to go back to my JavaScript or "hook up" these things thanks to the wonders of jQuery and CSS selectors.

I think this is something truly beautiful.

Hidden features of Visual Studio, enter watch expressions with format specifiers

2011-04-15T11:30:00.007+02:00

Every now and then I find myself getting annoyed at these little refresh squiggles that the Visual Studio watch expression get if they potentially have side-effects. I reckon that's a good thing, however, sometimes I need to express a quick way to drill down some data structure and get at that specific value.

Format Specifiers in C# (see MSDN)

These format specifiers are for C# but some of these are language agnostic and some variations of these work with other languages.

There's more information on MSDN. But these are my favourite ones, the ones I try to remember. Because when I need them, I have know that I can use them.

Specifier	Format	Value	Displays
ac	Force evaluation of an expression. This can be useful when implicit evaluation of properties and implicit function calls is turned off. *
d	Decimal integer	0x0065	101
h	Hexadecimal integer	61541	0x0000F065
nq	String with No Quotes	"My String"	My String

* See Side Effects and Expressions.

And this is how you use them...

Watch
Name	Value
Text,nq	string without quotes
source.Skip(2).First(),ac	The 3rd element of an enumerable source
0xa,d	10
10,h	0x0000000a

The d and h format specifiers are especially nice when you find yourself toggling Hexadecimal Display on and off. And as a bonus, If you'ven figured this out yourself, these expressions also work in the DebuggerDisplayAttribute string. Very handy.

VSX Mercurial

2011-03-07T23:32:00.007+01:00

I have a habit of leaving things unfinished if I find my self having solved the nitty gritty details. That last stretch stuff just isn't as fun as the technical challenges. Though that's a topic for another day.

This week I contracted a bad case of the flu. I've spent the last several days in bed trying to regain some strength. Laying there thinking (not really accomplishing much of anything) I thought I'd reboot that source code control plug-in for Visual Studio.

The thing about Mercurial (and this is equally true for git) is that it's super easy to get started. And with it you can distribute, patch and hack away effortlessly in mere minuets. What I felt was missing was a source control package for Visual Studio that embraced the same principles.

For all the SVN tools out there TortoiseSVN always struck me as odd because it attempts to turn every last folder in your computer into a version controlled system by integrating with the Windows Explorer Shell. I never got that and was somewhat surprised that TortoiseHg even existed. Mercurial not being SVN makes it somewhat less of an atrocity but I can't believe that people actually wants to use these GUI tools.

Unhappy with the performance of existing plug-ins for Visual Studio I decided to roll my own and I figured that it be a good opportunity to learn how to build Visual Studio Extensions.

I actually have decent statistics of the number of hours a day I actually spend within Visual Studio actively doing stuff. Learning more about the Visual Studio extensibility model felt like a smart move.

I have a very clear vision of where I wanna go from here, it's also my attempt at an open source project. I don't really have any strong feelings about open source but I've always learned a lot by reading source code. This could be a good opportunity to give something back.

~~If you're a Visual Studio user looking for a Mercurial SCC plug-in checkout VSX Mercurial on Codeplex~~.

I abandoned this project sometime ago. Mercurial is still a great tool but I use TFS and git today. I also find that the kind of integration I was planning with VSX Mercurial wasn't really what I needed, git extensions taught me this.

Microsoft PDC 2010

2010-11-09T07:17:00.005+01:00

Windows Azure is the big topic of this years Microsoft PDC. I haven't payed much attention to Windows Azure in the past but after the PDC I actually felt the urge to write some code in the cloud (hilarious isn't it?).

The developer experience for Windows Azure is what you'd expect with Visual Studio 2010. The transition is easy to make, and the big difference is primarily on the Windows Azure Storage side of things. That did not sit well with me, and it took me a while to figure out that the out-of-the-box tools don't really support the development storage, which is what you use if you haven't paid for an Windows Azure account.

There's a lot of pesky things about the Windows Azure Development Storage that's not entirely obvious. e.g. there's only one development storage account (which limits you to exactly one database at any given time), it can be completely wiped simply by clicking a button on the storage service client and there is no built-in tool for managing the storage (only querying from within Visual Studio is possible). Given that I might wanna switch between different projects, I thought I'd keep my schema and data primarily in scripts, that made sense until I realized the way to talk to the Windows Azure Storage service is through this REST API. That's fine though, but not if you're expecting to write T-SQL scripts. Windows Azure Storage, is a no-SQL storage and fundamentally different from what you get out-of-the-box with SQL Server. I realize that's a good thing for writing scalable software, but it did catch me off guard. I believe I was expecting something along the lines of what Microsoft is offering with SQL Azure.

I'm interested in learning more about Windows Azure, but this sort of thing doesn't not help with that. Eventually, I would put up the money for it, (truth be told, it's not that expensive). However, it's hard to get an overview of the what you need (in terms of computing power) and the final cost when running in the cloud is pretty guess work. Though, you can scale back to whatever supports your revenue stream as well as scale up, if necessary.

It's likely that my concerns right now, will fade away with time. As I learn more about the platform my ability to make the right decision will certainly increase. Though, as of today, the first-class developer experience I've come to expect from Microsoft with Windows Azure is not quite there.

The hardest problem with computing is people

2010-10-31T11:37:00.005+01:00

Gilad Bracha said this during the Programming Languages Panel at Microsoft PDC 2010.

The hardest problem really is this whole issue of compatibility that plagues software, that is so brittle, you can never take things out, you can never fix things early, you can just add stuff, and therefore contribute to bloat and the problem is not so much with the technology as it is with the people. The hardest problem with computing is people, and you know they need to be replaced but we cannot do that. Ultimately we need to find ways for software to evolve and update itself in such a way that it can not only grow but shrink.

Hopefully I'm not misrepresenting Bracha or quoting him out of context when I say this.

I've been looking into software composition and clearly found great use for dependency injection. Inversion of control IoC (IoC) represents a step in the right direction and dependency injection (DI) is the means to achieve both IoC and composition. As a proof of concept of this, I was refactoring some code written by co-worker this Friday, moving things into reusable services, and surprisingly yet satisfyingly so, I'm removing more code than I'm adding. The result is not necessarily a smaller code base, but the complexity is certainly more manageable (on top of being testable). I think this is what Bracha is getting at.

The benefit is that there's no longer a tight coupling between these objects. Which is what's allowing me to actually move this code into a separate assembly altogether. If I can partition my software into isolated compartments then that's a step in the right direction. What you get out of this is a problem which focuses on the management of several smaller instances instead of one very large code base and through experience, I've learned that managing complexity on a smaller scale is preferential to a big ball of twine (intermingled dependencies).

The compositional approach is of course not free. Nothing, is. For one, a loosely coupled software architecture that is compositional requires lots of abstractions and you should be aware that there's a fair amount of work to be done to orchestrate the composition of the software itself. Some work is going to focus on writing auxiliary software to make deployment and configuration easier.

I'm convinced that composition is the way to tackle software aiming at solving a diverse class of problems in some manner which is consistent on a higher level. It enables great flexibility without sacrificing architecture, testability or maintainability at the cost of initial bootstrapping when new requirements surface.

Windows Phone 7

2010-10-23T10:03:00.003+02:00

After attended the Windows Phone 7 launch event here in Stockholm, Sweden. I started thinking about the Windows Phone 7 experience in a new way. Having read everything I could find on the Internet actually using the phone didn't really change my opinion. It's still a solid touch device. But one thing that became abundantly clear was that the phone can not be experienced through others.

I say this because so much of the phone experience is tied into social networking, such as Facebook and Twitter. If the phone is hooked up to your contacts it's personal and talks to you in a different way. The problem is that you cannot not buy the phone and get that experience, you basically need to rent it and spend time on it, before you can make a decision.

The Windows Phone 7 experience appeals to me differently than the iPhone and Android and if you have a chance to check it out, you should!

Moving to the cloud

2010-07-26T18:28:00.008+02:00

I've more or less relied on my hosting company to advocate my own existence on the web. With the changes done to Blogger over the years I've been using it, it has pretty much come to a point where everything I need is in one place. All in all, it's really convenient.

I can make changes whenever and wherever I like. And there's no longer any source code to compile. Normally I'd have a small back-end that grabbed my Blogger and Twitter feeds. But as I recall I've misplaced that source tree at least three times already. With this change the source code is now up in the cloud. And if you're about to right-click and view source. I want to point out that I'm in no way responsible for the additional markup and scripts being generated by Blogger.

System.HardwareAccelerated

2010-07-04T12:17:00.009+02:00

Please support my campaign for adding SIMD extensions (and other hardware accelerated features) to the Microsoft CLR implementation. Follow this link and vote for this feature on the Microsoft Connect website.

The x87 floating-point math extensions have long been one of the ugliest legacy warts on x86. Stack-based and register-starved, x87 is hard to optimize and needs more instructions and memory accesses to accomplish the same task than comparable RISC hardware. Intel finally fixed this issue with the Pentium 4 by introducing a set of SSE scalar, single- and double-precision floating-point instructions that could completely replace x87, giving programmers access to more and larger registers, a flat register file (as opposed x87's stack structure), and, of course, floating-point vector formats.

Intel formally deprecated x87 in 2005, and every x86 processor from both Intel and AMD has long supported SSE. For the past few years, x87 support has been included in x86 processors solely for backwards compatibility, so that you can still run old, deprecated, unoptimized code on them. Why then, in 2010, does the CLR emit x87 instructions, and not scalar SSE or, even better, vector SSE?

I lifted this text from Ars Technica and changed one word "PhysX" for "CLR". It's totally out of context but totally applicable at the same time. The CLR is an abstraction, it has the power to emit any instruction set best suitable for any platform.

A Big shout out to VMware

2010-06-24T16:46:00.003+02:00

When travelling pack only the essential stuff. Well, I just down graded my MacBook installation from Windows 7 to OS X 10.5 becuase of all the driver troubles but without my development tools I don't dare leave my home.

Not surprisingly, I turned to VMware Fusion to bring the stuff over that I enjoy so much. But what's amazing is that I've set up a brand new VM running Windows 2008 R2 (which is 64-bit) on my 32-bit OS X installation and then installed Visual Studio 2010 in just under 2 hours, and it just works!

I can't possibly imagine my MacBook without VMware Fusion and I'm defintely going to upgrade to the 3.1 release as soon as the trial has played out it's role.

Operator Overloading Ad Absurdum

2010-06-11T09:56:00.002+02:00

Short post, and nothing more than a link of what someone else wrote, but a good read, none the less.

http://james-iry.blogspot.com/2009/03/operator-overloading-ad-absurdum.html

Microsoft, Google and Security

2010-06-06T01:07:00.006+02:00

On May 31 2010, Financial Times reported that Google was going to phase out internal use of Microsoft Windows due to security concerns. All new employees are given the opportunity to chose between a Linux or OS X workstation. This action has spurred some rather lively discussions about what the actual reason for ditching Windows and whether it ever had anything to do with security, or for that matter, if Windows is secure or not. Ultimately, many of these blog posts, articles and rants ended up spreading misleading information about security and Windows security in particular.

With Windows being the most widely used OS. The threat against Windows is bigger than any other OS. Everybody is using it, and everybody is watching. A potential exploit can have a big success simply because enough users don't know how to protect themselves. Yet, I've read articles were people argues that Linux and OS X are better alternatives because they don't get hacked as much. This argument has utterly nothing to do with security and it's an irritating piece of information used by people to fuel the debate.

That Windows is less secure simply because it's being targeted a lot is a naive to say the least. What you should be looking at is the success rate of attacks carried out against users running Windows, which has been in steady decline ever since the release of Windows Vista. People also tend to pick on the UAC, in all fairness it is/was annoying as hell. But the truth is that it was designed that way, to pull the majority of software away from running in an elevated mode, a potential security risk.

As an attacker, if you target Windows, that's where you'll most likely find a valuable target simply because that's where normal people, like you and me do our banking and what not. What I'm trying to say is that the threat against Windows is real, the threat against Linux and OS X, not so much. I would argue that most people running Linux are more tech savvy and less likely to make the same mistakes as the average user running Windows, but you can't hold that against Microsoft or Windows because it's software designed to run software. Not software designed to protect you from danger. Windows (or any other OS for that matter) is not intelligent enough to stop you from doing something stupid (creating new vulnerabilities).

In the case of the IE6, the exploit utilized by the attack, was carried out against a decade old deprecated browser. Not Windows it self. If you allow just any software to run inside your machine, you're begging for it. What fair chance do you think a vendor like Microsoft has to protect you from a potential threat if your not serious about protection yourself? The truth is, that you, the end user, is left with a lot of power, that if you make one mistake, it can be devastating no matter the design of the OS. Many use this same example and jump to the conclusion that Windows is flawed by design. Which is absurd. There are people at Microsoft both engineers and researches that focus their entire day on preventing potential exploits from ever materializing. And they do a lot of good. But they can not stop you from pressing the red button. They can only make the warning label bigger, but typically what happens is that the user ignores the warning and clicks the button anyway. What diligent system would ever exist to prevent you from running arbitrary programs inside you computer? Isn't that the purpose of computers? to run programs? Yet, people argue that running programs are less secure by design, but that's a risk we have to take.

Security, real security has always been about establishing a network of trust. If you don't know the origin of a program, file or document, you can never presume to say it is safe. You will take a risk, every time you click something from an unknown source. And the computer has been placed in your hands to do as instructed. Think twice about clicking random things and keep your software up to date.

An interesting side note also is that computer programs, typically viruses, can't install themselves or spread between computer unless there's something to exploit. In most cases that something, is you. You opened that attachment and you downloaded that file. In the unlikely event that there was such a serious bug in a already installed program would you be at risk, but then, you trusted that program because you installed it.

Real security asks a lot of you, and you have to be willing to face these challenges if you wanna stay secure, typically the best way to protect yourself is to educate your users and make them aware of the risks. Windows is not the problem, it hasn't been a problem for many years.

The Managed Extensibility Framework (MEF) More on Part Creation Policies -- Enter the ExportFactory

2010-04-12T20:48:00.010+02:00

Last I blogged about part creation polices and why it might be a good idea to not say anything about them, and leave them as their default, "Any". This time I'm gonna talk about the ExportFactory which is what you'll wanna use for your object factory needs in the future.

Unfortunately, the ExportFactory didn't make it into the .NET 4.0 RTM build and that's a darn shame. However, MEF being open source and all, allows one to easily rectify the problem.

You can find all relevant source here. It contains a link to compiled binaries for several .NET Framework versions and source code that you can drop into any existing framework/core assembly you have. I've done this with the .NET 4.0 RC version of MEF without much of a problem. But you'll need to configure your CompositionContainer some more.

var catalog = ... // get a catalog here
var exportFactoryProvider = new Microsoft.ComponentModel.Composition.Hosting.ExportFactoryProvider();
container = new CompositionContainer(catalog, exportFactoryProvider);
exportFactoryProvider.SourceProvider = container; // a bit odd, but necessary

The ExportFactory is used in the same way as the Lazy<T,TMetadata> type and will create instances when you ask for them. This conveniently avoids the hassle of having to manually compose objects created through the common factory pattern. The instances served by the ExportFactory will already have it's imports satisfied, so no additional steps are required. MEF makes it fun!

e.g.

[ImportMany]
public ICollection<ExportFactory<IMyObject, IMyObjectMetadata>> Objects { get; private set; }

I ran into problems when using constructor injection, if you follow the link above you'll find out how I worked around that.

The Managed Extensibility Framework (MEF) Part Creation Policies

2010-04-04T02:25:00.003+02:00

I'm gonna kick of a series on MEF partly for my own benefit, but maybe this will make sense to you as well. However, this won't be a beginners tutorial.

MEF ships with 3 types of part creation policies, Any, Shared and NonShared. By default all exports get the CreationPolicy.Any. What's a lesser known fact is that this allows the consumer -- imports -- to specify either CreationPolicy.Shared or CreationPolicy.NonShared.

Both attributes [Import] and [ImportMany] has a RequiredCreationPolicy property which let's you specify how the imports are satisfied. I tend to prefer to imply the correct behavior on the consumer side and not mandate that the producer has to think about it. With the default policy being CreationPolicy.Any a convenient factory pattern just imports with the CreationPolicy.NonShared. And if you're importing a collection you just don't have to worry about the composition failing, because if it's not working they're doing something wrong, it's not your problem.

Of course if you export a specific policy and import the other policy, the import is not satisfied which can lead to an unexpected exception being thrown.

Best of MIX2010

2010-04-01T15:20:00.004+02:00

DS13 The Elephant in the Room

About 5 minutes in I felt I had to stop watching this, but it turned out to be the best presentation I've seen at the MIX 2010 conference. Thank you Nishant for an hour and a half of delightful material.
DS16 An Hour With Bill Buxton
DS07 The Art, Technology and Science of Reading

I would also recommend Joe Belfiore presentation CL01 Changing our Game – an Introduction to Windows Phone 7 Series and the others that continue the Windows Phone 7 Series. It's a really interesting piece of technology and beautifully designed with great potential.

New unit for speed, courtesy of Microsoft Windows

2010-02-15T17:44:00.001+01:00

Difficult != time consuming

2010-02-14T09:10:00.004+01:00

There. I said it. Difficult problems are problems that lack a solution. Problems that can be solved, problems that have many existing solutions today, are not difficult. Not all problems are difficult, but they are time consuming and it would be preferable if they could be solved by someone else. The problem with such a solution is that things change, and when you rely entirely on other people for your solutions, you'll lose the ability to adapt in a situation when your out of the box (component) or solution can't sustain your domain. That's why you sometimes have to do things yourself, in your own way. Because no one else understands your problems that well.

Publishers want a law to control online news access

2009-07-12T22:14:00.003+02:00

I just recently stumbled upon this comment while reading this article over at Ars technica.

Maybe I'd feel a little more sympathy for the press if they actually did their jobs nowadays. Rewriting press releases in your own words is not journalism. Reporting on what is on Twitter is not journalism. Running yet another story on the most popular video on Youtube is not journalism. It's mimicry. Very little actual journalism happens anymore. The rare exception is local news, where you do get reporters going to committee meetings, calling multiple sources, and in general doing research. But national news? Eh. They do little more than rewrite press releases.

Evolve or die.

I couldn't agree more, I don't watch TV and generally avoid newspapers because of the sub-par writing. I tend to prefer my first-hand sources, here on the Internet.

What bothers me is that it seems like someone, or something is trying to control the flow of information... this is why we built the Internet and it's not about control, it's all about sharing!

MIX09

2009-03-18T13:10:00.004+01:00

Doug Purdy and Chris Sells talks about developing RESTful Services and Clients with "M"
Seema Ramchandani also gives two decent talks about Silverlight graphics and preformance, I reccomend both for anyone intrested in Siliverlight or WPF.
The return of Johnny Lee with more Wii and HCI stuff.

Go to the http://videos.visitmix.com/MIX09/ website and check out the rest of this year's presentations.

Rotten apples

2009-03-14T17:56:00.012+01:00

I bought my first Apple product (a MacBook) late 2007, I was a proud owner back then, and it was a absolutely wonderful design.

For roughly 6 months, I learned new things about Mac OS X and it was good. I was motivated to learn more about UNIX, and dug into Mac OS X development with Xcode. I had never intended that the MacBook would replace my desktop use enterily, but through a number of unforeseen events it has. However, it wasn't possible for me to entirely leave the Win32 platform behind, I'm a Win32 guy at heart, and I managed to bring that to Mac OS X with the help of VMware Fusion.

As of this date, I own, a late 20007 MacBook, a Apple Mighty Mouse and a 4G iPod nano. I've been using Apple products and Mac OS X for about 18 months now...

From this point on, it's all down hill... and I believe this Mac vs. PC debate is mostly silly, and in large, fueled by people with no knowledge of how a computer actually works. Some people will argue that they don't care about that, and they shouldn't be caring about it, because it is not in their job description to care about it and I respect that, but don't go make false accusations about things you don't understand.

The reason why I'm abandoning ship and going back to Windows, is because of Microsoft, they fix problems. I'll admit that Windows back in the day, was a troubled experience, but so was all pre-Mac OS X releases. Microsoft today, is very keen on listening and helping out, and since hardware, that runs Windows, ain't proprietary Apple branded and redicoulously expensive. Any third-party with the knowledge and experience can fix driver issues. I've had issues with OpenGL support on my late 2007 MacBook, and while there were driver updates for Windows, Apple left MacBook users like me, in the dark to rot. And that's another reason why I'm going back, Windows is an open platform, not open source or free as in freedom but accessible, by any hardware. And these graphics issues happen to have been solved by Intel, but there is no Mac OS X download, just a Windows version...

Mac OS X didn't really shine until they went POSIX and replaced their kernel with a UNIX variant. But what's distrubing about that, is that Linus Torvalds (in this Tech Talk) calls on Mac OS X to be an even worse platform for software development than Windows, and Linus knows a thing or two about UNIX. It's like Ballmer said, we're paying $500 more, for a brand! And while that's a feat in itself, I doubt Apple can keep this up for much longer. While the MacBook and Apple logo is everywhere, surprisingly few people run Mac OS X.

Right now, I'm in the process of installing Windows on my MacBook, but while downloading some required bits to format my iPod, I'm getting a disk full error. I bring up Finder, it says I have 166 GiB of disk space available... This is just one of the many problems I've experienced with Mac OS X, and it's about to go away.

HOWTO: Install Windows Vista/Server 2008 on a late 2007 MacBook3,1

First off, this is about liberating the MacBook, you'll only need some of the Boot Camp drivers and the firmware update which emulates a BIOS. Once you have this you're done with Mac OS X. You'll only need the Mac OS X installation DVD once. Derek Hatchard blogs about the specifics. You just use it to partion your hard drive with a MBR. To my knowledge no MacBook supports booting of an USB stick so you'll have to have a Windows installation DVD. The rest is simple, you can boot the Windows installation of the DVD as you would normally. My disc drive gave me a hard time but being presistant helped, after poping in and out many times it eventually completed the installation, be presistent!

I had major issues with the suttering audio and had to install a very specific driver to get around this. After manually and forcefully installing the Broadcom v5.10.38.26, "Broadcom 4322AG 802.11a/b/g/draft-n Wi-Fi Adapter" things began to work fine again. Unfortenetly, I could not find a reliable way to determine which hardware matches what driver, it looked like most MacBook users went with the Broadcom 4322AG driver so you could start with that, it worked for me.

Shearing my life

2008-12-27T20:40:00.006+01:00

I could seriously do for a indefinite timeout. With all the things in the world that needs to get done, 24 hour days ain't nowhere near enough.

I been wanting to make a website based around some 3D element. I got curious and looked at websites that did similar things, trying to figure out the simplest way of 3D-2D mapping. Skewing or shear mapping (which is another fancy name for a bizarre transform) actually preforms the kind of transform I've been wanting to do and it isn't really that difficult.

I began figuring out the geometric interpretation. I'm very much so a visual person in that I have to picture things before I can argue about their function. The above picture just came to by applying a really trivial skew transform over a period of time.

...but I need more time, all the things to do, all the things to see. I have to write up my Master's thesis, I got two jobs on the side which has to get done this week and I should really look for a third more permanent job. My time at DICE has been most wonderful, its truly a great place to be working at but I'm not certain I'll be able to continue once my internship is up. EA announced a 1000 people layoff 'til the end of March. My hope is that they let HR hire people along the side...

And apparently some stupid company got a 10-year old patent approved today. The consequence is that if you generate icons based on the contents of a file, you will be slapped with a lawsuit for patent infringement. Microsoft, Apple and apparently Google are on trail. What the hell is up with the USPTO!?

PDC2008

2008-10-30T08:56:00.001+01:00

The Future of C#
"Oslo" - The Language
Mono and .NET (kodus Miguel!)
Contract Checking and Automated Test Generation with Pex

Piracy, causality and PC gaming

2008-08-17T11:06:00.025+02:00

Piracy is the result of an elite group of reverse engineers that mostly got bored with everything else and took upon themselves to face a greater challenge. Software (and games in particular) is not 'cracked' because it's a good for you. It's the way some talented people choose to deal with boredom. And in a sense combating piracy in the realm of software engineering through legal actions is like a Sisyphean challenge. There will always exist bored, beautifully talented people that will be able to circumvent the inner workings of whatever DRM software you imagine.

If you want to pickup and play a game, the time it takes to get from that feeling in your brain to actually playing, will probably end up determining if you're going to play at all. There's this notion of availability which I do believe that consoles have in advantage but it's not exclusive to consoles. Getting from the point at which you decide to play a game up to the moment where you've bought and installed the game is an important deciding factor when combating piracy. Which ties together accessibility and distribution, that is, the very core of information technology.

The following quote was taken from the Hanselminutes #89 podcast (the context is console security).

"You know, one of the things people don't understand is some of us have like scanning electron microscopes in our living room."

He puts up a slide that has a picture of his living room and darn if there isn't a scanning electron microscope next to his couch and he uses that. He sands off the epoxy on the top of the chips of the things he's trying to hack and he uses this electron microscope to read the internal circuitry to find out what it does.

Outsmarting an army of people with an endless amount of resources, is just silly i.e. ByteShield which have written a very interesting and probably plausible way of truly protecting software, is flawed in that it's a system that people will take great pride in circumventing.

But what about availability? I'm congratulating Valve for their premium free of charge service Steam. Because it's everything the industry needs to be. Almost $2 billion in sales was collected from Steam-like services in 2007 (that roughly 21% of the grand total according to the ESA).

Download services that won't cut it, will basically sell you a single download opportunity and once that window closes your purchased content is forfeit. One of the reasons I enjoy Steam so much is that I can bring my games with me, anywhere I go. I just install their bootstrapping software Steam and let the software prepare the games I wish to play. It's as simple as that.

I'm saying that it all comes down to this very moment, were the sheer number of clicks you'll have to make to get the game to your hard drive is a deciding factor whether you're going to buy that game or not. People that understand piracy will be able to grab a seemingly bug free copy of the game for no money at all and start playing without having to invest any time into the process of acquire the game itself. This is where you (as a business exec) will have to be smart and competitive.

Now, Steam is free of charge, no subscription required, it's not pay-per-view and that's important, because you cannot compete with people that crack software just for the kicks of it. Not by taking money for something they'll provide for free. This is all about value and aggressive pricing. Even though these talented reverse engineers are great at what they do, they make mistakes and the quality can vary a lot. Many cracked games bring about new bugs, bugs that otherwise would not occur and any non-engineer would presume the game is faulty and might end up NOT buying the game based on false grounds. But with cleaver marketing, you'll pay that extra money to get that full experience.

Peter Moore is a man I've come to respect. He said this in a response to some rather harsh legal actions taken by Atari, Codemasters, (Topware, Reality Pump and Techland).

Now, if you want people to buy your software, you'll have to find value in the purchasing of your product as a full service package, not in the sequence of bytes alone that you'll end up shipping. For this, there's no single answer, instead there are numerous of carefully planned steps that need to take place.

Make a great game and if you think shipping NOW with what you got is the only way out, you're already fighting a losing battle. At this point you're better off saving what you can and starting over. It's hard yes but it is the right thing.

Michal Kicinski, co-founder of CD Projekt. Managed to sell 1 million units of The Witcher by sticking to a specific audience. You can read the rest here. What will you do to make sure your game sell?

Fast binary visualization (to hex conversions) and back

2008-07-01T23:48:00.010+02:00

This spin-off started with the immutable System.Data.Linq.Binary class. It's somewhat irritating because it does not expose any read-only capabilities. As such one must either rely on the ToString method which surrounds the base64 encoded value in quotes or the ToArray method which uses defensive copying.

At this point, I wonder why they never defined a public getter, like with the string class (string s; s[0]). Since what I need is to be able to serialize this object. Object serialization comes in many flavors, I've chosen one which utilizes reflection to emit a parser collection of strongly typed serializes. It's way faster than what you get with any of the built-in stuff but I've purposely chosen to ignore the System.Data.Linq.Binary because I didn't use it, up until now.

SQL to LINQ maps timestamps to the Binary data type and you need timestamps to apply optimistic concurrency. So I have to able to treat them as well. And this is where this hack'ish hex stuff comes in.

In the end, timestamps are 8-bytes in size. So I accept the defensive copying overhead that takes place. I rely on the ToArray method and .ctor(byte[]) for getting and setting the internal representation.

When you look at hex, the 4 high and low bits of a byte is usually referred to as a nibble. A nibble map to 1 hexadecimal character, usually in the range of [0-9ABCDEF] but why? There really no relation between the binary value or the ASCII representation.

I gave it some thought, looked at the bit patterns and this is what I came up with.

Each nibble represent a 4-bit integer in the range of 0-15. As such if I were to use a lookup table I could map this space to any 16 characters, but I chose to not do that. Instead I searched the byte range for a bit pattern which would not involve any lookup map and still remain a presumably safe ASCII representation.

Funny enough the commercial at '@' character happens to not use the low 4-bits of a byte and the following 16 characters constitutes "ABCDEFGIHJKLMNOP", the beginning of the alphabet. What's even more coincidental is that the caret notation, is exactly this (Ctrl+C is really ASCII character at zero-based index 3).

But enough talk, let's look at the code:

public static string Serialize( this System.Data.Linq.Binary binary )
{
    StringBuilder sb = new StringBuilder();
    byte[] bytes = binary.ToArray();
    for ( int i = 0 ; i < bytes.Length ; i++ )
    {
        // Unpack
        sb.Append( (char)('@' | (bytes[i] >> 4)) ); // '@' 0x40 | (i & 0xf)
        sb.Append( (char)('@' | (bytes[i] & 0xf)) );
    }
    return sb.ToString();
}

public static System.Data.Linq.Binary ToBinary( string s )
{
    byte[] bytes = new byte[s.Length >> 1];
    for ( int i = 0 ; i < bytes.Length ; i++ )
    {
        // Pack
        bytes[i] = (byte)((s[i << 1] << 4) | (s[(i << 1) + 1] & 0xf)); 
    }
    return new System.Data.Linq.Binary( bytes );
}

The above code in C# can be used as is. But those of you that think bit manipulating is a bit daunting, I recommend you to take a while to understand the code (make sure you fully understand the impact of explicit and implicit type casts in conjunction with bit arithmetic).

Each nibble does not occupy the 4 high bits of each byte and since we want to be able to represent them as strings, the bitwise | with '@' (0x40) will give us a very nice ASCII representation. One which is also very efficient to parse, as each nibble is stored unmodified together with the '@' character. We simply bitwise & with mask 0xf to get just the low part, and then bit shift everything else into place. Another nice property of this method is that the ordering of the binary value does not change with the textual representation.

This bi-directional relationship actually wants me to stop using the old fashion hexadecimal representation altogether and start using this caret notation stuff. I reckon it's just wrapping your head around another numerical representation.

There's 10 kinds of people, right?
Those who speak binary and those who don't.

Update July 25, '08: I've observed that the commercial at character '@' is sometimes encoded, despite the fact that any documents I could find on the subject says otherwise. So if you want a pure ASCII and URL-safe binary representation add 1 just before the serialization and subtract 1 just after the de-serialization. It will do the trick.

Type-casting as bugs in C#

2008-05-31T10:44:00.006+02:00

I've been noticing more and more that people use the 'as' keyword in C# to presumably do safe type casts. But enough is enough, it's not ideally the way to actually go about type conversions, nor does it have any benefits.

To start things of, the expression 'instance1 as SomeType' keyword is just the equivalent of '(instance1 is SomeType ? (SomeType)instance1 : null)' and the following code will throw an exception anyway, iff the type is not instance of SomeType... So why do you type cast this way?

[TestMethod]
public void Index()
{
    // Setup
    HomeController controller = new HomeController();

    // Execute
    ViewResult result = controller.Index() as ViewResult;

    // Verify
    ViewDataDictionary viewData = 
        result.ViewData as ViewDataDictionary;

    Assert.AreEqual( "Home Page", viewData["Title"] );
}

The above example is from the ASP.NET MVC Framework, and while it's just a sample from the testing framework this code is bad. Because when the right-hand side 'controller.Index() as ViewResult' returns null (the return type was not of type ViewResult) and this code should, because this expression implies that type does not have to be ViewResult. There is no error handling code and the result is the NullReferenceException, which at this point is a side effect of some other code that failed.

Do you know the null coalescing operator? It's about the most useless thing in the language, or is it?

(controller.Index() as ViewResult) ?? new ViewResult()

Now, if you really want the program to crash in the event of such a failure you should NOT do this, nor should you yield null in the case of a type test failure. But if you want to actually type cast and in the case of a problem fall-back to some instance, you can use the above snippet.

Think about it. The example is really stupid because it fails to capture that an error could (or could not occur) as well as that if it would, the wrong error would be reported. But this snippet, it can actually make sense of a failed type test and rely on any other code to provide an instance of some type that doesn't have to stop execution of your program.

Bending the Command Prompt to your will and why I don't use Microsoft's PowerShell

2008-05-18T10:31:00.014+02:00

I'm a Windows guy for sure, but I bought a MacBook because they are kind of cool. And I occasionally write code for both OS X (BSD) and Windows and often so use command line tools.

One thing that Windows kind of have been missing is a terminal, with capabilities more similar to that what you get from Unix/Linux (from now on referred to as -nix) based operating systems. Microsoft has noticed this and they believe PowerShell is the one solution to all your scripting and terminal needs. I have a couple of problems with Microsoft's PowerShell and that is why I write this.

Auto-completion
: PowerShell auto-completion is completely subpar, not only does it fail to recognize aliases or commands in your path environment variable, it rewrites partial paths to full suggestions which covers your prompt. You end up cycling several suggestions before you run into the right one, or the completion in the worst-case will suggest a complete name, while you only want part of that. It makes me think that the guys who wrote PS did not have any background experience in NIX based operating systems #!/bin/bash.
Output: PoweShell does have a somewhat nice way of pretty printing output from different commands, but it's bloated by too much white space. I'm not very keen on output that will take up my whole screen, I want something which is mean and lean as often as possible. PS fails to achieve this with several commands, such as the basic ls or mkdir commands.

That Microsoft currently does not support SSH is just sad, but if you want to be able to SSH to your Windows computer you can always rely on Cygwin (though, Cygwin is bundled with so much more). Anyhow, this post is about making the good old fashion Command Prompt behaving more like a terminal you're familiar with if you know -nix based operating systems. The Command Prompt itself is actually alright.

Some sugar

Font-rendering capabilities are getting better in Windows, with ClearType you get true subpixel rendering which makes things more readable, so why don't apply this to your terminal. All you need is a TrueType font face with ClearType support. A monospaced font you might know of which is excellent for terminals and code is Consolas. I use it for everything code and it is freely provided by Microsoft.

Install this font on your computer and fire up the registry editor.
Goto HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont
Add a string value "00"="Consolas"
Reboot (you'll have to reboot for the font to be available in the command prompt settings dialog)

Commands

What I do is that I, set up a PATH variable to point out a bunch of batch files that reminds me of -nix. I create a file ls.bat and put a single line: "@dir /w" in that file. This way the DOS command ls, will actually give something which is more similar to ls. You can tweak around with this to get everything you might need. (I find "dir /w" a lot more useful than just "dir")

This is just a few tricks I ended up using after realizing Microsoft's PowerShell annoyance. The main reason why I "downgrade" is that the auto-completion in PS truly is worse than the old fashion cmd.exe way. I'm quite certain that there is a way to just replace the way this is done in PS but I will not waste my time pursuing it. Cygwin does come with almost everything you need but if you don't want to install Cygwin, you could make do with these tricks.

I primarily use the Command Prompt for svn, because I loath TortoiseSVN. And I wrote this in hope of some realization from people using PowerShell, in that it falls short on some very basic things.

Happy prompting!

In response to Jeffrey Snover's comment

The above mentioned problems can be further exemplified by the following:

C:\Users> ls


    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Users


Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         5/15/2008   1:07 PM            leidegre
d-r--         11/2/2006   1:49 PM            Public
d----          3/1/2008  10:45 PM            sshd_server
d----          3/1/2008   9:35 PM            SvcCOPSSH


C:\Users>

leidegre@monkey-intel /c/Users
$ ls
All Users/   Default/  Default User/  Public/  SvcCOPSSH/  desktop.ini*  leidegre/
sshd_server/

leidegre@monkey-intel /c/Users

The above output illustrates the vast differance in PowerShell and Cygwin running bash. The Directory: followed by a CLR type? I definitively don't need to know when the directory was last modified, nor what attributes are set (if I was I would set a flag for that), moreover the second example is from Cygwin using bash, and it uses colors to show that file actually is folder. The old DOS command "dir /w" used square brackets around the [file name] both are fine mean and lean ways of displaying the appropriate information.

Here is more examples from that horrible output formatting, the pwd command in PS. Yeah I'm well aware that it's a path.

..\ItemTemplates\CSharp\Data> pwd

Path
----
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\Data


..\ItemTemplates\CSharp\Data> 

leidegre@monkey-intel /c/Users
$ pwd
/c/Users

leidegre@monkey-intel /c/Users

There is an excessive amount of line breaks taking place almost everywhere, hold back on a few, you don't need to feed the prompt that much. It's nice to not have to scroll a lot within the terminal itself.

I'm using the Windows PowerShell V2 (CTP) and I can't say that it any better with the tab completion.

C:\Users> S<TAB>

into

C:\Users> .\sshd_server

And the obvious problem here is that S would first of all match SvcCOPSSH due to case and secondly is ambiguous. Also note how S is not transformed into .\sshd_server\ which instead necessitates an every so annoying extra slash as sshd_server is a directory. So if I where to cd into leidegre/Documents I'll have to write cd leidegre<SLASH>Documents. On top of this, the second completion will turn leidegre/Documents into cd C:\Users\leidegre\Documents which is longer. Not that big of deal, right? Well, let's look at another example:

..\ItemTemplates\CSharp\Data> cd 1033/D<TAB>

is auto-completed into

..\ItemTemplates\CSharp\Data> cd 'C:\Program Files\Microsoft Visual Studio 9.0\Comm
on7\IDE\ItemTemplates\CSharp\Data\1033\DataSet.zip'

Note that my prompt is different from the default PS prompt in that it only shows the top 3 directories. The full prompt running the defult PS installation is...

C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\Data> 
cd 'C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\D
ata\1033\DataSet.zip'

That is quite a lot. Now take into consideration the fact that the terminal most likely is more narrow than you're currently set browser/window and that people more than so will be running several instances of PS. And what about white space in file names? I'd prefer the old fashion programming style of just \ escaping the white space. Mean and lean does have it's benefits. And so, terminals are not mean to be used by just anyone.

bash will suggest the longest match which is not ambiguous, any ambiguity is solved by putting in more input. That's i very important in auto-completion. I'm expected to put in some text, but I will be able to rapidly get exactly what I want.

Things like being able to press CTRL+D or <TAB><TAB> to get an auto-completion list on current input is also welcome. And note that if the list is very long, you'll be prompted that you will be shown +100 items or so.

If you can't unite on any of the above stuff, make it easily configurable. So people can use it however they want.

Best of MIX'08

2008-03-25T14:15:00.004+01:00

I love that Microsoft makes all this great content available for free, being unable to attend such a conference for many reasons, it nice to be able to listen to what people have to say without actually being there.
Notably there is one speaker, and one talk that I want to share with everybody.

Developing ASP.NET Applications Using the Model View Controller Pattern
by Scott Hanselman

Now, you might not think that ASP.NET is cool, or that MVC sucks, but this guys is not only hilarious (yeah, a bit off-topic but entertaining), he brings with him a lot more that conventional web development. The presentation runs about 1 hour 15 minutes, and features things from the ASP.NET MVC Framework and a lot of cool neat tricks, e.g. how to use a domain specific language (DSL) for HTML rendering, and more, I'll let you figure it out for yourself.

File decomposition (part 1)

2008-02-14T19:22:00.002+01:00

Almost every game share several proprietary file and/or archive formats. When bound by an underlying file system such as FAT, NTFS, HFS Plus or UFS (Mac OS X) you can only do so much, yet some of these come with features well beyond what which you are willing to implement.

I spend a decent amount of time reverse engineering the internals of things, and when it comes to file formats I've looked into a couple of formats that I have found to be more interesting. Among these are MPQ (Mo'PaQ/Blizzard Entertainment), GCF (Half-Life 2/Valve) as well as Scimitar (Assassin's Creed/Ubisoft).

In summary there are a couple things that you would consider when creating a archive/packing format and these are:

Performance, compression and integrity

This is my take on it, and why these things matter.

Performance

On Win32 platforms the WINAPI CreateFile function is not only used to create files but to open them as well. The performance implication of retrieving a single file handle through this function is negligible, but consider a scenario were there is thousands of small files that need to placed in memory. This is an entirely different case were it is much faster to have an in memory representation of the archive structure and do unbuffered sequential block read operations with this single file handle.

The MPQ archive is efficient as it keeps the entire archive structure in memory while maintaining a very small footprint ~64K. If I wanted to open a file in this archive, I would however, need to know the file name, because what I cannot do in a MPQ archive is browse.
It's simple, the MPQ archive uses a hash table to represent paths efficiently. e.g. 'ThisFolder/ThisFile.ext' transform into a 32-bit hash 0x7c14a6c9, which in modulus 32768 turns up 9929. This is a general case, but it means that the path is a file which is described by a block which is found at that index (it's all very efficient in practice).
This is also interesting as in Blizzard games the file is always queried in patch_rt.mpq before any other archive. This gives a way to override any file but it's a solution in which the initial construction failed to address replacement of legacy files within an archive. This became a challenge with the launch of World of Warcraft with massive monthly content updates.

MPQ archives did not initially support archives larger than ~4 GB. Something which became a problem. However, the initial version had almost no room for improvements. Extensions added after the initial release never looked as good as the initial release. A word of advice, if you think you might need it, make room for it. I'm talking about writing for any need you might want later. Mostly reserving binary space in headers. Backwards compatibility is nice, but not really necessary, the reusability is wroth a lot more.

I like the MPQ hash table approach, it's lean and efficient. But I realized while writing this, its a bit hefty to go through it all at once. I will be covering compression and integrity in the future.

I've been looking into the application of ternary search trees (TST) as well as partial match algorithms. To see if there isn't a even better way of creating even more efficient archiving methods.

Quaternions and spatial rotation

2007-12-11T13:03:00.003+01:00

Quaternions are non intuitive. A point along the surface of a 4-dimensional sphere, isn’t really something which is imaginable. But if you’re more interested in the application rather than research, well this article might just be for you.

When I started out researching quaternions and their application I was working with animation in games. There's a lot to it, and quite a bit of nonsense too. I found that a bit tiresome so I'll try to get right to it.

The thing about quaternions (I'll refer to them as quats or simply quat, from now on) is that they are defined in a 4-dimensional space referred to as H. This article is about application in 3-dimensions, so one thing you need to understand is that a quat compared to an scalar or vector is never the same thing. One way to look at quats is to visualize them as a rotation around an axis. However, this can lead to incorrect assumptions and wont make much sense in end. The way I chose to picture it, is more like a function with arguments. Input and output in this case is simply something useful in euclidean space (E for short).

A naive camera rotation generally suffers from a gimbal lock, a simple way of avoiding this is by representing the rotation by each axis as two separate quats. Creating these two quats is straight forward, now combine these rotations using the quaternion (Grassmann) product and then extract the rotation matrix from this product.

This was the step that initially confused me. But the trick is to create quats in H space from E values, then do the math (simple product) and finally converting back into E space (extracting the rotation matrix from the quaternion). The result is a rotation matrix which does not suffer from gimbal lock and can transform E space values for you.
In order for the calculations to work you need to use unit quats only.

Vector3 v = {0,0,1}
Q yaw = fromE({0,1,0}, yawAngle) // commonly known as the axis/angle representation
Q pitch = fromE({1,0,0}, pitchAngle)
Q result = mul(yaw, pitch) // magic!
Matrix3x3 m = fromQ(result) // convert back to E space from H space
Vector3 v1 = mul(m, v)

The result is a vector v1, rotated about the two angles yawAngle and pitchAngle around the origin ({0,0,0}). Most people when the first try to rotate things, end up rotating objects around a stick because they fail to realize that rotation is the result of rotation around the origin and translation. If you want the appearance of rotating around some other point you need to translate between that point and the origin before applying the rotation.

There are many great code examples out there and it shouldn't be a problem for you to figure out the precise math involved. Once you get going, it's easy. And hopefully this will help you in getting there faster.

Comments Alongside Code (CAC)

2007-10-05T16:37:00.002+02:00

While working on a game I’m currently developing I started to think about how poor some text editors really are. I usually stick with Visual Studio as my preferred environment and it’s a very powerful IDE. But one thing I’m currently missing is the ability to treat any certain range of characters across different lines as a group of text.

Allow me to explain. Back in the old days, any terminal had a fixed number of characters one could display on a single line (typically 72) and basically because of this; you limited any single line to that number of characters. While we still try to avoid writing long lines of code (for readability purposes) we tend to use more characters per line now a days and it’s more a question of what’s suitable rather than what’s readable. I tend to average 80 or 100 characters per line and the readability is still great. But with my screen resolution I can fit up to 148 characters per line and thus wasting about 50 characters, for something which I believe, can be put to better use.

Any modern text editors worth mentioning have a way of doing syntax highlighting (colors to certain keywords or expressions) most text editors also do code folding or outlining which is a way of grouping lines together (this requires some sort of semantic awareness). But I have yet not seen a text editor which can take any rectangular selection as a group of text. I think this is best illustrated by an example.

// A Direct 3D device object other than the one that returned this code caused the 
// hardware adapter to be reset by the OS. Delete all video memory objects (surfaces, 
// textures, state blocks) and call Reset() to return the device to a default state. 
printf( "IDirect3DDevice9::Present D3DERR_DEVICELOST\n" );

One can for instance write comments preceding a certain action which is very common and not wrong in any way at all. But what about writing comments like this:

g_Globals.BeginD3DDReset(); // NOTE to self: this is blocking. 
                            // Blocking the game loop becuase the graphics device
                            // is currently unavailable will totally mess up the
                            // fixed game updates, just ignore the rendering step
                            // until the device is successfully reset.

This is not ideal all the time but this way of writing a comment will allow me to write comments alongside code and I think that rather useful. For this to be really practical, I would expect a text editor to recognize the above comment as comments alongside code (CAC) and conveniently indent the whole thing if any code following the above statement exceed current indentation (see below). It should also adjust the input mode so that these whitespaces used to align the comments alongside code never interfere with the actual writing of additional code. One way to think of this, is to picture the text as a layer on top of your code. This layer, is directly synonymous with the actual text itself and is just a feature by the text editor for positioning comments alongside code.

g_Globals.BeginD3DDReset();             // NOTE to self: this is blocking. 
                                        // Blocking the game loop becuase the graphics device
DoMoreCode( NULL, NULL, NULL, NULL );   // is currently unavailable will totally mess up the
                                        // fixed game updates, just ignore the rendering step
                                        // until the device is successfully reset.

One additional thing which came to mind while writing this is that manual word wrapping is a tedious task. That being said, any implementation of comments alongside code should have the ability to word wrap a comment group through a simple user action. The idea is to augment the experience while still working with text files and never adding any clutter to the actual code. Whitespaces can thus not be considered clutter but storing information about a comment block inside the actual comment would be considered clutter. However, it would also be useful as it could provide sensible metadata about the comment itself. E.g. one could chose to treat a comment beginning with a certain word or pattern as a certain marker.

I think it’s obvious at this point why this would be a very useful and powerful feature. Not only as an extension but it could theoretically provide a lot more functionality than just comments alongside code when working with plain text files.

Hopefully I will be able to revisit this topic with a working demo sometime in the future but I leave no such promises.

To all you Emacs and VIM users, if you think you got it, then you're wrong. I use VIM myself and VIM does not do what I want to do. You can do rectangular text selection, but the editor needs to recognize the comments as more than just comments. It a block which needs to adjust to it's surroundings.

Game loops – Not to be taken lightly

2007-09-30T20:41:00.004+02:00

Not too far back, I started developing games. Back then I decided to hit the books for a while before getting upfront with the actual programming and one of my questions was about game loops. While googling the web I came across this article (a must read for anyone not familiar with different game loops) and must say that I was a bit surprised how a rather simple thing as a loop turned into something a bit more complex, but for the better!

I must begin to thank Koen Witters for his post about game loops as it helped me a lot. However, it still took me some time to wrap my head around the constant game speed independent of variable FPS loop, and decided to write this in order to work out some of the questions I had.

This is a modified version of the suggested game loop by Koen Witters:

typedef T_TIMEINFO struct {
    int updates_per_second;
    int frequency;
    int skip_ticks;
    int max_frameskip;
    int next_game_tick;
    int loops;
    float interpolation;
} TIMEINFO;

TIMEINFO t;
ZeroMemory( &t, sizeof( TIMEINFO ) );

t.updates_per_second = 25;
t.frequency          = get_frequency();
t.skip_ticks         = t.frequency / t.updates_per_second;
t.max_frameskip      = 5;
t.next_game_tick     = get_time();

while( true ) 
{
    t.loops = 0;

    while( get_time() > t.next_game_tick && 
        t.loops < t.max_frameskip ) 
    {
        update_game();

        t.next_game_tick += t.skip_ticks;
        t.loops++;
    }

    t.interpolation = float( get_time() + t.skip_ticks – 
        t.next_game_tick ) / float( t.skip_ticks );

    update_render( t.interpolation );
}

The variables here have been moved into a struct to make them more easily managed and accessible. Also, time is treated as an incremental value and only through the frequency can you find a relation to actual seconds. This is important since high-resolution timers come with a variable frequency (Hz) while the GetTickCount() function returns a discreet number of milliseconds (1000 Hz).

So without further due I present to you this diagram of the above running game loop:

Let’s start with the scheduling as it’s essential to any game loop.

Each game_update (red dot) occur after deadline (black dot) and thus there is always a difference between game_update and deadline denoted here by overdue. This relatively small value can be used in various ways e.g. in a multiplayer game to check that game state is synchronized among participants, as the overdue value is simply a metric of how far game_update is behind. Note that this value is only accumulated if game_update requires more time than skip_ticks and if so, another game_update is scheduled immediately after.

I bet you went – “Eh? What the… the timer is all backwards!?” when you first looked at the above code. It sure isn’t trivial but it’s essential. You have to think about it as two separate time lines; one continuous and one sampled. While we can plan ahead exact moments in time it’s unlikely that we’ll be that punctuational. If we’d assume we’re always late and plan our next deadline always skip_ticks after last deadline we not only prevent drifting, we also make up for any subtle interrupt game_update might cause. In practice we will either schedule game_update a little before or little after actual skip_ticks but as long as this overdue error remains small, it’s perfectly fine.

It’s possible that game_update might stall (be interrupted) and take very long to finish due to some unexpected behavior. When this occurs the next deadline might be scheduled within the overdue error and immediately run game_update until either caught up with real time or max_frameskip force game_render. A max_frameskip is good, it will give the impression that your game is actually still running, slow, but running. However, getting behind and accumulating an overdue is very bad and you should be aware of problems that might arise from this.

And that’s it! Now all the remaining time between each game_update is spent on game_render. The game_render have to implement prediction and interpolation if you want to smooth results. This is fairly simple and Witters covers that which you need to know in his article. This interpolation value is conveniently always between 0.0 and 1.0 depending on the intermediate step of the current iteration. So the only thing you need to be concern about is how to represent a progression between each fixed game_update.

Ad free Windows Live Messenger

2007-06-18T22:46:00.004+02:00

It's actually very simple. Windows Live Messenger get's all it's ad information from a host called rad.msn.com (remote advertisement?) and since it's possible to tell Windows to look up invalid IP addresses through the host file, one can quickly get rid of the ads in Messenger.

You need administrative privileges for this, and if you are running Vista with UAC enabled you need to make sure you launch both Notepad.exe and Cmd.exe with elevation (Shift right-click them in the start-menu and choose run as administrator).

First, find you host file, Microsoft Windows (NT/2000/XP/2003/Vista) is located in %SystemRoot%\system32\drivers\etc by default.
Second, open Notepad.exe as administrator (and with elevation if you use Vista and UAC) choose File > Open... and navigate to the host file (make sure you show all files) and open it. It's called "hosts" without a file extension.
Now add the following new line at the end of the file: "0.0.0.0 rad.msn.com". Save the file and close Notepad.

It will tell Messenger to look up an invalid IP address for ads, and thus display no ads. You need to flush the DNS cache for this to take immediate effect, and it can be done by opening the command prompt Cmd.exe (as administrator) and type "ipconfig /flushdns". If you wait long enough Messenger should stop displaying ads but the quickest path to salvation is restarting Messenger as well.

Enjoy your Ad free Windows Live Messenger.

World of Warcraft security and protocol analysis

2006-10-26T09:28:00.005+02:00

[This post was originally published as part of my developer blog back in late 2006. Back then I was reverse engineering network protocols for educational purposes. I found a flaw in the World of Warcraft encryption scheme that allowed me to tap into the network stream. The results presented here explains how to exploit this weakness as well as how to prevent it. Enjoy.]

When logging onto WoW the client and server computes an identical private key as part of the SRP6 authentication. This key is 40 bytes in length and is used as a session token as well as keystream. In an attempt to locate this key in client memory, I was unsuccessful. But recently I've figured out a way to subvert the encryption.

I've seen a lot of people making packet captures of serve/client traffic and trying to analyze the raw data. But as you might already know, the header of each message is encrypted. This encryption however, is flawed. While subverting the stream is simple in theory, it's a bit tricky to implement in practice.

The problem is that the underlying network layer (TCP) does not provide a reliable way to determine the actual length of application layer data. And since TCP itself is quite a hassle to handle, the process of keeping track of sequence numbers is frustrating. On top of this, the guesses used to crack the encryption becomes even more of a problem when the server bursts data (multiple packets per segment). The game will of course handle this correctly but the tap will not be in-sync.

The idea is based upon a more complex approach where the tap examines every piece of pushed TCP data and for each guess parts of the keystream. As soon as a continuous keystream is found we cycle the entire keystream to synchronize the cipher (in theory, the keystream is too short). This step does not take time, does not involve intense computations and it's a very neat way to being able to tap into the network stream (this works for any computer with a promiscuous network interface in the same collision domain).

Each packet uses a header which stores the actual length of packet. Since the client piggy backs each packet with the PSH flag we can use the TCP/IP headers to calculate the actual length value. This is how we guess the key stream. What's even worse is that the client sends 6 byte encrypted headers while the server uses 4 bytes. This will effectively allow us to cycle the keystream in no more than 20 packets. The problem is that we can guess parts of the plaintext. A quick and dirty solution here is to just pad every other packet or so with 1-3 bytes.

However, if you think about what you gain this system (despite being completely broken) still has some benefits. Even if you have complete control over the TCP stream, you can not just inject data it's more difficult than that. Either the client (will most likely crash) or server will de-sync and terminate the compromised connection but it's not impossible to act as a man-in-the-middle.

While Blizzard clearly states that the creation of third-party applications that enable players to gain an unfair advantage is strictly forbidden. This will not in any way make it possible to gain such an advantage. It will however allow you to read all in-game languages. The benefit of this is that an external application can log your play or more precisely see what you see and even further record your actions and later replay those actions.

While working on this I had an idea that could enable replay capabilities. I never finished reverse engineering the protocol but I knew enough about the game internals to impersonate a server. This effectively allowed me to pretend that the client was playing the game, while in fact you where watching a replay of some previously logged session. You can't interact with this session but you can re-experience any event as if you where playing the game, allowing you to freely move around and examine every angle of the situation.