Coding in London

Windows Service Gotchas

2011-11-23T20:19:00.001Z

the current directory is not the location of the executable, it’s System32! But that’s easy to change.
anything logged to the console will be lost. You can redirect anything you output with System.Diagnostics.Trace. That includes for instance the internal debugging output from log4net (useful when you don’t know why your service is not logging anything).

Quoted from log4net documentation:

There are 2 different ways to enable internal debugging in log4net. These are listed below. The preferred method is to specify the log4net.Internal.Debug option in the application's config file.
Internal debugging can also be enabled by setting a value in the application's configuration file (not the log4net configuration file, unless the log4net config data is embedded in the application's config file). The log4net.Internal.Debug application setting must be set to the value true. For example:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>
        <add key="log4net.Internal.Debug" value="true"/>
    </appSettings>
</configuration>
This setting is read immediately on startup an will cause all internal debugging messages to be emitted.
To enable log4net's internal debug programmatically you need to set the log4net.Util.LogLog.InternalDebugging property to true. Obviously the sooner this is set the more debug will be produced.
Internal debugging messages are written to the console and to the System.Diagnostics.Trace system. If the application does not have a console the messages logged there will be lost. Note that an application can redirect the console stream by setting the System.Console.Out. The Trace system will by default send the message to an attached debugger (where the messages will appear in the output window). If the process does not have a debugger attached then the messages are sent to the system debugger. A utility like DebugView from http://www.sysinternals.com may be used to capture these messages.

As log4net internal debug messages are written to the System.Diagnostics.Trace system it is possible to redirect those messages to a local file. You can define a trace listener by adding the following to your application's .config file:
<configuration>
    ...
    
    <system.diagnostics>
        <trace autoflush="true">
            <listeners>
                <add 
                    name="textWriterTraceListener" 
                    type="System.Diagnostics.TextWriterTraceListener" 
                    initializeData="C:\tmp\log4net.txt" />
            </listeners>
        </trace>
    </system.diagnostics>

    ...
</configuration>
Make sure that the process running your application has permission to write to this file.

Your Home Address on the Web

2011-11-21T22:27:00.001Z

I’ve always thought privacy concerns over Facebook were exaggerated -they usually come from your parents who have been publishing their home address and phone number on the white pages for decades anyway. There is much much worse.

The mere act of voting in the UK ends up in having your home address published on the web.

If you registered to vote on the Electoral Roll and ticked the wrong box, the one with the confusing caption, something like “tick here if you don’t want your details to appear in the private listing”, then all your past addresses are on the web, year by year. Yippee!

It’s not just your home address, but also the list of people who lived with you. That must be the most blatant breach of privacy I’ve ever seen. It’s like Facebook for flatmates with privacy settings set to ‘Everyone’.

Look at this. Only the first part of the postcode is available. To get the full info all you have to do is register at 192.com.

Currently Reading or Watching…

2011-10-31T19:05:00.001Z

Authentication, authorization and impersonation in WCF: WCF Training Kit (Safari Books Online)
Herb Sutter: Writing modern C++ code: how C++ has evolved over the years (video from the //build/ event)
Active, dormant and tombstoned apps: Multithreading in Mango

Current Reading About Hedging

2011-10-23T21:02:00.001+01:00

Where to invest in case of a catastrophe?

If you don’t own gold, now is a good time to buy

How Bill Gates is betting on inflation

Crisis Investing: Hedging for and Against Catastrophe

TeamCity, msbuild and Version Numbers

2011-10-15T18:01:00.001+01:00

How to automatically set all your solution’s projects to the same version number with msbuild and TeamCity?

We have a number of solutions mixing native and managed projects.

The mechanism we use to ensure all DLLs and EXEs have the same version number relies on modifying 2 files:

A VersionInfo.h file for native projects
A VersionInfo.cs file for managed projects

The .h file contains something like this:

#define PROJECT_VERSION 3,9,0,1234
#define PROJECT_VERSION_STR “3.9.0.1234”

All native projects in the solution have a resource file (.rc) that includes this unique VersionInfo.h. This is how the exe/dll obtain their version info.

On the .NET side, the VersionInfo.cs file contains:

[assembly: Assemblyversion(“3.9.0.1234”)]
[assembly: AssemblyFileversion(“3.9.0.1234”)]

All managed projects in the solution compile this VersionInfo.cs file (using a soft link, no duplication necessary).

So all we have to do in order to ensure that the 30+ binaries have exactly the same version number is to get our build script to modify the .h and the .cs just before building the solution.

Sounds easy!

Then there is the question of where to store the “3.9.0.1234” version. It should be in one single place of course. An obvious choice seems to be the TeamCity build number.

From Major.Minor.Build.Revision, we can set Major.Minor.Build by hand in TeamCity and have the Revision set automatically to the current SVN revision number, which is a handy way to relate builds to SVN revisions.

In the TeamCity project settings, the build number format field looks like this:

3.9.0.%build.vcs.number.MyProject_SVN_Root%

Then within our msbuild XML file we refer to the build number set in TeamCity like this:

$(BUILD_NUMBER)

The following msbuild lines update the version number stored in the VersionInfo.cs.

    <FileUpdateFiles Files="VersionInfo.cs"
      Regex="(\d+)\.(\d+)\.(\d+)\.(\d+)"
      ReplacementText="$(BUILD_NUMBER)"/>

This regex does wonders to replace “1.2.3.4” with “3.9.0.1234”. But now, more tricky: when doing the string replace in VersionInfo.h (it’s always the native projects that cause trouble ), how do you replace

#define PROJECT_VERSION 1,2,3,4

with

#define PROJECT_VERSION 3,9,0,1234

given the string in $(BUILD_NUMBER) “3.9.0.1234”.

Can’t do anything a bit subtle with msbuild without having to download and install third-party tasks… So I’m currently looking into the TextString task from the MsBuild Extension Pack to tokenise the version string and then concatenate it back to the appropriate format. It’s all a bit complicated, there must be an easier way to do this…

Update (16/10/2011):

It’s actually not that bad, here is the code that formats $(BUILD_NUMBER) and updates the .h /.cs VersionInfo files.

    <TextString TaskAction="Replace"
                OldString="$(BUILD_NUMBER)"
                OldValue="."
                NewValue=",">
      <Output PropertyName="FormattedBuildNumber"
              TaskParameter="NewString"/>
    </TextString>
    <Message Text="Replacing version in VersionInfo.h with $(FormattedBuildNumber) and $(BUILD_NUMBER)">
    </Message>
    <FileUpdateFiles Files="VersionInfo.h"
      Regex="(\d+)\.(\d+)\.(\d+)\.(\d+)"
      ReplacementText="$(BUILD_NUMBER)"/>
    <FileUpdateFiles Files="VersionInfo.h"
      Regex="(\d+)\,(\d+)\,(\d+)\,(\d+)"
      ReplacementText="$(FormattedBuildNumber)"/>
    <Message Text="Replacing version in VersionInfo.cs with $(BUILD_NUMBER)">
    </Message>
    <FileUpdateFiles Files="VersionInfo.cs"
      Regex="(\d+)\.(\d+)\.(\d+)\.(\d+)"
      ReplacementText="$(BUILD_NUMBER)"/>

Getting Ready For FOWA London 2011

2011-10-03T21:03:00.001+01:00

This is the schedule for tomorrow’s first day of FOWA (Forum of Web Apps) at the Brewery between Shoreditch and Whitechapel.

http://futureofwebapps.com/london-2011/schedule/

Body of Lies

2011-09-27T22:19:00.001+01:00

I just saw Body of Lies by Ridley Scott. At minute 1h38’ there is a scene where Di Caprio talks on the phone to the bad guys who kidnapped his girlfriend. The call is recorded so to make that obvious to the audience, there is a massive old-school tape recorder spinning in the background. Cell phones, drones and satellites… You can never go wrong with good old reel-to-reel tape recording.

I Own a Domain Name I Can’t Use!

2011-09-24T19:50:00.001+01:00

I bought a domain name through Blogger a few months ago, got my receipt through Google Checkout and went on holiday. The Google Checkout receipt has a link to retrieve the Google Apps account to manage the domain: that link doesn’t work any more and the domain does not appear in Blogger.

My domain fell into a black hole: I can prove I bought it, but I can’t use it!

Currently reading… (Read It Later)

2011-09-20T20:05:00.001+01:00

Windows 8: A bad picture is worth a thousand long discussions
Economy: Grece, questions sur une interminable crise (Le Point)
DB: Mythbusting: Concurrent Update/Insert Solutions
Misc: Italian scientists on trial for failing to predict earthquake

Fire-proof Software #2: Designing Fault Tolerant Systems

2011-09-10T13:34:00.000+01:00

Follow-up from part 1.

This is a few of the failure conditions we see with real production software, along with some approaches to deal with them. The general idea is to define activities.

Each activity is a unit of work that may either complete or fail and that can be retried if necessary.
An activity has a start and an end, as well as a status (Complete/Failed).
An activity also has success criteria that allow a process to check the activity actually completed.

The database contains a schedule of those activities along with a MaxRunTime value and a flag indicating if it's currently running. The MaxRunTime allows processes to spot an activity that timed out.

Input data not available (Failure of one of the upstream systems)

You can’t bet on the fact that upstream systems will work, regardless of the reason why, they will fail. There is no point trying to imagine the reasons behind the potential failure, what matters is the impact of the failure on your system and on the business.

Have a retry policy for each activity (retry period + retry window)
Use a stale version of the data (previous day for instance) if acceptable with the business.

Notification not available

Back-up the notification mechanism with an activity schedule: an activity should automatically start at a defined time if it hasn’t already.
If an activity has already started following a notification, the DB flag in the activity schedule table will guarantee that it doesn't start again.

Process crash

Have a Windows service detect the process crash and restart the process.
If the process crashes following an unhandled exception, the currently running activity will be automatically marked as failed. When the process starts again it should attempt to schedule or start the failed activity.
If the process is simply killed with no opportunity to mark the current activity as failed the activity will still appear as running in the ActivitySchedule. When the process starts again it will see that its activity is currently running and will simply schedule a check at StartTime + MaxRunTime. Obviously the check will fail and the process will restart the activity.

Process stalls

Kill it with a watchdog thread: this is a pattern used by embedded systems on real-time OSs to reset a stalled CPU. Inside the app server process the main thread –the one that does all the work- should periodically reset the watchdog flag. If the watchdog flag is not reset after a defined period of time the watchdog thread kills the process after failing the current activity.

Server down or unreachable

Use 2 servers, one primary and one backup.
Each server runs identical processes scheduling the same activities. Both processes will attempt to start their activity at the same time however the ActivitySchedule table in the DB will allow only one process to actually start the activity.
If one server goes down, all processes will simply attempt to reschedule their activities upon restart.
If one server goes down while an activity is running, the activity will still appear as running in the ActivitySchedule. Upon restart processes will see the activity is marked as running and will schedule a check at the expected completion time.

Database not available

Avoid maintaining a single DB connection for too long, that reduces the opportunities of the system to reconnect.
If you can detect the connection failure and don’t want the complexity of implementing a retry mechanism, at least fail the current activity to take advantage of the activity’s retry policy.

Sink-Proof Software #1: Designing Fault Tolerant Systems

2011-09-03T15:58:00.000+01:00

Middleware Enterprise apps usually do straightforward things: they pump data from here, crunch it a bit, dump it there. Complexity comes from the variety of data sources, their reliability and more importantly how much availability (up-time) the business requires.

Say you write an app that processes a daily list of financial instruments to perform a Present Value calculation. What if the input data is not available, i.e. if that list of instruments is not ready? Do you raise a critical fault and give up? Do you use a stale list of instruments? Do you wait a bit and retry? How long should you retry, should you retry forever until the list is available or is there a point where you should give up? Should the retry period be constant or should it increase exponentially to save resources?

What if one of your processes was waiting for a notification from another system and that notification was never received?

What do you do if a process crashes because of an unhandled exception, an out-of-memory exception or a third-party library bug? Is there a system in place to restart the process automatically? Does it take over its task where it left off or does it restart from scratch?

What do you do if a process stalls? Because of a blocked DB call for instance, or some synchronous API call that never comes back…

What if that process was handling client requests? Do you have a backup process to handle the requests while the primary process recovers? Can the system handle a down time?

What do you do if the entire application server goes down either because an engineer tripped on the power cable, the server room was flooded by hurricane Irene, a plane crashed on your main datacentre, the datacentre's power supply failed or there is no more network connectivity to the datacentre?

What do you do if the database is not available for some time? Do you store data in memory temporarily? Do you detect it and try and reconnect or do you just fail on all subsequent DB calls? Do you reroute the DB logging to log files so that logs remain accessible?

Do I need this config file?

2011-08-17T22:35:00.001+01:00

Config files are all over the place with .NET: WCF, Log4Net, everything comes with an XML config file even if you didn't ask for one.

But when you think about it, why do you need config files really? Usually that's because you want to change some behaviour without having to rebuild the code. Ok. But what is so bad about rebuilding the code?

In the Enterprise world if you intend to make a change to a config file in production, you’re not going to just change the file on the production server (unless you're Jack Bauer and you have a death wish after your wife was murdered). You're going to make the change in the staging environment first then test it. If the system didn’t collapse then you'll raise a change request, have it approved by change management and eventually release it in prod. Well now compare that with rebuilding the code... mmm? Same amount of hassle, you'd have to go through the same steps anyway.

I find the main benefit of config files in Enterprise server apps is to ease the pain involved in moving from development to staging and from staging to production. This is where the config files are really useful because you have one code base for many environments. Connection strings, timeout values, environment identifiers, log file paths, simulated dates, weekend definitions, job start times, retry windows, SMTP servers, Tibco RV transport settings... They all depend on where you run the code from.

Using Mobile Broadband in Europe with Huawey Mifi

2011-06-30T21:08:00.001+01:00

I was in Singapore recently and used the little Huawey with a local SIM card to get mobile internet for the iPhone, iPad and Vaio.

To do the same in Europe, I’m trying out MaxRoam. You pay 10 Euros for the SIM card then you top it up as you go.

Steps:

Buy the card
Activate it on the MaxRoam website
Insert the SIM in the Mifi device
Plug the Mifi device to your PC over USB
Install the drivers located on the Mifi. This also creates a little shortcut on the desktop.
Launch the shortcut that opens the configuration page of the Mifi.
There you can unlock the SIM by typing its code and specify the MaxRoam APN details.
The MaxRoam website tells you how much you have left on your account and lets you top-up.

UK tech.days Day 3–Client Apps

2011-05-25T18:19:00.001+01:00

The last day was all about writing Windows GUIs.

During the intro Mark Quirk listed the technologies currently available for building client apps. He gave his personal recommendation about which one to choose: “if you’re not sure which one to go for give a try to Silverlight and see if it works for you.” When he asked “show of hands, who here has written code in Silverlight?” most of the attendees in the packed room raised their hand (I didn’t).

Amazing how Silverlight has quickly moved from something that was rather niche to a mainstream Windows technology. Now it is the toolkit you use by default when you don’t have a compelling reason to use any other (a bit like MFC 15 years ago). What used to be just a cut-down version of the .NET framework for browser plug-in is now much more…

Mike Taulty went over what’s coming in Silverlight 5:

access to low-level XNA APIs for 3D graphics
video: playback rate control and pitch correction
the client networking stack is faster and was moved to its own thread.
faster app startup through multi-core JITing
faster XAML parser
WS-Trust support for SAML security tokens
support for multiple top-level windows
trusted browser application: if you sign the XAP you can run an in-browser Silverlight app in fully trusted mode and therefore be able to do nasty things such as P/Invoke…

Impressive… One by one the limitations associated with Silverlight are going away.

Laurent Bugnion talked about MVVM, showing a solution where 3 projects (Silverlight, WPF and Web) were accessing the same service. Some videos of his previous talks here and here.

Windows Phone 7–Day 2 of UK tech.days

2011-05-24T18:37:00.001+01:00

Today’s track was all about Windows Phone 7.

Brendan Watson talked about the Windows Phone Marketplace.

There are currently 17000 apps registered in the market place. The average selling price of an app is $2.93.

The Nokia deal will bring more carrier partnerships with countries where it’s difficult to get into (China for instance).

You can currently create apps in 30 countries but sell them in only 16. This number is expected to go up to 35 by the time Mango comes out.

Something clever about the market place: devs have the possibility to initiate beta testing programs.

Most of the other talks centred around UI design and performance. We heard a good number of tips and tricks from coders who faced specific performance issues when building apps and showed how to solve them.

Gergely Orosz described how his team spent a month discussing how Cocktail Flow would look like, building photoshop prototypes.

small team, rapid prototyping and iteration.
full visual plans with Photoshop and Flash. Several iterations.
found inspiration in design guidelines for Windows Phone (MSDN)…
…and YouTube videos of app demos

Some of his graphic design tips:

You don’t have to stick with black (even if this is the default for most apps)
Use images in menu items. Square images with grey frame look good.
Use non dominant background images
Keep dominant images for headers
Spice up the buttons with gradients or different typo

Quotes from Gergely:

“Allow your designer gene to come out”

“Content is king but don’t ignore the chrome”

Oren Nachman gave lots of tips to optimise performance:

Don’t use the ProgressBar control that comes out the box. It’s jerky. Instead use the PerformanceProgressBar from the Silverlight Toolkit.
Enable RedrawRegions flag in debug mode to make it obvious when Silverlight is redrawing regions for no reason.
Use the CacheMode property (on the WrapPanel for instance) to put images on the GPU and avoid redrawing them.
Alternatives to the current ListBox: LongListSelector, LowProfileImageLoader

Also Oren mentioned that in the coming Mango release,

the ListBox was completely revisited and will be faster.
Networking: HttpWebRequest will take place on a background thread.

Overview of HTML5 – Day 1 of UK tech.days

2011-05-23T18:44:00.001+01:00

It’s day 1 of UK Tech Days at the Vue cinema of Fulham Broadway where I attended the Web track.

I heard a number of talks today, one of them about HTML5 by Bruce Lawson.

HTML5 is an on-going effort and is not yet complete but lots of bits can already be used now.

In 1999 the W3C decided that HTML was finished and that the future was XML with XHTML1. Then it started planning XHTML2 where a lot of importance was given to language purity. No one cared.

Following work initiated by Ian Hickson with WHATWG, W3C decided to start working on HTML again and created HTML5. XHTML2 was eventually deprecated. So how come did HTML5 prevail on XHTML2? Bruce said:

it’s backward compatible with existing websites and
it’s more tolerant to error handling: the spec precisely defines what to do when markup does not validate.

The design principles that guide HTML5 are based around:

the 80/20 rule and
the belief that users are more important than authors, authors are more important than theoretical purity.

Other technologies not part of HTML5 but often used together: SVG, geolocation and CSS3.

HTML5 defines markup elements but also many JavaScript APIs.

HTML5 introduces the commonly used <div> as proper elements that actually mean something (header, footer, side bar). This makes it easier to build accessible apps because the browser can immediately identify the footer, the header, the side bar…

It also standardises some commonly used input forms (such as date pickers for instance) with the <input type=”…”/> element.

The <video> element displays play/pause controls on top of the video, works with many formats and remove the need for a plugin.

Media queries allow you to define several files with various resolutions depending on the capabilities of the device.

The <track> element shows subtitles over a video. The subtitles are in plain text therefore searchable and you can define several languages.

Other features:

detection of connection loss
offline application cache
server-sent events (to avoid polling)
web sockets

Good quotes by Bruce:

“HTML5 wants you to go to the pub”

“Nobody likes their bits to be in a black box, everybody likes their bits to be manipulated”

Upgrading from iPad1 to iPad2

2011-05-17T21:01:00.000+01:00

It’s a bit thinner and a bit lighter, the screen is a bit brighter but nothing very spectacular.
The most striking difference is with the app start-up time. All apps that take a little while to load initially, such as BBC iPlayer, Google Earth, Friendly, Evernote, the New York Times, MobileRSS, BBC News all take less time to start-up on the iPad2.

And it's a bit more slippery as well when handled bare.

Facetime on a tablet is fun (although the image quality for the other party is better when using the iPhone 4, because the iPad2 camera is so poor).

The New Safari Books Online app for iPad!

2011-05-15T17:09:00.000+01:00

At last!!! The reason why I bought an iPad in the first place! Being able to browse the wonderful Safari Books Online library from the iPad and download entire books for offline reading in the tube.

Last year O’Reilly released a first version of Safari To Go that was one of the worse apps on the planet at the time. They removed it from the app store less than a month after release.

So far the least worse way to browse the books from an iPad was to use the full web front-end, which wasn’t too bad with a browser supporting full-screen. But reading a book on the web was sluggish: you had to wait every time you clicked the next page, offline reading was not possible so no way to read in the London tube.

Version 2.0.1 of Safari To Go is a proper native iPad app that uses the whole screen and allows you to touch swipe between pages (compare that to the wasted space and sluggish web feel of version 1.0).

When flipping pages you have access to a very handy pop-up Table of Contents (I wish iBooks had that).

You can cache the entire book you’re currently reading. This is a very useful functionality that even the web front-end did not allow: all you could do with Safari Books Online was download individual chapters as PDF. Because there was a limit on how many chapters you could download each year you always had to think twice before downloading an entire book. That made offline reading very tedious.

Saving a book for offline use looks like this:

Conferences List

2011-05-07T18:41:00.000+01:00

Conference	When	Next	Where
FOWA Miami	February		Miami
SXSW	March		Austin
DevWeek	March		London
Qcon	March		London
DevTeach	March		Toronto
MIX	March	April 12-14, 2011	Las Vegas
VBug4Thought	April		Midlands, UK
Microsoft Tech Days UK	April	May 23-27, 2011	London
Future of Web Design	May	May 16-28, 2011	London
FOWA Dublin	May		Dublin
DDD Southwest	June		Bristol
Norwegian Developers Conference	May	May 25-27, 2011	Oslo
TechEd US	May	May 16-19, 2011	Atlanta
Code Generation	June		Cambridge
TechEd Europe	June	June 25-29 2012	Amsterdam
FOWA London	October		London
DDD	Any time of the year		Reading
PDC	November		Redmond
LeWeb	December		Paris

Safari Books Online on the Ipad

2011-04-18T21:01:00.001+01:00

The best way I know to read Safari Books Online on the Ipad is the full web interface (not the mobile web version).

Install an alternative to Safari called Perfect Browser that allows full-screen browsing.
Browse to a Safari book.
Enable HTML View and Full Screen. Without HTML View, the text doesn’t scale properly when you zoom.
In Perfect Browser, hide the address bar and the tab bar.
Display the table of contents on the left.

You are now browsing your Safari books full screen with a table of contents in sync. Double tap on a paragraph to zoom in and hide the TOC. Double-tap again and the TOC comes back.

Obviously flicking pages is not as smooth as with a PDF and you can’t read the books offline. We’ll have to wait for the next version of Safari To Go for that…

By the way Perfect Browser is not perfect: if it was it would be able to post to Twitter and Read It Later.

Each Technology Has Its Place

2011-04-16T17:21:00.000+01:00

You would think software engineering is an activity where the decision to use such or such tool is based on pragmatic considerations: short term development costs and long term maintenance costs. In reality there is often a mix of religion (belief that ONE language is better than all others because it’s the language I learnt from an old bold master in a temple somewhere in Tibet) and fashion (technology perceived as cool because its name often appears in blogs and job specs).

Software libraries form a giant, ever-evolving toolset where each tool is optimal only for a narrow range of applications. This is in spite of the massive efforts by standard bodies to build all-purpose languages. No language is truly all-purpose, each one has its preferred applications.

Languages:

C++ stack: native programming with Win32, STL, Boost… Best for situations where the CPU or the memory usage becomes a bottleneck: embedded systems, server apps handling large volumes of requests. The focus is on execution speed and low memory footprint.
C# stack: .NET, CLR, Linq, WCF, Windows Forms… Best for situations where the performance bottleneck is in the database or the network. Need for high level of customisation while keeping development costs down: the focus is on ease of development.

GUI toolsets:

MFC: extremely responsive front-end for consumer-grade applications (as opposed to Enterprise apps). Has been around for ages and is still actively maintained by Microsoft.
Windows Forms: all-purpose front-end, ideal for Enterprise apps. Plenty of of 3rd party controls available. Very easy to code (when you compare to MFC for instance). Acceptably responsive. Deployment over ClickOnce easy as long as no admin rights are required and the target machines have the right .NET framework.
WPF: suitable for front-ends with relatively simple functionality, with an accent on good looks or advanced interaction. Feels a little bit overkill for Enterprise apps in terms of ratio between programming effort / result. Model apps for WPF should be those cool-looking interfaces you see in films. You can also do boring-looking Windows apps (with a menu, drop-down lists, trees and grids…, but there is no much point, you might as well use WinForms). Deployment over ClickOnce similar to Windows Forms.
Silverlight: suitable for extremely simple Enterprise front-ends (think IPhone apps but on the desktop), very easy Enterprise deployment via the web browser. The architecture guarantees that the client-side set of DLLs will be lean: you can’t deploy any 3rd party DLL, only Silverlight classes. Any 3rd party functionality has to run on a server, behind a WCF endpoint. In an enterprise context this type of constraint is actually a good thing because it forces you to leave complexity on servers.
HTML5/Javascript: web apps with a large reach, required to work on any browser/platform. Would sound like a really tedious choice in a Windows-based Enterprise environment where much better tools are available. However for pure web apps HTML5/Javascript makes more sense than Flash or Silverlight.

Using Mobile Broadband in Singapore with Huawey Mifi

2011-03-24T07:36:00.001Z

How to use mobile internet when visiting Singapore?

I’m currently on holiday in Singapore and I carry around an Iphone, an Ipad and a Vaio (as you do when you’re on holiday). I bought a Mifi device (Huawey E585) a few weeks ago, which means I can create my little Wifi hotspot anywhere I go as long as I have a 3G SIM card.

Starhub sells a pre-paid SIM card called MaxMobile Prepaid. It contains a credit of SGD12.

You can buy it in any Starhub outlet (they tend to run out of stock at the weekend). You will need to show your passport during registration.
The SIM card goes into the Huawey (I use the E585) and connects to Starhub network immediately.
However it’s not operational until you activate the credit. Because you registered using your passport, the only way you can activate the credit is by talking directly to customer care on 1633.
To identify yourself you will need to quote the 8-digit number that appears on the package of the pre-paid SIM that you bought.
The lady on the phone will ask how many days you want to activate using your available credit. After she activates it, you will be able to connect to the internet immediately.

To top up your account,

buy a SGD18 Happy Top-up card from Starhub or Seven-Eleven.
call 1633 to credit your account and activate more days.
You will need to quote your 8-digit SIM identifier as well as the codes that appear on the Happy Top-up scratch card.

I used Starhub with the Huawey without any problem during my stay in Singapore and could connect the IPhone, IPad and Vaio to it.

Self-Defense Techniques

2011-03-19T09:00:00.001Z

It took me some time before I got to a stable and convenient backup system. I think I got there now. It’s hassle-free, complete and entirely automatic. This is the setup I use now for backup and synchronisation between all my portable gizmos.

1st level

Dropbox to synchronise my essential documents between

home PC,
2 VAIO laptops,
IPhone
IPad.

I have less than 2 GB of essential documents so the free version of Dropbox is fine. I find the synchronisation very fast and more reliable than using the Windows offline files feature or Live Sync. This acts as a first form of backup since all files are saved on the Dropbox server by default. Dropbox files are always available offline under Windows. On the IPad/IPhone versions of Dropbox you must manually tick the files to make them available offline. By essential files I mean

technical Books
travel documents, flight tickets, travel insurance policy, conference tickets….
Visual Studio projects
OneNote files

2nd level

Daily incremental backup on an external hard drive of all documents on my D drive using the backup tool that comes with Windows 7. The drive is a 1TB Western Digital. This protects absolutely everything in case my main drives fail, which is a very likely occurrence.

3rd level

Off-site backup with Mozy, just in case my flat burns or I get burgled. This is an industrial-scale backup: documents, video, raw sound files, VM files, everything gets saved. I currently have 130GB saved on Mozy. At the time I bought the subscription, the backup size was unlimited. No matter how big a video is it will save it, it’s just a matter of time. It took me about one week to finish the initial backup. After that it does very quick daily automatic incremental backups.

I’m happy about this system so far, it’s entirely automatic and very exhaustive.

Books I currently flip through #9

2011-02-26T12:51:00.000Z

Test Driven Development with Fitness (Gojko Adzic – free e-book)
C# in Depth (Jon Skeet)

Introducing Silverlight 4 (Ashish Ghoda)

Programming Windows Phone 7 (Charles Petzold) Free e-book that now works on the IPad (the version from a few months ago had trouble being displayed on Ipad/Iphone)

Preparing a trip to South East Asia…

2011-01-24T07:07:00.001Z

I’m gathering some info about the countries around Singapore, with a mix of web links, books and Kindle books.

General

Lonely Planet: South East Asia on a Shoestring

southeastasia.org

Singapore

Singapore Journey Planner (SMRT)

Taiwan

Lonely Planet: Taiwan, ebook Edition

Taipei Insight City Guide

Malaysia

Timeout Kuala Lumpur

Diving Spots

Sipadan: Dive the World.com, sipadanvacations.com

Currently Listening to…

2011-01-18T21:31:00.001Z

C9 Going Deep: Herb Sutter and Erik Meijer - Perspectives on C++
C9 Lectures: Stephan T. Lavavej - Type Traits in STL
C9 Lectures: Stephan T. Lavavej – RValue References in STL
.NET Rocks: DotNetRocks and JavaPosse Together at CodeMash
Hanselminutes: Executable Specifications with Gojko Adzic, Jonas Bandi and Aslak Hellesoy.

How to re-build your PC in less than 2 hours

2011-01-10T07:17:00.001Z

Pre-requisites:

Your data is on D:, the OS on C:
Your backup is done (nothing to do because you have an automatic full C+D-drive incremental back-up scheduled to run every day )
Your CD case with all legally acquired original CDs for software and drivers
Access to the spreadsheet where you carefully store all serials for the above.

Go:

Boot from DVD
Format C:.
Click Next, OK, next, Ok, I agree, OK, Next, London GMT
Install your favourite software. For me it is:

Microsoft Security Essentials
Slysoft Virtual Clone Drive
Office
Chrome, Firefox
Set-up backup with Mozy

Check the time: if it took you more than 2 hours, start all over again.

The Mystery of Safari To Go: why is it so bad?

2010-12-31T12:04:00.001Z

A lot of applications make the IPad shine: they’re fast, responsive, elegant and take full advantage of the touch screen: Flipboard, Ocado, Twitter and most magazines (Wired, Economist, New York Times, …). Swiping is smooth, the whole screen is used.

And then you have things like BBC IPlayer, Amazon Touch Shopping or Safari To Go that in terms of user interface are plain rubbish.

Take Safari To Go: I was expecting to be able to easily flick through my collection of Safari books stored offline, it was the reason why I bought the IPad in the first place. Well it turns out you can’t even swipe the pages!!! You have to press left/right arrow buttons to navigate. It’s full of strange rounded dialogs that waste space. It’s very buggy. The table of contents appears in an annoying modal dialog right in the middle of the screen, the table hierarchy is flatten and slow to scroll down… I actually find the web version of Safari Online Books to be much more usable than the IPad app, even if the web version does not support offline storage.

Amazon Touch Shopping is not as buggy but you will find similar clunky rounded dialogs, wasted space, slow scrolling plus some very annoying sounds that acknowledge every user action.

Although it’s fantastic to be able to watch BBC programs on the IPad for the price of a TV license, the BBC IPlayer app suffers from the same sort of web-like sluggishness.

What’s frustrating is that we’re not talking about cheap farting apps here: Safari Books Online, BBC IPlayer and Amazon are all extremely useful platforms supported by companies that have a habit of being innovative. I guess it’s just poor technological choices. I don’t know how they’re designed but I suspect that all 3 apps are actually web browsers running javascript… At least that’s what they feel like.

Update: (13/02/2011)

I read that OReilly removed Safari To Go from the app store on 24/11/2011, 3 weeks after it introduced it. They’re currently working on a new version (yeah!).
There is now a proper BBC IPlayer (released 10/02/2011) designed for the IPad from the ground up. It’s much much much nicer than the previous one.

Currently Listening to…

2010-11-02T21:48:00.000Z

Mark Russinovich: Mysteries of Windows Memory Management Revealed (PDC10)
Mads Torgersen: Inside C# Async (Channel 9 Going Deep)
Anders Hejlsberg: The Future of C# and Visual Basic (PDC10)
IPad Today 17: Radioliffic Apps, Mute Button Woes, iPad Cat Stands, Fruit Ninjas!
Net@Night 174: Instagram
Roy Osherove: Unit-Testing C++ with Isolator++ Webinar

Watching PDC10…

2010-10-28T21:46:00.001+01:00

I’m impressed with the live streaming video system that Microsoft set-up for the PDC.

You can watch live or past sessions. I’m watching the keynote right now that took place just a few hours ago.

The player lets you switch between slide view and stage view.

I wish they did that for the previous TechEd events… Too bad the Ipad doesn’t do Silverlight.

Update: when the US sleeps (now 07:54 BST), quality switches to HD, see below:

Notes from DDD8a

2010-10-24T21:58:00.000+01:00

The DeveloperDeveloperDeveloper event at the Microsoft offices in Reading was split in two tracks of simultaneous sessions.

Notes from some of the sessions:

It’s Time to Look at Entity Framework

Speaker: Julie Lerman (twitter @julielerman) thedatafarm.com

The Entity Framework was initially designed to reverse engineer existing databases but you can now also create them from scratch.

The designer generates a DDL. It doesn’t actually create the DB, it creates a script. Then you give the script to a DBA (Not sure how you specify paths for database and log files ?).

Support for POCO classes: your classes don’t have to inherit from Entity Framework classes, you can keep your model totally separate from EF and still work with the designer. The way you do this is through T4 templates, some of them you can download from Visual Studio Gallery.

If you tweak the T4 template you can totally abstract the EF layer (ObjectSet, etc…). In other words your code can interact with EF only through interfaces, which makes unit-testing possible.

TODO:

check out T4, CodeFirst

Packaging in the .NET World

Speaker: Seb Lambla (twitter @serialseb)

Seb demoed OpenWrap www.openwrap.org (the site is going live live in a few days), a packaging tool that you use to inject dependencies into your build. The dependencies must be available as “wraps” for this to be possible. You inject the dependencies with a simple command-line tool.

The openwrap commands can also be used from within an msbuild file.

There is another open source tool called Nupack (maintained by Microsoft) released on codeplex.

TODO:

Check out Scott Hanselman’s blog about Nupack

Is NoSQL the Future of Data Storage?

Speaker: Gary Short (twitter @garyshort)

Term introduced by a programmer from LastFM.
NoSQL
- often does not implement ACID
- avoids joins
- no fixed schema
- scales horizontally (adding more machines)
Types of NoSQL DBs
- document Store
- graph storage: nodes and edges
- key/value stores: on disk/on RAM
- “eventually consistent”
- object DBs. You provide your own indexing rules.
Good for:
- geographic regions, large quantities of data, game server sharding (what’s sharding?)
- often written, rarely read > Key/Value
- binary data
Example: Twitter
- they tried RDB > didn’t scale.
- built FlockDB
  - not optimized for transversal, because not needed.
  - optimized for adjacency lists: graph stored as set of edges
  - idempotency: useful for computing set unions and intersections.
- Lessons learned: use aggressive timeouts

TODO:

check story of Twitter (DB angle)

Modern C#: this is not your grand-daddy’s language

Speaker: Jon Skeet and his pony. (twitter: @jonskeet)

Inspiring talk about the evolution of the language features of C#.

Jon illustrated how powerful C# has got by writing a MaxBy() implementation in various versions of C#. MaxBy() should return the element of a collection containing the maximum value of a specified field. It should work with any field and any collection.

He wrote the solution in C# 4 use generics, lambdas, method extensions and inferred types.

In C# 1.0, he started with a simple for loop: this lead to code specific to the class being compared. To make the code a bit more generic he used a delegate and some downcasting from System.Object to the type compared: it worked but was somewhat verbose and not as type safe as the C# 4 version.

Jon’s quotes:

“Knowing how things work under the hood is important. However you need to be able to shift gears so that you think at a higher level when you need to”

“We need to be jolted out of the idea of what a language should look like.”

“You can express yourself more clearly without having to write things you don’t need. What needs to be expressed is expressed only once.”

“Learning F# helps you understand the new features of C# 4.”

Things you should know about SQL as a developer

Speaker: Simon Sabin (twitter: @simon_sabin)

Do not truncate the transaction log > you loose point-in-time recovery
Re-indexing
- You may just need to update the statistics.
Shrinking
- usually bad, files grow for a reason
- shrinking makes sense after a big import job only
Clustered indexes on dates
- make them small and unique
Types of joins
- Loop: for small datasets
- Hash: large datasets
- Merge: requires sets to be ordered
user-defined functions
- bad performance!
  - interpreted
  - can’t use parallelism
- demoed a padding operation done with UDF vs one done with a CLR function. The CLR one significantly faster.
- Prefer CLR functions over UDF.

TODO:

Look into taking snapshots vs restoring from backup.

More blog posts about the event:

Danny-T.co.uk

The Ninja Ferret

Slides:

Chris Hardy’s WP7 talk

Ipad/Iphone Apps List

2010-10-20T21:56:00.000+01:00

IPad

Ocado

Very well suited to the Ipad, here is a typical use case:

Book your delivery slot (slots for the following evening are usually free).
Add all items from the previous order
Display basket and go to your kitchen
Remove unwanted items (check your fridge and cupboards)
Add items you know are missing
Go back to your sofa and browse your favourites for things you might have forgotten
Checkout: type in the voucher (Ocado often emails 15% or 20% reduction vouchers)
Type in the 4-digit pin to authorise the transaction, and that’s it.

Reeder

Quickly syncs with Google Reader.
Elegantly displays groups of feeds
How I use it: I star articles I want to read during the week and take the time to read them at the week-end.
No pictures, no colors :-(
Too bad you can’t subscribe to a new RSS feed from within Reeder.

Pulse

Unlike Reeder, this one will show the pictures included in the RSS feeds.

You can create tabs to arrange your feeds. Inside each tab you can add feeds imported from Google Reader. Note you can only import from Google Reader, it does not sync.

Pleasant to look at but from a functional point of view, Reeder is still better because it syncs with any change you make to your subscriptions in Google Reader.

DropBox

There is a small set of files I always want to have with me. I use DropBox to keep them in sync. I installed it on my PC, 2 laptops, an Iphone and an Ipad. Changes made on any device are immediately propagated to the others if they’re on.

The Windows version gives a version history of each file.

IPhone

2Do Lite

Clever little app for TODOs. You can associate a TODO with a geographical location. Click the nearby button to filter TODOs by current location. This is good for the situations where you say “Next time I go to Boots, I should get some shampoo”. Elegant, fast, good-looking.

TimeOut London

Search films by location, by date, by cinema.

I use it to look-up the films available in a short list of of favorite cinemas.

FourSquare

This is what Google Latitude should have been. Check-in to pre-defined places (restaurants, bars, companies, hotels, train station platforms, forests…) to tell where you are or add tips about a particular place. This app would be great if only more people used it: the nearby tips tend to age a bit.

It’s all I wanted to do with Google maps: see a place I like, make a note of it to come back later or go to a place and leave a note about it. I used to do it with Google maps, it’s much more clever with FourSquare.

A friend talks to you about this new restaurant he tried: look-up the name, add it as todo and attach a tip “Remi recommended the Savoy bar because it was recently refurbished and just re-opened”

A number of little apps are available to make it easier to use FourSquare, in particular to automate the check-ins: CheckMate and FourMinder.

TimeOut London

Search films by location, by date, by cinema.

I use it to look-up the films available within my short list of favorite London cinemas.

DropBox

The Iphone version allows you to cache some of the files for offline view (by making them favorites). You can’t view the version history though.

UK TechDays Special Event with Steve Ballmer

2010-10-01T21:57:00.001+01:00

I’ll go hear about Windows Phone 7 and Azure on Tuesday. It will happen at the ICC in the Docklands (Royal Victoria Dock).

It seems it’s still possible to register.

Notes about Snapshot Isolation

2010-07-03T15:02:00.001+01:00

MSDN: Using Snapshot Isolation

The SQL92 standard defines four isolation levels, two of them being READ_COMMITTED and REPEATABLE_READ.

READ_COMMITTED prevents the current transaction from seeing data changed by uncommitted transactions.

REPEATABLE_READ guarantees that the data read since the start of the current transaction will not be changed by other transactions. This way the state of the database remains the same from the point of view of the current transaction.

By default, SQL Server uses locks to implement READ_COMMITTED. That means any query attempting to read a row that is being changed by a transaction will block. Not great really. Oracle uses rollback segments to achieve the same thing and queries are never blocked by writes!

Snapshot Isolation (from SQL Server 2005 onwards) is another way to implement REPEATABLE_READ and READ_COMMITTED without using locks. Instead row versions are saved in tempdb every time an insert or update takes place.

To allow all future sessions to use snapshot isolation:

ALTER DATABASE MyDatabase
SET ALLOW_SNAPSHOT_ISOLATION ON

ALTER DATABASE MyDatabase
SET READ_COMMITTED_SNAPSHOT ON

To enable REPEATABLE_READ in snapshot mode for the current connection:

SET TRANSACTION ISOLATION LEVEL SNAPSHOT
Articles

MSDN:
Isolation levels in SQL Server 2005
Understanding Row Versioning-Based Isolation Levels
Cost of Row Versioning
Stack Overflow:
Why is READ_COMMITTED_SNAPSHOT not on by default?

A week-end in the South of France using the Ipad

2010-06-28T08:46:00.001+01:00

I enjoyed a long and sunny week-end in the south of France at a friend’s place (Gordes, Provence). I spent many hours near the swimming pool toying with the Ipad (the first Apple device I’ve ever bought).

I like the sexiness of the hardware but plenty of little things keep annoying me.

Intended home usage:

read my RSS feeds
browse Oreilly Safari
read kindle books

I’m still looking for:

A better browser, because Safari lacks some really obvious features such as tabs or page text search.
- A1 Perfect browser: would be great if it didn’t reload the last page every time you switch back to it.
- ICab Mobile
  - To be tried
a RSS Feed reader that compares to Feedly. I use Feedly on the desktop but it does not exist on the Ipad yet so I looked for alternatives:
- Feedler:
  - the experience is similar to Google Reader, hence functional but a bit boring.
- Reeder: could not install
- Early Edition:
  - Visually pleasant although not as much as feedly
  - Very slow to refresh
  - imports feeds from Google Reader but does not sync automatically.
  - No mark for later function
  - Doesn’t remember what you read.
  - Displays original posts with embedded browser instead of opening Safari (which a good thing).

Some random observations:

You can’t see anything on the screen if you wear polarising sunglasses. :D Not a problem but it’s the first time I notice something like this on any device…. Update: (12/09/2010) When using polarised RayBan sunglasses, the IPad screen becomes invisible if you hold it in portrait mode. In landscape mode it’s fine.
The absence of notice means you have to play around to discover some features of the hardware/OS/virtual keyboard, that’s actually quite fun.
The WIFI issue
- I observed that my Ipad loses the Wifi connection if the connection is unused for more than 5 minutes (does not occur at 4 minutes).
- Router tweaks I tried with no success:
  - Disabled security
  - Changed the wireless channel
  - Switched down to 802.11b.
  - Used a static IP address
  - Updated the firmware of my Belkin F5D8635v1 from 1.00.17 to 1.00.23

Apps reviews:

Wired
PadGadget

Update: (07/08/2010)
Safari Books Online plans on releasing an app for the Ipad this summer.

SQL Server and Notifications

2010-06-20T10:09:00.000+01:00

Chaps from Microsoft gave us a presentation of StreamInsight at work the other day.

Following this I started digging around the whole SQL Server notification/event thing: how can an application be notified of a data change instead of polling the database like a drowsy lobotomised lemming?

Service Broker

Messaging infrastructure for SQL Server. To enable it on a database, right-click the database in SSMS, click Properties > Options > Broker Enabled

Service Broker
Servicer Broker Programming

T-SQL has a few constructs dedicated to using the service broker.

Query Notifications

Requires Service Broker to be installed on the current SQL Server instance and enabled for the current database.

Using query notifications
Query Notifications in SQL Server (ADO.NET)

SqlDependency

This is the class you use to implement Query Notifications in a .NET app. It is supposed to be used by server apps (ASP.NET for instance). The documentation warns it is not designed for client apps. It probably doesn’t scale too well.

SqlDependency class
Detecting Changes with SqlDependency (ADO.NET)

Notification Services

SQL Server 2005 only, not 2008. This is a publish/subscribe messaging service.
Notification Services Tutorial
What does Notification Services do?

StreamInsight

Big Diagram
StreamInsight Installation
Download
Deployment model
Concepts

Other Stuff…

Change Data Capture / Change Tracking

This is more for auditing purposes: allows you to easily query historical changes made to the data without having to write tons of error-prone code in triggers (as I saw being done on an Oracle database by a client I used to work for…)

Data Capture and Change Tracking compared

WAITFOR

Can be used to delay a statement: block for 2 minutes before running a statement for instance or can be used to schedule a statement to run at a specified time.

Using WAITFOR
WAITFOR (Transact-SQL)

FOWA Dublin 2010 – Notes

2010-05-16T09:09:00.001+01:00

Some notes scribbled down during the talks.

Twitter and Geolocation

Raffi Krikorian (twitter, blog)

Yaketee: very simple web app that automatically detects your location and allows you to create chat groups with people in the same location.

What makes a place?

longitude, latitude. Sometimes this is a bit too precise.
A polygon (area rather than exact location). According to Raffi defining locations with polygons is currently what Twitter does for named areas.
Area name / town / country
WOEID: Where on Earth Identifier, defined by Yahoo.
TwID (?)
IP-based location
w3c location
brokered location: redistributes location info to other apps. Example: Yahoo’s fire eagle.
location databases
- Yahoo (WOE db)
- Twitter

Cloud Computing: Innovation vs Commodity

Simon Wardley (twitter, blog)

Simon based his talk on a graph showing the relationship between ubiquity and certainty (slides). It’s an asymptotic curve. On one end of the curve innovative technologies have low ubiquity and low certainty. At the other end of the graph, commoditized technologies have infinite certainty (no more change) and a capped ubiquity.

How can a technology move from the innovation stage to the commodity stage and become a service? Four requirements:

concept
attitude (willingness to see IT as a commodity)
suitability
technology

Hierarchy and componentisation: stable components act as building blocks and accelerate innovation.

Any technical progress that makes something more efficient increases the demand for its use.

Risks associated with move from innovation to service:

highly disruptive transition, how do you manage the new supplier?
outsourcing risks: lock-in, second sourcing…

Myths around cloud computing:

You have a choice (actually you don’t).
It will reduce IT spending. It doesn’t because you’ll end-up spending the same amount of money, you’ll simply do more stuff with it.
Cloud is an innovation. It’s not, it’s a commodity.

Accessibility

Robin Christopherson (twitter, blog)

Flash is really bad for accessibility: nothing can be done with the keyboard.
Internet Explorer has a function to bring up all links of a page inside a dialog. Not all browsers support that.
Chrome not accessible.

Robin did his talk and demos without being able to see the screen. He used keyboard shortcuts and text-to-speech. When he scrolls a list of items the system enunciates the links at very high speed (much too fast for me to understand, but I guess if it’s the only way you can use a PC you must end up getting pretty good at it).

Automated caption and automated timing. Youtube makes it easier to produce videos with captions: you upload the video and the transcript. Youtube uses voice recognition to associate time markers with captions (so you don’t have to do it yourself). It sort of works, enough to be useful.
Internet TV: projectcanvas.info: joint venture between the BBC, ITV, C4 and Five to build an internet TV platform with accessibility as a prime requirement.
Avoid horizontal scrolling: Opera mini can force single column browsing.
Siruna: can crunch down a website for any mobile platform.
Textcaptcha.com. Accessible CAPTCHAs. The distorted pictures of random words are not usable by blind people! This site suggests an alternative using logic questions. They even provide a REST web service.
Accessibility testing budget encompasses usability testing. i.e. if you test an app for accessibility for the blind you will spot usability issues affecting everyone else.

FOWA Dublin 2010 – Overall Impressions

2010-05-14T21:10:00.001+01:00

The Irish edition of FOWA in Dublin lasts only one day. But because the presentations are short (about 20 minutes) you end-up seeing a lot of talks. And as usual what makes FOWA different is the opportunities you get for casual conversations with the speakers.

The venue was a building in the grassy campus of University College Dublin. Everyone could sit outside in the sun for lunch.

The talks that stood out for me were those by Raffi Krikorian (Twitter) and Christian Heilmann (Yahoo), both dealing with geolocation. When it’s about geo I always think of Google Latitude and Bing Maps but Yahoo actually does a lot too, especially in the area of APIs and web services.

IPv6: we’ve been hearing about the exhaustion of IPv4 addresses for a long time. I remember my teachers talking about this at school back in 1996. Well Owen Delong explained that this is now less than two years away. More info on the ARIN site. You can even get counter badges that give the current estimate for the exhaustion date.

I was sitting next to the CEO of TicTacDo, a collaborative how-to site that looks very useful. You could think of it as a Stack Overflow where each question would get a single answer in the form of an actionable TODO list. The app has been live for a while and the enterprise version will launch soon.

Databases: there are many alternatives to the classic relational database. Chris Lea described the NoSQL approach (slides): a range of tools (CouchDB, MongoDB, JackRabbit) should be considered before jumping on MySQL, SQL Server or Oracle.

Some excellent non technical talks by professional speakers: Alex Hunter about taking care of your brand and the lovely Relly Annett-Baker about content creation.

The after conference party at Solas was great: free bar from 7.30pm to midnight (you can never go wrong with that), very good atmosphere and good crowd.

One thing I noticed: during the last FOWA I attended there was an army of bloggers taking notes live on their laptops and posting as the conference was progressing (I could find the posts in Google 30 min after a presentation was finished). Not this time! I think I know why: three years ago people were using blogs to post live information, now they use Twitter, which is better suited for this kind of thing. Somehow Twitter has made blogs more quiet.

Maps at the British Library

2010-05-09T09:39:00.001+01:00

The Jubilee line is working this Sunday. I’ll make the most of this exceptional event by going to the new maps expo at the British Library.

In September there will be a talk by Ed Parsons geospatial technologist at Google Maps and Steve Chilton from OpenStreetMap.

Insight Into an Agile Team

2010-04-18T17:34:00.001+01:00

It’s one thing to read about agile in books. It’s another to see it working in practice. And it’s impressive!

I had the opportunity to take part in meetings with a team that applies agile methods: they’ve been doing this for years and their process is very mature. I was impressed with the discipline. The methods the team uses are agile in spirit and re-use some elements from scrum, extreme programming and lean.

They hold at least three types of meetings:

stand-up (daily):
- quick status update by each pair
- decide the new pairs for the day
- quickly go over yesterday’s annoyances (written on a board)
planning (every iteration): give a time estimate to each piece of work in the next iteration.
retrospective (every two iterations): list what went well, what didn’t and what needs to change.

Six programmers work in pairs and interact with two business analysts. The pairs change every day.

The business analysts identify early little chunks of functionality called stories. Stories are small enough to be designed/coded/tested in less than two days and still provide business value. The BAs prepare a list of the stories that will make it into the next iteration. After discussion with the developers they assign a rough weight to each story. A story is weighted in relation to a “standard” story, which takes about 1.5 day for a pair to complete. A story with weight 1/3 takes a third of the time of a standard story. Later the devs will give a more precise estimate to the story during the planning meeting.

The most striking aspect of the approach is that the BAs write and complete the FIT tests before each iteration. That means even before the dev pair starts coding a story, there is already a FIT test for that story. Obviously at this stage the test fails because the solution does not yet exist. Then the job of the devs is to get the FIT tests to pass by actually coding the functionality (writing the unit-tests first and using TDD’s Fail>Pass>Refactor cycle). If the devs feel a need to change something to the FIT test itself, they discuss it with the BAs.

The dev pairs change every day and the new pairs are decided during the stand-up.

More about the planning meeting: before each new iteration the team meets to discuss the stories going into the next iteration. They sit around a table and go through the stack of stories one by one. It’s literally a stack of cards with a brief description of the story on it. Because the stories are very lightweight both in terms of functionality and coding complexity, you can get away with a very short description on the card. A few lines of handwritten text are enough to express what the story is about. No heavy specs documents: fine level story granularity + lots of team communication replace the need for detailed specs. The FIT tests and unit-tests capture the detailed specifications.

The process they use is self-evolving: once a month they hold a formal meeting called a retrospective. They discuss what went well, what didn’t and then fine-tune their methods.

The structure of the retrospective is straightforward: they write three lists of items on a board:

GOOD
BAD
TODO

They also bring the board with last month’s TODOs to tick off those that were actually done and carry over the rest. Everyone sits around a table with tea, coffee and biscuits. Not surprisingly that meeting can get quite animated, especially given that those guys are not exactly the shy type. You can tell they’re used to doing that by the way they structure the discussions: list the items first on the board, close the list and then discuss the items one by one rather than dwell on the first items and run out of time.

Feedly, Google Map Layers and Bing Maps

2010-04-10T15:33:00.001+01:00

I’m an avid user of Google Reader because it’s fast and straightforward. Unfortunately the layout is a bit boring. The problem with a boring layout is that you can end up missing posts either because there is no pic to catch your attention or you fell asleep.

While I was in Istanbul I tried feedly. It works as a Chrome extension and uses your Google Reader feeds (so no need to import the feeds, they’re all there). It lays out RSS posts with a picture, a title and a preview text. Several layouts are available, digest, cover, popular and latest, all using some obscure algorithm to display a subset of the posts available in your RSS feeds. The result is visually appealing and the keyboard shortcuts are similar to those in Google Reader.

The digest layout (notice how you get some Kandinsky-like abstract painting when a picture is not available from the post):

The cover layout:

Compare that with the equivalent Google Reader layout:

Google maps for mobile

At last, the latest version of Google maps for Windows mobile is what it should have been at the beginning: the places you save in My Maps on the desktop appear on your mobile. The places you create as starred items on the phone appear on the desktop. It’s good because I create starred items on my phone whenever I walk past a restaurant I want to remember. But until this new version of Google Maps I couldn’t save them in the cloud, which was a bit daft.

So on the mobile Google maps you can now display layers. A layer can be anything: your saved maps, saved searches, maps of favorites places by famous people, your friends on latitude, tube lines… There is a Wikipedia layer that displays an icon on places related to a geolocated article and a Buzz layer that displays icons where people left a buzz message…

New Bing maps

The new bing maps available at www.bing.com/maps/explore work with Silverlight and are clearly faster than Google maps. Try bringing up London in two chrome sessions side by side, one with Google and one with Bing. It’s pretty obvious: panning, zooming in and zooming out feels faster and more fluid with Bing.

The map apps are pretty good: there is a Restaurants finder that works really well (at least for London that is) and a Twitter Map to display geolocated tweets on the current map in real time. Today’s front pages is exciting if you’re into worldwide news.

Currently the data in the new Bing lags behind… For instance the Restaurant Finder didn’t have anything to show for Dublin or Istanbul. But in terms of functionality, it’s very promising.

This is some of the available Map apps:

The Restaurants finder with category filtering:

The Ordnance Survey maps are not displayed in the new Bing maps apparently. However they still are in the normal bing maps maps.bing.com. It still blows my mind that the raster version of those 1/25000 maps (those that cost you an arm and two legs at the bookshop) are up there for free.

Comparing Result Sets with Linq

2010-03-29T21:59:00.000+01:00

Linq is really handy. Once you’re familiar with the syntax and the way it works, it becomes very useful.

Recently I had to write a front-end to compare lists of financial instruments coming from two different sources. One source was a SQL Server database, the other source was an in-house market data system.

The result set from the database was extracted by a stored procedure into a collection. The collection’s type is IEnumerable<Result1> where Result1 is a type automatically generated by the Linq designer using the stored procedure definition.

The other result set comes from the in-house market data system. It is extracted with an API that constructs Result2 objects and accumulates them into a collection List<Result2>.

The purpose of the little app is to show the instruments present in the first result set and not the other one. The types Result1 and Result2 are defined as follows:

/// <summary>
 /// In my real project this class was generated by the Linq designer
 /// </summary>
 class Result1
 {
     public string Isin { get; set; }
     public string Sedol { set; get; }
 }
 
 /// <summary>
 /// Manually created class to store objects from the in-house market data system
 /// </summary>
 class Result2
 {
     public string Isin { get; set; }
     public string Sedol { set; get; }
     public string Location { set; get; }
 }

Fill both collections with some dummy data:

List<Result1> coll1 = new List<Result1>();
 List<Result2> coll2 = new List<Result2>();
 
 coll1.Add(new Result1 { Isin = "ABCD", Sedol = "123" });
 coll1.Add(new Result1 { Isin = "EDFG", Sedol = "234" });
 
 coll2.Add(new Result2 { Isin = "EDFG", Sedol = "234", Location = "Set2" });

I want to compute result1 MINUS result2. In Linq this is done with Except. So I’d like to write something like this:

var diff =
     (from c in result1
      select c).
     Except
     (from c in result2
      select c);

…but that does not compile because result1 and result2 have two different types. Therefore I need to do a projection, like this:

var diff =
     (from c in result1
      select c).
     Except
     (from c in result2
      select new Result1 { Isin = c.Isin, Sedol = c.Sedol });

The above compiles fine, but there is a problem: the result of the diff is the original result1 collection. It doesn’t remove the elements present also present in result2! This is because at this stage Except does not know how to test Result1 objects for equality.

Except takes an IEqualityComparer<T> as second argument. Let’s define one:

class Result1Comparer : EqualityComparer<Result1>
 {
 
     /// <summary>
     /// Two elements are considered equal if they match on at least one identifier
     /// </summary>
     /// <param name="x"></param>
     /// <param name="y"></param>
     /// <returns></returns>
     public override bool Equals(Result1 x, Result1 y)
     {
         return ((x.Isin == y.Isin) || (x.Sedol == y.Sedol) );
     }
 
     public override int GetHashCode(Result1 obj)
     {
         // Just re-use Object's default GetHashCode
         return obj.GetHashCode();
     }
 }

The diff now looks like this:

var diff =
     (from c in result1
      select c).
     Except
     (from c in result2
      select new Result1 { Isin = c.Isin, Sedol = c.Sedol },
      new Type1Comparer());

It’s still not working! Why? When Except compares the result sets, it uses the comparer’s GetHashCode first. If two objects have two different hashcodes then it doesn’t bother calling Equals().

In the code above the GetHashCode() override calls the default Object.GetHashCode(). Object.GetHashCode() is useless: it generates an index as a function of the object reference. Even if two objects have the same value Object.GetHashCode()returns two different hashes. And Except thinks the objects are different!

No need to burn too much brainpower on writing the hash function as long as:

it’s fast
two objects with the same value return the same hashcode.

The following should do:

class Result1Comparer : EqualityComparer<Result1>
 {
 
     /// <summary>
     /// Two elements are considered equal if they match on at least one identifier
     /// </summary>
     /// <param name="x"></param>
     /// <param name="y"></param>
     /// <returns></returns>
     public override bool Equals(Result1 x, Result1 y)
     {
         return ((x.Isin == y.Isin) || (x.Sedol == y.Sedol) );
     }
 
     public override int GetHashCode(Result1 obj)
     {
         // Objects with different Isins will be considered as different
         // If 2 objects have same Isin then Equals is used
         return obj.Isin.GetHashCode();
     }
 }

No the MINUS operation with Except works.

Limits of Linq:

The SQL-like way of comparing collections is elegant but it’s nothing more than a nice way of writing for loops. Don’t expect miracles on the performance side…

The best place to perform queries is still in the database.

In the case of the app described above performing queries on the client is just fine. However there are situations where importing into the database and having all the queries take place in SQL Server would be faster.

Books I currently Flip through #8

2010-03-07T21:42:00.000Z

Tech

LINQ to Objects Using C# 4.0: Using and Extending LINQ to Objects and Parallel LINQ (PLINQ)

Others

The Cul-de-Sac Syndrome: Turning Around the Unsustainable American Dream (Bloomberg) (John F. Wasik)

Outliers (Malcolm Gladwell)

Udi Dahan about Simplifying Multi-Tier Architecture

2010-02-15T20:12:00.001Z

I listened to Udi Dahan talk about an alternative pattern to the classic multi-tier architecture during a Skills Matter event in London.

I knew the name was familiar but I couldn’t put my finger on it until I entered the room: he was at TechEd Barcelona in November 2008! Back then he gave a very sarcastic (and enjoyable) talk about interface-based programming, dependency injection and the abuses of the strategy pattern.

Udi is a good speaker. That’s quite a treat to have a TechEd-level presentation for free at Skills Matter.

The pattern is called CQRS. Despite the boring 4-letter acronym it is based on a few very simple ideas: take out from the standard UI/Services/Business logic/DAL everything that is related to queries (= pull data off the database to show it to the user) and leave only the commands (= use business rules to change data). Queries can simply use a 2-tier model with persistent views.

The video of the talk is available on Skills Matter website.

Command Query Responsibility Segregation

View more presentations from Skills Matter.

Presentation notes: (those are Udi’s ideas rephrased with my own words…)

Queries

Data is always stale, but why not show how stale it is with a timestamp? Doesn’t cost anything.
Representing data with objects is a bit of a waste because as far as queries go, data is not an object, it has no behaviour, it’s just data!
Consequence: go back to the simplest thing that works, the 2-tier architecture.
The UI can talk directly to the DB since layers don’t add any value to queries. For each view in the UI there is one persistent view in the DB. The UI does a simple select *, no calculation. The persistent views have a column for every piece of data to be shown in the UI view.
Data duplication between persistent views is fine. Entities are covered separately by an OLTP model.

Search

Avoid generic search screens that allow users to compose an ad-hoc query with plenty of fields. Focus on the user’s intent only: design the most likely queries to be used and make them available by default with very limited amount of parameters.
For anything that’s not covered by a pre-designed query, use a very simple google-style 1-field full text search. You’d need a data model dedicated to this kind of search (SQL Server full-text search service for instance).

Persistent view model

The views don’t need foreign keys, but they do need indexes.

Security

Use the database for role-based access: user, superviser, each one has its own view with its own set of columns. Data is duplicated but that’s ok.
That’s actually more secure than a memory-based web cache.

Validation

Use the persistent view model to do preliminary validation (as the user types his name for instance). No need to wait until submission.
Validation is done within components identical on the client and the server.
Examples of things that can be checked early: uniqueness, related entity existence…
Validation result in correct in 99% of cases, which is good enough for preliminary validation. The purpose is to answer the question: does the data have chances to be good?
Validation does not have to be mixed with business rules: ranges, lengths, etc…

Commands

Commands encapsulate the business rules.
Rules answer the question: should we do this based on what’s currently in the DB?
Commands can use the usual UI/Services/Business Logic/DAL layers.
Because commands are successful most of the time you can get away with giving an immediate positive answer to the user. Notify him asynchronously if something goes wrong (per email for instance). “Thank you, we’ll let you know if there is a problem”.

User interface design

Tips to capture the user’s intent in the UI while taking advantage of the preliminary validation on the client:

a grid should be able to accept a column and reject another. There is not necessarily a transaction at the row level. Changing a user’s status is a different task from changing a shipping address and can be done separately.
a reservation system should allow people to book blocks of seats rather than forcing the user to tick seats individually until the finds an available block. The system could even look for an available block and notify the user when one is found. A good command ends with: “Thanks, you’ll get a confirmation soon”.
posting a comment to blog. The best way to confirm is to display the user’s comment in the page. You can do that on the client directly, no need to wait until the server updated the page.
cash machine: you can do a preliminary validation too (credit card valid, user authenticated, yesterday’s balance good…). If the balance was not good and the user is overdrawn, no big deal, it makes money to the bank anyway (just make sure there is a clause in the credit card’s small print).

Domain Models

They are not for validation (use components instead), not for queries (remove all objects used only to pull data and use a persistent view model instead).
Keep the domain model for the interesting stuff: the business logic. Remove from the domain model everything (tables or columns) that’s not directly used by business rules.

Coding in Istanbul 3: Pair Coding with a Friend is Fun

2010-02-06T18:02:00.001Z

Helping Jeff with his PHP work was a chance to practice some pair coding. I don’t get to do this at work!

I know Jeff since the engineering school. We graduated from the same place in Paris. At the time (about 14 years ago) we were always coding in pairs: nothing to do with agile, we simply had to! The school labs were equipped with one PC/Solaris station for two people at best. We often had to finish projects late at night. Eating pizza at someone’s place, three or four people on the same computer. Of course we couldn’t afford laptops. I even remember having to carry around my tower PC to a mate’s flat in order to finish a late project! Those were tough times… and pairing with people was natural: it didn’t look like an exotic feature from a new methodology. It was just what all students did.

During our excursions in the coffee-shops and restaurants of Istanbul we both have our netbook. We tried phases where we worked separately, Jeff coding and me looking up some info. But what turns out to work best is to code on the same netbook taking turns at the keyboard. Less chances to get distracted this way. If one loses focus, the other puts him back on track. If one does a typo the other spots it immediately. If we both loose focus we just order another Latte.

We can sustain the pace for about two hours before we become useless.

Quite a good laugh actually… Just like good old times! Too bad we don’t do that at work…

Coding in Istanbul 2: Reverse Engineering a LAMP Project

2010-02-01T17:21:00.000Z

We had a salad at Belvu restaurant, Fenerbahçe then carried on coding in True Blue, a cafe overlooking the Sea of Marmara with a nice view to the Princes Islands.

The stack of Jeff’s site is based on PHP/MySql/Apache. I’m more used to the world of C#/ADO.NET/Linq/T-SQL/SQL Server.

It’s funny how web development differs from enterprise apps!

I got into the habit of encapsulating everything and separating layers but the approach used by some PHP devs is more pragmatic. Business logic, data retrieval, building SQL queries and presentation: everything is done in the same .PHP file!

In Windows client development you typically let a data layer retrieve some data objects (via ADO.NET, Linq or something else) then you assign the data object to a grid control. The control takes care of displaying the content of the object, dynamically creating columns and adding rows… In the PHP files I saw the data layer bit (building the SQL, performing the query, retrieving data) appears just a few lines before a while loop that echoes <td> tags to generate a table for each row of the result set. All layers in the same place.

Also in the code I saw SQL queries were always built on the fly. The first downside of building SQL queries with string concatenation is vulnerability to SQL injection (followed by performance loss due to always rebuilding the query plan and no transactional integrity)… The devs protected the code from injections by using specific PHP functions that parse all user inputs. Why not use stored procedures? Apparently stored procedures are a new thing in MySql: they arrived only in version 5. So I guess they got into the habit of doing away without them.

Regarding the development tools, when working with PHP in Dreamweaver -apart from a bit of color coding- you get absolutely ZERO compiler support. Indentation is manual and there is no intellisense. Comes pretty close to handcarving on stone tablets.

Another funny thing with MySQL: it comes with a large choice of database engines, each one having its pros and cons. It turns out that the default db engine in MySQL is optimised for speed and does not support foreign keys. So by default you work in a system where you can delete parents without deleting childrens and where children can contain a reference to a non existing parent. You’d better be sure that’s what you want, most of the time it’s not…

Coding in Istanbul 1: PHP in Turkey

2010-01-30T10:32:00.000Z

I’m having a çoban kavurma in a restaurant with my old mate Jeff who lives in Istanbul. We stopped for lunch in one of the busy pedestrian streets of Kadiköy on the Asian side of the city.

We spent most of yesterday coding on Jeff’s doctors appointment website. We added a small admin functionality that will save him time when mailing doctors. He’s currently working on getting more doctors registered: it involves finding their details and emailing them one by one. Obviously spamming them all in one go is out of the question so each email is slightly personalised (quoting the doctor’s own website). Yet they all follow the same template so there is some room for automation.

Jeff also needs to track how doctors open and view the campaign emails. Each email contains a number of links: to the appointment’s website, press articles, facebook and to twitter. Pretty clever: each link calls a php on his website that logs the doctor’s identifier along with the identifier of the link. Then he can run a report that shows various stats about who clicked what link…

Good places to take the netbooks out in Turkey include a growing number of restaurants, Starbucks, Gloria Jeans… and Wifi is almost always free!

Gloria Jeans is a great place to sit down with the laptop: more comfortable than Starbucks and less crowded. The one in Ortaköy is perfect: beautiful view to the Bosphorus, 3 levels, smoothies, cheesecakes, tosts (paninis) and free wifi.

View Istanbul in a larger map

Setup, TearDown, When to Mock?

2010-01-12T09:49:00.001Z

I’m writing this while eating at the Launceston Place, a very comfy restaurant near Gloucester Road. Warm decor, soft lighting and excellent service.

Setup and TearDown

Only use those to respectively initialise then destroy mock objects. Methods with [Setup] and [TearDown] attributes are called before and after each test so they’re the only methods that guaranty that things get cleaned up.

Best to avoid the [SetupFixture] attribute (to mark a class that contains Setup and Teardown methods called just once before ALL tests), as well as [TestFixtureSetup]/[TestFixtureTeardown] (called once only before all tests in a fixture). I tried to be clever and used them and ended up having to debug nasty side effects. Unit tests are supposed to run in isolation, so just one [Setup] before and one [TearDown] after each of them, that’s it.

What should be mocked?

Yesterday while writing a UT I bumped into this question: should I bother writing a mock for that object or can’t I just use the real thing?

One answer: objects that access external systems (DB, hard-drive, client API to some server-based system, …) should always be mocked (for unit-tests that is, not integration tests), otherwise the test is not repeatable, and it’s slow…

If a method under test accesses an injected object and this object does only in-memory stuff, then no need to mock it. It would make sense to mock it because after all this injected object is not actually the thing you want to test… Still, the more real objects are covered by the test, the better. As long as they all run in-memory and do not prevent the test from being repeatable, all is well. So no need to waste time writing a mock in that case.

Got to Treat Yourself

2010-01-09T21:32:00.002Z

I bought myself a Christmas present. Next week I’ll receive this neat little thing:

It’s a VAIO X-Series.

This is a replacement for the VGN-SZ5 that I carried around every day in a rucksack for more than two years. Now the rucksack will disappear to be replaced by a tiny messenger bag.

Premium carbon finish, SSD drive, embedded 3G. Extremely thin and less than 900g. This is going to be pure pleasure.

In 2007 netbooks were not around yet and I needed something to write code on the go, not just in my lounge. I got the VGN-SZ5 and a Wenger SwissGear rucksack, both of which I’m still using today. I went everywhere with them: Hertfordshire, TechEd Barcelona, Egypt, the French Pyrenees (picture), Paris, Istanbul… In London I use the Vaio in the tube, in Starbucks, Costa, in pubs, in restaurants (which is extremely annoying to people I go out with).

Last year the rucksack felt a bit heavy on my shoulders so I started looking around for a netbook. I read reviews and benchmarks in magasines, tried out all sorts of models in the shop… Samsung, Asus Eee, Acer, Lenovo, MSI Wind… It wasn’t quite that. Either the 3G card was not embedded, or the thing was too bulky… Not sexy enough.

The first time I saw the X-Series in a Sony shop (in October last year two days before the official UK launch), I immediately thought “I want that”. I gave myself some time to think it over (it’s a bit of an investment). In its October 2009 review, PC Pro said:

But the real problem for the X-Series is its price. (…) Who, though, can lavish £1,130 exc VAT on a vanity laptop such as this?

I just ordered one.

Cheap IoC in native C++

2010-01-03T21:50:00.000Z

In a 2008 episode of dnrTV James Kovac describes how to create a very simple IoC container. All the container does is mapping the name of an interface to an instance of this interface.

Easy to do in .NET with a generic Dictionary, but what about native C++? It turns out it's not that much more difficult: all you need is a STL map and C++'s typeid.

What is an IoC container?
A class that given an interface ISomething returns the concrete object implementing ISomething.

Why do you need an IoC container ?

It's one way to do dependency injection.

Let's say you want to unit-test some business layer that relies on a data layer. The business layer sees the data layer through an interface (let's call it IDataLayer). The actual instance of IDataLayer is created by either:

the entry point of the code that uses the business layer (an executable for instance). This is production code and it has to create the object connecting to the real database.

int _tmain(int argc, _TCHAR* argv[])

{

    // Create the concrete DataLayer object

    DataLayer dataLayer;

    // Register it with the resolver

    Resolver::Register<IDataLayer>(dataLayer);

    BusinessStuff businessStuff;

    businessStuff.DoStuff();

    return 0;

}

the unit-test code. The IDataLayer object in that case is a fake object that doesn't connect to the real database and therefore makes unit-tests easily reproducible and fast.

void BusinessStuffTest::DoStuff_Nominal_DoesNotRaiseException()

{

    // Arrange

    // Create the concrete DataLayer object

    FakeDataLayer dataLayer;

    // Register it with the resolver

    Resolver::Register<IDataLayer>(dataLayer);

    BusinessStuff businessStuff;

    // Act

    businessStuff.DoStuff();

    // Assert

    dataLayer.AssertStuffWasDone();

}

Implementation of the Resolver class
The Resolver class has two methods Register and Resolve. Here is the code for the Resolver class (very straightforward):

#pragma once

#include 

#include 

using namespace std;

///

/// Allows you to Register and Resolve global objects

///

class Resolver

{

    static map<string, void* > typeInstanceMap;

public:

    template<class T> static void Register(const T& object)

    {

        typeInstanceMap[typeid(T).name()] = (void*)&object;

    }

    template<class T> static T& Resolve()

    {

        return *((T*)typeInstanceMap[typeid(T).name()]);

    }

};

This is a just a starting point. In production Resolver could do with methods such as Unregister(), IsRegistered(), UnregisterAll()...

An IoC container is one way to do dependency injection. Other ways include:

pass the injected object using the constructor
pass the injected object using a setter method

Both methods above can become tedious very quickly in a big project.
I tried all 3 methods and I find the IoC approach scales rather well. I've used it for 10 months in a production project and it works nicely with unit tests.

Resources:

Jean-Paul Boodhoo on the gateway pattern
Roll your own IoC container (dnrTV)
The Art of Unit-Testing where Roy Osherove tells everything about dependency injection and fake objects.

Windows 7 and the Press

2009-11-08T21:49:00.001Z

I was replacing Windows 7 beta with Windows 7 Release Candidate on my laptop a few months ago... And I was joking with colleagues about the future success of this new version. Having used Vista for two years
Windows 7 didn't strike me as an earth-shattering evolution. Yet we predicted that people would rush on Windows 7 like bees on cupcakes for two main reasons:

the press was very positive about Windows 7 after it slaughtered Vista
the Release Candidate was as good as the final thing and free to use until March 2010. In other words people could use a fully working OS for free for 10 months.

I don't think Vista deserved the bad press it received. I used it for two years: RC1 and RC2 on an old PC then the RTM 32-bit and eventually 64-bit on my main machine.
Driver issues? A bit at the beginning but nothing that couldn't be overcome.
Speed issues? Desktop: nothing that I could notice on a 4GB dual-processor Dell with raid-striped 15000rpm SAS drives. Laptop: browsing the web was fine, Visual studio was slow. After I moved to Windows 7 browsing the web was still fine and Visual Studio still slow.

When I removed Vista 32-bit from my Vaio and installed Windows 7 RC, it indeed felt a bit snappier. But again, ANY version of Windows feels snappier after a fresh install.

Whether you run XP, Vista or Windows 7 the OS you're running on a machine does not matter as much as the following factors:
- how fresh the install is: re-install your OS often, with a bit of organisation it can be quick.
- the amount of RAM: install more than you think you'll need.
- the hard drive speed: one 15000rpm drive is good, two are better.

I might oversimplify a bit, still I think there are mostly two reasons why you want to upgrade to a new OS:

the untold one: it looks better than the previous version.
the one you tell people to look clever: the OS is faster, contains bug fixes and new useful features.

Thanks to unbalanced press opinions Windows 7 at last gives people an excuse to cave in to the first reason while being covered by the second.

Books I currently flip through #7

2009-11-01T21:10:00.002Z

Technology

Methods

Apprenticeship Patterns, 1st Edition

Finance

The (Mis)behaviour of Markets (Benoit Mandelbrot)

Others

Blink (Malcolm Gladwell)
The Black Swan (Nassim Nicholas Thaleb)

Trying out TeamCity and CruiseControl

2009-11-01T13:12:00.000Z

A few teams use Cruise Control at work. It seems to be a fairly standard choice when it comes to continuous integration. Roy Osherove recommends TeamCity over CruiseControl because he doesn't like getting his hands dirty with XML configuration (can't blame him).

I tried both to get a feel of what you can do with them. I ran TeamCity of my main machine and CruiseControl on a VM to avoid clashes.

I managed to get a build running in TeamCity without too much difficulty. I installed the tray notifier.

CruiseControl is a bit more tricky. After I edited the config files I kept getting exceptions when trying to startup ccnet.exe. Had to go through several install iterations before getting something running.

Installing TeamCity:

Install Tortoise SVN
Install VisualSVN Server
Run the TeamCity installer
Start the build agent manually (rather than through a Windows service).
Install the TeamCity Windows tray notifier

Installing Cruise Control inside a virtual machine.

Windows Virtual PC RC, wich is a new version of Virtual PC for Windows 7.
Install Virtual Server 2008
Install SVN command line
Edit the ccnet.config file

Added a project block
Added a source control block
Added a tasks block. Good news: you can add task where you simply run devenv.exe against the solution file.

Get the Web Dashboard working:

Install IIS: under Windows Server 2008, it's not a Windows feature any more, it's a server role. You have to go to Server Manager > Roles > Add Roles and follow the wizard to add IIS.
Run the CruiseControl.Net installer
Create a new application in IIS for the ccnet webdashboard. In Server Manager, go Roles > Web Server (IIS) > Internet Information Services, open Sites > Default Web Site. Right-click Default Web Site and choose Add Application. Set Application Pool to Classic .NET AppPool.

How to re-build your PC in less than 2 hours

2009-10-30T23:52:00.001Z

Pre-requisites:

Your data on D:, the OS on C:
Your backup ready (nothing to do because you have an automatic full D-drive incremental back-up scheduled to run every day )
Your CD case with all legally acquired original CDs for software and drivers
Access to the spreadsheet where you carefully store all serials for the above.

Go:

Boot from DVD
Format C:.
Click Next, OK, next, Ok, I agree, OK, Next, London GMT
Install your favorite software. For me it is:

Kaspersky
Office
Chrome, Firefox
Skype, Live Messenger
Visual Studio

Check the time: if it took you more than 2 hours, start all over again.

Reading the SQL Server Execution Plan

2009-09-27T16:28:00.104+01:00

We're having a pretty good weather in London: still sunny and about 18C. I took the bicycle to Hampstead Heath with the VAIO in the rucksack and started looking into SQL performance tuning, as you do.

Went through SQL Server 2008 Query Performance Tuning Distilled in Safari.

Here are a few reading notes...

What are the different types of joins?

nested loop join: the most intuitive one. This is the sort of join you would write if you were to code it in C++: iterate over the smallest table first and for each row, look for a match in the other table. Efficient only if the first input is small, and the second is large and indexed.

hash join: used if the largest input is not indexed. This is done in two steps:

step 1: builds a hash table with the smallest of both inputs. This hash table uses a hash function to associate a value in the joined column with an index to a bucket. Go through the whole input row by row and add each row to its appropriate bucket using the hash function.
step 2: go through the second input row by row. For each value in the joined column, work out the index to the bucket in the hash table using the hash function. If a row is present, then there is a match and the row is kept in the result set.

merge join: used if an index exists on the join columns of both tables. The join columns are sorted in both tables using the indexes. Then comparing columns is relatively fast because it takes advantage of the ordering.

What is a RID Lookup?

If a table does not have a clustered index, data pages are on the heap.
If a table has a clustered index, they are inside the clustered index.

Non clustered indexes contain pointers to table rows: this pointer is either

a RID (Row ID) if the table is on the heap
or a clustered index key if the table has a clustered index.

A RID lookup takes place on a heap table (table without clustered index). In order to locate data using a non clustered index SQL Server uses the RID to locate the data row in the heap. A RID lookup is costly because it involves an extra page read (on top of the page read needed for the non clustered index). You wouldn't get this extra page read with a clustered index.

How to see the execution plan directly from the SQL profiler?
This is very handy! No need to try and re-run a slow query in SSMS. Simply track the event ShowPlan XML in SQL Profiler under the Performance group. When you run the trace you can see the actual execution plan of any statement. The plan is displayed in a graphical way as in SSMS.
Warning, not to be used in production! It slows down the performance of the database quite a lot.

More about indexes:
Index A nalysis
Index Design Recommendations

Other resources:
Checklist for analysing slow-running queries

Using XML in SQL Server

2009-09-06T20:12:00.027+01:00

Investigating the possibilities around storing and retrieving native XML data in SQL Server...

... and looking into table partitioning

Books I currently flip through #6

2009-06-20T11:38:00.010+01:00

Methods

Finance

XPath

2009-06-17T23:22:00.000+01:00

XPath came handy lately as I needed to look-up an in-memory XML document.
I know it's probably smoother with Linq to Xml but my project had to compile under VS2005 so I used the .NET 2.0 library which does not contain Linq but contains XPath.

Starting from the following xml file:

<?xml version="1.0" encoding="utf-8"?>
 <pigs>
   <piggy infected="false" name="bob"/>
   <piggy infected="false" name="alfred"/>
   <piggy infected="true" name ="rodrigo">
     <disease name="swine flu"/>
     <disease name="boredom"/>
     <disease name="pig blues"/>
     <address>confidential</address>
     <phonenumber>01234546576</phonenumber>
   </piggy>
 </pigs>

To load the XML in memory:

XmlDocument doc = new XmlDocument();
try
{
doc.Load("piggy.xml");
}
catch (XmlException e)
{
Console.WriteLine("Could not load the file. Detail: " + e.Message);
}

To query elements based on their name:

XmlNodeList allPigs = doc.SelectNodes("/pigs/piggy"); // Returns all nodes called 'piggy' located inside the root-level node called 'pigs'.
foreach (XmlNode node in allPigs)
Console.WriteLine(node.Name + " " + node.Attributes["name"].Value);

To query elements based on their attribute name:

// Returns only infected pigs
XmlNodeList infectedPigs = doc.SelectNodes("/pigs/piggy[@infected='true']"); 
foreach (XmlNode node in infectedPigs)
Console.WriteLine(node.Name + " " + node.Attributes["name"].Value);

To return a single node (same query as above but only one node is expected):

// Returns the single infected pig
XmlNode infectedPig = doc.SelectSingleNode("/pigs/piggy[@infected='true']");
if (infectedPig != null)
Console.WriteLine(infectedPig.Name + " " + infectedPig.Attributes["name"].Value);

To do a query relative to the current node:
All queries above were made relative to the top of the document. But it you call SelectNodes against an XmlNode, you can do a query relative to that node. Just ommit the '/':

// Query relative to the current node. Returns all diseases for the infected pig

XmlNodeList diseases = infectedPig.SelectNodes("disease");

foreach (XmlNode node in diseases)

Console.WriteLine(node.Name + " " + node.Attributes["name"].Value);

Resources:
MSDN: XPath Syntax
LINQ to XML queries
XML Support in SQL Server 2005

Webforms vs MVC (London .NET User Group)

2009-05-31T22:01:00.005+01:00

100 people listened to Sebastien Lambla and Phil Wistanley at the Microsoft customer centre in Victoria. They talked for 2 hours without a break. Two hours is a long time but the formula they adopted was entertaining and fun. Seb was supporting MVC, Phil Webforms. They kept rotating at the mike, going through humorous slides, throwing jokes at each other and interacting with the audience. Seb:

Webforms has a tendency to hide HTML and actually generates a lot of goo.
MVC relies on you knowing HTML but once you learn it, things get pretty easy.
Webform's page lifecycle is complex
Webforms is for morons.

Phil:

MVC is too complicated,
Webforms has a lot of ready-made controls, lots of 3rd party vendors
Uses a familiar event model
MVC is for hippies.

To sum it up, Webforms is for building apps quickly (better suited for the financial world), MVC is good if you need a high level of quality and testability.

C++/CLI Cheat Sheet

2009-05-14T21:22:00.011+01:00

My C++/CLI pocket reference...

Declare a string

    System::String^ myString = "";

Declare a null reference

    System::String^ myString = nullptr;

Pass a string by reference to a method (the reference to the string will be modified, not the string itself since strings are immutable). Use %:

void MyMethod(System::String^% myString)
{
 
}

Declare an array of strings

    cli::array<System::String^>^ stringArray = gcnew cli::array<System::String^>{"Skype","Blogger"};

Declare a managed member inside a native class

class IAmNative{    gcroot<IAmManaged^ > managedMember; public:    IAmNative():managedMember(gcnew IAmManaged())    {     }};

Declare a managed member that will self destroy

#include <msclr\auto_gcroot.h>

class IAmStillNative

    msclr::auto_gcroot<IAmStillManaged^> managedMember; // Will be disposed when IAmStillNative is destroyed

public:

    IAmStillNative():managedMember(gcnew IAmStillManaged())

};

Convert a native STL string to a managed string

System::String^ ToManaged(const std::string& nativeString)

    return gcnew System::String(nativeString.c_str());

The same backwards:

std::string ToNative(System::String^ managedString){   char* str = (char*)(void*)System::Runtime::InteropServices::Marshal::StringToHGlobalAnsi(managedString).ToPointer();    std::string result(str);    System::Runtime::InteropServices::Marshal::FreeHGlobal((System::IntPtr)str);   return result;}

Things you can do

Instantiate managed types, call managed methods from native code provided it compiles with /CLR.
Instantiate native types, call native methods from managed code
Link to the native types of a mixed static lib (/clr)
Declare managed methods that have native types in their signature

Things you can't do

Link to the managed types of a mixed static lib
Link to managed types in a DLL if those managed types have methods with native types in their signature.

First Contact with DevExpress

2009-04-28T22:04:00.010+01:00

Recently I had the opportunity to use DevExpress controls for WinForms. Overall I wouldn't say those controls save you time if you compare with regular Windows Forms controls. The time you save in coding you spend it reading documentation and experimenting with the controls. However the results are definitely superior to WinForms: better looking and with plenty of niceties that come for free (grouping, sorting, customizing, dragging and dropping, advanced tooltips, etc...) The controls I used to far:

GridControl
VGridControl
PropertyGridControl
TreeList

GridControl Comes with plenty of default features such as very powerful grouping and filtering options accessible to users at runtime. To populate it, simply set its DataSource to a DataTable or an IList.

If all you do is setting the DataSource then the control doesn't know anything about the data layer, which is good.
However if you add columns from the designer (either manually or from a datasource) then you create a tight coupling with the data layer, which might be ok depending in the type of app you're creating. I find the designer is great for discovering functionality but when given a choice, it is better to write code: this avoids having the UI know too much about the data.

VGridControl and PropertyGridControl From a distance VGridControl looks like a Property grid control. The main difference is you can display many records at a time while a property grid control displays a single record.

Both theVGridControl and PropertyGridControl use reflection to display the properties of an object.
For some reason PropertyGridControl works fine for public properties with exotic types (such as collections) while VerticalGridControl simply doesn't display collection properties -unless there is something I missed...

TreeList The first column contains the tree, the other columns display data in a grid. Handy when each node of the tree contains usefull info and you don't want users to click each node to see the info. Instead you display the info directly in the grid and users can sort and filter as they see fit. Most of the effort with the DevExpress controls is working out what they do. Two main sources:

The DemoCenter that comes with the DevExpress install
The blonde and brunette from DevExpress TV

T-SQL vs PL-SQL

2008-12-30T12:14:00.000Z

"Oracle PL-SQL is the same as Microsoft T-SQL. If you know one, you know the other"

Not really.

Here are a few differences, some of them can have a serious impact of the way you write sproc code.

Good:

T-SQL: stored procedures return result sets very easily: all you have to do is write a select statement without any 'INTO'. PL-SQL: you have to select into a ref cursor and define this ref cursor as output parameter.

Bad:

T-SQL: stored procedures do not rollback automatically if something fails. Until SQL Server 2005 you had to test @@ERROR after each and every statement and goto a handler to rollback. Ugly, tedious and error-prone. In 2005 you can use a TRY CATCH, which is much more elegant. However you still have to rollback explicitly
PL-SQL: sprocs are atomic. Any error inside a sproc rolls everything back up to the point where the sproc was called.
T-SQL: no %TYPE! You can't refer to the type of a column without repeating it.
T-SQL: RAISERROR does not break the flow. It simply returns an error string or message but the sproc still returns normally. Unless you use it within a TRY block, in which case the flow is diverted to the CATCH block (for SQL 2005 and beyond). Depending on the severity level you specify, RAISERROR within a TRY block either

returns an error number without breaking the flow
jumps to the CATCH block
breaks the current database connection (wow!) (provided you have sysadmin role)

PL-SQL: raise_application_error throws an exception, exits the current sproc, rolls back till implicit savepoint at the beginning of the sproc...

Other differences:

Oracle does not have a BOOLEAN column type, although you can define a BOOLEAN variable in PL/SQL. SQL Server has a BIT column type where values can be only 0 or 1.

Using NUnit with C++ - Part 2

2008-12-27T19:00:00.005Z

Follow up from part 1 We've been experimenting at work with writing unit-tests against native code for 2 months now. We're using NUnit together with CruiseControl and it works fine.

the test project is a C++/CLI DLL compiled with /CLR option.
the project containing the code under test has an additional project configuration that generates a static LIB instead of an executable.
the test project links to the native LIB above.

What if the code under test is C++ compiled with /CLR? For some reason I thought it wasn't possible to create a static LIB when the /CLR option was active. C++/CLI is very flexible and you can generate a static library containing a mix of managed and unmanaged code. The purpose of building a mixed static lib is to easily import native code into a test project. We keep the test project and the code under test in separate solutions.

Get time in HH:mm format regardless of the local culture

2008-11-30T17:55:00.003Z

DateTime.Now.ToShortTimeString() returns a string that depends on the culture. DateTime.Now.ToString("HH:mm") is deterministic. Use DateTime.Now.ToUniversalTime.ToString("HH:mm") to get the current HH:mm time in UTC.

Using NUnit with native C++

2008-11-16T21:19:00.004Z

NUnit was designed to be used with managed apps. So what?

All you need to do in order to test native code is create a C++/CLI project to host the test files. To link the test project to the native C++ project, simply add a configuration to the solution and call it ReleaseUnitTests for instance. This configuration will build the native C++ project as a static library as opposed to an executable. The C++/CLI test project links to this library and can call into any public method of the native project.

Job done! Who needs cppunit?

Detailed steps:

Install NUnit.
In Visual Studio Pro, create a native C++ project and call it CatHouse
Add a class called Cat with a public method void Feed()
Add another project to the solution: choose Visual C++ > CLR > Class Library and call it CatHouseTests. This will create a mixed assembly containing both native and managed code.
Add a C++ managed class called CatTest. Declare it public because we want it to be a managed type that can be seen by Nunit. Add a public method called TestFeed().
Open CatHouseTests project properties, go to Common Properties > References and add the nunit.framework.dl assembly.
In CatTests.h, add the line using namespace NUnit::Framework.
Now you can add the [TestFixture] and [Test] attributes to the class and test method declarations respectively.
By default the CatHouse project has a Debug and Release configurations. Add a new one called ReleaseUnitTests: in Build > Configuration Manager in the Active Solution Configuration drop-down, select New. In the dialog, type ReleaseUnitTests and in Copy settings from choose Release. Click OK. Now all projects have a configuration called ReleaseUnitTests.
Using Configuration Manager, ensure that the Debug and Release solution configurations build CatHouse only and not CatHouseTests. Ensure that ReleaseUnitTests builds both CatHouse and CatHouseTests, both having project configuration ReleaseUnitTests.
Open the properties of project CatHouse and in Configuration Properties > General, select Configuration ReleaseUnitTests. Change configuration type to Static Lib (that will make it possible to link all the content with the test program).
Make CatHouseTests link to the .lib generated by CatHouse. In Project Properties of CatHouseTests, go to Common Properties > Add New Reference > Projects > CatHouse.
Add a #include "Cat.h" in CatTest.cpp and change the project properties so that it knows where to find the include.
In the TestFeed() method, instantiate a Cat object on the native stack and call its Feed() method.
Build the solution in Release mode then in ReleaseUnitTests mode.
In the NUnit GUI type Ctrl O, select the CatHouseTests.dll. The TestFeed test should appear in the tree view.

TechEd EMEA 2008 Developers Wrap

2008-11-14T17:56:00.001Z

What I got out of this TechEd Saw a total of 15 sessions and did 4 labs. If I had to pick just one thing, that would be VC9 DBPro (VSTS2008 Database Edition) with its database deployment and sproc unit-testing features. I saw a talk about it then played with it at the lab. It has the potential to remove a LOT of pain from schema upgrades. Because most sessions I picked were centered around unit-testing I'm a bit less ignorant about the subject:

now I know there is a descent framework for unit-testing native C++ code in VSTS2008 Dev. Ok you have to write C++/CLI but the next best thing is cppunit so... VSTS2008 Dev is apparently the best tool around for the moment.
database unit-testing in VC9 DBPro where it creates a C# class for you that automatically calls a T-SQL sproc where you write your test.
other tools: I had an overview of the tools available for managing unit-tests and IOC containers: Pex, TypeMock, TestDriven. They're on my list of things to try out next.

Integration Testing:

It's worth digging into SpecExplorer 2007 and see what can be done with it. The idea of creating a simplified state model of your system and having the tool generate all possible paths for you is very interesting.

Methodologies

I am now a bit more familiar with IOC frameworks, interface-based coding and TDD so I feel more confident to start writing my first unit-tests. One day maybe I'll write tests before the code...

Friday - TDD and Interface-based Programming

2008-11-14T15:45:00.001Z

DVM303 - Understanding Test Driven Development Speaker: Roy Osherove This is my 3rd session with Roy following from Designing for Testability and Future of Unit-Testing. TDD helps with

trusting your tests
designing the app (because it forces you to take the point of view of the client)
implementing stuff that works partially as opposed to not at all
code quality
documenting the code
reducing integration testing time, reducing number of bugs in production

How to write a unit test with TDD?

add attribute [Test] to methods, [TestFixture] to classs
naming: tests must have a good name to document their intention: method + scenario + expected behaviour.

Content of the test:

arrange (instatiate objects)
act (call the method under test) ,
assert (check the expected result)

Writing process That's the interesting bit, the part that's totally couter-intuitive and requires a good dose of self-discipline.

Make it Fail: write a test that fails because the method is not doing what's expected. Purpose: test your test
Make it work: with minimum code, make the test pass by writing anything even if it's stupid as long as it doesn't break any other test.
Make it better: refactor but do not add functionality and make sure the test still passes.
Want to add more functionality? Ok but write another test first.

Tools:

use Testdriven to run test by right-clicking method in the middle of the method. Apparently TestDriven allows you to run a test even if you don't have a fully working executable.
If you use VSTS, use the built-in MSTest because it brings a lot of integration benefits. It runs slower though. Usually unit-tests are kept in a separate VS project.

Tip from Roy: it's a bit of a culture change so implement it incrementally and don't mention it by name to avoid scaring people off! ARC213 Intentions & Interfaces - Making Patterns Concrete Speaker: Udi Dahan Very sarcastic talk by Udi, making fun of the abuses of the visitor and strategy patterns. "Make your roles explicit" Instead of overriding virtual methods, use generic interfaces. You then retrieve concrete objects for those interfaces from service locators. In the end he was recommending Inversion of Control containers as Roy did in his testability session but with a different approach. Roy's emphasis was on writing code so that unit-tests stick to PC-COF while Udi's angle was on making it easy to change code in large projects. Slides.

Friday - DB Deployment and Unit-Testing with DBPro

2008-11-14T15:40:00.005Z

TLA03-HOL - Introduction to Visual Studio Team System 2008 Database Edition I took this lab as a follow-up to Brian Randell's talk on Tuesday. This is by far the most useful tool I learned about at this TechEd. You start by creating an empty database project.

Then you import the schema from an existing database. VSTS automatically creates a folder struture for schema objects in the solution folder. Each db object has got its own script therefore you can commit them all into CVS if you need to (you don't have to use Foundation Server).
To add an object, right-click Add Table in the Schema View. This generates a script skeleton that you edit as you see fit (add columns, add options, etc...). Save the file and the Schema View is updated automatically. Change an object in the Schema View and the script is updated automatically.

Schema compare That's where it gets interesting:

Choose a target database to compare the offline schema to the target schema.
A view appears that shows all changed, new or missing objects.
Select one of the changed object in the lists and view the difference (new column, changed line in sproc, etc..)
You can also preview the DDL script that VSTS is about to generate
You can enable/disable individual differences from the list if you don't want them to be part of the script or force some to be in the scripts if you really want them to be there.
Export the DDL script to a .sql file then edit if you need to tweak it before testing it against a pre-prod database.

So the tool is nice because it does the dirty work for you, leaves you a high degree of control and makes the schema differences absolutely crystal clear. Database unit-testing

You can create a unit-test for a sproc.
VSTS automatically creates a test class that calls the sproc that contains the unit test. All you have to do is write the T-SQL for the sproc that does the unit-test. DB Sprocs tests are therefore perfectly integrated to all other tests.
You can attach a Pre-Test and Post-Test to a sproc test in order to put the db into an known state.

Thursday - Testing in Team System

2008-11-13T18:46:00.006Z

TLA13-HOL - Visual Studio Team System code name "Rosario": Team Development Unit Test management Everytime you run tests it automatically creates a new test run with a default name and records all the test results. So you keep a history of what tests were run and when and which ones passed or failed. Types of tests you can create VS is used to managed both unit-tests and integration tests. You can create:

coded UI tests (where you write UI tests using automation as opposed to record/replay)

Database unit-tests to test T-SQL stored procedures.

Generic tests. This is actually an external program that will look like an ordinary test from within visual studio. You specify the command line arguments to call the external program. You have the option to redirect the standard output / error to the test results or not.
Load tests. That's quite fun: you can simulate heavy load conditions with many users. Among other things you can define a load pattern (constant load or increasing load) and the distribution (percentage of appareance for each test).
Manual tests: simple text file describing a manual test procedure. This is there for auditing purposes only obviously.
Ordered test: specify a list of tests to be run in a specific sequence in the situation where order counts (that's for integration tests only, unit-tests should not be order dependent)

Impact analysis: as you change code, VS gives you the list of recommended tests (tests that should be re-run as a result of your changes)

Thursday - SQL Tips and Tricks

2008-11-13T14:29:00.003Z

Before he ran to the post office to mail stuff that Carl Franklin had left behind, Stephen Forte demonstrated a few tips:

To write a fancy UPDATE statement that needs a complex subquery, use WITH (common table expression)
To keep 2 tables in sync with one single statement, use the new MERGE syntax (new in 2008).
To filter out the results of a ranking query without affecting the rank values, use WITH again.

Thursday - Test Creation Automation with SpecExplorer 2007

2008-11-13T09:28:00.006Z

DVP302 - Automating Test Creation Something clever to automate creation of integration tests: SpecExplorer 2007 Keith Stobie demonstrated a tool that automatically generates tests based on a simple model of a system. You start by defining states and actions. Actions have an effect on the System Under Test and cause a transition to another state. You also define expectations (the expected results of those actions). The evaluation of those expectations either causes an error (and the test fails) or results in a transition to another state. In other words you model your system with a state machine. The whole point of the tool is to generate all possible ways of navigating through the state transition graph. You end up with non deterministic tests that go through scenarios you didn't necessarily think of. This makes the tool relevant to integration testing, not unit-testing. Last night's UK Country drinks at BroadBar badly affected my attention span so all I remembered right after Keith's talk was:

Lots of little tests is better than a single big one There are no perfect models, only useful ones Start small: even very small models can find bugs

DVP01-IS Model Based Testing with Spec Explore In the afternoon Keith did another demo of SpecExplorer 2007 followed by Q&A session.

How does SpecExplorer interact with the System Under Test ?

You use adapters that hide the concrete implementation of the actions (such as calling an executable, sending a message, calling a sproc, clicking a button, etc...)

SpecExplorer is a free add-in to Visual Studio that will be available as a powertoy in the beta of VS2010.

There is a video about SpecExplorer here.

Wednesday - More Unit Testing and more C++

2008-11-12T14:04:00.005Z

ARC308 - The future of unit testing Following up yesterday's session about designing for testability, Roy gave an opinionated overview of the existing unit-testing tools/frameworks (there are lots of them), comparing those he thinks will stick around against those that might fade away. His final advice:

if you're new to unit-testing, don't get into TDD immediately, start with writing a few unit-tests incrementally to get the hang of it.

His powerpoint is here. TLA401 - Microsoft Visual C++ 2008 for Unrepentant C++ Developers The last time I saw Kate Gregory was at TechEd 2007 in Orlando. She hasn't changed: her demos go really fast :-). VS2008 SP1 comes with TR1 which is a set of proposed additions to the next C++ standard C++0x. It includes stuff that is currently in the boost library. Kate explained and demoed each of the following language additions:

shared_ptr: safer, more intuitive, more powerful than auto_ptr.
lambda expressions: [] or [&] or [=] equivalent of => in C#. Allows you to define short functions inline. She showed how elegantly they integrate with STL.
auto: equivalent of C# var so you don't have to spell out a type when the compiler can work it out by itself.

Kate's question to the audience: "who in this room learned C++ in this century?" Just one person raised his hand (that wasn't me). Kate's final message: "C++ is not dead!". Indeed, new versions of Visual Studio keep adding new things for C++:

TR1 additions
New MFC classes, ribbon toolbar
New libraries for parallel development
Some Vista features are only available to C++
C++ is the most practical language to do interop

Tuesday - Unit Testing in Visual Studio

2008-11-12T12:50:00.005Z

DVP304 - Building fast and secure native applications with VSTS 2008 The good news from this session by Lin Xu: yes, you can run unit-tests against native C++ from VSTS! The catch is you have to write them in C++/CLI and compile the test project with the /clr option. VSTS 2008 also comes with the sort of features that tools like DevPartner offer: Profiling:

choose between sampling (takes snapshots at intervals) and instrumentation (records all function calls).
instrumentation generates a call tree view
performance reports show how much time was spent in each function.

Static analysis:

compile with the /analyse option to raise warnings highlighting potential coding mistakes. This tool is based on another one called Prefast.

Coverage:

display the portions of code that were executed by the unit-tests.

Lin Xu's powerpoint presentation: https://emea1.msteched.com/resources/presentations1/DVP304_Xu.pptx It turns out that the unit-testing feature alone is also available in the Pro version. Visual Studio Pro Academic on Amazon costs less than £120.

Tuesday - Designing for Testability

2008-11-12T07:17:00.007Z

ARC307 - Designing for Testability: Bridging the Gap between Design and Testing in Object-Oriented Software Speaker: Roy Osherove Probably the most inspiring session so far. It struck a chord because I recently gave some serious thought to coding unit-tests and using TDD. What has been holding me back so far is that most unit-testing frameworks seem to be targeting the .NET framework as opposed to native C++. For each piece of coded logic says Roy Osherove you should follow the PC-COF rules:

Partial runs possible
No need for Configuration
Consistent pass/fail
Order does not matter
Tests must be Fast

Dependencies between objects should be based on interfaces (to enable creation of mock objects), which then raises the problem of dependency injection: how and when do you pass a concrete object A to an object B that expects interface A? You can do injection in various ways:

Constructor (forces you to pass a concrete object)
Property (passing the concrete object is optional)
Factory (passing the concreate object is done as part of a factory method)

It can become very clunky to do code injections for all objects unless you use inversion of control containers. Those containers hold associations between abstract interfaces and concrete objects. Inversion of control frameworks:

Spring.net (I met one of the authors of Spring.net at TechEd Orlando)
Castle.Windsor
StructureMap

Unit-testing tools:

MSTest (the one integrated in Visual Studio Team System)
Nunit
TestDriven.NET
Typemock isolator

Slides

Tuesday - Schema Compare in VSTS2008

2008-11-11T11:07:00.007Z

TLA313 - End-to-End Database Development Using Microsoft Visual Studio Team System 2008 Database Edition Brian Randell talked about how to manage change with SQL Server. That was a useful session! I've been struggling with this problem for years with Oracle and SQL Server: you have to manually write DDL scripts when doing database upgrades. Those scripts make visualising changes difficult and reversing changes impossible unless you write undo scripts. DBPro (VSTS 2008 Database edition) generates those DDL scripts automatically: you edit your schema offline within your solution (all under source control) then when you need to deploy the schema, DBPro compares the target DB model to the offline model and generates the scripts automatically. So no need to store the DDL scripts in source control any more (ALTER statements to add a column for instance). What you store in source control is the declarative model (i.e. the CREATE statements for the successive versions of the table). If a DBA makes a change in production, you can have DBPro compare the schemas and highlight the differences. You can then select an individual difference and bring it back into the offline model to keep it in sync. Sounds brilliant. By the way, this is available in VSTS2008 but works against SQL Server 2005. Brian did the whole demo against 2005 to make a point that there is nothing specific to SQL Server 2008. Resources:

Gert Drapers blog (Data Dude): http://blogs.msdn.com/gertd/
Will take ages to download the trial version of DBPro from here
The list of features in VSTS2008
Power Tools for DBPro

Monday - Agile Development in VS2008 and VS2010

2008-11-11T07:21:00.005Z

DVP03-IS - Agile development with Visual Studio Interactive session with Stephanie Saad who demonstrated some features of VS2008/2010. There are very interesting things for database development in the DB edition of Visual Studio Team System:

DB refactoring: changing a table name automatically updates all dependencies
DB deployment: you can generate a script that contains all differences between the current schema and the production schema.
DB testing: C# Unit-tests for T-SQL stored procedures where you define setup, test and teardown Test data generation (that's really handy for the setup part of the unit tests)

Apparently the dev and db versions of Visual Studio Team System have been merged into one license for 2008 so if you buy one, you get the other one for free. Other notes: VSTS 2008

see author/changes directly in code
define check-in policies (to force you to pass unit-tests for instance)
define triggers such as on check-in build
setup build notifications popups (through powertools)
code metrics: show the number of lines, cyclomatic complexity
all of the above present in VSTS for DB, Team System Server is not required

Coming up in VSTS 2010

Planning, task management: creation of user stories / tasks hierarchy, capacity planning, reports to show test results per user story / task.
Gated check-in: define check-in conditions, for instance you can't check-in if it doesn't build or if the unit-tests don't pass

Monday - Keynote

2008-11-11T07:04:00.004Z

The CCIB got much busier after lunch, delegates with orange badges running everywhere, and the auditorium was packed for the keynote. The Keynote Session Although Christopher Lloyd didn't appear on stage in a De Lorean as happened in TechEd 2007 in Orlando, this keynote was a pretty good one. Nothing about adding-business-value-by-leveraging-customer-statisfaction bs: Jason Zander knew his audience so he did a demo-based presentation, going through some of the new features of Visual Studio 2010, changing, building and running code. To show that VS still provides support for unmanaged C++ he imported a VC6 implementation of Pong into VS2010 then

sexed-it up with the MFC C++ ribbon classes,
parallelised a lengthy for loop across 8 cores with parallel_for
and eventually showed off the multi-touch capabilities of Windows 7 by moving both pong racquets with his fingers simultaneously, under a round of applause.

video of the keynote

Monday - Silverlight 2.0

2008-11-10T09:48:00.004Z

I am in Barcelona for TechEd. The keynote is planned for the afternoon, no sessions until after the keynote so my morning will be spent at the self-paced labs. I had planned to do some labs about multithreading in Visual Studio but it looks like they've gone from the schedule! Nevermind, I just did a lab on Silverlight Foundations lab (WUX11-HOL) Notes from the lab: • Silverlight projects produce xap files, wich are just zip files that contain dlls and a manifest. • When you create a Silverlight project in Visual Studio, it generates a new ASP.NET project by default to host the silverlight control. • Layout: VS contains some sort of read-only designer to preview the results of changes to the XAML file. Editing the layout WYSIWYG-style is done in Expression Blend. • Blend much more powerful than Winforms editor: add rectangles, ellipses, bezier curves, clever resize/skew commands... Apparently you can switch back and forth from Blend to Visual Studio using the same solution/project files. • Expression Design is used to build vector graphics that produce a XAML file. I'm considering Silverlight to build business apps so I also started the Communications and Data with Microsoft Silverlight 2.0 (WUX14-HOL)

Free Wifi in Istanbul

2008-11-04T09:07:00.003Z

It seems that in Istanbul, many coffee/restaurant chains provide free Wifi access.

Palladium shopping center:

Cafe Nero

Midpoint restaurant

SushiCo restaurant

Capitol shopping center:

Kitchenette

Sugar Sync

2008-11-02T17:05:00.005Z

Currently visiting a friend in Istanbul who showed me a useful on-line storage tool: for $50 a year you can store 26GB of data online.

your data is encrypted with AES, which is the least you should ask for, really.
the really nice thing: it comes with a client that automatically syncs your PC folders with the remote storage.
the sync works accross many PCs, which is cool as well.

That means you don't need to schedule back-ups of your important files, this is an automatic instant off-site backup (much more useful than my daily incremental Vista-based on-site backup).

An there's more:

you can access your files online over the web.
you can, and that's really convenient, send links to large files per email to other people. The recipient doesn't need to login to retrieve the file, he just clicks it. You get notified when he receives it.

Timezone conversion

2008-10-12T17:30:00.004+01:00

How to convert a local time from a given time zone into UTC?

For instance how to convert the time in New York (Time zone: Eastern Standard Time) into UTC time, taking into account the fact that daylight saving is currently active in New York (until November 2)? Sounds trivial? Well it's not, at least not until .NET Framework 3.5.

The standard time zones are listed in the Windows registry: each time zone string is a key. The values under that key describe the timezone. For instance the "Eastern Standard Time" key contains info such as UTC offset and daylight saving start/end dates. Unless you use 3.5, you have to write code yourself to retrieve that info from the registry. It looks like the simplest way to do it is to use System.TimeZoneInfo in .NET 3.5. Example:

DateTime time = new DateTime(2008, 11, 7, 11, 46, 0, 0); TimeZoneInfo info = TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time"); DateTime utcTime = TimeZoneInfo.ConvertTimeFromUtc(time, info); Console.WriteLine("New york time: " + time.ToString()); Console.WriteLine("UTC time : " + utcTime.ToString());

Some resources:

General info about time zones: http://www.timeanddate.com/
Upcoming daylight saving changes: http://www.timeanddate.com/time/dst2008b.html

Curly Braces

2008-09-28T12:19:00.000+01:00

Visual Studio Add-in to highlight curly braces:

http://www.codeproject.com/KB/macros/matchingbraces.aspx

Currently Listening To...

2008-09-02T21:19:00.004+01:00

Hanselminutes: .NET 3.5 SP1 is out...what's inside
Mark Russinovitch, the Case of the Unexplained http://www.microsoft.com/emea/spotlight/sessionh.aspx?videoid=722 (TechEd Barcelona)

Code-Tourism

2008-06-16T16:35:00.003+01:00

Code-tourism in London... After a few hours of coding at home, I usually get bored and feel like I'm missing out. So here is a new concept: coding while visiting London. I take my laptop with me and start wandering around. You need a 3G connection for this, Wifi is not enough. Visit a museum, have a bicycle ride, then sit down somewhere in a park, a library or a coffee-shop to do some coding for an hour. Good places to tourist-code:

Waterstones Picadilly: huge book-shop with comfy sofas.
Starbucks, Holland Park avenue. Comfortable seats and relaxing music. Bicycle racks outside. Crap 3G signal though.
Foyles, Charing Cross Road: the most complete bookshop for computing I know in London. No sofas though. You cannot take books to the coffee area.
Pret-a-Manger, Kensington High Street. Good seats. Bicycle racks outside.
Bincho, OXO Tower. Japanese Grill Restaurant. Cool view to the Thames, very quiet during the week.
British Library: the cafe inside has plenty of tables next to a huge books tower.

Currently listening to...

2008-05-28T23:04:00.004+01:00

Finding Passion for Software *or* The Return of the "Franklinator"

2 new episodes of Mondays....

Books I currently flip through #5

2008-05-27T22:55:00.002+01:00

A colleague recently recommended Windows via C/C++ (Jeffrey Richter)
Derivatives demystified

Things missing from C++

2008-05-10T17:06:00.005+01:00

I'm using unmanaged C++ for a new project at work. For a while we hesitated between C++ and C# and eventually went for unmanaged C++ because of legacy libraries. I miss some of the C# features such as:

No cpp/h separation
Automatic creation of properties from a member field with snippets
Automatic implementation of interface methods in concrete classes.
The .NET collections (I am currently using the STL collections, they're a real pain in the bum)
Interfaces: although you can simulate interfaces in C++ by creating a class only just virtual pure methods and no field, there is still a risk it turns into a base class if someone adds some code to a method...

.NET Framework 3.5

2008-05-10T07:27:00.002+01:00

The .NET Framework 3.5 complete install package: http://download.microsoft.com/download/6/0/f/60fc5854-3cb8-4892-b6db-bd4f42510f28/dotnetfx35.exe The install DVD download for Visual C# Express 2008 http://www.microsoft.com/express/download/ at the bottom of the page

Installing Windows Vista 64

2008-03-23T13:25:00.004Z

13:30 Started install, booted from DVD.
14:26 OS Installed ok. Had a crash during install after creation of the admin user but carried on with install. My NVIDIA Quadro FX3500 is recognised immediately and works in 2500x1600. Did not have to install the NVIDIA driver yet, using the driver provided by Microsoft. Now running Windows update.
15:32 Windows updated completed. Rebooting OS.
16:00 Ran Windows update again. Service Pack 1 does not appear in Windows Update yet. Installed Kaspersky 7.0.1.325.
16:40 Installed Office Ultimate and ran Windows update. Office Service Pack 1 appears in the list of updates. Started installing it.
17:26. Office Service Pack 1 installed ok.
18:15. Finished configuring Outlook email. I have a Hauppage WinTV-HVR. Does it support Vista 64? Apparently yes according to this page: http://www.hauppauge.co.uk/pages/whatsnew/vista.html
18:45 Downloaded the software and Vista 64 drivers for WinTV-HVR 1100 from the Hauppage support website. Driver install failed: it could not find the hardware. Interestingly, Windows Media Center can use the card: though downloading TV guide failed, scanning of freeview channels worked perfectly.
19:00 Understood why downloading the TV guides with Windows Media Center failed: Kaspersky antivirus was being to keen. Disabled it and it worked fine.
Installed the logitech drivers of my S510 keyboard to be able to control Media Center from the Logitech remote. It works brilliantly.

Summary: I installed the 64 bit version of Windows Vista today and everything seems to be working fine. My Nvidia card hasn't crashed yet. So far the only driver that caused me grief was the Hauppage card when using it with WinTV software. The WinTV software is a big pile of poo anyway, who wants to use that when you have something like Windows Media Center? I might discover more hardware problems this week.

Currently Listening To

2008-02-06T22:24:00.000Z

Dot Net Rocks: Andy Leonard on Unit Testing your Database

Usefull Links

2008-02-03T14:12:00.000Z

List of Visual Studio shortcut keys: http://www.dofactory.com/ShortCutKeys/ShortCutKeys.aspx#textsel

Pandora not available in the UK

2008-01-12T12:45:00.001Z

I started using Pandora in April 2006 and it quickly became the only way I listened to music at home. It simply matches my approach music: I tend to get bored of tracks fairly quickly and need to renew my list of favorites very often. Also my tastes are not too mainstream. The Find similar or bin it model of Pandora is perfect for me: I built a collection of genres (electronic, heavy metal, dark ages...) and keep switching from one to the other depending on the mental state I want to be in. I discovered artists I never heard of before: KMFDM, Pro-Pain, Darkwell, Leave's Eyes, Kill Hannah, Nightwish, Theatre of Tragedy, Elis, Lacuna Coil... I even ended up buying the CDs of those I wanted to listen in loop. Unfortunately, because of licensing issues the service will very soon stop in the UK...

Usefull Tools

2007-11-30T10:35:00.001Z

editor: Notepad2 (free)
screen capture: Screen Hunter 5 (free for business use)
secure FTP client: Core FTP LE (free for business use)
refactoring for C++: Refactor! (free for business use)
cheap Photoshop-like: Paint.NET (free)
browse .NET assemblies: Reflector (free)
Web Apps: http://www.dumblittleman.com/2008/12/21-excellent-web-apps-for-college.html

Currently listening to...

2007-11-23T22:08:00.000Z

Martin Wolf - FT.com - Welcome to a world of runaway energy demand
Channel 9 - MFC is back! - Pat Brenner: New Updates to MFC in Visual Studio 2008. This is quite an event. Nothing happened in MFC since Visual Studio 6 in 1998. I'm really surprised since I talked with a Visual Studio developer at TechEd who told me there would be nothing new for unmanaged C++ in Visual Studio 2008.

There's Perception and There's Reality

2007-10-28T21:37:00.000Z

Joe Wilcox on Apple and Microsoft: http://www.microsoft-watch.com/content/operating_systems/the_great_double_standard.html?kc=MWRSS02129TX1K0000535

The blogosphere will praise Leopard as the next best thing ever and use it as more proof why Vista sucks (It doesn't). Meanwhile, there will be little good said about Microsoft's colossal 2008 fiscal first quarter results. Those people acknowledging the earnings results will blame Microsoft for trying to kill Linux and babies in Africa as reasons for its success. The perception: When Microsoft competes, it cheats. There is a double standard.

Performance Measurement for Financial Systems - Messaging

2007-10-07T14:49:00.000+01:00

Performance Measurement for Financial Systems

Seanet: http://www.seanet-tech.com/products.html
Wombat: http://www.wombatfs.com/
Endace: http://www.endace.com/

Messaging Systems

Martin Fowler: Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions

Got my TechEd DVD

2007-09-23T21:09:00.000+01:00

Received the 6DVDs per post. Will never run out of stuff to listen to...

TechEd DVD

2007-09-13T21:35:00.000+01:00

The TechEd DVD is on its way!

Dear Tech.Ed 2007 Attendee, This e-mail serves as confirmation that your Tech.Ed 2007 DVD set has been shipped and you should be receiving it within 1-2 weeks (International locations may take longer). The DVD set has been sent via the U.S. Postal Service to the address included with your registration information.

Currently listening to...

2007-07-21T21:44:00.000+01:00

Hanselminutes: Jeff Atwood overclocks the Ultimate PC
.NET Rocks: Dan Ciruli's Grid Computing Redux

Currently Reading...

2007-07-14T08:01:00.000+01:00

Dr Dobb's: The Pillars of Concurrency
ft.com: The dream machine: invention of credit derivatives