Neil Rees's Blog

I blogged previously on setting up a simple Ubuntu based router and load balancer for a virtual test environment we were building.  We’ve now reached the point that we want to clone the entire test system so that we can test different releases in the different environments.  The Windows based virtual machines cloned without issue, however the Ubuntu VM did not appear have any network adapters after having been cloned.

Running

$ ifconfig

yielded just the loopback adapter and nothing else:

Because the new VM has different MAC addresses on its virtual network adapters, the VM has configured them as new interfaces, and there’s no config for them in /etc/network/interfaces.  The new interfaces are numbered as eth2 and eth3 so we could possibly repeat our configuration steps from the first blog post substituting the new NIC names as we went, alternatively if we:

$ sudo rm /etc/udev/rules.d/70-persistent-net.rules

And then reboot, it will redetect the network adapters and they’ll then be installed as eth0 and eth1 again, so running ifconfig again gives the correct answers:

I’ve recently been building a fresh test environment to mimic a live environment as closely as possible using the virtualisation technology that we have here.  My design goals are:

• The test environment should have the same network topology as live
• The test web servers should be load balanced the same as the live servers
• The resulting set of machines should be able to be cloned in order to produce multiple test environments
• This has to work on both Hyper-V and VMWare Infrastructure.

The complexity in the solution stems from the fact that this particular solution has a number of SSL websites which are served up by a pair of web servers, SSL sites cannot share an IP address like non-encrypted sites can so there are a large number of IP addresses and load balancing being done.

I’m going to build a Linux server to act as a router and load balancer between my virtualised test environment and the real world, hopefully by using Linux for this piece I can get this working whilst using as few resources as possible on my virtualisation platform and get it up and running quickly and cheaply.

Here’s the environment I’m aiming for:

This article is mainly an aide-mémoire for me as I’ve built a few of these systems now, and each time it takes me half a day to remember how to configure each of the steps, so here’s how I went about building it.

Firstly download Ubuntu 9.10 Server from Ubuntu’s download site.  Create a new virtual machine and add two network adapters to it – attach the first adapter (which will become eth0) to the main network and attach the second adapter to your internal network (which will become eth1):

Start the install and choose the default server install.  The kernel that gets installed needs to have support for IPVS, the minimum install option (underneath the F4 option) does not have this support in the kernel so I went for the standard install instead.  Work through the setup and don’t add any extra software to the server when it asks what packages to add.  Once it’s installed and rebooted, we need to log in and install ipvsadm

$ sudo apt-get install ipvsadm

That’s the only extra piece of software we need, so now we just need to configure it.

Switch To Static IP Address

The server is by default installed with a dynamic DHCP IP address, to save any issues with the server being allocated a different address in the future I’ve had a set of addresses on the network reserved and can now assign a static IP address to this machine.

To change the configuration of the network interface we need to edit /etc/network/interfaces, by default it will look like

auto eth0
iface eth0 inet dhcp

To change it to use a static IP address the networking information can be inserted here

auto eth0
iface eth0 inet static
address 172.21.5.200
netmask 255.255.248.0

To get changes that are made to the interfaces file applied we can restart the network:

$ sudo /etc/init.d/networking restart

We can also configure the network interface for the internal network to have the 192.168.1.1 address:

auto eth0
iface eth0 inet static
address 172.21.5.200
netmask 255.255.248.0

auto eth1
iface eth1 inet static
address 192.168.1.1
netmask 255.255.255.0

At this point we can configure the internal windows servers to use this machine as their default gateway:

Configuring Routing/NAT

We now need to teach it to pass traffic between it’s two network interfaces, one on the internal virtual network and the other on the physical network.  For this to work IP Forwarding must be enabled in the Kernel, to enable it you run

# sysctl -w net.ipv4.ip_forward=1

This is not a persistent change though, to make the change permanent make the corresponding change in /etc/sysctl.conf, the following will be presented but commented out:

net.ipv4.ip_forward=1

After making the change to sysctl.conf run the following to pick up the change:

# sysctl –p /etc/sysctl.conf

The virtual network is on eth1 with the physical network on eth0, this will allow all the virtual machine to send traffic to the physical network:

# iptables –A FORWARD –i eth1 –j ACCEPT
# iptables –A FORWARD –o eth0 –j ACCEPT

However, in our environment we require the router to perform network address translation as the addresses on the virtual network are not routable from the physical network. So also:

# iptables –t nat –A POSTROUTING –o eth0 –j MASQUERADE

This should then allow the internal virtual machines to ping machines on the physical network.  However these settings are not persistent yet, if the router is rebooted this configuration will be lost. So we can save the current configuration to a file and then put that file somewhere handy:

# iptables-save > /etc/iptables.rules

These rules can be loaded when the network interface is brought online by adding an extra line to /etc/network/interfaces:

auto eth0
iface eth0 inet static
address 172.21.5.200
netmask 255.255.248.0
pre-up iptables-restore < /etc/iptables-rules

The routing rules should now be persistent and survive reboots.

Additional IP Addresses

However we need to have multiple IP addresses on the public side of the router so that each one can have an SSL web site associated with it.  To add extra addresses to existing interfaces you can create virtual interfaces again by editing /etc/network/interfaces.  The definition in the interfaces file now looks like:

auto eth0
iface eth0 inet static
address 172.21.5.200
netmask 255.255.248.0
pre-up iptables-restore < /etc/iptables-rules

auto eth0:0
iface eth0:0 inet static
address 172.21.5.201
netmask 255.255.248.0

Add Load Balancing onto each of the Virtual IPs

Firstly, define the service that is to be exposed load balanced:

# ipvsadm –A –t 172.21.5.201:443 –s rr

The –s rr uses Round Robin scheduling when assigning new connections to servers.  Secondly need to add the servers in that are going to handle requests to the service:

# ipvsadm –a –t 172.21.5.201:443 –r 192.168.1.3 –m –w 100
# ipvsadm –a –t 172.21.5.201:443 –r 192.168.1.4 –m –w 100

Again this configuration isn’t permanent, to make these settings apply automatically at boot time the config needs to be placed into /etc/ipvsadm.rules :

# rules file for ipvsadm
–A –t 172.21.5.201:443 –s rr
-a –t 172.21.5.201:443 –r 192.168.1.3 –m –w 100
–a –t 172.21.5.201:443 –r 192.168.1.4 –m –w 100

This can then be rinsed and repeated for eth0:1, eth0:2 etc depending on how many addresses you need.

So that’s it - we’re good to go, we can now deploy our test environment onto the virtual servers and get on with the testing.  Users on the main networks can point a browser at https://172.21.5.200 and https://172.21.5.201 (which will get a nice DNS names assigned to them) and the router and load balancer will forward the requests to the virtual server farm.

Remember :

$ man

is your friend :)

It’s been a long time since I’ve had the opportunity to geek with a bit of new hardware, and this didn’t really look like this was going to provide much of an opportunity at first.  The back story is that the power supply in the Media Center has been breaking down slowly and emitting a nasty whistling noise which has been driving me up the wall, let alone anyone with more acute hearing than me.

I set about hunting down a quiet power supply to replace it with, one that didn’t whistle and didn’t emit a huge amount of fan noise.  I settled on the Arctic Cooling Fusion 550R as it promises silent operation and has an airflow path compatible with the narrow HTPC case that I’m using.

Now the thing that caught my attention and is the real reason that I’m writing this is that it claims to be high efficiency energy saving unit.  Now normally I ignore the bold claims made on packaging, however this one has piqued my curiosity and was worthy of experiment.

The web site says:

High Efficiency = Less Power Loss = Quiet Cooling
With an efficiency of at least 82 to 86%, less heat is generated inside the PSU. As a result of this a single 80-mm-fan is enough to cool the PSU. The rubber-mounted fan is spinning in a range between 700 and 2000 RPM making it virtually silent. Additionally to this the PSU comes along with two external fan-connectors giving the possibility to control the case ventilation based on load and temperature and thus lowers the noise level of the complete system to an absolute minimum.

Effective energy-use preserves nature and saves money 
By making appreciated use of natural resources, especially energy you can contribute to lower CO2-Emissions. Additionally you save electricity cost of about 100 €, so the Fusion 550R is a true Payment Saving Unit*.

* In 4 years, 200 d/y with 4 h/d full load at 0,15€/kWh

Ok, understood.  I used my power meter to measure the power the Media Center uses using it’s old power supply (an old Q-Technology 400W unit, circa ~2003).

87.8W at idle at desktop, and 5.1W when off.  This stunned me, that’s not on standby, that’s 5W been consumed when OFF – proper off.  I’m amazed at the phantom load that a switched off PC creates, I’ll not mock my Grandmother for switching her electronics of at the mains overnight ever again!

I swapped out the QTechnology PSU with the new Arctic Cooling PSU, and repeated the test:

Well I’ll be damned.  Not that I doubted the advertising you understand.

Let’s verify their maths.  4 years, 200 days/year with 4 hours/day.  So that’s a 4 x 200 x 4 = 3,200 hours of operation during the four year sample.  Based on my readings I’m saving 22.9W, which means I’ll save 1KWh every 43.6 hours of operation.  So, 3200 / 43.6 = 73.3 KWh saved.

And 73.3 KWh costs the princely sum of £8.80 (at £0.12/KWh).  So I’m not sure about the €100 claim that they make, however I’m happy - the unit is silent in operation which was the goal, I’m using at least 25% less power (without changing any other components!) which can’t be bad, and it’ll pay for a takeaway curry over the course of it’s life.

Nice.

If you want to reset PocketYam back to it’s ‘factory fresh’ state, it’s as simple as deleting a single file…

You may want to do this if:

• You want to change the yammer identity associated with your device
• The authorisation process was so much fun you want to do it again

To clear the configuration, launch File Explorer and navigate to \Application Data\Yammer\Data and delete the ‘settings’ file that’s in there:

The next to you start PocketYam you’ll be guided through the Authorisation process again.

Microsoft’s new search engine has opened it’s doors to the public early, see what you think:

www.bing.com

Dave Friend, CEO of Carbonite the online backup provider, has given out some stunningly bad advice on how to check the integrity of the files contained in your online backup.

In response to the question: “How do I know if the files on Carbonite's servers really match my files?”

Dave answers:

Pick any backed up file on your PC. Delete it. Now open the Carbonite Backup Drive from the icon on your desktop. Find the file you just deleted – the status column will say "Right-click to restore latest backup copy (Original file deleted)." Right-click on the file, and select "Restore." In a few seconds, you'll see a Carbonite pop-up saying that the file has been restored. Go back to your C: drive and open the restored file. You'll see that it's perfect.

Let me get this right, to check that a file is backed up, delete it and see if you can restore it?  And if it’s not backed up, what then Dave?

How about: restore a folder that you want to test to a different location, and then use a tool like Beyond Compare to compare the two folders for files with differences or files that are missing? If you find any discrepancies between the original folder and the folder that you’ve restored you need to question the software that’s managing the backup.

It's in the release notes ( http://download.microsoft.com/download/4/5/6/45605938-AC68-4FF7-A092-10AFF0F7E5C7/Windows%20Server%202008%20R2%20Beta%20Release%20Notes.htm#ID0E6G ) however make sure that you are aware of this:

When MP3 files are added (either manually or automatically) to either the Windows Media Player or the Windows Media Center library, or if the file metadata is edited with Windows Explorer, several seconds of audio data may be permanently removed from the start of the file

Eeek.  So if you want to let Windows 7 anywhere near your MP3 collection, either make them read only like the release note suggests or make sure you have installed the patch (http://go.microsoft.com/fwlink/?LinkId=139391) manually or visited Windows Update to get it.

Since writing some time ago about how you would in theory go about creating a web site that could remotely wake up PCs that have gone to standby to conserve power it has generated a lot of interest.

If you're interested in deploying a system like this in your organisation, or at home, the source to this system is available below.  Here's what you'll need to deploy it.

1.  You need to identify a suitable server for this, it needs to be either Windows XP, Vista or Server 2003 or 2008.  It also must have IIS installed, .net 2.0 or higher, have ASP.NET enabled, and have SQL 2005 Express Edition installed.

2.  Download the application from here and extract it into it's own folder.  In this case I've extracted it into c:\inetpub\WOL

3.  I'll create a standalone web site for the application - there's no reason why it couldn't exist as a virtual directory underneath another app, but you may have to tweak the web.config slightly in that scenario.  Launch IIS Manager (I'm using Vista), Click Sites then Add Web Site.  Give the site a name, point the directory at the location we just unzipped the solution into and choose a port to listen on (I'm using 8080):

4. Click ok, and it should be that easy.  In practice you'll need to make sure that the user the application is running as has read/write access to the database files in the App_Data folder.  You then just need to point a browser at it:

Then configure all your workstation to sleep after 20 mins of inactivity and you'll be able to sleep at night safe in the knowledge you are burning much less electricity, and also saving money!

The binary distribution can be downloaded here (200kb)

You can download the full source from here (200kb)

UPDATE: The project is now up on Codeplex

We have recently upgraded our installation of TFS from TFS2005 straight to TFS2008 SP1, this involved quite a number of steps, collating a lot of information from various sources and a lot of dry runs in a virtual test environment before we were happy to do an in-place upgrade on the production servers.  If you are researching a similar deployment then hopefully the following will upgrade process will be of some use to you.

iMeta's TFS deployment consists of an Application Tier server which is also hosting WSS and a separate SQL Server - I don't know how much of this process holds true for single server deployments.

Do remember to test this process out in a isolated environment before trying it with any live servers and also make sure you have a rollback plan in case something does fail!

Anyhow here goes:

1 Create Install Media

Create an ISO image with all the tools on in that we are going to need to perform the upgrade. The ISO should to contain the following:

• Visual Studio 2005 Power Tools (September 2007 Release)
  http://www.microsoft.com/downloads/details.aspx?familyid=7324c3db-658d-441b-8522-689c557d0a79&displaylang=en
• Visual Studio 2008 Power Tools (July 2008 Release)
  http://msdn.microsoft.com/en-us/tfs2008/bb980963.aspx
• Powershell 1.0
  http://support.microsoft.com/kb/926139
• Visual Studio 2008 Service Pack 1
  http://msdn.microsoft.com/en-us/vstudio/products/cc533447.aspx
• Visual Studio Team Foundation Server 2008 Service Pack 1
  http://www.microsoft.com/downloads/details.aspx?familyid=9e40a5b6-da41-43a2-a06d-3cee196bfe3d&displaylang=en
• Windows Sharepoint Services 3.0 with SP1
  http://www.microsoft.com/downloads/details.aspx?FamilyID=ef93e453-75f1-45df-8c6f-4565e8549c2a&displaylang=en
• Visual Studio 2008 Team System Web Access SP1 (August 2008 Release)
  http://www.microsoft.com/downloads/details.aspx?FamilyId=3ECD00BA-972B-4120-A8D5-3D38311893DE&displaylang=en
• The TFC Folder from the TFS2008 install media
• The WssExt Folder from the TFS2008 install media
• The AT folder from the TFS2008 install media

Before burning that onto a disk, slipstream the TFS2008SP1 install into the AT folder:

• To slipstream the install:
  • Extract the SP using
    • TFS90sp1-KB949786-ENU.exe /extract:[destination]
  • Install the service pack into the AT folder using:
    • msiexec /a [RTM_AT_FOLDER]\vs_setup.msi /p [SP1_FOLDER]\TFS90sp1-KB949786.msp TARGETDIR=[RTM_AT_FOLDER]
  • That will update the AT install with the SP1 bits

Get that lot onto an ISO and you’re ready to go. VMWare Workstation comes with a windows implementation of mkisofs which you can use:

The contents of the ISO looks like this:

2 Backup the TFS Databases

Take a backup of the TFS databases on the Data Tier, including the Analysis Services Database.

3 Run the best practices analyser

Install powershell, then install the Visual Studio 2005 Power Tools and run the Best Practices Analyser.

Ensure that the best practices are being met before we start the upgrade.  Don't attempt the upgrade without addressing any issues that the BPA uncovers.

4 Uninstall old versions

Uninstall the following from the TFS app tier

• Microsoft Visual Studio Team System Web Access 2005
• Microsoft Visual Studio 2005 Premier Partner Edition
• Microsoft Visual Studio 2005 Team Explorer
• Microsoft Visual Studio 2005 Tools for Office Runtime

And uninstall from the TFS data tier:

• Microsoft Visual Studio 2005 Team Foundation Server Data Tier (sp?)

5 Reboot the TFS Application Tier

It helps, trust me.

6 In-place Upgrade of TFS2005 to TFS2008 SP1

NB. The upgrade of the TFS databases uses a lot of disk space on the data tier server, expect the Version Control database’s log file to grow to the same size as the Version Control database, also expect TempDB to grow to a number of GB in size whilst the upgrade is running. Ensure the data tier server has sufficient free space to cope with this growth, ensure that there is enough space for the databases to at least double in size on the drive(s) holding the databases.

Login into to AT as the TFS setup account

Run the setup.exe in the AT folder.

Enter the product key and verify the destination folder:

Verify that it correctly detects the existing data tier server, in our case SQL01:

Be happy that you have a backup of the SQL Server, this is a good point to double check the backup succeeded.

Make sure the system passes the pre-reqs:

Now enter the passwords for the TFS Service and Reporting Accounts when prompted.

Then, enable TFS alerts and enter the address of your SMTP server and from email address:

Ready to go, click install and wait (database upgrade takes can take a number of hours) until....

The installer writes a log file into %TEMP% as it goes, and this gets updated with much more regularly than the progress bar.

Then reboot.

7 Upgrade WSS2.0 to WSS3.0

Procedure taken from:

http://blogs.msdn.com/sudhir/archive/2007/05/31/upgrade-2005-with-wss2-0-to-orcas-wss3-0.aspx

Nb. I have renamed the install executable from SharePoint.exe to SharePointWithSp1.exe, to distinguish from the RTM WSS release which is also called Sharepoint.exe

Extract the SharePoint install into the folder D:\sharepoint

Open d:\sharepoint\global\wss\sts.cab and extract prescan.exe into d:\sharepoint

(Taken from http://blogs.msdn.com/joelo/archive/2007/05/01/your-friend-prescan-what-it-does-part-2.aspx )

Run prescan.exe /all and review results, the log file is written into %TEMP%, the desired result is “Scan finished without failure”:

If you see errors check this article to try to resolve the issues:

http://blogs.technet.com/wbaer/archive/2006/12/22/prescan-errors-what-they-mean.aspx

Run d:\sharepoint\setup.exe, accept the EULA, and then choose the perform an automated in-place upgrade:

Server type, should be the default and only option, Web Front End.

Click “Install Now”, wait for the install to finish.

Leave “Run configuration wizard” checked and click close.

In a few moments the wizard will launch:

Acknowledge the warning that appears about needing to restart services.

Also take note of the next warning if you have Windows Sharepoint Language Template Packs in use.

Make a note of the port number it has chosen for us, and ensure that NTLM authentication is chosen:

Verify all the settings are correct and click install.

If there are multiple servers in the farm it’s time to upgrade the others now.

Wait for the upgrade to complete:

Wait for the Configuration Successful message:

The installer will then open a browser to show the status of the database upgrade that is running, it should say “Job in progress”.  You can see how far through the process it has got by inspecting the “Step within the action” and “total steps in this action” fields, expect the job to take around ten to fifteen minutes to complete:

Wait until it says, upgrade succeeded:

Next need to install the SharePoint Extensions for TFS. On the install media in the WssExt directory run the setup.exe. Note the WssExt installation is not touched by TFS 2008 SP1.

This is a simple setup, agree to the EULA, accept the default and you should be ready to install.  Click install, wait for the install to complete:

Now we need to tell TFS the new WSS Admin URL, as the upgrade will have changed it.

Run Sharepoint Central Administration ( Start -> Administrative Tools -> Sharepoint 3.0 Central Administration ). It will open a new browser window, make a note of the URL in the browser.

Open a command prompt and make the current directory c:\program files\Microsoft Visual Studio 2008 Team Foundation Server\Tools

Then run TfsAdminUtil configureconnections /SharepointAdminURI:[url from browser]

WSS3.0 is now configured.

8 Install TSWA 2008

Firstly you need to install Team Explorer from the TFS 2008 media. In the TFC folder run the setup.exe. Click install, and wait for it to complete.

Next install Visual Studio 2008 Service Pack 1 as it includes required updates to Microsoft Visual Studio 2008 Team Explorer:

Installer is \VS2008SP1\vs90sp1\SPInstaller.exe

Again, agree to the EULA and wait for the install to finish.  This one takes a long time but eventually you reach the other end.

Now reboot.

After the reboot we can, at last, run the Visual Studio Team System Web Access 2008 SP1 install, run the setup.exe in the TSWA folder.

Next, next through it accepting the default locations, and wait again:

When it asks, choose a port to put the Team System Web Access site on, the default is 8090 but we prefer 8000:

Use Integrated Windows Authentication when installing on the Application Tier, forms authentication otherwise.

( See the TSWA install guide http://msdn.microsoft.com/en-us/library/bb822178.aspx )

Accept the default cache location,then fill the details of the mail server - this is probably the same information as used earlier during the TFS installer.

Review and click install, then wait for it to complete.  Team System Web Access is now installed

Now run IISRESET as the TSWA installer leaves IIS in a bad state.

9 Update security settings

Team Foundation Server 2005’s default setting was to allow all users to see all the projects that were available on the server, this is no longer the default on TFS 2008 and we want to apply the new behaviour to the old projects.

Launch Microsoft Visual Studio 2008 and using Team Explorer add all the projects on the server.

For each project, click Team Project Settings -> Security, then remove the “Team System Valid Users” group from the project:

Do this for every project.

I wonder if there’s a command line tool for doing this?

The implication of doing this is that only members of a project will be able to see that the project exists on the server.

10 Delete any unneeded projects (optional)

This is also the ideal opportunity to delete any unneeded team projects from the server that may have been created either by accident or to experiment on. TFS 2005 cannot delete projects, but TFS 2008 now has the TFSDeleteProject tool.

Use TFSDeleteProject to delete any accidental or experimental team projects that may have been created.

11 Run TFS2008 Analyser

Powershell should already be installed, but if not install it now, then install the Team Foundation Power Tools for TFS2008 from the TPFP2008 folder.

Do a complete install, and wait for it to complete.

Now run the Team Foundation Server Best Practices Analyser (TfsBpa.exe):

Start a new scan, give it a name, and choose ‘Health Check’ for ‘Team Foundation Server Only’:

Then start the scan, and hopefully it produces a nice empty report.

The installation tasks are now complete, time to invite the users back.  I hope you get to bed earlier than we did!

Following on from Steve and Hadi's recent posts on the correct naming of tests in particular, and methods and variables in general, I've read this example a few times:

[TestMethod]
public void GetAllEmployeesByCompanyId_When_Employees_Are_Marked_Returns_Non_Marked_Employees()
{
   // implementation
}

The thing I've noticed is that I, personally, have difficulty parsing the name of that particular test so I'm still a bit unsure as to what is is really testing or why.  Don't get me wrong, the name is considerably better than calling it TestMethod1, and if I'm looking for all the tests related to the behaviour of the GetAllEmployeesByCompanyId method I can find them all pretty quickly - but I'd argue that I shouldn't have to parse any method names or read any code to find the reason and purpose behind a test, all I should have to do is read the comments:

        /// <summary>
        /// Validate that when <c>GetAllEmployeesByCompanyId</c> is called 
        /// only records not marked for deleted are returned.
        /// </summary>
        /// <remarks>
        ///    <para>Operation of <c>GetAllEmployeesByCompanyId</c> is described in section 4.3.2 of the <a href="#">Repository Detailed Design</a></para>
        ///    <para>This scenario is described in test case 7.6.5 in the <a href="#">Test Plan</a></para>
        /// </remarks>
        /// <example>
        ///    <code>
        ///       var employees = employeeRepository.GetEmployeesByCompanyId(Guid.Empty);
        ///    </code>
        /// </example>
        [TestMethod]
        public void GetAllEmployeesByCompanyId_When_Employees_Are_Marked_Returns_Non_Marked_Employees()
        {
            // implementation
        }

That way if I see a failing test in my Team Build reports, I can look up the test in the documentation that Sandcastle produced for me to see what it's testing (everyone has Sandcastle integrated into their team build scripts, yes? ;-) )

It also gives a place to include words that could help to explain why the test needs to exist, what is important about this behaviour or why this is an edge case - or link to the documents that include the information if you prefer.

That way people can get a feeling as to the reason behind the test and how it intends to run without having to open Visual Studio or read any code.

(I'll stop now as I've just realised I'm advocating not opening the compiler and reading code - I clearly have reached a certain age :( )

I've needed to create an ISO image as part of something that I was doing and needed to try and find a tool to do it for me, there are a number of commercial tools including our favourite PowerISO, but me being me I wanted a free one.  Also found ISO Recorder which is a neat little shell extension allowing you to right click a folder and create an ISO image of it, also burns ISOs to your CD/DVD-ROM burner - but whilst free for personal use Alex makes no mention of what commercial users are expected to do.

I could really do with a Windows port of mkisofs or genisoimage, there are a few pages claiming to have ports, and there is certainly a port contained within Cygwin, but it turns out that I already have it as VMWare Workstation also includes a port of it:

So if you need to create an ISO to feed to your VM, and you're using VMWare Workstation you already have the tools you need!

And if you care, here's my tried and trusted combination of switches for creating reliable images to be read in Windows:

mkisofs -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames -V "VOLUME-NAME" -o [output iso name] [input folder]

Controlling Power Management and preventing standby from C#

Following on from last time Going slightly green - Wake on LAN from ASP.NET I and a few other people have come across another problem which only occurs occasionally, but when it does is rather irritating.  For example, I have a need to calculate Pi to several thousand decimal places (I can't imagine why this would ever be the case) - it's going to take my Pi generator a few hours to work it out, so I kick it off when I go home confident in the knowledge that it'll be done by the morning.  In the morning I discover my PC has conscientiously  gone to sleep and my process is only twenty minutes closer to it's goal, which is a shame as I need my Pi.  I could have turned the power management off, but I know that I'll forget to enable it again, so that falls into my category of "Bad Things" to do.

I need something like Presentation Mode, however with a timeout.  As far as I can tell such as thing doesn't exist, so I've made one.

The power management functions in kernel32.dll contain a function called SetThreadExecutionState which allows an application to inform the system that it is in use, thereby preventing the system from entering sleep or turning off the display while the application is running.  I need a simple little application that will tell the system not to go to sleep, and then tell it it's OK again some period of time later.

First off I need wrap up the native call in a managed wrapper:

[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
static extern EXECUTION_STATE SetThreadExecutionState(EXECUTION_STATE flags);

[Flags]
public enum EXECUTION_STATE : uint
{
    ES_SYSTEM_REQUIRED = 0x00000001,
    ES_DISPLAY_REQUIRED = 0x00000002,
    ES_CONTINUOUS = 0x80000000
}

public static void PreventSleep()
{
    SetThreadExecutionState(EXECUTION_STATE.ES_SYSTEM_REQUIRED | EXECUTION_STATE.ES_CONTINUOUS);
}

public static void AllowSleep()
{
    SetThreadExecutionState(EXECUTION_STATE.ES_CONTINUOUS);
}

I also need a little form to allow me to tell the PC to stay awake, it need not be complicated - just a form and a notification icon to let me access it, and some timers to wait.

When the I click the OK button I can tell Windows to not go to sleep and also start a timer such that when it ticks enables power management again:

private void btnOK_Click(object sender, EventArgs e)
{
    int duration;

    // check that we have a a number entered by the user
    if (int.TryParse(comboBox1.Text, out duration) == false)
    {
        MessageBox.Show("Numbers only please", "Validation failed");
        return;
    }

    // keep the system awake for many hours
    Insomnia.PreventSleep();
  
    stayAwake.Interval = (int)new TimeSpan(duration, 0, 0).TotalMilliseconds;
    stayAwake.Enabled = true;

    this.Hide();
}

When the stayAwake timer goes off it has waited for the desired amount of time and we can turn power management on again:

private void timer1_Tick(object sender, EventArgs e)
{
    stayAwake.Enabled = false;
    Insomnia.AllowSleep();
}

The only other thing we need to do is to make the notification icon let me know what's going and what mode the PC is in at any point in time:

  followed later by

And as I'm likely forget when I have a job running and it's home time, I'll get the other timer to nag me just before I leave to go home:

I'm happy with that. If it looks useful to you too you can install it using clickonce, or download the source if you prefer.

Enjoy, it comes with my usual warranty.

The first release of Photsynth went public yesterday, I've been wanting to have a play with it since I first saw the Notre Dame demo tat was put together using it. 

Photosynth was massively overloaded yesterday on release day according to the blog, it's now working again on day 2 but I thought I'd give it a go before the US wakes up!

Generating a synth (new tool - new noun) is dead easy:

Click start a new synth...

Add a ton of photos, here's a load I took around the outside of iMeta this morning:

Wait whilst generating image tiles...

... and wait and wait, until you get to extracting image features, wait some more...

... matching images ...

... still going an hour later, laptop's getting quite hot now :-)  ...

... reconstructing scene ( 1 hour 20 minutes elapsed ) ...

Unfortunately I didn't see the rest of the process, but after three hours:

But the result is certainly worth it:

http://photosynth.net/view.aspx?cid=16BDBD49-8102-4EFE-B21D-5E4271E7EF58

(See if you can spot how many photos Dan got into!)

Team Explorer RTM has issues with VS2008 SP1, you get the error:

Could not load type 'Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemTypeDeniedOrNotExistException' from assembly 'Microsoft.TeamFoundation.WorkItemTracking.Client, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

whenever you go near work item tracking.

I got into this state by installing Team Explorer after installing VS2008 SP1.  The visual studio service pack also updates Team Explorer if it finds it installed,  however if you install Team Explorer after the service pack you'll have the RTM version of Team Explorer and the SP1 version of visual studio who don't play well together.

Running the VS2008 SP1 install for a second time will patch Team Explorer up to the required level.

I'm often torn between the desire to switch my PC off when I'm not using it as an attempt to perform some token gesture towards being green (every little helps) and the desire to leave it on so that I can access it remotely from where ever I happen to be.

Wouldn't it be handy if you could allow your PC to sleep automatically but somehow have the ability to wake it up without having to push the power button in the rare event that I would like to use it when I'm nowhere near it?  Turns out there is - however it isn't new, it's just only recently that I've been bothered/driven to investigate this (delete as appropriate) and have been pleasantly surprised as to how easy it is to make it work and how reliably it seems to work.

Most PCs with modern motherboards support Wake On LAN, which is why when your PC goes to standby or hibernates there is still a link light on the network card - it is still awake and waiting for packet to arrive which would require it to signal the PC to wake.  The packet it's looking out for is known as the magic packet and is specially laid to make it easy for the little brain on board the network card to spot it while the CPU and rest of the PC is turned off.

The Magic Packet is 102 bytes long made up of a 6 byte header of all 0xFF followed by the target computer's MAC address sixteen times (6 bytes each) and is sent out using UDP.

We can build and send one of these from C# reasonably easily using the UdpClient class:

/// <summary>
    /// Send a magic packet
    /// </summary>
    /// <param name="macAddress"></param>
    public static void Wake(byte[] macAddress)
    {
        // A Wake on LAN magic packet contains a 6 byte header and
        // the MAC address of the target MAC address (6 bytes) 16 times
        byte[] wolPacket = new byte[17 * 6];

        MemoryStream ms = new MemoryStream(wolPacket, true);

        // Write the 6 byte 0xFF header
        for (int i = 0; i < 6; i++)
        {
            ms.WriteByte(0xFF);
        }

        // Write the MAC Address 16 times
        for (int i = 0; i < 16; i++)
        {
            ms.Write(macAddress, 0, macAddress.Length);
        }

        // Broadcast the magic packet
        UdpClient udp = new UdpClient();
        udp.Connect(IPAddress.Broadcast, 0);
        udp.Send(wolPacket, wolPacket.Length);
    }

Note, by using a UDP Broadcast the sender and recipient must be on the same subnet, otherwise you'll have to teach your router what's going on.

This then leads to the question of how to obtain the MAC address of the remote computer.  If you are sat in front of it you can run ipconfig /all at a command line to get the address:

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : imeta.co.uk
   Description . . . . . . . . . . . : Broadcom NetXtreme 57xx Gigabit Controller
   Physical Address. . . . . . . . . : 00:1C:23:27:CE:C0
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : ... etc

Now we have a string with the MAC address we can wake that machine up from someplace else:

public static void Wake(string macAddress)
{
    Wake ( macAddress.Split(':').Select(c => byte.Parse(c, NumberStyles.HexNumber)).ToArray() );
}

However, if I don't want to have to run a command on the local computer to wake it up remotely I can ask it over the network what its MAC address is using a protocol called ARP.  ARP is used to find out which MAC address currently owns a particular IP address.

So given the I know the name of the remote PC, I'd like to be able to discover the MAC address and wake it up.  Unfortunately there is no managed class for generating and sending ARP requests, we need to make use of a native DLL:

internal static class NativeMethods
{
    [DllImport("iphlpapi.dll", ExactSpelling = true)]
    internal static extern int SendARP(int DestIP, int SrcIP, byte[] pMacAddr, ref uint PhyAddrLen);
}

/// <summary>
/// Summary description for ARP
/// </summary>
public static class Arp
{
    public static byte[] GetMACAddress(string hostNameOrAddress)
    {
        IPHostEntry hostEntry = Dns.GetHostEntry(hostNameOrAddress);

        byte[] macAddr = new byte[6];
        uint macAddrLen = (uint)macAddr.Length;

        // Find the first IPV4 address for that host
        IPAddress ipAddress = hostEntry.AddressList.First<IPAddress>(ip => ip.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork);

        byte[] addressBytes = ipAddress.GetAddressBytes();
        int address = BitConverter.ToInt32(addressBytes, 0);

        if (NativeMethods.SendARP(BitConverter.ToInt32(addressBytes, 0), 0, macAddr, ref macAddrLen) != 0)
        {
            return null;
        }

        return addressBytes;
    }
}

Great, so now given the name of a PC we can use DNS to lookup it's IP address, send an ARP request to obtain its MAC address and then wake up the remote computer using a Magic Packet?  Sorted?  No... ARP tells you which MAC address currently owns an IP address, when a computer is in standby it doesn't have an IP address - so it doesn't need to respond to ARP requests.

While understandable, this is a real shame as it means if a remote computer is asleep there is no means of obtaining it's MAC address; you can only find a computers MAC address if the computer is on!

In order to produce a useful system to wake up PCs when they're asleep the system needs to know ahead of time which machines I'm likely to want to wake.  By creating a little ASP.NET application we can register the machines that are likely to sleep while they're awake and store the MAC address so that we have it handy when it's asleep.  By placing this application on a central server everyone can register their computers on it and there is one central place to visit to turn them back on again:

Clicking the add computer button will use the hostname string that I've entered, obtain the MAC address for it and squirrel it away into a database.  If the remote machine isn't on at the moment it can't be added to the collection until it is.  However once my PC has been added we can then view the machines I've added and wake any machines of mine that are asleep:

By clicking on the "Wake up" link, the asp.net page can use the MAC address stored in the database to create a magic packet and wake the PC up.  We can also do some little touches like pinging the machine as the page loads and tailoring the icons and links available; there's no point waking a machine that's awake!

When my computer is awake it shows up as online:

So now are we finished?  Almost!  The web application now has everything it needs to make it work, however if you try it the remote PC probably won't wake up, as Wake-On-LAN is not enabled by default! You need to enable it in your BIOS and in Windows. 

In the BIOS it will probably be under power management or similar, like this example:

... and then the driver for the network card needs to be taught what's going on.  Locate your network card under Device Manager, and click properties and Power Management and then check "Allow this device to wake the computer"

And finally, it seem to help if you tell the network card to expect a Magic Packet as the wake-up signal rather than a ping or any other type of packet:

And now we're done!  Your PC should now wake when instructed to by a magic packet!

So that was a lot of heart-ache, why bother?  A quick burst of maths demonstrates the scale of the cost savings that are possible through a bit of green IT:  Working on the assumption that an idle desktop computer pulls 150w at idle, it means that a powered on PC will burn 1 KWh every 6 hours 40 minutes.  For a PC on 24/7 that's 25.2 KWh per week or 1,310 KWh per year.  Assuming a rough price of 10p per KWh a single PC costs £131 per year to run 24/7.  Now how many PC's do you have around the place?  Two? Five?  Maybe a hundred (£13,100 per year) - a thousand maybe?  

By making the PC's switch to standby the figures change, let's say each one is on  9AM till 6PM, 5 days a week, 48 weeks of the year (people go on hols!).  The PC is on for 9 hours a day, 2,160 hours per year which consumes 324KWh, which costs £32.40.   A saving of just shy of a hundred pounds per PC per year - assuming the cost of electricity remains constant, and that's probably the worst assumption I've made so far!  As the cost of energy goes up, the savings will increase proportionally too. 

It's probably worth factoring in the amount of time that the PC is on standby into the equation, as they are still pulling some current:  if the PC is on for 2,160 hours per year, it is off or on standby for 4,200.  Assuming a draw of 5w in that state it will consume 21KWh over the year - costing £2.  I think the case for the savings still holds!

At iMeta we have this live now on our intranet site (http://intranet/wake for iMeta people) but it's probably something that anyone who leaves a desktop PC on 24/7 should consider setting up.

Credits: Thanks go to Bart and Lukas that got me down the road to this solution!

26/11/08 Update:  This solution and the source are available to download

Got the latest version of the Sysinternals tools on your machine?  Are you sure?  You can now access the current version of the tools via a UNC share!

That's useful.  If you're in PowerShell even more so!

I bumped against this just the other day and unfortunately had to go down the virtualisation route.  The summary is Visual Studio needs Sharepoint Server installed in order to create any Sharepoint Projects, and Sharepoint will only install on a server operating system.

These chaps seem to have come up with a workaround: http://community.bamboosolutions.com/blogs/bambooteamblog/archive/2008/05/21/how-to-install-windows-sharepoint-services-3-0-sp1-on-vista-x64-x86.aspx

This kept me amused for a few minutes:

http://www.roesler-ac.de/wolfram/hello.htm

Thanks to Eric for that one!

If you're running Vista or Server 2008 in a virtual machine, you're unlikely to ever want to hibernate, however more virtual disk space might be useful.

powercfg -h off

Disables hibernation, removes hiberfil.sys, and gives you back a chunk of disk.

Enjoy.

Obvious really, however it just had me foxed for a while doing exactly this:

http://blogs.msdn.com/carloc/archive/2007/11/24/the-error-indicates-that-iis-is-not-installed-on-the-machine-please-install-iis-before-using-this-tool.aspx

When you're on 64bit version of Windows all the goodness is in %windir%\Microsoft.Net\Framework64, Framework contains the 32bit version for apps running in WoW.

Hope it helps, Neil.

Then do both!

#if !NUNIT
using Microsoft.VisualStudio.TestTools.UnitTesting;
using TextFixture = Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute;
using SetUpAttribute = Microsoft.VisualStudio.TestTools.UnitTesting.TestInitializeAttribute;
using TearDownAttribute = Microsoft.VisualStudio.TestTools.UnitTesting.TestCleanupAttribute;
using TestAttribute = Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute;
#else
using NUnit.Framework;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
#endif

Superfetch is a funky new service in Vista that tries to pre-cache all the files that you use on a regular basis to make your apps run faster and snappier.  While I'm sure this is a good idea in general, if you make heavy use of VMs on your machine you may have noticed a rather heavy load on the HDD.

On my laptop I've narrowed this down to Superfetch trying to pre-cache all of my .vhd and .vmdk files into memory as they're the files that I access the most.  Suffice to say a) they don't fit so it's a waste of time, and b) it does this at the expense of loading other apps. I guess it's logic is that I load Outlook once a week, but I nail the .vmdk files all day every day so that's what it spends the time loading.

Disabling Superfetch has transformed the system, and now the hard disk light goes out sometimes.  Try it out if you're seeing similar issues.

As always, YMMV,

Neil.

They almost play well together.

*If* you need to install all these at the same time you'll find the install for Team Explorer blows up with a rather cryptic error - ask Dan how cryptic it was...

To make it install, fire up the Orcas that you've just installed, create a new Windows Service project, build and install the service ensuring it's service name is "WebClient".  Then when you run the Team Explorer install it'll work just fine.

Enjoy!

The characters who visited us doing the Vista launch all have blogs which are worth watching:

Mike Taulty : http://mtaulty.com
Martin Parry : http://martinparry.com
Daniel Moth: http://danielmoth.com

and Mark Johnston, you can guess the URL.

No?  It's http://blogs.msdn.com/markjo/   !!

One of the day to day things that I miss from XP in Vista is the "move up a level" button in explorer windows, in Vista you have to mess with the address bar to navigate up the folder structure.  Backspace goes back and not up... grrr

Anyway:   ALT + UP ARROW   - Up a level.

Hurray.  Small victories.