Friday, 27 June 2008

MSBuild Tasks - DLL or EXE

Quite often I find that I need the reuse the functionality that is in a custom msbuild task within small tool elsewhere. Usually the way round that issue is to refactor the nuts an bolts of the task into another dll, and then referencing this where needed. This is still the best option in my opinion. However this morning we discovered that you put your tasks assembly in exe wrappers also. This allows you to have a static main etc etc to allow to you call the nuts and bolts of the task from, say, the command line. Thanks to colleague Martin for verifying my ad hoc musings on this subject.

Wednesday, 25 June 2008

MSTest fails with UTA058: Trust problem

I kept getting this error for all my unit test dlls even though I was absolutely  certain that their locations were 100% fully trusted on my local cdrive. I even checked using the .net config tool in control panel. Finally I had a go at specifying the full paths to the dlls and 'viola'  it worked! You'll see that in the previous posting we use /testcontainer:%%~ff . The necessary bit is ~f. To find out more about this you can type 'for /?' in the a command window.

In addition, substitution of FOR variable references has been enhanced.
You can now use the following optional syntax:

    %~I         - expands %I removing any surrounding quotes (")
    %~fI        - expands %I to a fully qualified path name
    %~dI        - expands %I to a drive letter only
    %~pI        - expands %I to a path only
    %~nI        - expands %I to a file name only
    %~xI        - expands %I to a file extension only
    %~sI        - expanded path contains short names only
    %~aI        - expands %I to file attributes of file
    %~tI        - expands %I to date/time of file
    %~zI        - expands %I to size of file
    %~$PATH:I   - searches the directories listed in the PATH
                   environment variable and expands %I to the
                   fully qualified name of the first one found.
                   If the environment variable name is not
                   defined or the file is not found by the
                   search, then this modifier expands to the
                   empty string

The modifiers can be combined to get compound results:

    %~dpI       - expands %I to a drive letter and path only
    %~nxI       - expands %I to a file name and extension only
    %~fsI       - expands %I to a full path name with short names only
    %~dp$PATH:I - searches the directories listed in the PATH
                   environment variable for %I and expands to the
                   drive letter and path of the first one found.
    %~ftzaI     - expands %I to a DIR like output line

MSTest.exe on the command line

Heres a little batch file to run your tests locally (neatly) from test containers (courtesy of Paul)

rmdir TestResults /S /Q
for %%f in ("..\bin\debug\*UnitTest*.dll") do del "%%~nf".trx & mstest /testcontainer:%%~ff /noisolation /resultsfile:"%%~nf".trx
for /f "Tokens=*" %%f in ('dir *.trx /b') do @nxslt2.exe "%%f" trxsummary.xsl


Works for me!!!

Tuesday, 24 June 2008

Debugger.IsAttached

Have just discovered System.Diagnostics.Debugger.IsAttached after years of using using #if. This way I can check if my unit tests are being debugged and thus alter timeouts as needed! As they say down here - Sweet As!

Sunday, 22 June 2008

Visual Studio 2005 to 2008 Migration - Collating the upgrade logs

I've been involved in the migration of a fairly large code base from VS2005 to VS2008 this past few weeks. Well kudos to MS for making this a simple process, however one little thing really got on my nerves. I wanted to run the devenv /upgrade xxxxx.sln across the entire code base and not a solution at a time. Next I wanted all the upgrade reports to be amalgamated into one report that I could search easily for errors/issues/warnings etc.

The first bit was solved by my colleague Larry in Tallahassee building a batch file that ran through all the solution files in a DOS for loop and running  down across the whole code base looking for solution files and then running the above command over them.

for /f "tokens=*" %s in ('dir *.sln /s/b') do devenv /upgrade "%s"



Next I wanted to collect all the upgrade xml reports (all called upgradelog.xml), but I didn't want to generate a different format to the one that MS creates. The ms report is actually just a xsl transformation that is run on the upgradelog.xml. It takes the contents of log file (an xml file with the root node <Log />, and generates some nice html with some expand/collapse JavaScript.



So I figured my best bet would to extend this template making is work over a series of logs rather than just one. This left the next issue, how to amalgamate all the XMLs. My first choice was hindered by the fact that XSLT to me a write only language. Try as I might I cannot seem to get the syntax to stick :-)



Anyway my next option was to create use python to create a simple XML file that listed all the logs. This was trivial but again fun as I don't know much python either.




dir UpgradeLog.XML /s/b | \Python25\python.exe generatemasterlog.py > upgrade.xml.list


The python is trivial



import sys
print '<?xml version="1.0" encoding="utf-8"?>'
print '<Logs>'
while True:
logfile = sys.stdin.readline().strip()
if not logfile: break
print '\t<UpgradeLog report="%s" />' % logfile
print '</Logs>'




this upgrade.xml.list has the form




<?xml version="1.0" encoding="utf-8"?>
<Logs>
<UpgradeLog report="c:\development\SomeSln\UpgradeLog.XML" />
.
.
.
<UpgradeLog report="c:\development\AnOtherSln\UpgradeLog.XML" />
</Logs>



The using my old favourite nxslt2.exe I transformed this into a master upgrade log xml file.




<?xml version="1.0" encoding="utf-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl='urn:schemas-microsoft-com:xslt'>
<xsl:output method='xml' indent='yes'/>
<xsl:template match='/*'>
<xsl:processing-instruction name="xml-stylesheet">
<xsl:text>href="UpgradeReport.xslt" type="text/xsl"</xsl:text>
</xsl:processing-instruction>

<xsl:element name="Logs">
<xsl:for-each select='/Logs/UpgradeLog'>
<xsl:copy>
<xsl:copy-of select='document(@report)//UpgradeLog/*'/>
</xsl:copy>
</xsl:for-each>
</xsl:element>
</xsl:template>
</xsl:stylesheet>



What this does is run through all the report="" attributes in the above upgrade.xml.list and pull the out the contents of that xmlfile and output it in with in xml file within the root node of <Logs />. Hence I now have a master XML file with all the logs in the form




<Logs>
<UpgradeLog> ....
</UpgradeLog>
<UpgradeLog> ....
</UpgradeLog>
</Logs>



I also output a processing statement (href="UpgradeReport.xslt" type="text/xsl") that allows a browser automatically transform this xml into the nice html view.



The final step was to update the Microsoft XSLT UpgradeReport.xslt to support the new amalgamted xml. Thats too big to put here, but it basically change the master match from Log to Logs (<xsl:template match="Logs">) and then change the existing <body> to apply template (<xsl:apply-templates select="/Logs/UpgradeLog"/>). This meant that is applies to all the <UpgradeLog /> elements rather than just one.



I'll email it to anyone who wants it (preetsangha gmail etc), but remember is mostly Microsoft's work.

Thursday, 19 June 2008

Getting round the Visual Studio 2008 MSTest Appbase issue/bug

The Problem

There is a known bug in visual studio 2008 (and here)  Well actually its in how MSTest runs the unit tests. The problem as I see it is that the AppDomain that your unit tests run in is not the same one that is initially created in the VSTestHost.exe. Its this initially created domain cannot find your types/assemblies. This is caused by VSTestHost.exe having an APPBASE of "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE (or the equivalent on 32 bits "C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE).

Debugging through with the .net framework reference source, led me to the following explanation. When the tests run there is an AppDomain transition from your testing AppDomain. This can be caused by may things, I my case it was a System.Configuration call that was resulting a GetEvidence() call deep in framework. Once this transition had occurred, the assembly resolver delegate chain only had one resolver delegate within 

Microsoft.VisualStudio.TestTools.TestTypes.Unit.AssemblyResolver.

This is the one that cannot find the unit test dlls. Our resolver on the other hand was in our AppDomain and was therefore never called.

My simple workaround

I couldn't find a complete workaround, but I have what I think is one that is fairly unobtrusive. However this one relies on a known locatation for the Unit test dlls to reside (which is only really true when you done a complete build).

What you do is create a Vista symbolic link to the required appbase inside the the problematic appbase, and then tell VSTestHost.exe to probe this path as well. This fools the resolver into following this link to your required appbase.

The simplest way to do this use the mklink, on the command line, and then edit the VSTestHost.exe.config in the above directory to include this symlink. For example if my unit test dll was in directory c:\UnitTests. I'd use (mind the wrap)

mklink /D mklink /d "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MyUnitTestsDir" C:\UnitTests


and I'd add edit the C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\VSTestHost.exe.config and replace




<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="PrivateAssemblies;PublicAssemblies"/>
</assemblyBinding>
</runtime>
</configuration>



with



<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="PrivateAssemblies;PublicAssemblies;MyUnitTestsDir"/>
</assemblyBinding>
</runtime>
</configuration>


You can see in both cases the symbolic link is called MyUnitTestsDir.



My better workaround



In the case of running the tests from the command line, or via the IDE I picked a similar method, but actually create and delete the symlink in the [AssemblyInitialize]/[AssemblyCleanup] methods. I created a small helper class (thanks to Creating symlinks in C# by Bart De Smet), that can be used to create and delete the symbolic links.




public class SymLink : IDisposable{

[System.Runtime.InteropServices.DllImport("kernel32.dll")]
static extern bool CreateSymbolicLink(string lpSymlinkFileName, string lpTargetFileName, int dwFlags);


private int SYMLINK_FLAG_DIRECTORY = 1;
private bool linkCreated;

public bool LinkCreated {
get { return linkCreated; }
}

string symbolicLinkPath;

/// <summary>
/// Create a symbolic link of the given name to in a given directory to an actual location
/// </summary>
/// <param name="linkName">name of the link. </param>
/// <param name="root">this is the directory within which the symlink should be created</param>
/// <param name="actual">this is the actual location where the symlink points</param>
public SymLink(string linkName, string root, string actual) {
symbolicLinkPath = System.IO.Path.Combine(root, linkName);
linkCreated = CreateSymbolicLink(symbolicLinkPath, actual, SYMLINK_FLAG_DIRECTORY);
}

/// <summary>
/// Create a symbolic link of the given name to in a given directory to an actual location
/// </summary>
/// <param name="linkName">name of the link. </param>
/// <param name="root">this is the directory within which the symlink should be created</param>
/// <param name="actual">this is the actual location where the symlink points</param>
/// <param name="force">Where the link should be deleted before use</param>
public SymLink(string linkName, string root, string actual, bool force) {
symbolicLinkPath = System.IO.Path.Combine(root, linkName);
if (force) {
Dispose();
}
linkCreated = CreateSymbolicLink(symbolicLinkPath, actual, SYMLINK_FLAG_DIRECTORY);
int getLastError = System.Runtime.InteropServices.Marshal.GetLastWin32Error();
}


#region IDisposable Members

/// <summary>
/// Dispose
/// </summary>
public void Dispose() {
if (System.IO.Directory.Exists(symbolicLinkPath)) {
System.IO.Directory.Delete(symbolicLinkPath);
}
}

#endregion
}



I  then added the appropriate calls with in the [AssemblyInitialize]/[AssemblyCleanup] methods.




private static SymLink link;

[AssemblyInitialize()]
public static void AssemblyInitialize(TestContext testContext) {

// I suggest that symLinkName should be a guid or uniqueness without } as this messes up string.format unless you think about it.
string symLinkName = "SymLink-CF06174B-AA2A-4890-9209-ECCD0F9D172E";
string oldAppBase = (string)AppDomain.CurrentDomain.GetData("APPBASE");
string requiredAppBase = (new System.IO.FileInfo(new Uri(System.Reflection.Assembly.GetExecutingAssembly().CodeBase).LocalPath)).DirectoryName;
link = new SymLink(symLinkName, oldAppBase, requiredAppBase, true);

Assert.IsTrue(link.LinkCreated, "MS Test directory symlnk creation failed");

}

[AssemblyCleanup()]
public static void AssemblyCleanup() {
link.Dispose();
}



You still have to add a probe path entry to 'SymLink-CF06174B-AA2A-4890-9209-ECCD0F9D172E' like in the first workaround, but atleast this version will take into account the funny deployment directories




...\TestResults\SomeDymanicName[xxx]\Out




And there you go.



Some thoughts



This is only a hack, as it requires you to edit your C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\Vstesthost.exe.config file with known identifiers. But got me working until we see the real solution from MS.










 


Tuesday, 17 June 2008

Decoding the VS2008 Trx file

I've just updated my script to support 2008 and the of course if you look at the internals you're going to have to upgrade on new release! Here's the new xslt. I've updated it to include the errors too (mind the wrap):

<xsl:stylesheet version="2.0"  
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:t="http://microsoft.com/schemas/VisualStudio/TeamTest/2006"
>
<xsl:output method="text" />
<xsl:template match="/">
<xsl:text>&#xA;&#x9;</xsl:text><xsl:value-of select="substring(t:TestRun/t:TestRunConfiguration/t:NamingScheme/@baseName, 1, 20)"/>: <xsl:value-of select="t:TestRun/t:TestRunConfiguration/t:Deployment/@runDeploymentRoot"/>.trx
<xsl:choose>
<xsl:when test="(t:TestRun/t:ResultSummary/@outcome) = 'Failed'">
Failed <xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@failed"/> (Executed [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@executed"/>], Found [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@total"/>], Passed [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@passed"/>])
<xsl:for-each select="/t:TestRun/t:Results/t:UnitTestResult" >
<xsl:if test="@outcome = 'Failed'" >
<xsl:text>&#xA;&#x9;&#x9;</xsl:text><xsl:value-of select="@testName"/>
<xsl:text>&#xA;&#x9;&#x9;&#x9;</xsl:text>
<xsl:value-of select="substring(t:Output/t:ErrorInfo/t:Message, 1, 170)"/>
<xsl:text>&#xA;&#x9;&#x9;&#x9;</xsl:text>
<xsl:value-of select="substring(t:Output/t:ErrorInfo/t:Message, 171, 340)"/>
<xsl:text>&#xA;</xsl:text>
</xsl:if>
</xsl:for-each>
</xsl:when>
<xsl:otherwise>
Passed (Executed [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@executed"/>], Found [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@total"/>], Passed [<xsl:value-of select="t:TestRun/t:ResultSummary/t:Counters/@passed"/>])
</xsl:otherwise>
</xsl:choose>

</xsl:template>
</xsl:stylesheet>