Querying TFS using Roslyn

Roslyn is an ambitious new project by Microsoft. It is the fully written in .Net replacement of the old compilers used in Visual Studio and should be shipped in a future version of the IDE. The current compilers csc.exe and vbc.exe were first shipped in version 1.0 of .Net more than 10 years ago.

Roslyn offers a lot of new possibilities. As of today Roslyn in still is in CTP and the latest release was in September 2012. For this first post on Roslyn I want to explore the new C# Interactive window (you will need to download and install the CTP for that).

For this demonstration I'll use one of my previous query for LinqPad, Searching in all work items.

First we need to open the C# Interactive window (from menu VIEW -> Other Windows -> C# Interactive)



Then we start by referencing the required libraries using the special #r instruction

> #r "Microsoft.TeamFoundation.Client"
> #r "Microsoft.TeamFoundation.Common"
> #r "Microsoft.TeamFoundation.VersionControl.Client"
> #r "Microsoft.TeamFoundation.WorkItemTracking.Client"

Next we need to import the namespaces we are going to use with the using statement like this

using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.WorkItemTracking.Client;

Now we are ready to write some real code. I'll be using my online Team Foundation Service account for this. You can register for a free account here.

> var uri = new Uri("https://my.visualstudio.com/DefaultCollection/");

> var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(uri);

> tfs.EnsureAuthenticated();

> var workItemStore = tfs.GetService<WorkItemStore>();

> var title = "phone";

> var query = string.Format(@"
    Select [Id], [Work Item Type], [Title], [State]
    Where [Title] Contains '{0}'
    From WorkItems",
    title);

> var results = workItemStore.Query(query).Cast<WorkItem>()
    .Select(wi => new
    {
        wi.Id,
        wi.AreaPath,
        Type = wi.Type.Name,
        wi.Title,
        wi.State
    })
    .OrderBy(wi => wi.AreaPath).ThenBy(wi => wi.Type).ThenBy(wi => wi.Title);

Of course we don't have access to LinqPad's Dump function so we have to print the results ourself.

> foreach (var r in results)
{
    Console.WriteLine("{0} - {1} [{2}] {3} <{4}>", 
        r.Id, r.AreaPath, r.Type, r.Title, r.State);
}
2 - Project1 [Bug] Bug #1 - Fix the phone number on the Contact page <New>

I think using LinqPad is still easier for this kind of work but those who don't have a paid version of LinqPad will like the facts that Roslyn is free, offers a way to write queries using Intellisense and also a more interactive way to work and find the data you are looking for.

How to reference and reuse LinqPad queries

One missing feature from LinqPad is the ability to easily reuse code in a query we wrote in other queries.

If we really want to do it we have the following options:

• Copy and paste the code into the new query
• Open Visual Studio, create a project, create a class and copy the code you want to reuse. Compile the project and finally reference the generated dll from your query.
• Compile the LinqPad query directly (using a special script), then reference the generated dll from your query.

Today we will look at the third option. Found in the .Net framework is a very interesting API called the CodeDOM API. This API can be used to compile .Net code at run-time, like the compiler does. With this, we will be able to parse a LinqPad query and compile it.

This is what we will look into right now.

Compiling a LinqPad query

Those are the steps we need to do to accomplish query compilation:

1. Read the query and extract the options, usings and references
2. Create the CodeDOM objects and set the options and references
3. Create a string of the code file including the usings and wrap the query code inside a class
4. Compile the code

I'll explain in details each steps in future blog posts but for now I'll give you my query to compile LinqPad queries. You will notice that this is a self compiling query as the query compile itself to a dll I can reuse it to compile other queries!




You can figure out the Compiler class usage from the Main function for now.

Additional Namespace Imports

System.CodeDom.Compiler
System.Runtime.InteropServices
Microsoft.CSharp

Query (C# Program)

	void Main()
	{
	Environment.CurrentDirectory = Path.GetDirectoryName(Util.CurrentQueryPath);
	Compiler.CompileFiles(Options.CreateOnDiskDll(
	@namespace: "LinqPad.QueriesCompiler",
	outputFile: "LinqPad.QueriesCompiler.dll")
	.AddCodeFile(CodeType.LinqProgramTypes, Util.CurrentQueryPath))
	.Dump("Successfully created assembly at " + DateTime.Now.ToLocalTime());
	}
	// Define other methods and classes here
	public static class Compiler
	{
	public static Assembly CompileFiles(Options options)
	{
	var outputPath = options.IsInMemory ? "" : options.OutputFile;
	if (!options.CodeFiles.Any())
	throw new InvalidOperationException("Should add at least one file to compile.");
	foreach (var codeFile in options.CodeFiles)
	{
	codeFile.RawContent = File.ReadAllLines(codeFile.FilePath);
	codeFile.Query = GetQuery(new FileInfo(codeFile.FilePath).DirectoryName, codeFile.RawContent);
	GetCode(codeFile, options.Namespace);
	}
	return BuildAssembly(options.CodeFiles, options, outputPath);
	}
	public static Query GetQuery(string folder, IEnumerable<string> content)
	{
	var xml = string.Join("\r\n", content.TakeWhile(l => l.Trim().StartsWith("<")));
	var queryElement = XDocument.Parse(xml).Element("Query");
	if (queryElement == null) throw new InvalidOperationException("Missing <Query> header definition");
	var query = new Query
	{
	Kind = queryElement.Attribute("Kind").Value,
	Namespaces = queryElement.Elements("Namespace").Select(n => n.Value).ToList(),
	GACReferences = queryElement.Elements("GACReference").Select(n => n.Value).ToList(),
	RelativeReferences = queryElement.Elements("Reference").Where(e => e.Attribute("Relative") != null)
	.Select(n => n.Attribute("Relative").Value)
	.Select(x => new FileInfo(Path.Combine(folder, x)).FullName)
	.ToList(),
	OtherReferences = queryElement.Elements("Reference").Where(e => e.Attribute("Relative") == null)
	.Select(n => n.Value.Replace("<RuntimeDirectory>", RuntimeEnvironment.GetRuntimeDirectory()))
	.Select(n => n.Replace("<ProgramFilesX86>", Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86)))
	.ToList()
	};
	return query;
	}
	public static IEnumerable<string> GetCode(IEnumerable<IEnumerable<string>> contents, Query query, string ns)
	{
	return contents.Select(c => GetCode(c, query, ns));
	}
	private static void GetCode(CodeFile codeFile, string @namespace)
	{
	IEnumerable<string> result = null;
	if (codeFile.Query.Kind != "Program" && codeFile.Query.Kind != "Statements")
	throw new InvalidOperationException("Only queries of type C# program and C# statements are supported");
	var filteredContent = codeFile.RawContent.SkipWhile(l => l.Trim().StartsWith("<"));
	switch (codeFile.Type)
	{
	case CodeType.LinqStatements:
	result = WrapInClass(WrapInMain(filteredContent));
	break;
	case CodeType.LinqProgramTypes:
	result = FilterTypes(filteredContent);
	break;
	case CodeType.LinqProgram:
	result = WrapInClass(filteredContent);
	break;
	default:
	throw new InvalidOperationException("Only queries of type C# program and C# statements are supported");
	}
	codeFile.Content = GetCode(result, codeFile.Query, @namespace);
	}
	private static IEnumerable<string> WrapInClass(IEnumerable<string> inputCode)
	{
	var s = new[] { "public class Program {" };
	var e = new[] { "}" };
	return s.Concat(inputCode).Concat(e);
	}
	private static IEnumerable<string> WrapInMain(IEnumerable<string> inputCode)
	{
	var s = new[] { "public static void Main() {" };
	var e = new[] { "}" };
	return s.Concat(inputCode).Concat(e);
	}
	private static IEnumerable<string> FilterTypes(IEnumerable<string> inputCode)
	{
	return inputCode.SkipWhile(l => l.Trim() != "// Define other methods and classes here");
	}
	public static string GetCode(IEnumerable<string> content, Query query, string ns)
	{
	var code = string.Join(Environment.NewLine, content);
	var codeBuilder = new StringBuilder();
	codeBuilder.AppendLine("using " + string.Join(";\r\nusing ", query.Namespaces.Union(StandardNamespaces)) + ";");
	codeBuilder.AppendLine(string.Format("namespace {0} {{", ns));
	codeBuilder.AppendLine(code);
	codeBuilder.AppendLine("}");
	return codeBuilder.ToString();
	}
	public static Assembly BuildAssembly(IEnumerable<CodeFile> codeFiles, Options options, string outputPath)
	{
	var providerOptions = new Dictionary<string, string> { { "CompilerVersion", "v4.0" } };
	var provider = new CSharpCodeProvider(providerOptions);
	var assemblies = new[]
	{
	codeFiles.SelectMany(c => c.Query.GACReferences.Select(s => Assembly.Load(s).Location)),
	codeFiles.SelectMany(c => c.Query.RelativeReferences.Select(s => Assembly.LoadFrom(s).Location)),
	codeFiles.SelectMany(c => c.Query.OtherReferences),
	Assemblies
	};
	var compilerparams = new CompilerParameters
	{
	GenerateExecutable = !string.IsNullOrWhiteSpace(options.StartupObject),
	OutputAssembly = options.IsInMemory ? null : outputPath,
	GenerateInMemory = true,
	IncludeDebugInformation = true,
	MainClass = options.StartupObject
	};
	compilerparams.ReferencedAssemblies.AddRange(assemblies.SelectMany(a => a).ToArray());
	var results = provider.CompileAssemblyFromSource(compilerparams, codeFiles.Select(f => f.Content).ToArray());
	if (results.Errors.HasErrors)
	{
	var errors = new StringBuilder("Compiler Errors:\r\n");
	foreach (CompilerError error in results.Errors)
	{
	errors.AppendFormat("File {0}, Line {1},{2}\t: {3}\r\n",
	error.FileName, error.Line, error.Column, error.ErrorText);
	}
	throw new Exception("Errors compiling:\r\n" + errors + "\r\n\r\n" +
	string.Join(Environment.NewLine, codeFiles.Select(f => f.FilePath + ":\r\n" + AddLineNumber(f.Content))));
	}
	return results.CompiledAssembly;
	}
	private static string AddLineNumber(string code)
	{
	var lines = code.Split(new[] { Environment.NewLine }, StringSplitOptions.None);
	return string.Join(Environment.NewLine, lines.Select ((x, i) => (i + 1).ToString().PadLeft(4) + ": " + x));
	}
	private static List<string> StandardNamespaces
	{
	get
	{
	return new List<string>
	{
	"System",
	"System.IO",
	"System.Text",
	"System.Text.RegularExpressions",
	"System.Diagnostics",
	"System.Threading",
	"System.Reflection",
	"System.Collections",
	"System.Collections.Generic",
	"System.Linq",
	"System.Linq.Expressions",
	"System.Data",
	"System.Data.SqlClient",
	"System.Data.Linq",
	"System.Data.Linq.SqlClient",
	"System.Xml",
	"System.Xml.Linq",
	"System.Xml.XPath"
	};
	}
	}
	private static List<string> Assemblies
	{
	get
	{
	return new List<string>
	{
	"System.dll",
	"System.Core.dll",
	"System.Data.dll",
	"System.Xml.dll",
	"System.Xml.Linq.dll",
	"System.Data.Linq.dll",
	"System.Drawing.dll",
	"System.Data.DataSetExtensions.dll"
	};
	}
	}
	}
	public class Query
	{
	public string Kind { get; set; }
	public List<string> Namespaces { get; set; }
	public List<string> GACReferences { get; set; }
	public List<string> RelativeReferences { get; set; }
	public List<string> OtherReferences { get; set; }
	}
	public enum CodeType
	{
	LinqStatements, // Wrap inside class and Main() method
	LinqProgramTypes, // Code after 'Define other methods and classes here' directly inside namespace
	LinqProgram, // Wrap all code inside class
	}
	public class CodeFile
	{
	public CodeType Type { get; set; }
	public string FilePath { get; set; }
	public string[] RawContent { get; set; }
	public Query Query { get; set; }
	public string Content { get; set; }
	}
	public class Options
	{
	private Options(string @namespace)
	{
	if (string.IsNullOrWhiteSpace(@namespace)) throw new ArgumentNullException("namespace");
	this.CodeFiles = new List<CodeFile>();
	this.Namespace = @namespace;
	}
	public static Options CreateInMemoryDll(string @namespace)
	{
	var options = new Options(@namespace);
	options.OutputFile = null;
	options.IsInMemory = true;
	options.StartupObject = null;
	return options;
	}
	public static Options CreateOnDiskDll(string @namespace, string outputFile)
	{
	if (string.IsNullOrWhiteSpace(outputFile)) throw new ArgumentNullException("outputFile");
	var options = new Options(@namespace);
	options.OutputFile = outputFile;
	options.IsInMemory = false;
	options.StartupObject = null;
	return options;
	}
	public static Options CreateOnDiskExe(string @namespace, string outputFile, string startupObject)
	{
	if (string.IsNullOrWhiteSpace(outputFile)) throw new ArgumentNullException("outputFile");
	if (string.IsNullOrWhiteSpace(startupObject)) throw new ArgumentNullException("startupObject");
	var options = new Options(@namespace);
	options.OutputFile = outputFile;
	options.IsInMemory = false;
	options.StartupObject = startupObject;
	return options;
	}
	public Options AddCodeFile(CodeType type, string filePath)
	{
	this.CodeFiles.Add(new CodeFile { Type = type, FilePath = filePath });
	return this;
	}
	public string Namespace { get; private set; }
	public string OutputFile { get; private set; }
	public List<CodeFile> CodeFiles { get; private set; }
	public string StartupObject { get; private set; }
	public bool IsInMemory { get; private set; }
	}

view raw SelfCompile hosted with ❤ by GitHub

	void Main()
	{
	Environment.CurrentDirectory = Path.GetDirectoryName(Util.CurrentQueryPath);
	Compiler.CompileFiles(Options.CreateOnDiskDll(
	@namespace: "LinqPad.QueriesCompiler",
	outputFile: "LinqPad.QueriesCompiler.dll")
	.AddCodeFile(CodeType.LinqProgramTypes, Util.CurrentQueryPath))
	.Dump("Successfully created assembly at " + DateTime.Now.ToLocalTime());
	}
	// Define other methods and classes here
	public static class Compiler
	{
	public static Assembly CompileFiles(Options options)
	{
	var outputPath = options.IsInMemory ? "" : options.OutputFile;
	if (!options.CodeFiles.Any())
	throw new InvalidOperationException("Should add at least one file to compile.");
	foreach (var codeFile in options.CodeFiles)
	{
	codeFile.RawContent = File.ReadAllLines(codeFile.FilePath);
	codeFile.Query = GetQuery(new FileInfo(codeFile.FilePath).DirectoryName, codeFile.RawContent);
	GetCode(codeFile, options.Namespace);
	}
	return BuildAssembly(options.CodeFiles, options, outputPath);
	}
	public static Query GetQuery(string folder, IEnumerable<string> content)
	{
	var xml = string.Join("\r\n", content.TakeWhile(l => l.Trim().StartsWith("<")));
	var queryElement = XDocument.Parse(xml).Element("Query");
	if (queryElement == null) throw new InvalidOperationException("Missing <Query> header definition");
	var query = new Query
	{
	Kind = queryElement.Attribute("Kind").Value,
	Namespaces = queryElement.Elements("Namespace").Select(n => n.Value).ToList(),
	GACReferences = queryElement.Elements("GACReference").Select(n => n.Value).ToList(),
	RelativeReferences = queryElement.Elements("Reference").Where(e => e.Attribute("Relative") != null)
	.Select(n => n.Attribute("Relative").Value)
	.Select(x => new FileInfo(Path.Combine(folder, x)).FullName)
	.ToList(),
	OtherReferences = queryElement.Elements("Reference").Where(e => e.Attribute("Relative") == null)
	.Select(n => n.Value.Replace("<RuntimeDirectory>", RuntimeEnvironment.GetRuntimeDirectory()))
	.Select(n => n.Replace("<ProgramFilesX86>", Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86)))
	.ToList()
	};
	return query;
	}
	public static IEnumerable<string> GetCode(IEnumerable<IEnumerable<string>> contents, Query query, string ns)
	{
	return contents.Select(c => GetCode(c, query, ns));
	}
	private static void GetCode(CodeFile codeFile, string @namespace)
	{
	IEnumerable<string> result = null;
	if (codeFile.Query.Kind != "Program" && codeFile.Query.Kind != "Statements")
	throw new InvalidOperationException("Only queries of type C# program and C# statements are supported");
	var filteredContent = codeFile.RawContent.SkipWhile(l => l.Trim().StartsWith("<"));
	switch (codeFile.Type)
	{
	case CodeType.LinqStatements:
	result = WrapInClass(WrapInMain(filteredContent));
	break;
	case CodeType.LinqProgramTypes:
	result = FilterTypes(filteredContent);
	break;
	case CodeType.LinqProgram:
	result = WrapInClass(filteredContent);
	break;
	default:
	throw new InvalidOperationException("Only queries of type C# program and C# statements are supported");
	}
	codeFile.Content = GetCode(result, codeFile.Query, @namespace);
	}
	private static IEnumerable<string> WrapInClass(IEnumerable<string> inputCode)
	{
	var s = new[] { "public class Program {" };
	var e = new[] { "}" };
	return s.Concat(inputCode).Concat(e);
	}
	private static IEnumerable<string> WrapInMain(IEnumerable<string> inputCode)
	{
	var s = new[] { "public static void Main() {" };
	var e = new[] { "}" };
	return s.Concat(inputCode).Concat(e);
	}
	private static IEnumerable<string> FilterTypes(IEnumerable<string> inputCode)
	{
	return inputCode.SkipWhile(l => l.Trim() != "// Define other methods and classes here");
	}
	public static string GetCode(IEnumerable<string> content, Query query, string ns)
	{
	var code = string.Join(Environment.NewLine, content);
	var codeBuilder = new StringBuilder();
	codeBuilder.AppendLine("using " + string.Join(";\r\nusing ", query.Namespaces.Union(StandardNamespaces)) + ";");
	codeBuilder.AppendLine(string.Format("namespace {0} {{", ns));
	codeBuilder.AppendLine(code);
	codeBuilder.AppendLine("}");
	return codeBuilder.ToString();
	}
	public static Assembly BuildAssembly(IEnumerable<CodeFile> codeFiles, Options options, string outputPath)
	{
	var providerOptions = new Dictionary<string, string> { { "CompilerVersion", "v4.0" } };
	var provider = new CSharpCodeProvider(providerOptions);
	var assemblies = new[]
	{
	codeFiles.SelectMany(c => c.Query.GACReferences.Select(s => Assembly.Load(s).Location)),
	codeFiles.SelectMany(c => c.Query.RelativeReferences.Select(s => Assembly.LoadFrom(s).Location)),
	codeFiles.SelectMany(c => c.Query.OtherReferences),
	Assemblies
	};
	var compilerparams = new CompilerParameters
	{
	GenerateExecutable = !string.IsNullOrWhiteSpace(options.StartupObject),
	OutputAssembly = options.IsInMemory ? null : outputPath,
	GenerateInMemory = true,
	IncludeDebugInformation = true,
	MainClass = options.StartupObject
	};
	compilerparams.ReferencedAssemblies.AddRange(assemblies.SelectMany(a => a).ToArray());
	var results = provider.CompileAssemblyFromSource(compilerparams, codeFiles.Select(f => f.Content).ToArray());
	if (results.Errors.HasErrors)
	{
	var errors = new StringBuilder("Compiler Errors:\r\n");
	foreach (CompilerError error in results.Errors)
	{
	errors.AppendFormat("File {0}, Line {1},{2}\t: {3}\r\n",
	error.FileName, error.Line, error.Column, error.ErrorText);
	}
	throw new Exception("Errors compiling:\r\n" + errors + "\r\n\r\n" +
	string.Join(Environment.NewLine, codeFiles.Select(f => f.FilePath + ":\r\n" + AddLineNumber(f.Content))));
	}
	return results.CompiledAssembly;
	}
	private static string AddLineNumber(string code)
	{
	var lines = code.Split(new[] { Environment.NewLine }, StringSplitOptions.None);
	return string.Join(Environment.NewLine, lines.Select ((x, i) => (i + 1).ToString().PadLeft(4) + ": " + x));
	}
	private static List<string> StandardNamespaces
	{
	get
	{
	return new List<string>
	{
	"System",
	"System.IO",
	"System.Text",
	"System.Text.RegularExpressions",
	"System.Diagnostics",
	"System.Threading",
	"System.Reflection",
	"System.Collections",
	"System.Collections.Generic",
	"System.Linq",
	"System.Linq.Expressions",
	"System.Data",
	"System.Data.SqlClient",
	"System.Data.Linq",
	"System.Data.Linq.SqlClient",
	"System.Xml",
	"System.Xml.Linq",
	"System.Xml.XPath"
	};
	}
	}
	private static List<string> Assemblies
	{
	get
	{
	return new List<string>
	{
	"System.dll",
	"System.Core.dll",
	"System.Data.dll",
	"System.Xml.dll",
	"System.Xml.Linq.dll",
	"System.Data.Linq.dll",
	"System.Drawing.dll",
	"System.Data.DataSetExtensions.dll"
	};
	}
	}
	}
	public class Query
	{
	public string Kind { get; set; }
	public List<string> Namespaces { get; set; }
	public List<string> GACReferences { get; set; }
	public List<string> RelativeReferences { get; set; }
	public List<string> OtherReferences { get; set; }
	}
	public enum CodeType
	{
	LinqStatements, // Wrap inside class and Main() method
	LinqProgramTypes, // Code after 'Define other methods and classes here' directly inside namespace
	LinqProgram, // Wrap all code inside class
	}
	public class CodeFile
	{
	public CodeType Type { get; set; }
	public string FilePath { get; set; }
	public string[] RawContent { get; set; }
	public Query Query { get; set; }
	public string Content { get; set; }
	}
	public class Options
	{
	private Options(string @namespace)
	{
	if (string.IsNullOrWhiteSpace(@namespace)) throw new ArgumentNullException("namespace");
	this.CodeFiles = new List<CodeFile>();
	this.Namespace = @namespace;
	}
	public static Options CreateInMemoryDll(string @namespace)
	{
	var options = new Options(@namespace);
	options.OutputFile = null;
	options.IsInMemory = true;
	options.StartupObject = null;
	return options;
	}
	public static Options CreateOnDiskDll(string @namespace, string outputFile)
	{
	if (string.IsNullOrWhiteSpace(outputFile)) throw new ArgumentNullException("outputFile");
	var options = new Options(@namespace);
	options.OutputFile = outputFile;
	options.IsInMemory = false;
	options.StartupObject = null;
	return options;
	}
	public static Options CreateOnDiskExe(string @namespace, string outputFile, string startupObject)
	{
	if (string.IsNullOrWhiteSpace(outputFile)) throw new ArgumentNullException("outputFile");
	if (string.IsNullOrWhiteSpace(startupObject)) throw new ArgumentNullException("startupObject");
	var options = new Options(@namespace);
	options.OutputFile = outputFile;
	options.IsInMemory = false;
	options.StartupObject = startupObject;
	return options;
	}
	public Options AddCodeFile(CodeType type, string filePath)
	{
	this.CodeFiles.Add(new CodeFile { Type = type, FilePath = filePath });
	return this;
	}
	public string Namespace { get; private set; }
	public string OutputFile { get; private set; }
	public List<CodeFile> CodeFiles { get; private set; }
	public string StartupObject { get; private set; }
	public bool IsInMemory { get; private set; }
	}

view raw SelfCompile hosted with ❤ by GitHub

Using NuGet with LinqPad for free

Recently LinqPad got a great integration with NuGet in the paid version of the software

Unfortunately for the user of the free version it is a bit more difficult to use NuGet but it's not impossible.

Today I'm going to show you a way to use NuGet to get libraries into your LinqPad queries.

NuGet Package Explorer

You can download the NuGet Package Explorer from CodePlex

The primary use for this tool is to explorer, edit and publish NuGet packages. You could also use this tool to extract the files contained in the packages and put them in a folder, and that's what we'll do now.

First we need to Open a package from online feed

Here we could query for the package we want.

Select a package and click OK. We can now take a peek inside the package if we want. For us, we want to Export the content from the File menu.

You need choose a folder where to export the files of the package. Finally, in LinqPad we need to add a reference to the DLL by going in the Query Properties (F4) and clicking Browse. After that we are ready to start writing our queries.

Using the NuGet local cache folder

One another tip I've got for you is to add the local NuGet cache folder to the list of online feed as a shortcut for when we want to access a package we previously downloaded with the NuGet Package Explorer or even Visual Studio.

First, from the Tools menu select View NuGet download cache.

This will open a Windows Explorer on the folder %AppData%\Local\NuGet\Cache.

Now copy this path, go back to NuGet Package Explorer and Open a package from online feed. Then paste the path into Package source field and click Reload.

From now on you will be able to choose between the official NuGet.org feed and your local cache when you want to open NuGet packages.

NuGet.exe

Of course for the hardcore user of the command line interface one could use nuget.exe directly to download and extract the files rather than using the Package Explorer. I'm definitely not that hardcore. :-)

NuGet.exe is also available from the CodePlex site.

Using log4net MemoryAppender for unit testing

Sometimes unit testing existing code can be really hard. Especially when coding by exceptions (using Try.. Catch.. then do nothing!).

This will look a bit like this

	public void DoSomething()
	{
	try
	{
	ReallyDoSomething();
	}
	catch (Exception ex)
	{
	this.logger.LogError("An error occured!", ex);
	}
	}

view raw TryCache.cs hosted with ❤ by GitHub

From the outside we might not be able to test whether ReallyDoSomething has thrown an exception except if we can validate state changes done by ReallyDoSomething. And if we can't change the code much (like adding dependencies using parameters or change the return type of the method) we won't be able to write that test at all.

Another way is to use the logger to validate that.

The piece of code above use Log4Net as the logging framework and allow us to use it in our unit tests like this

	[SetUp]
	public void SetUp()
	{
	this.memoryAppender = new MemoryAppender();
	BasicConfigurator.Configure(this.memoryAppender);
	...
	}
	[Test]
	public void TestSomething()
	{
	...
	// Assert
	Assert.IsFalse(
	this.memoryAppender.GetEvents().Any(le => le.Level == Level.Error),
	"Did not expect any error messages in the logs");
	}

view raw MemoryAppenderUT.cs hosted with ❤ by GitHub

Just like that we can intercept all the logged messages by our production code.

I've used this trick a few times now when other options were not available. Hope it helps.

TFS Queries : Searching in all work items

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

To search for work items effectively we can use a special TFS API on the WorkItemStore class. The query format is based the Work Item Query Language (or WIQL ) and we can't use Linq directly, but still, we can use LinqPad to write a quick little query.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var workItemStore = tfs.GetService<WorkItemStore>();

var title = "phone";

var query = string.Format(@"
    Select [Id], [Work Item Type], [Title], [State]
    Where [Title] Contains '{0}'
    From WorkItems",
    title);

workItemStore.Query(query).Cast<WorkItem>()
    .Select(wi => new
    {
        wi.Id,
        wi.AreaPath,
        Type = wi.Type.Name,
        wi.Title,
        wi.State
    })
    .OrderBy(wi => wi.AreaPath).ThenBy(wi => wi.Type).ThenBy(wi => wi.Title)
    .Dump();

Result

Here we get the list of all the work items from all projects containing the word 'phone' in their title.

To find more about WIQL please take a look at the MSDN section on it.

IOrderedEnumerable<> (1 item)
Id	AreaPath	Type	Title	State
2	Project1	Bug	Fix the phone number on the Contact page	Done

TFS Queries: Listing all the branches

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

When it comes to branches, using the TFS Web Access or Team Explorer we can navigate around the source code explorer to see branches from various Team Project and related information. But to get the big picture there is an easier way using the TFS API.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory.
    GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

versionControl.QueryRootBranchObjects(RecursionType.Full)
    .Where(b => !b.Properties.RootItem.IsDeleted)
    .Select(s => new
    {
        Project = s.Properties.RootItem.Item
            .Substring(0, s.Properties.RootItem.Item.IndexOf('/', 2)),
        Properties = s.Properties,
        DateCreated = s.DateCreated,
        ChildBranches = s.ChildBranches
    })
    .Select(s => new 
    {
        s.Project,
        Branch = s.Properties.RootItem.Item.Replace(s.Project, ""),
        Parent = s.Properties.ParentBranch != null ?
            s.Properties.ParentBranch.Item.Replace(s.Project, "") : "",
        Version = (s.Properties.RootItem.Version as ChangesetVersionSpec)
            .ChangesetId,
        DateCreated = s.DateCreated,
        Owner = s.Properties.Owner,
        ChildBranches = s.ChildBranches
            .Where (cb => !cb.IsDeleted)
            .Select(cb => new
            {
                Branch = cb.Item.Replace(s.Project, ""),
                Version = (cb.Version as ChangesetVersionSpec).ChangesetId
            })
    })
    .OrderBy(s => s.Project).ThenByDescending(s => s.Version)
    .Dump();

Result

Here we can see a bit more information on the branches like related parent and child branches in a simple list.

IOrderedEnumerable<> (4 items)
Project	Branch	Parent	Version	DateCreated	Owner	ChildBranches
$/Project1	/Release v1.1.0.0	/Main	14	04/07/2012 9:18:01 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project1	/Main		8	28/06/2012 10:24:05 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Release v1.1.0.0 14
$/Project2	/Team	/Main	13	04/07/2012 9:17:24 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project2	/Main		12	04/07/2012 9:17:02 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Team 13
			47

TFS Queries: Generating a changelog from the branch history

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

If you ever need to produce a changelog for a release you could do it by hand, looking at all the work items associated with the changesets or you could use the TFS API to generate it automatically.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var branch = "$/Project1/Main";
var fromVersion = new ChangesetVersionSpec(1);
var toVersion = VersionSpec.Latest;

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

var history = versionControl.QueryHistory(branch, VersionSpec.Latest, 0, 
    RecursionType.Full, null, fromVersion, toVersion, int.MaxValue, false, 
    false, false);

history.OfType<Changeset>()
    .Select(x => new
    {
        x.ChangesetId,
        x.CreationDate,
        x.Committer,
        x.Comment,
        WorkItems = x.WorkItems.Select(wi => new
        {
            wi.Id,
            wi.Title,
            wi.Description,
            wi.State,
            wi.Reason
        })
    })
    .Dump("Branch history of: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);
 
history.OfType<Changeset>()
    .OrderBy(x => x.ChangesetId)
    .SelectMany(x => x.WorkItems)
    .Select(wi => string.Format("[{0}] {1} #{2} - {3}", 
        wi.Reason, wi.Type.Name, wi.Id, wi.Title))
    .Dump("Changelog of branch: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);

Result

With this we will get the list of all the changesets and associated work items from the history of the given branch.

Branch history of: $/Project1/Main from C1 to T

IEnumerable<> (4 items) ChangesetId CreationDate Committer Comment WorkItems 11 28/06/2012 10:42:14 PM Domain\Pascal Change the phone number IEnumerable<> (1 item) Id Title Description State Reason 2 Fix the phone number on the Contact page   Done Work finished 10 28/06/2012 10:39:58 PM Domain\Pascal Changed the version to 1.1.0.0 IEnumerable<> (1 item) Id Title Description State Reason 1 Change the version number to 1.1.0.0   Done Work finished 9 28/06/2012 10:31:55 PM Domain\Pascal Created a MVC4 Mobile application IEnumerable<> (0 items) 8 28/06/2012 10:23:52 PM Domain\Pascal Added Main folder for the branch IEnumerable<> (0 items)

Changelog of branch: $/Project1/Main from C1 to T
IEnumerable<String> (2 items) [Work finished] Task #1 - Change the version number to 1.1.0.0 [Work finished] Bug #2 - Fix the phone number on the Contact page

NuGet integration in LinqPad

NuGet is amazing in Visual Studio. With it I can easily download and add references to popular libraries in my project with only a few clicks in Manage NuGet Packages or keystrokes when using the Package Manager Console. I really like that because I can try out new libraries without too much effort. I also like LinqPad to try out snippets of code. But if I want to try a library I have to download the binaries from a web site, unzip them on my disk and finally add references to the assemblies from my query before I can start to play around with it.

The new version of LinqPad (v4.42.04 currently still in beta) finally gets an integration with NuGet directly in the tool but only with the Premium Edition. Let's take a look at how it works.

In the Query Properties (F4) we have a new option: Add NuGet...

This will open the NuGet Manager window

On the left we can see all the NuGet packages we already downloaded. Those will be stored in our local AppData folder and be available for all queries. In the middle section we have the NuGet feed search where we can browse for packages to download. Finally in the section on the right we see all the information on the currently selected package.

To add a package to the current query simply click Add To Query. If the package was not already downloaded LinqPad will do it now and after that it will appear in the left section. Finally we can select namespaces to use in the query by clicking Add namespace.

LinqPad will also automatically check for updates for us so we just have to click Update next to the package. All the queries using this package will get updated to the latest version too.

We are not limited to the official NuGet feed but we could also add new feeds to connect to, even our own if we wish. We can add new feeds by clicking the Settings button in the lower left of the window.

Here we can add a remote feed or a local folder if we wish. After that we'll have the option to choose which feed to use in the main NuGet Manager window using the list down the middle section.

Going back to the Query Properties window all the packages we added will be shown as a reference with the location set to NuGet. If we click on NuGet we can see the current version of the package and an option to Show assemblies in this package.

Here we can exclude some assemblies from the package if we need to.

One additional FREE new feature this time we get in this version is the ability to browse for namespaces to add from existing references. In the Additional Namespace Imports tab click on Pick from assemblies

We can now select one assembly to see the list of available namespaces from it. Then simply select the namespaces we want and click Add Selected Namespaces.

Summary

I know that not everyone have the Premium version of LinqPad but still I wanted to show you what appends when 2 amazing productivity tools collide! I guess it is possible to use some NuGet Powershell cmdlet to do part of the job. It think a stand alone NuGet client could be a good idea for someone to come up with too.

Anyway, hope you get the chance to play with those two together

TFS Queries: Build agents status and build queue

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

Sometimes it's hard to figure out exactly why a build is pending and not running right after it was queued. It can be because the build agents are currently busy building other builds, or because build agents are not started or even because the windows services for the build agents are not running. Looking at the Build Explorer we can only see builds from one team project at a time which is not ideal when you have a lot of them.

Fortunately we can create a LinqPad query to get all this information in one report.

Required references

Microsoft.TeamFoundation.Build.Client
Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Framework.Client
System.ServiceProcess

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
 .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var buildServer = tfs.GetService<IBuildServer>();

var queryAgents = buildServer.QueryBuildAgents(buildServer.CreateBuildAgentSpec());
queryAgents.Agents
 .Select(x => new
 {
  x.MachineName,
  Controller = x.Controller.Name,
  x.Enabled,
  x.Status,
  x.QueueCount
 })
 .Dump("TFS build agents");

queryAgents.Agents
 .SelectMany(x => ServiceController.GetServices(x.ServiceHost.Name)
  .Where(s => s.ServiceName.StartsWith("TFSBuildServiceHost")))
 .Select(x => new
 {
  x.MachineName,
  x.ServiceName,
  x.Status
 })
 .Dump("Windows Services of build agents");
 
buildServer
 .QueryQueuedBuilds(buildServer.CreateBuildQueueSpec("*"))
 .QueuedBuilds
 .Select(x => new
 {
  x.TeamProject,
  BuildDefinition = x.BuildDefinition.Name,
  x.QueuePosition,
  x.QueueTime,
  x.Priority,
  x.CustomGetVersion,
  x.RequestedFor,
  x.Reason,
  x.Status
 })
 .OrderBy(x => x.QueuePosition)
 .Dump("Queued builds");

Result

We get a list of the build agents status. Then the state of the windows services running the build agents. Finally the full content of the build queue regardless of the team project.

TFS build agents
IEnumerable<> (2 items) MachineName Controller Enabled Status QueueCount Server4 Server4 - Controller False Available 0 Server1 Server1 - Controller True Available 3 3

Windows Services of build agents
IEnumerable<> (2 items) MachineName ServiceName Status Server4 TFSBuildServiceHost Running Server1 TFSBuildServiceHost Running

Queued builds
IOrderedEnumerable<> (3 items) TeamProject BuildDefinition QueuePosition QueueTime Priority CustomGetVersion RequestedFor Reason Status Project1 CI 1 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI InProgress Project1 Daily 2 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued Project1 Performance 3 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued 6

TFS Queries: Recent builds of all team projects

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

We can see all the build results of a team project easily inside Team Explorer or with the Web Access but there is no way to see the build results of all team projects at the same time. Using a simple query and a few Linq operators we can get a useful little report in LinqPad.

Required references

Microsoft.TeamFoundation.Build.Client.dll
Microsoft.TeamFoundation.Client.dll
Microsoft.TeamFoundation.Framework.Client.dll

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
 .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var buildServer = tfs.GetService<IBuildServer>();

var spec = buildServer.CreateBuildDetailSpec("*");
spec.MinFinishTime = DateTime.Now.Subtract(TimeSpan.FromDays(7));
spec.MaxFinishTime = DateTime.Now;
spec.QueryDeletedOption = QueryDeletedOption.IncludeDeleted;

var builds = buildServer.QueryBuilds(spec).Builds;
var total = builds.Sum(b => b.FinishTime.Subtract(b.StartTime).TotalMinutes);

builds.Select(x => new
{
 Project = x.TeamProject,
 Definition = x.BuildDefinition.Name,
 Version = x.SourceGetVersion,
 Developer = x.RequestedFor,
 Type = x.Reason,
 Start = x.StartTime,
 Duration = x.FinishTime.Subtract(x.StartTime),
 Build = x.CompilationStatus,
 Tests = x.TestStatus,
 Result = x.Status,
 StyleCopViolations = InformationNodeConverters.GetBuildWarnings(x)
  .Count(inc => inc.Message.StartsWith("SA")),
 FxCopViolations = InformationNodeConverters.GetBuildWarnings(x)
  .Count(inc => inc.Message.StartsWith("CA")),
 Warnings = InformationNodeConverters.GetBuildWarnings(x)
  .Select(inc => inc.Message)
  .Where(m => !m.StartsWith("SA") && !m.StartsWith("CA")),
 Errors = InformationNodeConverters.GetBuildErrors(x)
  .Select(inc => inc.Message)
})
.OrderByDescending(b => b.Start)
.Dump("All builds of the week.  Total build duration: " + total);

Result

You will get a list of the build results of all the team projects for the last week. The total build time is at the top of list. Each build result include the build errors, warnings and the number of StyleCop and FxCop violations found (if you have included those static analysis tools in your project template). Also, on line 9 you can change the query to get build results over a longer period of time.

All builds of the week.  Total build duration: 20.666666666666667

IOrderedEnumerable<> (1 item) Project Definition Version Developer Type Start Duration Build Tests Result StyleCopViolations FxCopViolations Warnings Errors Project 1 Main 25 Pascal Manual 09/05/2012 9:34:07 PM 00:12:06 Success Success Success 4 1 IEnumerable<String> (0 items) IEnumerable<String> (0 items) Project 2 Main 28 Pascal Manual 09/05/2012 11:25:26 PM 00:08:34 Success Success Success 0 0 IEnumerable<String> (0 items) IEnumerable<String> (0 items)

TFS Queries: Searching in all files of the source control

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

Searching in all files of a solution inside Visual Studio is very useful. In TFS you can search for files of a collection but there is no built in functionality to search for text inside those files. Using the TFS API we can do this kind of thing.

Required references

Microsoft.TeamFoundation.Client.dll
Microsoft.TeamFoundation.Framework.Client.dll
Microsoft.TeamFoundation.VersionControl.Client.dll

Query (C# Program)

string[] textPatterns = new[] { "Main(string", "(this " };
string[] filePatterns = new[] { "*.cs", "*.xml", "*.config" };

void Main()
{
 var tfs = TfsTeamProjectCollectionFactory
  .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

 var versionControl = tfs.GetService<VersionControlServer>();
 
 var teamBranches = versionControl.QueryRootBranchObjects(RecursionType.Full)
  .Where (s => !s.Properties.RootItem.IsDeleted)
  .Select(s => s.Properties.RootItem.Item)
  .ToList()
  .Dump("Searching in the following branches");
 
 filePatterns.Dump("File patterns");
 textPatterns.Dump("Text patterns");

 foreach (var teamBranch in teamBranches)
  foreach (var filePattern in filePatterns)
   foreach (var item in versionControl.GetItems(teamBranch + "/" 
    + filePattern, RecursionType.Full).Items)
    SearchInFile(item);
}

// Define other methods and classes here
private void SearchInFile(Item file)
{
 var result = new List<string>();
 var stream = new StreamReader(file.DownloadFile(), Encoding.Default);
 
 var line = stream.ReadLine();
 var lineIndex = 0;
 
 while (!stream.EndOfStream)
 {
  if (textPatterns.Any(p => line.IndexOf(p, StringComparison.OrdinalIgnoreCase) >= 0))
   result.Add("Line " + lineIndex + ": " + line.Trim());
 
  line = stream.ReadLine();
  lineIndex++;
 }
 
 if (result.Count > 0) result.Dump(file.ServerItem);
}

Result

You will get all the lines and line numbers matching the text search patterns for all files matching the file search patterns. This may take a lot of time (30 minutes on some occurrence so be careful!). You might want to add more restrictions, especially the the branches to query around line 12.

Setup your first TFS LinqPad query

 

Prerequisites

• LinqPad (of course)
• Team Explorer (as part of your complete VS2010 installation)

Setup

• Inside LinqPad create a new empty query and in the Language combo box select C# Statements.

• Go to the Query Properties page by the Query menu or by pressing F4.
• You can add assembly references using two ways in the Additional References tab, by browsing or adding them from the GAC

Adding assembly reference by browsing

• Press the Browse button
• The assemblies we want are under C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.

Adding assembly references by adding from the GAC

• Press the Add button
  • Check the Show GAC assemblies option
  • Enter TeamFoundation in the textbox

Usually we need at least:

• Microsoft.TeamFoundation.Client.dll
• Microsoft.TeamFoundation.Common.dll

But we might also want:

• Microsoft.TeamFoundation.VersionControl.Client.dll
• Microsoft.TeamFoundation.Build.Client.dll

Now, in the Additional Namespace Imports add the following lines

• Microsoft.TeamFoundation.Client
• Microsoft.TeamFoundation.Framework.Client

And maybe those too

• Microsoft.TeamFoundation.VersionControl.Client
• Microsoft.TeamFoundation.Build.Client

Finally press Ok
We are now ready to write our first query

A first query

Let’s start with something simple

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.Dump();

Now press F5 and you should get a result like below



That’s it.  You are now ready to start writing useful LinqPad queries against your TFS server.

What's next

You should take a look at useful queries I wrote using LinqPad and the TFS API here.