Packaging and distributing tools using NuGet

NuGet is an amazing tool to manage dependencies for external libraries. As part of Visual Studio it gives us an easy way to install and update those libraries over time. Now even Microsoft use it extensively to release updates to us. But the power of NuGet doesn't stop here. Some people found imaginative ways to use NuGet like the ScriptCs project and Chocolatey.

Another usage is to package and distribute utilities via NuGet packages like FAKE and xUnit.Runners. To create your own tool package you need to author a NuSpec file like this:

<package>
  …

  <files>
    <file src="tooling\app.exe" target="tools\" />
  </files>
</package>

The key here is to set all your file's target to tools\. Doing that the package will be considered a solution-level NuGet package and will be available solution wide instead of only for one project.

For example if I install the xUnit.Runners package to my solution like this:

PM> Install-Package xunit.runners

You will see that only a new package.config file will be created in the .nuget folder at the root of your solution. This is where all solution-level packages will be referenced. Nothing will actually change inside your projects.

After that all the files required to run the xUnit runner from the command line, PowerShell or a build script will be available from the \packages\xunit.runners.1.9.2\tools folder.

The power of NuGet doesn't stop there but we'll check that in another blog post.

TFS Queries : Searching in all work items

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

To search for work items effectively we can use a special TFS API on the WorkItemStore class. The query format is based the Work Item Query Language (or WIQL ) and we can't use Linq directly, but still, we can use LinqPad to write a quick little query.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var workItemStore = tfs.GetService<WorkItemStore>();

var title = "phone";

var query = string.Format(@"
    Select [Id], [Work Item Type], [Title], [State]
    Where [Title] Contains '{0}'
    From WorkItems",
    title);

workItemStore.Query(query).Cast<WorkItem>()
    .Select(wi => new
    {
        wi.Id,
        wi.AreaPath,
        Type = wi.Type.Name,
        wi.Title,
        wi.State
    })
    .OrderBy(wi => wi.AreaPath).ThenBy(wi => wi.Type).ThenBy(wi => wi.Title)
    .Dump();

Result

Here we get the list of all the work items from all projects containing the word 'phone' in their title.

To find more about WIQL please take a look at the MSDN section on it.

IOrderedEnumerable<> (1 item)
Id	AreaPath	Type	Title	State
2	Project1	Bug	Fix the phone number on the Contact page	Done

TFS Queries: Listing all the branches

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

When it comes to branches, using the TFS Web Access or Team Explorer we can navigate around the source code explorer to see branches from various Team Project and related information. But to get the big picture there is an easier way using the TFS API.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory.
    GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

versionControl.QueryRootBranchObjects(RecursionType.Full)
    .Where(b => !b.Properties.RootItem.IsDeleted)
    .Select(s => new
    {
        Project = s.Properties.RootItem.Item
            .Substring(0, s.Properties.RootItem.Item.IndexOf('/', 2)),
        Properties = s.Properties,
        DateCreated = s.DateCreated,
        ChildBranches = s.ChildBranches
    })
    .Select(s => new 
    {
        s.Project,
        Branch = s.Properties.RootItem.Item.Replace(s.Project, ""),
        Parent = s.Properties.ParentBranch != null ?
            s.Properties.ParentBranch.Item.Replace(s.Project, "") : "",
        Version = (s.Properties.RootItem.Version as ChangesetVersionSpec)
            .ChangesetId,
        DateCreated = s.DateCreated,
        Owner = s.Properties.Owner,
        ChildBranches = s.ChildBranches
            .Where (cb => !cb.IsDeleted)
            .Select(cb => new
            {
                Branch = cb.Item.Replace(s.Project, ""),
                Version = (cb.Version as ChangesetVersionSpec).ChangesetId
            })
    })
    .OrderBy(s => s.Project).ThenByDescending(s => s.Version)
    .Dump();

Result

Here we can see a bit more information on the branches like related parent and child branches in a simple list.

IOrderedEnumerable<> (4 items)
Project	Branch	Parent	Version	DateCreated	Owner	ChildBranches
$/Project1	/Release v1.1.0.0	/Main	14	04/07/2012 9:18:01 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project1	/Main		8	28/06/2012 10:24:05 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Release v1.1.0.0 14
$/Project2	/Team	/Main	13	04/07/2012 9:17:24 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project2	/Main		12	04/07/2012 9:17:02 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Team 13
			47

TFS Queries: Generating a changelog from the branch history

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

If you ever need to produce a changelog for a release you could do it by hand, looking at all the work items associated with the changesets or you could use the TFS API to generate it automatically.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var branch = "$/Project1/Main";
var fromVersion = new ChangesetVersionSpec(1);
var toVersion = VersionSpec.Latest;

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

var history = versionControl.QueryHistory(branch, VersionSpec.Latest, 0, 
    RecursionType.Full, null, fromVersion, toVersion, int.MaxValue, false, 
    false, false);

history.OfType<Changeset>()
    .Select(x => new
    {
        x.ChangesetId,
        x.CreationDate,
        x.Committer,
        x.Comment,
        WorkItems = x.WorkItems.Select(wi => new
        {
            wi.Id,
            wi.Title,
            wi.Description,
            wi.State,
            wi.Reason
        })
    })
    .Dump("Branch history of: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);
 
history.OfType<Changeset>()
    .OrderBy(x => x.ChangesetId)
    .SelectMany(x => x.WorkItems)
    .Select(wi => string.Format("[{0}] {1} #{2} - {3}", 
        wi.Reason, wi.Type.Name, wi.Id, wi.Title))
    .Dump("Changelog of branch: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);

Result

With this we will get the list of all the changesets and associated work items from the history of the given branch.

Branch history of: $/Project1/Main from C1 to T

IEnumerable<> (4 items) ChangesetId CreationDate Committer Comment WorkItems 11 28/06/2012 10:42:14 PM Domain\Pascal Change the phone number IEnumerable<> (1 item) Id Title Description State Reason 2 Fix the phone number on the Contact page   Done Work finished 10 28/06/2012 10:39:58 PM Domain\Pascal Changed the version to 1.1.0.0 IEnumerable<> (1 item) Id Title Description State Reason 1 Change the version number to 1.1.0.0   Done Work finished 9 28/06/2012 10:31:55 PM Domain\Pascal Created a MVC4 Mobile application IEnumerable<> (0 items) 8 28/06/2012 10:23:52 PM Domain\Pascal Added Main folder for the branch IEnumerable<> (0 items)

Changelog of branch: $/Project1/Main from C1 to T
IEnumerable<String> (2 items) [Work finished] Task #1 - Change the version number to 1.1.0.0 [Work finished] Bug #2 - Fix the phone number on the Contact page

TFS Queries: Build agents status and build queue

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

Sometimes it's hard to figure out exactly why a build is pending and not running right after it was queued. It can be because the build agents are currently busy building other builds, or because build agents are not started or even because the windows services for the build agents are not running. Looking at the Build Explorer we can only see builds from one team project at a time which is not ideal when you have a lot of them.

Fortunately we can create a LinqPad query to get all this information in one report.

Required references

Microsoft.TeamFoundation.Build.Client
Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Framework.Client
System.ServiceProcess

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
 .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var buildServer = tfs.GetService<IBuildServer>();

var queryAgents = buildServer.QueryBuildAgents(buildServer.CreateBuildAgentSpec());
queryAgents.Agents
 .Select(x => new
 {
  x.MachineName,
  Controller = x.Controller.Name,
  x.Enabled,
  x.Status,
  x.QueueCount
 })
 .Dump("TFS build agents");

queryAgents.Agents
 .SelectMany(x => ServiceController.GetServices(x.ServiceHost.Name)
  .Where(s => s.ServiceName.StartsWith("TFSBuildServiceHost")))
 .Select(x => new
 {
  x.MachineName,
  x.ServiceName,
  x.Status
 })
 .Dump("Windows Services of build agents");
 
buildServer
 .QueryQueuedBuilds(buildServer.CreateBuildQueueSpec("*"))
 .QueuedBuilds
 .Select(x => new
 {
  x.TeamProject,
  BuildDefinition = x.BuildDefinition.Name,
  x.QueuePosition,
  x.QueueTime,
  x.Priority,
  x.CustomGetVersion,
  x.RequestedFor,
  x.Reason,
  x.Status
 })
 .OrderBy(x => x.QueuePosition)
 .Dump("Queued builds");

Result

We get a list of the build agents status. Then the state of the windows services running the build agents. Finally the full content of the build queue regardless of the team project.

TFS build agents
IEnumerable<> (2 items) MachineName Controller Enabled Status QueueCount Server4 Server4 - Controller False Available 0 Server1 Server1 - Controller True Available 3 3

Windows Services of build agents
IEnumerable<> (2 items) MachineName ServiceName Status Server4 TFSBuildServiceHost Running Server1 TFSBuildServiceHost Running

Queued builds
IOrderedEnumerable<> (3 items) TeamProject BuildDefinition QueuePosition QueueTime Priority CustomGetVersion RequestedFor Reason Status Project1 CI 1 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI InProgress Project1 Daily 2 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued Project1 Performance 3 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued 6

TFS Queries: Recent builds of all team projects

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

We can see all the build results of a team project easily inside Team Explorer or with the Web Access but there is no way to see the build results of all team projects at the same time. Using a simple query and a few Linq operators we can get a useful little report in LinqPad.

Required references

Microsoft.TeamFoundation.Build.Client.dll
Microsoft.TeamFoundation.Client.dll
Microsoft.TeamFoundation.Framework.Client.dll

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
 .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var buildServer = tfs.GetService<IBuildServer>();

var spec = buildServer.CreateBuildDetailSpec("*");
spec.MinFinishTime = DateTime.Now.Subtract(TimeSpan.FromDays(7));
spec.MaxFinishTime = DateTime.Now;
spec.QueryDeletedOption = QueryDeletedOption.IncludeDeleted;

var builds = buildServer.QueryBuilds(spec).Builds;
var total = builds.Sum(b => b.FinishTime.Subtract(b.StartTime).TotalMinutes);

builds.Select(x => new
{
 Project = x.TeamProject,
 Definition = x.BuildDefinition.Name,
 Version = x.SourceGetVersion,
 Developer = x.RequestedFor,
 Type = x.Reason,
 Start = x.StartTime,
 Duration = x.FinishTime.Subtract(x.StartTime),
 Build = x.CompilationStatus,
 Tests = x.TestStatus,
 Result = x.Status,
 StyleCopViolations = InformationNodeConverters.GetBuildWarnings(x)
  .Count(inc => inc.Message.StartsWith("SA")),
 FxCopViolations = InformationNodeConverters.GetBuildWarnings(x)
  .Count(inc => inc.Message.StartsWith("CA")),
 Warnings = InformationNodeConverters.GetBuildWarnings(x)
  .Select(inc => inc.Message)
  .Where(m => !m.StartsWith("SA") && !m.StartsWith("CA")),
 Errors = InformationNodeConverters.GetBuildErrors(x)
  .Select(inc => inc.Message)
})
.OrderByDescending(b => b.Start)
.Dump("All builds of the week.  Total build duration: " + total);

Result

You will get a list of the build results of all the team projects for the last week. The total build time is at the top of list. Each build result include the build errors, warnings and the number of StyleCop and FxCop violations found (if you have included those static analysis tools in your project template). Also, on line 9 you can change the query to get build results over a longer period of time.

All builds of the week.  Total build duration: 20.666666666666667

IOrderedEnumerable<> (1 item) Project Definition Version Developer Type Start Duration Build Tests Result StyleCopViolations FxCopViolations Warnings Errors Project 1 Main 25 Pascal Manual 09/05/2012 9:34:07 PM 00:12:06 Success Success Success 4 1 IEnumerable<String> (0 items) IEnumerable<String> (0 items) Project 2 Main 28 Pascal Manual 09/05/2012 11:25:26 PM 00:08:34 Success Success Success 0 0 IEnumerable<String> (0 items) IEnumerable<String> (0 items)

TFS Queries: Searching in all files of the source control

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

Searching in all files of a solution inside Visual Studio is very useful. In TFS you can search for files of a collection but there is no built in functionality to search for text inside those files. Using the TFS API we can do this kind of thing.

Required references

Microsoft.TeamFoundation.Client.dll
Microsoft.TeamFoundation.Framework.Client.dll
Microsoft.TeamFoundation.VersionControl.Client.dll

Query (C# Program)

string[] textPatterns = new[] { "Main(string", "(this " };
string[] filePatterns = new[] { "*.cs", "*.xml", "*.config" };

void Main()
{
 var tfs = TfsTeamProjectCollectionFactory
  .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

 var versionControl = tfs.GetService<VersionControlServer>();
 
 var teamBranches = versionControl.QueryRootBranchObjects(RecursionType.Full)
  .Where (s => !s.Properties.RootItem.IsDeleted)
  .Select(s => s.Properties.RootItem.Item)
  .ToList()
  .Dump("Searching in the following branches");
 
 filePatterns.Dump("File patterns");
 textPatterns.Dump("Text patterns");

 foreach (var teamBranch in teamBranches)
  foreach (var filePattern in filePatterns)
   foreach (var item in versionControl.GetItems(teamBranch + "/" 
    + filePattern, RecursionType.Full).Items)
    SearchInFile(item);
}

// Define other methods and classes here
private void SearchInFile(Item file)
{
 var result = new List<string>();
 var stream = new StreamReader(file.DownloadFile(), Encoding.Default);
 
 var line = stream.ReadLine();
 var lineIndex = 0;
 
 while (!stream.EndOfStream)
 {
  if (textPatterns.Any(p => line.IndexOf(p, StringComparison.OrdinalIgnoreCase) >= 0))
   result.Add("Line " + lineIndex + ": " + line.Trim());
 
  line = stream.ReadLine();
  lineIndex++;
 }
 
 if (result.Count > 0) result.Dump(file.ServerItem);
}

Result

You will get all the lines and line numbers matching the text search patterns for all files matching the file search patterns. This may take a lot of time (30 minutes on some occurrence so be careful!). You might want to add more restrictions, especially the the branches to query around line 12.

Setup your first TFS LinqPad query

 

Prerequisites

• LinqPad (of course)
• Team Explorer (as part of your complete VS2010 installation)

Setup

• Inside LinqPad create a new empty query and in the Language combo box select C# Statements.

• Go to the Query Properties page by the Query menu or by pressing F4.
• You can add assembly references using two ways in the Additional References tab, by browsing or adding them from the GAC

Adding assembly reference by browsing

• Press the Browse button
• The assemblies we want are under C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.

Adding assembly references by adding from the GAC

• Press the Add button
  • Check the Show GAC assemblies option
  • Enter TeamFoundation in the textbox

Usually we need at least:

• Microsoft.TeamFoundation.Client.dll
• Microsoft.TeamFoundation.Common.dll

But we might also want:

• Microsoft.TeamFoundation.VersionControl.Client.dll
• Microsoft.TeamFoundation.Build.Client.dll

Now, in the Additional Namespace Imports add the following lines

• Microsoft.TeamFoundation.Client
• Microsoft.TeamFoundation.Framework.Client

And maybe those too

• Microsoft.TeamFoundation.VersionControl.Client
• Microsoft.TeamFoundation.Build.Client

Finally press Ok
We are now ready to write our first query

A first query

Let’s start with something simple

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.Dump();

Now press F5 and you should get a result like below



That’s it.  You are now ready to start writing useful LinqPad queries against your TFS server.

What's next

You should take a look at useful queries I wrote using LinqPad and the TFS API here.

TFS automation using LinqPad

On LinqPad

I love LinqPad. I do a lot of stuff with it like files renaming and crawling web pages to download stuff. Basically, I use it as my scripting platform. Some people uses Powershell or other bash languages and tools, I use LinqPad.

TFS automation

We’ve got TFS at work and sometimes I need to do spot checks on multiple projects, I could use the UIs and power tools built-in Visual Studio or the Team Web Access web site but it’s tedious and error prone. I prefer an automated way especially if I need the same information on a regular basis. So I’ve started to look at automating TFS using LinqPad queries. I hope to write a series of posts on this subject, one bit of API at the time and show the power queries I’ve been writing up that help me with my recurring tasks as the TFS guys.

The Basic

The basic stuff you need to know is that if you install Team Explorer alone or as part of Visual Studio you will now have access to the TFS automation API assembly directly from the GAC. They all starts with Microsoft.TeamFoundation name prefix. So adding those to your references you could write queries in LinqPad or write a tool in Visual Studio to automate some of your tasks too.

In the Series

- Setup your first TFS LinqPad query
- TFS Queries: Searching in all files of the source control
- TFS Queries: Recent builds of all team projects
- TFS Queries: Build agents status and build queue
- TFS Queries: Generating a changelog from the branch history
- TFS Queries: Listing all the branches
- TFS Queries: Searching in all work items