Visualizing Nuget packages dependencies without Visual Studio Ultimate

In my previous post I've shown the Package Visualizer tool. Unfortunately, it's only available in the Ultimate version of Visual Studio. But all is not lost because even with a Pro version with can open DGML files.

I've created a LinqPad query that analyse packages.config files and create a DGML diagram like Package Visualizer does. I've also added things like GAC libraries and normal file based library to the mix. You can get the full Gist here. Now let's take a look at some code…

Main

Here we set a few options for our query: some file extensions to ignore when scanning for projects and more importantly the root folder path to start scanning for project files.

private string[] projectExtensionExclusions = new[] { ".vdproj", ".ndproj" };
private string rootFolder = @"C:\Users\Pascal\Dev\MyProject";

void Main()
{
  LoadAllProjects();
  LoadAllPackagesConfig();
  GenerateDGML(Path.Combine(rootFolder, "Dependencies.dgml"));
}

Data structures to uses

Then we define some fields and basic classes to help us gather the information

private List<Project> projects = new List<Project>();
private List<Package> packages = new List<Package>();
private List<Library> libraries = new List<Library>();

public class Project
{
  public Project()
  {
    this.Projects = new List<Project>();
    this.Libraries = new List<Library>();
    this.Packages = new List<Package>();
  }
  public string Path { get; set; }
  public string Name { get; set; }
  public List<Project> Projects { get; private set; }
  public List<Library> Libraries { get; private set; }
  public List<Package> Packages { get; private set; }
}

public class Package
{
  public string Name { get; set; }
  public string Version { get; set; }
}

public class Library
{
  public string Name { get; set; }
  public bool IsGAC { get; set; }
}

LoadAllProjects

Now we can start scanning for projects to load. Next we open each project files and extract all dependencies like other project, a local library or a GAC reference. We keep all this info in the project instances for later.

private void LoadAllProjects()
{
  XNamespace ns = "http://schemas.microsoft.com/developer/msbuild/2003";
 
  var projectFiles = Directory.GetFiles(rootFolder, "*.*proj", 
    SearchOption.AllDirectories)
    .Where (pf => !projectExtensionExclusions.Any(ex => pf.EndsWith(ex)));
 
  foreach (var pf in projectFiles)
    this.projects.Add(
      new Project { Path = pf, Name = Path.GetFileNameWithoutExtension(pf) });

  // Get all projects, local libraries and GAC references
  foreach (var project in this.projects)
  {
    var projectDoc = XDocument.Load(project.Path);

    foreach (var pr in projectDoc.Descendants(ns + "ProjectReference"))
    {
      var prj = projects.SingleOrDefault(p => 
        p.Name == pr.Element(ns + "Name").Value);
      if (prj != null) 
        project.Projects.Add(prj);
      else
        (pr.Element(ns + "Name").Value 
          + " project reference not found in file " + project.Path).Dump();
    }

    foreach (var r in projectDoc.Descendants(ns + "Reference")
      .Where (r => !r.Value.Contains(@"\packages\")))
      project.Libraries.Add(GetOrCreateLibrary(
        r.Attribute("Include").Value, !r.Elements(ns + "HintPath").Any()));
  }
}

LoadAllPackagesConfig

Finally we scan for packages.config files, the ones responsible for maintaining the NuGet packages dependencies for a project. Again we extract the dependencies information from the files and keep it for later.

private void LoadAllPackagesConfig()
{
  foreach (var pk in Directory.GetFiles(rootFolder, "packages.config",
    SearchOption.AllDirectories)
    .Where (pc => !pc.Contains(".nuget")))
  {
    var project = this.projects.SingleOrDefault(p =>
      Path.GetDirectoryName(p.Path) == Path.GetDirectoryName(pk));
    if (project == null)
      ("Project not found in same folder than package " + pk).Dump();
    else
    {
      foreach (var pr in XDocument.Load(pk).Descendants("package"))
      {
        var package = GetOrCreatePackage(
          pr.Attribute("id").Value, pr.Attribute("version").Value);
        project.Packages.Add(package);
      }
    }
  }
}

GenerateDGML

Here we generate the final DGML file which is simply an XML file. The schema is quite simple: a root element DirectedGraph, a Nodes section and a Links section, all of which are mandatory. We also add a Styles section to colorize the different kind of nodes: projects, packages, libraries and GAC libraries.

private XNamespace dgmlns = "http://schemas.microsoft.com/vs/2009/dgml";

private void GenerateDGML(string filename)
{
  var graph = new XElement(dgmlns + "DirectedGraph", 
    new XAttribute("GraphDirection", "LeftToRight"),
    new XElement(dgmlns + "Nodes",
      this.projects.Select (p => CreateNode(p.Name, "Project")),
      this.libraries.Select (l => CreateNode(l.Name, 
        l.IsGAC ? "GAC Library" : "Library", l.Name.Split(',')[0])),
      this.packages.Select (p => CreateNode(p.Name + " " + p.Version, "Package")),
      CreateNode("AllProjects", "Project", label: "All Projects", @group: "Expanded"),
      CreateNode("AllPackages", "Package", label: "All Packages", @group: "Expanded"),
      CreateNode("LocalLibraries", "Library", label: "Local Libraries", @group: "Expanded"),
      CreateNode("GlobalAssemblyCache", "GAC Library", label: "Global Assembly Cache", @group: "Collapsed")),
    new XElement(dgmlns + "Links",
      this.projects.SelectMany(p => p.Projects.Select(pr => new { Source = p, Target = pr } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name, "Project Reference")),
      this.projects.SelectMany(p => p.Libraries.Select(l => new { Source = p, Target = l } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name, "Library Reference")),
      this.projects.SelectMany(p => p.Packages.Select(pa => new { Source = p, Target = pa } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name + " " + l.Target.Version, "Installed Package")),
      this.projects.Select (p => CreateLink("AllProjects", p.Name, "Contains")),
      this.packages.Select (p => CreateLink("AllPackages", p.Name + " " + p.Version, "Contains")),
      this.libraries.Where (l => !l.IsGAC).Select (l => CreateLink("LocalLibraries", l.Name, "Contains")),
      this.libraries.Where (l => l.IsGAC).Select (l => CreateLink("GlobalAssemblyCache", l.Name, "Contains"))),
    // No need to declare Categories, auto generated
    new XElement(dgmlns + "Styles",
      CreateStyle("Project", "Blue"),
      CreateStyle("Package", "Purple"),
      CreateStyle("Library", "Green"),
      CreateStyle("GAC Library", "LightGreen")));

  var doc = new XDocument(graph);
  doc.Save(filename);
}

Conclusion

All that is left is to open the Dependencies.dgml file in Visual Studio

I've left a few utility methods out of the inline code in this post but you can get all the code from the Gist. Feel free to grab a copy of the file and adapt it to your heart's content. It would be easy to create a small Console Application and call it from command line if you don't like LinqPad.

There is still a lot more I could add to the query like extracting projects and library versions from the DLL, dependencies between NuGet packages from .nupkg files and highlighting duplicates NuGet packages with different version. Still, it's enough for me in it's current form.

I hope this will help you figure out your NuGet packages usage and dependencies in your solution.

Implementing a Treemap in C#

I was wondering how to implement a treemap in C#. A treemap is a data visualisation technique that looks like this



I know a few examples exist online but between a very abstract paper like this one and a javascript implementation on GitHub I was wondering does one could write an algorithm from scratch using a naive approach.

Let's start with a simple example, the geometric series 1/2 + 1/4 + 1/8 + 1/16 + 1/32 + 1/64… would give us something like this



My guess would be to use a recursive algorithm to create our treemap. First we need to order all the elements from biggest to smallest. Then we start to divide the area for each elements starting with the first element which takes half the available space. After that we divide the remaining area with the rest of the elements. We repeat this process until we have all the pieces.

Now if we take another more realistic example



Here every time we need to slice the area we need to make sure it won't be too small or won't look so great. If the element represent 50% or more of the total it won't be a problem but what about only 10% or 4%? I think we should set a minimum threshold for our slice, let say 25% for now. We will experiment with this value once we finish our algorithm.

So what happen if the largest value is below our 25% threshold? I think we should include more elements in the slice until we reach at least 25%. For example if we have 14%, 8% and 5% the total is 27%, so our first slice will be 27% of the available space. Then we need to distribute the 3 elements in that slice. This is in essence a subset of our original problem so we can repeat the process just for that slice.

By the way what orientation the first slice should be? I think we should always slice on the longest side of the area rectangle. If we have a square then it doesn't matter really.

Next, how do we divide the first slice? We have 3 elements: 14%, 8% and 5%. If we look for a solution from the start we now have to following proportions: 51.8%, 29.6% and 18.5%. We can take a new slice only for the first element. And we repeat the process for the last 2 elements. Which is now 61.5% and 38.5%. At each step we need to evaluate if the slice will be horizontal or vertical depending on the the shape of the area we have.

Finally, all we have to do is repeat this until all the elements are placed!

Here is my implementation in LinqPad in 3 parts

Slice calculation

public Slice<T> GetSlice<T>(IEnumerable<Element<T>> elements, double totalSize, 
 double sliceWidth)
{
 if (!elements.Any()) return null;
 if (elements.Count() == 1) return new Slice<T> 
  { Elements = elements, Size = totalSize };
 
 var sliceResult = GetElementsForSlice(elements, sliceWidth);
 
 return new Slice<T>
 {
  Elements = elements,
  Size = totalSize,
  SubSlices = new[]
  { 
   GetSlice(sliceResult.Elements, sliceResult.ElementsSize, sliceWidth),
   GetSlice(sliceResult.RemainingElements, 1 - sliceResult.ElementsSize, 
    sliceWidth)
  }
 };
}

private SliceResult<T> GetElementsForSlice<T>(IEnumerable<Element<T>> elements,
 double sliceWidth)
{
 var elementsInSlice = new List<Element<T>>();
 var remainingElements = new List<Element<T>>();
 double current = 0;
 double total = elements.Sum(x => x.Value);
 
 foreach (var element in elements)
 {
  if (current > sliceWidth)
   remainingElements.Add(element);
  else
  {
   elementsInSlice.Add(element);
   current += element.Value / total;
  }
 }
 
 return new SliceResult<T> 
 { 
  Elements = elementsInSlice, 
  ElementsSize = current,
  RemainingElements = remainingElements
 };
}

public class SliceResult<T>
{
 public IEnumerable<Element<T>> Elements { get; set; }
 public double ElementsSize { get; set; }
 public IEnumerable<Element<T>> RemainingElements { get; set; }
}

public class Slice<T>
{
 public double Size { get; set; }
 public IEnumerable<Element<T>> Elements { get; set; }
 public IEnumerable<Slice<T>> SubSlices { get; set; }
}

public class Element<T>
{
 public T Object { get; set; }
 public double Value { get; set; }
}

Generating rectangles using leaf slice (slice with only one element in it)

public IEnumerable<SliceRectangle<T>> GetRectangles<T>(Slice<T> slice, int width, 
 int height)
{
 var area = new SliceRectangle<T>
  { Slice = slice, Width = width, Height = height };
 
 foreach (var rect in GetRectangles(area))
 {
  // Make sure no rectangle go outside the original area
  if (rect.X + rect.Width > area.Width) rect.Width = area.Width - rect.X;
  if (rect.Y + rect.Height > area.Height) rect.Height = area.Height - rect.Y;
  
  yield return rect;
 }
}

private IEnumerable<SliceRectangle<T>> GetRectangles<T>(
 SliceRectangle<T> sliceRectangle)
{
 var isHorizontalSplit = sliceRectangle.Width >= sliceRectangle.Height;
 var currentPos = 0;
 foreach (var subSlice in sliceRectangle.Slice.SubSlices)
 {
  var subRect = new SliceRectangle<T> { Slice = subSlice };
  int rectSize;
  
  if (isHorizontalSplit)
  {
   rectSize = (int)Math.Round(sliceRectangle.Width * subSlice.Size);
   subRect.X = sliceRectangle.X + currentPos;
   subRect.Y = sliceRectangle.Y;
   subRect.Width = rectSize;
   subRect.Height = sliceRectangle.Height;
  }
  else
  {
   rectSize = (int)Math.Round(sliceRectangle.Height * subSlice.Size);
   subRect.X = sliceRectangle.X;
   subRect.Y = sliceRectangle.Y + currentPos;
   subRect.Width = sliceRectangle.Width;
   subRect.Height = rectSize;
  }
  
  currentPos += rectSize;
  
  if (subSlice.Elements.Count() > 1)
  {
   foreach (var sr in GetRectangles(subRect))
    yield return sr;
  }
  else if (subSlice.Elements.Count() == 1)
   yield return subRect;
 }
}

public class SliceRectangle<T>
{
 public Slice<T> Slice { get; set; }
 public int X { get; set; }
 public int Y { get; set; }
 public int Width { get; set; }
 public int Height { get; set; }
}

Drawing the rectangles in WinForm

public void DrawTreemap<T>(IEnumerable<SliceRectangle<T>> rectangles, int width, 
 int height)
{
 var font = new Font("Arial", 8 );

 var bmp = new Bitmap(width, height);
 var gfx = Graphics.FromImage(bmp);
 
 gfx.FillRectangle(Brushes.Blue, new RectangleF(0, 0, width, height));

 foreach (var r in rectangles)
 {
  gfx.DrawRectangle(Pens.Black, 
   new Rectangle(r.X, r.Y, r.Width - 1, r.Height - 1));

  gfx.DrawString(r.Slice.Elements.First().Object.ToString(), font, 
   Brushes.White, r.X, r.Y);
 }

 var form = new Form() { AutoSize = true };
 form.Controls.Add(new PictureBox()
  { Width = width, Height = height, Image = bmp });
 form.ShowDialog();
}

And finally to generate a Treemap in LinqPad

void Main()
{
 const int Width = 400;
 const int Height = 300;
 const double MinSliceRatio = 0.35;

 var elements = new[] { 24, 45, 32, 87, 34, 58, 10, 4, 5, 9, 52, 34 }
  .Select (x => new Element<string> { Object = x.ToString(), Value = x })
  .OrderByDescending (x => x.Value)
  .ToList();

 var slice = GetSlice(elements, 1, MinSliceRatio).Dump("Slices");
 
 var rectangles = GetRectangles(slice, Width, Height)
  .ToList().Dump("Rectangles");
 
 DrawTreemap(rectangles, Width, Height);
}

References:

http://en.wikipedia.org/wiki/Treemapping

http://www.win.tue.nl/~vanwijk/stm.pdf

http://stackoverflow.com/questions/3438252/creating-a-treemap-visualization

https://github.com/imranghory/treemap-squared/blob/master/treemap-squarify.js

Using Azure Table Storage with dynamic table entities

I've been working with Windows Azure for a few months now and I was trying to figure out a way to use the Azure Table Storage Service with POCOs and complex types rather than only classes inheriting from the TableEntity base class. Turns out that the only thing the CloudTableClient cares about is the ITableEntity interface. DynamicTableEntity also implements ITableEntity but is only used for querying and updating entities. You can see it in action in the example on how to query only a subset of an entity's properties.

So I started to wonder if it was possible to create class that implements ITableEntity and offer the dynamic features of an ExpandoObject. After a bit of hacking around in LinqPad I have this solution.



In this snippet I also implemented the ICustomMemberProvider which is part of the LinqPad extensions API for queries (more on this here). In Visual Studio we'll need to remove that code.

We can now use the ElasticTableEntity class like this:



Please note that you need to use the dynamic keyword to be able to define properties dynamically. You can also use the entity indexer like I did with the LastName property.

Result
List<ElasticTableEntity> (1 item) PartitionKey RowKey Timestamp ETag FirstName Number Bool Date TokenId LastName Partition123 2520391787589766073 2013-03-13 1:00:40 AM +00:00 W/"datetime'2013-03-13T01%3A00%3A40.619873Z'" Pascal 34 False 1912-03-04 12:00:00 AM +00:00 50604c02-f01c-48fc-862e-7ea66153f434 Laurin

Result with projection
List<ElasticTableEntity> (1 item) PartitionKey RowKey Timestamp ETag Date FirstName Partition123 2520391787589766073 2013-03-13 1:00:40 AM +00:00 W/"datetime'2013-03-13T01%3A00%3A40.619873Z'" 1912-03-04 12:00:00 AM +00:00 Pascal

The ElasticTableEntity allows us to define properties at run time which will be added to the table when inserting the entities. Tables in the Azure Table Storage have flexible schema so we are free to store entities with different properties as long a we respect some limitations:

• Entities can have no more than 252 different properties (that's for the Table)
• An Entity's data can be up to 1 MB in size
• A property must be one of the following types : byte[], bool, DateTime, double, Guid, int, long or string
• A property value can be up to 64 KB in size (for string and byte array)
• A property name is case sensitive and can be no more than 255 characters in length

You can store about any kind of data as long as it is one of the supported data type.  You could also encode other kind of date type in a byte array or a string (like a json document).  Just be careful to always stick to one data type for a property (yes, we can store like int, bool and string in the same column using different entities!)

That's it for now. Next time I'll show you how to use the Windows Azure Table Storage Service as a document-oriented database with the ElasticTableEntity.

See also

- Document oriented database with Azure Table Storage Service
- Using Azure Blob Storage to store documents

How to reference and reuse LinqPad queries

One missing feature from LinqPad is the ability to easily reuse code in a query we wrote in other queries.

If we really want to do it we have the following options:

• Copy and paste the code into the new query
• Open Visual Studio, create a project, create a class and copy the code you want to reuse. Compile the project and finally reference the generated dll from your query.
• Compile the LinqPad query directly (using a special script), then reference the generated dll from your query.

Today we will look at the third option. Found in the .Net framework is a very interesting API called the CodeDOM API. This API can be used to compile .Net code at run-time, like the compiler does. With this, we will be able to parse a LinqPad query and compile it.

This is what we will look into right now.

Compiling a LinqPad query

Those are the steps we need to do to accomplish query compilation:

1. Read the query and extract the options, usings and references
2. Create the CodeDOM objects and set the options and references
3. Create a string of the code file including the usings and wrap the query code inside a class
4. Compile the code

I'll explain in details each steps in future blog posts but for now I'll give you my query to compile LinqPad queries. You will notice that this is a self compiling query as the query compile itself to a dll I can reuse it to compile other queries!




You can figure out the Compiler class usage from the Main function for now.

Additional Namespace Imports

System.CodeDom.Compiler
System.Runtime.InteropServices
Microsoft.CSharp

Query (C# Program)

Using NuGet with LinqPad for free

Recently LinqPad got a great integration with NuGet in the paid version of the software

Unfortunately for the user of the free version it is a bit more difficult to use NuGet but it's not impossible.

Today I'm going to show you a way to use NuGet to get libraries into your LinqPad queries.

NuGet Package Explorer

You can download the NuGet Package Explorer from CodePlex

The primary use for this tool is to explorer, edit and publish NuGet packages. You could also use this tool to extract the files contained in the packages and put them in a folder, and that's what we'll do now.

First we need to Open a package from online feed

Here we could query for the package we want.

Select a package and click OK. We can now take a peek inside the package if we want. For us, we want to Export the content from the File menu.

You need choose a folder where to export the files of the package. Finally, in LinqPad we need to add a reference to the DLL by going in the Query Properties (F4) and clicking Browse. After that we are ready to start writing our queries.

Using the NuGet local cache folder

One another tip I've got for you is to add the local NuGet cache folder to the list of online feed as a shortcut for when we want to access a package we previously downloaded with the NuGet Package Explorer or even Visual Studio.

First, from the Tools menu select View NuGet download cache.

This will open a Windows Explorer on the folder %AppData%\Local\NuGet\Cache.

Now copy this path, go back to NuGet Package Explorer and Open a package from online feed. Then paste the path into Package source field and click Reload.

From now on you will be able to choose between the official NuGet.org feed and your local cache when you want to open NuGet packages.

NuGet.exe

Of course for the hardcore user of the command line interface one could use nuget.exe directly to download and extract the files rather than using the Package Explorer. I'm definitely not that hardcore. :-)

NuGet.exe is also available from the CodePlex site.

TFS Queries : Searching in all work items

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

To search for work items effectively we can use a special TFS API on the WorkItemStore class. The query format is based the Work Item Query Language (or WIQL ) and we can't use Linq directly, but still, we can use LinqPad to write a quick little query.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var workItemStore = tfs.GetService<WorkItemStore>();

var title = "phone";

var query = string.Format(@"
    Select [Id], [Work Item Type], [Title], [State]
    Where [Title] Contains '{0}'
    From WorkItems",
    title);

workItemStore.Query(query).Cast<WorkItem>()
    .Select(wi => new
    {
        wi.Id,
        wi.AreaPath,
        Type = wi.Type.Name,
        wi.Title,
        wi.State
    })
    .OrderBy(wi => wi.AreaPath).ThenBy(wi => wi.Type).ThenBy(wi => wi.Title)
    .Dump();

Result

Here we get the list of all the work items from all projects containing the word 'phone' in their title.

To find more about WIQL please take a look at the MSDN section on it.

IOrderedEnumerable<> (1 item)
Id	AreaPath	Type	Title	State
2	Project1	Bug	Fix the phone number on the Contact page	Done

TFS Queries: Listing all the branches

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

When it comes to branches, using the TFS Web Access or Team Explorer we can navigate around the source code explorer to see branches from various Team Project and related information. But to get the big picture there is an easier way using the TFS API.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory.
    GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

versionControl.QueryRootBranchObjects(RecursionType.Full)
    .Where(b => !b.Properties.RootItem.IsDeleted)
    .Select(s => new
    {
        Project = s.Properties.RootItem.Item
            .Substring(0, s.Properties.RootItem.Item.IndexOf('/', 2)),
        Properties = s.Properties,
        DateCreated = s.DateCreated,
        ChildBranches = s.ChildBranches
    })
    .Select(s => new 
    {
        s.Project,
        Branch = s.Properties.RootItem.Item.Replace(s.Project, ""),
        Parent = s.Properties.ParentBranch != null ?
            s.Properties.ParentBranch.Item.Replace(s.Project, "") : "",
        Version = (s.Properties.RootItem.Version as ChangesetVersionSpec)
            .ChangesetId,
        DateCreated = s.DateCreated,
        Owner = s.Properties.Owner,
        ChildBranches = s.ChildBranches
            .Where (cb => !cb.IsDeleted)
            .Select(cb => new
            {
                Branch = cb.Item.Replace(s.Project, ""),
                Version = (cb.Version as ChangesetVersionSpec).ChangesetId
            })
    })
    .OrderBy(s => s.Project).ThenByDescending(s => s.Version)
    .Dump();

Result

Here we can see a bit more information on the branches like related parent and child branches in a simple list.

IOrderedEnumerable<> (4 items)
Project	Branch	Parent	Version	DateCreated	Owner	ChildBranches
$/Project1	/Release v1.1.0.0	/Main	14	04/07/2012 9:18:01 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project1	/Main		8	28/06/2012 10:24:05 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Release v1.1.0.0 14
$/Project2	/Team	/Main	13	04/07/2012 9:17:24 PM	Domain\Pascal	IEnumerable<> (0 items)
$/Project2	/Main		12	04/07/2012 9:17:02 PM	Domain\Pascal	IEnumerable<> (1 item) Branch Version /Team 13
			47

TFS Queries: Generating a changelog from the branch history

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

If you ever need to produce a changelog for a release you could do it by hand, looking at all the work items associated with the changesets or you could use the TFS API to generate it automatically.

Required references

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.WorkItemTracking.Client

Query (C# Statements)

var branch = "$/Project1/Main";
var fromVersion = new ChangesetVersionSpec(1);
var toVersion = VersionSpec.Latest;

var tfs = TfsTeamProjectCollectionFactory
    .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var versionControl = tfs.GetService<VersionControlServer>();

var history = versionControl.QueryHistory(branch, VersionSpec.Latest, 0, 
    RecursionType.Full, null, fromVersion, toVersion, int.MaxValue, false, 
    false, false);

history.OfType<Changeset>()
    .Select(x => new
    {
        x.ChangesetId,
        x.CreationDate,
        x.Committer,
        x.Comment,
        WorkItems = x.WorkItems.Select(wi => new
        {
            wi.Id,
            wi.Title,
            wi.Description,
            wi.State,
            wi.Reason
        })
    })
    .Dump("Branch history of: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);
 
history.OfType<Changeset>()
    .OrderBy(x => x.ChangesetId)
    .SelectMany(x => x.WorkItems)
    .Select(wi => string.Format("[{0}] {1} #{2} - {3}", 
        wi.Reason, wi.Type.Name, wi.Id, wi.Title))
    .Dump("Changelog of branch: " + branch + 
        " from " + fromVersion.DisplayString + " to " + toVersion.DisplayString);

Result

With this we will get the list of all the changesets and associated work items from the history of the given branch.

Branch history of: $/Project1/Main from C1 to T

IEnumerable<> (4 items) ChangesetId CreationDate Committer Comment WorkItems 11 28/06/2012 10:42:14 PM Domain\Pascal Change the phone number IEnumerable<> (1 item) Id Title Description State Reason 2 Fix the phone number on the Contact page   Done Work finished 10 28/06/2012 10:39:58 PM Domain\Pascal Changed the version to 1.1.0.0 IEnumerable<> (1 item) Id Title Description State Reason 1 Change the version number to 1.1.0.0   Done Work finished 9 28/06/2012 10:31:55 PM Domain\Pascal Created a MVC4 Mobile application IEnumerable<> (0 items) 8 28/06/2012 10:23:52 PM Domain\Pascal Added Main folder for the branch IEnumerable<> (0 items)

Changelog of branch: $/Project1/Main from C1 to T
IEnumerable<String> (2 items) [Work finished] Task #1 - Change the version number to 1.1.0.0 [Work finished] Bug #2 - Fix the phone number on the Contact page

NuGet integration in LinqPad

NuGet is amazing in Visual Studio. With it I can easily download and add references to popular libraries in my project with only a few clicks in Manage NuGet Packages or keystrokes when using the Package Manager Console. I really like that because I can try out new libraries without too much effort. I also like LinqPad to try out snippets of code. But if I want to try a library I have to download the binaries from a web site, unzip them on my disk and finally add references to the assemblies from my query before I can start to play around with it.

The new version of LinqPad (v4.42.04 currently still in beta) finally gets an integration with NuGet directly in the tool but only with the Premium Edition. Let's take a look at how it works.

In the Query Properties (F4) we have a new option: Add NuGet...

This will open the NuGet Manager window

On the left we can see all the NuGet packages we already downloaded. Those will be stored in our local AppData folder and be available for all queries. In the middle section we have the NuGet feed search where we can browse for packages to download. Finally in the section on the right we see all the information on the currently selected package.

To add a package to the current query simply click Add To Query. If the package was not already downloaded LinqPad will do it now and after that it will appear in the left section. Finally we can select namespaces to use in the query by clicking Add namespace.

LinqPad will also automatically check for updates for us so we just have to click Update next to the package. All the queries using this package will get updated to the latest version too.

We are not limited to the official NuGet feed but we could also add new feeds to connect to, even our own if we wish. We can add new feeds by clicking the Settings button in the lower left of the window.

Here we can add a remote feed or a local folder if we wish. After that we'll have the option to choose which feed to use in the main NuGet Manager window using the list down the middle section.

Going back to the Query Properties window all the packages we added will be shown as a reference with the location set to NuGet. If we click on NuGet we can see the current version of the package and an option to Show assemblies in this package.

Here we can exclude some assemblies from the package if we need to.

One additional FREE new feature this time we get in this version is the ability to browse for namespaces to add from existing references. In the Additional Namespace Imports tab click on Pick from assemblies

We can now select one assembly to see the list of available namespaces from it. Then simply select the namespaces we want and click Add Selected Namespaces.

Summary

I know that not everyone have the Premium version of LinqPad but still I wanted to show you what appends when 2 amazing productivity tools collide! I guess it is possible to use some NuGet Powershell cmdlet to do part of the job. It think a stand alone NuGet client could be a good idea for someone to come up with too.

Anyway, hope you get the chance to play with those two together

TFS Queries: Build agents status and build queue

For an intro on LinqPad and the TFS API please read this post
For the list of all the posts in this series please read this one

Context

Sometimes it's hard to figure out exactly why a build is pending and not running right after it was queued. It can be because the build agents are currently busy building other builds, or because build agents are not started or even because the windows services for the build agents are not running. Looking at the Build Explorer we can only see builds from one team project at a time which is not ideal when you have a lot of them.

Fortunately we can create a LinqPad query to get all this information in one report.

Required references

Microsoft.TeamFoundation.Build.Client
Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Framework.Client
System.ServiceProcess

Query (C# Statements)

var tfs = TfsTeamProjectCollectionFactory
 .GetTeamProjectCollection(new Uri("http://localhost:8088/tfs"));

tfs.EnsureAuthenticated();

var buildServer = tfs.GetService<IBuildServer>();

var queryAgents = buildServer.QueryBuildAgents(buildServer.CreateBuildAgentSpec());
queryAgents.Agents
 .Select(x => new
 {
  x.MachineName,
  Controller = x.Controller.Name,
  x.Enabled,
  x.Status,
  x.QueueCount
 })
 .Dump("TFS build agents");

queryAgents.Agents
 .SelectMany(x => ServiceController.GetServices(x.ServiceHost.Name)
  .Where(s => s.ServiceName.StartsWith("TFSBuildServiceHost")))
 .Select(x => new
 {
  x.MachineName,
  x.ServiceName,
  x.Status
 })
 .Dump("Windows Services of build agents");
 
buildServer
 .QueryQueuedBuilds(buildServer.CreateBuildQueueSpec("*"))
 .QueuedBuilds
 .Select(x => new
 {
  x.TeamProject,
  BuildDefinition = x.BuildDefinition.Name,
  x.QueuePosition,
  x.QueueTime,
  x.Priority,
  x.CustomGetVersion,
  x.RequestedFor,
  x.Reason,
  x.Status
 })
 .OrderBy(x => x.QueuePosition)
 .Dump("Queued builds");

Result

We get a list of the build agents status. Then the state of the windows services running the build agents. Finally the full content of the build queue regardless of the team project.

TFS build agents
IEnumerable<> (2 items) MachineName Controller Enabled Status QueueCount Server4 Server4 - Controller False Available 0 Server1 Server1 - Controller True Available 3 3

Windows Services of build agents
IEnumerable<> (2 items) MachineName ServiceName Status Server4 TFSBuildServiceHost Running Server1 TFSBuildServiceHost Running

Queued builds
IOrderedEnumerable<> (3 items) TeamProject BuildDefinition QueuePosition QueueTime Priority CustomGetVersion RequestedFor Reason Status Project1 CI 1 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI InProgress Project1 Daily 2 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued Project1 Performance 3 2012-06-01 16:58:32 Normal C3243 Pascal IndividualCI Queued 6