Visualizing Nuget packages dependencies without Visual Studio Ultimate

In my previous post I've shown the Package Visualizer tool. Unfortunately, it's only available in the Ultimate version of Visual Studio. But all is not lost because even with a Pro version with can open DGML files.

I've created a LinqPad query that analyse packages.config files and create a DGML diagram like Package Visualizer does. I've also added things like GAC libraries and normal file based library to the mix. You can get the full Gist here. Now let's take a look at some code…

Main

Here we set a few options for our query: some file extensions to ignore when scanning for projects and more importantly the root folder path to start scanning for project files.

private string[] projectExtensionExclusions = new[] { ".vdproj", ".ndproj" };
private string rootFolder = @"C:\Users\Pascal\Dev\MyProject";

void Main()
{
  LoadAllProjects();
  LoadAllPackagesConfig();
  GenerateDGML(Path.Combine(rootFolder, "Dependencies.dgml"));
}

Data structures to uses

Then we define some fields and basic classes to help us gather the information

private List<Project> projects = new List<Project>();
private List<Package> packages = new List<Package>();
private List<Library> libraries = new List<Library>();

public class Project
{
  public Project()
  {
    this.Projects = new List<Project>();
    this.Libraries = new List<Library>();
    this.Packages = new List<Package>();
  }
  public string Path { get; set; }
  public string Name { get; set; }
  public List<Project> Projects { get; private set; }
  public List<Library> Libraries { get; private set; }
  public List<Package> Packages { get; private set; }
}

public class Package
{
  public string Name { get; set; }
  public string Version { get; set; }
}

public class Library
{
  public string Name { get; set; }
  public bool IsGAC { get; set; }
}

LoadAllProjects

Now we can start scanning for projects to load. Next we open each project files and extract all dependencies like other project, a local library or a GAC reference. We keep all this info in the project instances for later.

private void LoadAllProjects()
{
  XNamespace ns = "http://schemas.microsoft.com/developer/msbuild/2003";
 
  var projectFiles = Directory.GetFiles(rootFolder, "*.*proj", 
    SearchOption.AllDirectories)
    .Where (pf => !projectExtensionExclusions.Any(ex => pf.EndsWith(ex)));
 
  foreach (var pf in projectFiles)
    this.projects.Add(
      new Project { Path = pf, Name = Path.GetFileNameWithoutExtension(pf) });

  // Get all projects, local libraries and GAC references
  foreach (var project in this.projects)
  {
    var projectDoc = XDocument.Load(project.Path);

    foreach (var pr in projectDoc.Descendants(ns + "ProjectReference"))
    {
      var prj = projects.SingleOrDefault(p => 
        p.Name == pr.Element(ns + "Name").Value);
      if (prj != null) 
        project.Projects.Add(prj);
      else
        (pr.Element(ns + "Name").Value 
          + " project reference not found in file " + project.Path).Dump();
    }

    foreach (var r in projectDoc.Descendants(ns + "Reference")
      .Where (r => !r.Value.Contains(@"\packages\")))
      project.Libraries.Add(GetOrCreateLibrary(
        r.Attribute("Include").Value, !r.Elements(ns + "HintPath").Any()));
  }
}

LoadAllPackagesConfig

Finally we scan for packages.config files, the ones responsible for maintaining the NuGet packages dependencies for a project. Again we extract the dependencies information from the files and keep it for later.

private void LoadAllPackagesConfig()
{
  foreach (var pk in Directory.GetFiles(rootFolder, "packages.config",
    SearchOption.AllDirectories)
    .Where (pc => !pc.Contains(".nuget")))
  {
    var project = this.projects.SingleOrDefault(p =>
      Path.GetDirectoryName(p.Path) == Path.GetDirectoryName(pk));
    if (project == null)
      ("Project not found in same folder than package " + pk).Dump();
    else
    {
      foreach (var pr in XDocument.Load(pk).Descendants("package"))
      {
        var package = GetOrCreatePackage(
          pr.Attribute("id").Value, pr.Attribute("version").Value);
        project.Packages.Add(package);
      }
    }
  }
}

GenerateDGML

Here we generate the final DGML file which is simply an XML file. The schema is quite simple: a root element DirectedGraph, a Nodes section and a Links section, all of which are mandatory. We also add a Styles section to colorize the different kind of nodes: projects, packages, libraries and GAC libraries.

private XNamespace dgmlns = "http://schemas.microsoft.com/vs/2009/dgml";

private void GenerateDGML(string filename)
{
  var graph = new XElement(dgmlns + "DirectedGraph", 
    new XAttribute("GraphDirection", "LeftToRight"),
    new XElement(dgmlns + "Nodes",
      this.projects.Select (p => CreateNode(p.Name, "Project")),
      this.libraries.Select (l => CreateNode(l.Name, 
        l.IsGAC ? "GAC Library" : "Library", l.Name.Split(',')[0])),
      this.packages.Select (p => CreateNode(p.Name + " " + p.Version, "Package")),
      CreateNode("AllProjects", "Project", label: "All Projects", @group: "Expanded"),
      CreateNode("AllPackages", "Package", label: "All Packages", @group: "Expanded"),
      CreateNode("LocalLibraries", "Library", label: "Local Libraries", @group: "Expanded"),
      CreateNode("GlobalAssemblyCache", "GAC Library", label: "Global Assembly Cache", @group: "Collapsed")),
    new XElement(dgmlns + "Links",
      this.projects.SelectMany(p => p.Projects.Select(pr => new { Source = p, Target = pr } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name, "Project Reference")),
      this.projects.SelectMany(p => p.Libraries.Select(l => new { Source = p, Target = l } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name, "Library Reference")),
      this.projects.SelectMany(p => p.Packages.Select(pa => new { Source = p, Target = pa } ))
        .Select (l => CreateLink(l.Source.Name, l.Target.Name + " " + l.Target.Version, "Installed Package")),
      this.projects.Select (p => CreateLink("AllProjects", p.Name, "Contains")),
      this.packages.Select (p => CreateLink("AllPackages", p.Name + " " + p.Version, "Contains")),
      this.libraries.Where (l => !l.IsGAC).Select (l => CreateLink("LocalLibraries", l.Name, "Contains")),
      this.libraries.Where (l => l.IsGAC).Select (l => CreateLink("GlobalAssemblyCache", l.Name, "Contains"))),
    // No need to declare Categories, auto generated
    new XElement(dgmlns + "Styles",
      CreateStyle("Project", "Blue"),
      CreateStyle("Package", "Purple"),
      CreateStyle("Library", "Green"),
      CreateStyle("GAC Library", "LightGreen")));

  var doc = new XDocument(graph);
  doc.Save(filename);
}

Conclusion

All that is left is to open the Dependencies.dgml file in Visual Studio

I've left a few utility methods out of the inline code in this post but you can get all the code from the Gist. Feel free to grab a copy of the file and adapt it to your heart's content. It would be easy to create a small Console Application and call it from command line if you don't like LinqPad.

There is still a lot more I could add to the query like extracting projects and library versions from the DLL, dependencies between NuGet packages from .nupkg files and highlighting duplicates NuGet packages with different version. Still, it's enough for me in it's current form.

I hope this will help you figure out your NuGet packages usage and dependencies in your solution.

Managing NuGet packages dependencies with the Package Visualizer tool

If you ever used NuGet on a large enough solution you know you can get into trouble when projects reference different versions of the same NuGet package. That happens a lot in Azure projects as the libraries/packages get updated all the time.

I'm really surprise when I talk to people using NuGet everyday that they don't know about the Package Visualizer tool. (update: I've been told that this feature requires VS Ultimate and is not available in the Pro version. I'm still going to show it to you but stay tune for another post with a free alternative later)

NuGet Package Visualizer in Visual Studio

Once you open up a solution in Visual Studio you can go to the Tools menu, NuGet Package Manager and Package Visualizer.

This will analyse all the packages.config files in the solution and generate a DGML diagram of all NuGet packages and projects of the solution.  The diagram will help us see packages usage in the solution and find the ones with different versions.  Below you can see that I've tried this on the Roslyn (open source C# compiler) solution.

The first thing to note (and a surprise to me!) is that Roslyn use the XUnit testing framework and not MsUnit! More seriously we can quickly see that we have no duplicate packages with different versions. If we compare that to this sample solution I created we can see I'm using two versions of the Json.NET library. Now I know I should update the ClassLibrary1 project to use the new version of the package.

Of course, this only work for NuGet packages but it would be useful to have something like this for regular DLL references.  I'll try to work on a LinqPad query to generate such a DGML graph with all projects, libraries and packages.  Stay tune till next time.

Looking inside a NuGet package with NuGet Package Explorer

When I really want to learn something new (like a new tool, technology or a programming language) I do two things

• I try it myself
• Check out what others have done

I'm currently learning how to create my own NuGet packages so I'm trying to do a lot of things on my own, but I would also like to see how existing packages are made.

After installing a NuGet package in your project you can go in the packages folder and unzip the .nupkg file (yes, it's only a zip file with a different extension).  Fortunately, there is an easier. NuGet Package Explorer is an open source tool available on CodePlex.

With it you can load a package from the official NuGet feed or any other feeds you want even local feeds.

Then when we open a package we can explore its content and even go inside individual files

NuGet Package Explorer also allow us to edit files and the package itself if we want.

One trick I like to do is to add my local NuGet package download cache as a feed. To do that in the Tools menu select View NuGet download cache.

This is the folder where all the packages you previously downloaded are cached (from Visual Studio, NuGet Package Explorer and any other NuGet based tools). Simply copy the path and paste it in the Package Source field like this

This way I can quickly get to a package I just installed in my solution.

NuGet Package Explorer is a powerful tool I use a lot to understand how NuGet packages are made.

I hope it will help you too.

Packaging and distributing tools using NuGet

NuGet is an amazing tool to manage dependencies for external libraries. As part of Visual Studio it gives us an easy way to install and update those libraries over time. Now even Microsoft use it extensively to release updates to us. But the power of NuGet doesn't stop here. Some people found imaginative ways to use NuGet like the ScriptCs project and Chocolatey.

Another usage is to package and distribute utilities via NuGet packages like FAKE and xUnit.Runners. To create your own tool package you need to author a NuSpec file like this:

<package>
  …

  <files>
    <file src="tooling\app.exe" target="tools\" />
  </files>
</package>

The key here is to set all your file's target to tools\. Doing that the package will be considered a solution-level NuGet package and will be available solution wide instead of only for one project.

For example if I install the xUnit.Runners package to my solution like this:

PM> Install-Package xunit.runners

You will see that only a new package.config file will be created in the .nuget folder at the root of your solution. This is where all solution-level packages will be referenced. Nothing will actually change inside your projects.

After that all the files required to run the xUnit runner from the command line, PowerShell or a build script will be available from the \packages\xunit.runners.1.9.2\tools folder.

The power of NuGet doesn't stop there but we'll check that in another blog post.

Using NuGet with LinqPad for free

Recently LinqPad got a great integration with NuGet in the paid version of the software

Unfortunately for the user of the free version it is a bit more difficult to use NuGet but it's not impossible.

Today I'm going to show you a way to use NuGet to get libraries into your LinqPad queries.

NuGet Package Explorer

You can download the NuGet Package Explorer from CodePlex

The primary use for this tool is to explorer, edit and publish NuGet packages. You could also use this tool to extract the files contained in the packages and put them in a folder, and that's what we'll do now.

First we need to Open a package from online feed

Here we could query for the package we want.

Select a package and click OK. We can now take a peek inside the package if we want. For us, we want to Export the content from the File menu.

You need choose a folder where to export the files of the package. Finally, in LinqPad we need to add a reference to the DLL by going in the Query Properties (F4) and clicking Browse. After that we are ready to start writing our queries.

Using the NuGet local cache folder

One another tip I've got for you is to add the local NuGet cache folder to the list of online feed as a shortcut for when we want to access a package we previously downloaded with the NuGet Package Explorer or even Visual Studio.

First, from the Tools menu select View NuGet download cache.

This will open a Windows Explorer on the folder %AppData%\Local\NuGet\Cache.

Now copy this path, go back to NuGet Package Explorer and Open a package from online feed. Then paste the path into Package source field and click Reload.

From now on you will be able to choose between the official NuGet.org feed and your local cache when you want to open NuGet packages.

NuGet.exe

Of course for the hardcore user of the command line interface one could use nuget.exe directly to download and extract the files rather than using the Package Explorer. I'm definitely not that hardcore. :-)

NuGet.exe is also available from the CodePlex site.

NuGet integration in LinqPad

NuGet is amazing in Visual Studio. With it I can easily download and add references to popular libraries in my project with only a few clicks in Manage NuGet Packages or keystrokes when using the Package Manager Console. I really like that because I can try out new libraries without too much effort. I also like LinqPad to try out snippets of code. But if I want to try a library I have to download the binaries from a web site, unzip them on my disk and finally add references to the assemblies from my query before I can start to play around with it.

The new version of LinqPad (v4.42.04 currently still in beta) finally gets an integration with NuGet directly in the tool but only with the Premium Edition. Let's take a look at how it works.

In the Query Properties (F4) we have a new option: Add NuGet...

This will open the NuGet Manager window

On the left we can see all the NuGet packages we already downloaded. Those will be stored in our local AppData folder and be available for all queries. In the middle section we have the NuGet feed search where we can browse for packages to download. Finally in the section on the right we see all the information on the currently selected package.

To add a package to the current query simply click Add To Query. If the package was not already downloaded LinqPad will do it now and after that it will appear in the left section. Finally we can select namespaces to use in the query by clicking Add namespace.

LinqPad will also automatically check for updates for us so we just have to click Update next to the package. All the queries using this package will get updated to the latest version too.

We are not limited to the official NuGet feed but we could also add new feeds to connect to, even our own if we wish. We can add new feeds by clicking the Settings button in the lower left of the window.

Here we can add a remote feed or a local folder if we wish. After that we'll have the option to choose which feed to use in the main NuGet Manager window using the list down the middle section.

Going back to the Query Properties window all the packages we added will be shown as a reference with the location set to NuGet. If we click on NuGet we can see the current version of the package and an option to Show assemblies in this package.

Here we can exclude some assemblies from the package if we need to.

One additional FREE new feature this time we get in this version is the ability to browse for namespaces to add from existing references. In the Additional Namespace Imports tab click on Pick from assemblies

We can now select one assembly to see the list of available namespaces from it. Then simply select the namespaces we want and click Add Selected Namespaces.

Summary

I know that not everyone have the Premium version of LinqPad but still I wanted to show you what appends when 2 amazing productivity tools collide! I guess it is possible to use some NuGet Powershell cmdlet to do part of the job. It think a stand alone NuGet client could be a good idea for someone to come up with too.

Anyway, hope you get the chance to play with those two together