Custom Code Analysis Rules in VS2010 (and how to make them run in FxCop and VS2008 too)

Back in 2002 Microsoft released FxCop, a static code analysis tool. At the time it was shipped as a separate product and received a bit of buzz. It used .NET reflection and a series of pre-defined rules to detect and report coding issues that wouldn’t normally be picked up by the compiler. Since this initial release, FxCop has undergone an amazing amount of work and become more mainstream with its integration into Visual Studio under the title of ‘Code Analysis’.

Recently I’ve been developing some custom extensions to FxCop – my own code analysis rules. While extremely powerful, this isn’t yet a fully documented or supported scenario. Until it is, this post shows you how to do it all.

Why should we care?

Lately I’ve been working on the ASP.NET Web Forms Model-View-Presenter framework. It’s not quite ready for launch yet, which is why I haven’t been blogging about it, but it is already in use by a number of high traffic websites. As more and more people have started to adopt the project in its relative infancy, documentation hasn’t been up to standard. To try and keep everybody in line I contemplated writing up some ‘best practices’ documentation but then figured that this probably wouldn’t get as much attention as it should and had a high chance of rapidly becoming stale.

Code analysis rules were the perfect solution. They would allow me to define a series of best practices for use of the library in a way that could be applied across multiple projects by the developers themselves. Code analysis rules are also great because they produce a simple task list of things to fix – something that appeals to developers and managers alike.

Over the course of developing these rules I’ve increasingly come to realise that custom rules are something that should be considered in any major project – even if it’s not a framework that will be redistributed. All projects (should) have some level of consistency in their architecture. The details of this are often enforced through good practice and code reviews, but from time to time things slip through. In the same way that we write unit tests to validate our work, I think we should be writing code analysis rules. Think of them like an “architectural validity test” or something.

The Basics

A quick note about versioning: First we’ll create some rules in VS2010, to be executed in VS2010. Later in the post we’ll look at how to compile these same rules in a way that makes them compatible with FxCop 1.36 (and thus VS2008). If you’re only targeting VS2008 then all the same concepts will apply but you’ll be able to skip a few steps.

  1. Start with a new class library project. Make sure you choose to target “.NET Framework 4”, even if the rest of your solution is targeting an earlier framework. Because we’re going to be loading these rules inside VS2010, and it uses .NET 4.0, we need to use it too.

    New Class Library using .NET Framework 4 

  2. Add references to FxCopSdk.dll, Microsoft.Cci.dll and Microsoft.VisualStudio.CodeAnalysis.dll. You’ll usually find these in C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop, or an equivalent location. Don’t worry about copying these libraries to a better location or anything – we’ll look at a smarter way of referencing them shortly. (If you’re doing this in VS2008, you’ll need to download and install FxCop 1.36 first and then find these references in that folder. Also, you’ll only need the first two.)
  3. Add a new XML file to your project called Rules.xml. This will be a manifest file that describes each of our individual rules. To get us started, paste in the following content:

    <?xml version="1.0" encoding="utf-8" ?>
    <Rules FriendlyName="My Custom Rules">
      <Rule TypeName="AllTypeNamesShouldEndInFoo" Category="CustomRules.Naming" CheckId="CR1000">
        <Name>All type names should end in 'Foo'</Name>
        <Description>I like all of my types to end in 'Foo' so that I know they're a type.</Description>
        <Url>http://foobar.com</Url>
        <Resolution>The name of type {0} does not end with the suffix 'Foo'. Add the suffix to the type name.</Resolution>
        <MessageLevel Certainty="95">Warning</MessageLevel>
        <FixCategories>Breaking</FixCategories>
        <Email />
        <Owner />
      </Rule>
    </Rules>
    

    This XML file is pretty self-explanatory, but there are a few things I should point out:

    The type name needs to match the name of the class that we define the actual rule in, so make it appropriate (don’t use special characters, use Pascal casing, etc).

    The check id must be unique within the namespace of your rules, but really should be unique across the board. Microsoft uses the letters “CA” followed by a four digit number, and we use a similar scheme for Web Forms MVP.

    The resolution message is stored in the XML here, and not in your own code, but you want it to be as specific as possible so that the developer on the receiving end of it knows exactly what they need to do. Use it like a formatting string – you’ll soon see that it works really nicely.

  4. Go to the properties for the XML file and change the Build Action to EmbeddedResource so that it gets compiled into our DLL.

    Build Action: Embedded Resource

  5. Create a class called BaseRule and paste in the following code:

    using Microsoft.FxCop.Sdk;
    
    public abstract class BaseRule : BaseIntrospectionRule
    {
        protected BaseRule(string name)
            : base(
    
                // The name of the rule (must match exactly to an entry
                // in the manifest XML)
                name,
    
                // The name of the manifest XML file, qualified with the
                // namespace and missing the extension
                typeof(BaseRule).Assembly.GetName().Name + ".Rules",
    
                // The assembly to find the manifest XML in
                typeof(BaseRule).Assembly)
        {
        }
    }
    

    There are three pieces of information we’re passing into the base constructor here. The first is the type name of the rule which the framework will use to find the corresponding entry in the manifest, the second is the namespace qualified resource name of the manifest file itself and the last is the assembly that the manifest is stored in. I like to create this base class because the last two arguments will be the same for all of your rules and it gets ugly repeating them at the top of each rule.

  6. Create a class called AllTypeNamesShouldEndInFoo and paste in the following stub code:

    using Microsoft.FxCop.Sdk;
    using Microsoft.VisualStudio.CodeAnalysis.Extensibility;
    
    public class AllTypeNamesShouldEndInFoo : BaseRule
    {
        public AllTypeNamesShouldEndInFoo()
            : base("AllTypeNamesShouldEndInFoo")
        {
        }
    }
    

That’s all of the boilerplate code in place. Before we start writing the actual rule, let’s take a brief detour to the world of introspection.

Um ... ‘introspection’?

The first version of FxCop used basic .NET reflection to weave its magic. This approach is relatively simple, familiar to most developers and was a quick-to-market solution for them. As FxCop grew, this approach couldn’t scale though. Reflection has two main problems: First and foremost, it only lets you inspect the signatures of types and members – there’s no way to look inside a method and see what other methods it’s calling or to identify bad control flows. Reflection also inherits a major restriction from the underlying framework – once loaded into an app domain, and assembly can’t be unloaded. This restriction wreaks havoc in scenarios where developers want to be able to rapidly rerun the tests; having to restart FxCop every time isn’t the most glamorous of development experiences.

At this point we could fall back to inspecting the original source code, but that comes with a whole bunch of parsing nightmares and ultimately ties us back to a particular language. CIL is where we want to be.

Later versions of FxCop started using an introspection engine. This provided a fundamentally different experience, light-years ahead of what reflection could provide. The introspection engine performs all of its own CIL parsing which means that it can be pointed at any .NET assembly without having to load that assembly into the runtime. Code can be inspected without ever having the chance of being executed. The same assembly can be reloaded as many times as we want. Better yet, we can explore from the assembly level right down to individual opcodes and control structures through a unified API.

Jason Kresowaty has published a nice write up of the introspection engine. Even cooler yet, he has released a tool called Introspector which allows us to visualise the object graph that the introspection engine gives us. I highly recommend that you download it before you get into any serious rules development.

Introspector

Back to our rule...

Now that we know some of the basics of introspection, we’re ready to start coding our own rule. As a reminder, this is what we have so far:

using Microsoft.FxCop.Sdk;
using Microsoft.VisualStudio.CodeAnalysis.Extensibility;

public class AllTypeNamesShouldEndInFoo : BaseRule
{
    public AllTypeNamesShouldEndInFoo()
        : base("AllTypeNamesShouldEndInFoo")
    {
    }
}

The FxCop runtime manages the process of ‘walking’ the assembly for us. It will visit every node that it needs to, but no more, and it’ll do it across multiple threads. All we need to do is tell the runtime which nodes we’re interested in. To do this, we override one of the many Check methods.

As much as possible, use the most specific override that you can as this will give FxCop a better idea of what you’re actually looking at and thus provide better feedback to the end user. For example, if you want to look at method names don’t override Check(TypeNode) and enumerate the methods yourself because any violations you raise will be raised against the overall type. Instead, override Check(Member member).

In our scenario, because we want to check type names, we’ll override Check(TypeNode type).

The actual code for this rule is quite simple:

public override ProblemCollection Check(TypeNode type)
{
    if (!type.Name.Name.EndsWith("Foo", StringComparison.Ordinal))
    {
        var resolution = GetResolution(type.Name.Name);
        var problem = new Problem(resolution, type)
                          {
                              Certainty = 100,
                              FixCategory = FixCategories.Breaking,
                              MessageLevel = MessageLevel.Warning
                          };
        Problems.Add(problem);
    }

    return Problems;
}

All we’re doing here is checking the name of the type, and then adding a problem to a collection on the base type. The GetResolution method acts like string.Format and takes an array of parameters then formats them into the resolution text we defined in the XML file.

The second argument that we pass to the Problem constructor is the introspection node that the problem relates to. In this case it’s just the type itself, but if we were doing our own enumeration then we would pass the most specific node possible here so that FxCop could return the most accurate source reference possible to the end user.

Let’s start ‘er up.

At the time of writing, the latest standalone version of FxCop is 1.36 which still targets .NET 2.0 – 3.5. Because we’ve written our rule in .NET 4.0, our only option is to test it within Visual Studio. Luckily, that’s not as hard as it sounds. (If you’re writing your rules in VS2008, jump over this section.)

  1. Create another class library in your solution called TestLibrary. We won’t put any real code in here – we’re just going to use it as the library to execute our rules against.
  2. Add a new Code Analysis Rule Set file to the project:

    New Code Analysis Rule Set

  3. When the file opens in the designer you’ll see a list of all the built-in rules. Because custom rules aren’t really supported yet, there’s no nice way of adding our own rules into this list.

    Default Rules

  4. In Solution Explorer, right click on the .ruleset file, choose Open With and select XML Editor from the options. This will show you the raw contents of the file, which is currently pretty boring. To point Visual Studio in the direction of your custom rules, you then add a series of hint paths.

    This is what my rule set XML looks like:

    <?xml version="1.0" encoding="utf-8"?>
    <RuleSet Name="New Rule Set" Description="" ToolsVersion="10.0">
      <RuleHintPaths>
        <Path>C:\Temp\CARules\BlogDemo\BlogDemo.CodeAnalysisRules\bin\Debug</Path>
      </RuleHintPaths>
    </RuleSet>
    

    Hint paths can be absolute, or relative to the location of the rule set file. They should point at the exact folder that your compiled rules sit in. Because Visual Studio fails silently if it can’t load a rule, I prefer to start with an absolute folder path first, then change it to a relative path once everything is working.

  5. Make sure you have compiled your rules project, then go back to Solution Explorer, right click on the .ruleset file, choose Open With and select Code Analysis Rule Set Editor.

    (If you have file locking issues, close Visual Studio, delete all of your bin folders, reopen the solution, build the rules project, then attempt to open the Code Analysis Rule Set Editor again.)

Now, you should see your custom rule loaded into the list:

6CustomRules

Running the rule is now easy. Open the project properties for your test library project, go to the Code Analysis tab, enable Code Analysis and select our new rule set:

7EnableCodeAnalysis

Now when we build the project, the output from our new rule will appear in the Errors List just like any of the default rules:

8ErrorList

A Bit of Clean-up

Back when we first created the project file for our rules we referenced a couple of DLLs from a system location. This isn’t very maintainable, particularly in a team environment, so let’s clean that up quickly.

  1. Right click on the rules project and select “Unload Project”
  2. Right click on the rules project again and select “Edit .csproj” – this will show you the raw XML definition for the project
  3. Find these three references:

    <Reference Include="FxCopSdk">
      <HintPath>..\..\..\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\FxCopSdk.dll</HintPath>
      <Private>False</Private>
    </Reference>
    <Reference Include="Microsoft.Cci">
      <HintPath>..\..\..\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\Microsoft.Cci.dll</HintPath>
      <Private>False</Private>
    </Reference>
    <Reference Include="Microsoft.VisualStudio.CodeAnalysis, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\..\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\Microsoft.VisualStudio.CodeAnalysis.dll</HintPath>
      <Private>True</Private>
    </Reference>
    

    And replace them with this:

    <Reference Include="FxCopSdk">
      <HintPath>$(CodeAnalysisPath)\FxCopSdk.dll</HintPath>
      <Private>False</Private>
    </Reference>
    <Reference Include="Microsoft.Cci">
      <HintPath>$(CodeAnalysisPath)\Microsoft.Cci.dll</HintPath>
      <Private>False</Private>
    </Reference>
    <Reference Include="Microsoft.VisualStudio.CodeAnalysis">
      <HintPath>$(CodeAnalysisPath)\Microsoft.VisualStudio.CodeAnalysis.dll</HintPath>
      <Private>False</Private>
    </Reference>
    

    The build system populates the $(CodeAnalysisPath) variable for us automatically. This way, our references will be valid on every developer’s machine.

  4. Save and close the file, then right click the project and select “Reload Project”

Do the shuffle. The two-step, multi-framework shuffle...

For Web Forms MVP we want to support users on both VS2008 and VS2010. The work we’ve done so far in this post is all exclusively targeted towards VS2010 and not compatible with VS2008 or FxCop 1.36.

To make the compiled rules compatible with both IDEs we’ll need to compile two different versions of it. The VS2008 version will use .NET 3.5 and only two references while the VS2010 version will use .NET 4 and a third reference, Microsoft.VisualStudio.CodeAnalysis.

  1. Right click on the rules project and select “Unload Project”
  2. Right click on the rules project again and select “Edit .csproj” – this will show you the raw XML definition for the project
  3. Find both your Debug and Release property groups and add a DEV10 constant to each:

    <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
      ...
      <DefineConstants>TRACE;DEBUG;CODE_ANALYSIS;DEV10</DefineConstants>
      ...
    </PropertyGroup>
    <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
      ...
      <DefineConstants>TRACE;DEV10</DefineConstants>
      ...
    </PropertyGroup>
    
  4. Find the Microsoft.VisualStudio.CodeAnalysis reference and make it conditional based on the framework version being compiled against:

    <Reference Include="Microsoft.VisualStudio.CodeAnalysis" Condition=" '$(TargetFrameworkVersion)' == 'v4.0' ">
      <HintPath>$(CodeAnalysisPath)\Microsoft.VisualStudio.CodeAnalysis.dll</HintPath>
      <Private>False</Private>
    </Reference>
    
  5. Save and close the file, then right click the project and select “Reload Project”
  6. Go to AllTypeNamesShouldEndInFoo.cs and wrap the using statement for Microsoft.VisualStudio.CodeAnalysis.Extensibility in an #if construct like so:

    using System;
    using Microsoft.FxCop.Sdk;
    #if DEV10
        using Microsoft.VisualStudio.CodeAnalysis.Extensibility;
    #endif
    
  7. Make sure that your project still compiles with VS2010

At this point our project is still only building for VS2010 but it now contains all of the hook points we need to perform a second build for VS2008. The reference to Microsoft.VisualStudio.CodeAnalysis.dll will only be included if we’re building against .NET 4 and the using statements will only be compiled if the DEV10 compilation constant is present.

Normally, we would build the project using a simple call to MSBuild (which is exactly what VS2010 does under the covers):

MSBuild "BlogDemo.CodeAnalysisRules.csproj" /p:Configuration=Release /maxcpucount

To compile the FxCop 1.36 version, we just pass some extra arguments:

MSBuild  BlogDemo.CodeAnalysisRules.csproj " /p:Configuration=Release /maxcpucount /p:CodeAnalysisPath="..\Dependencies\FxCop136\" /p:DefineConstants="" /p:TargetFrameworkVersion="v3.5"

The CodeAnalysisPath parameter is normally supplied by MSBuild, but we are now overriding it with the location of the FxCop 1.36 SDK. We’re also overriding TargetFrameworkVersion.

Of course, there are nicer ways to script the build process using technologies like PowerShell. The ZIP file below contains a nice little Build-CARules.ps1 which you can use as a template.

The Resources

Download all of the sample code from this blog post and a PowerShell build script here:

Download CodeAnalysisRulesBlogDemo.zip

37 comments

  1. Tatham,

    Very good post – good to see someone else with enthusiam around custom CodeAnalysis rules :). Have you looked at unit testing custom rules? That’s always been a real PITA in the past and while I doubt that’s changes in 2010 I’m hoping there’s an easier way that firing up another project.

  2. Hi,

    I got an error using “StringComparison.Ordinal” ,
    says that undeclared element . please help me :)

    Thanks

      1. What’s the error?

        StringComparison.Ordinal is an enum value in the .NET Framework. If you’re having a compilation issue then you’re probably missing a namespace or using an old framework version.

  3. Hi again,

    Please help me with this.

    I need to create ruleset that check if sub or function is declared as private or public , etc

    sample:

    Private —> should be – Private Overridable
    Public —> should be – Public Overridable
    Friend —-> should be – Friend Overridable
    Protected —> should be – Protected Overridable

    Thank you very much in advance.
    Trex

    1. Hi Brandon,

      Code Analysis rules use introspection against the compiled assembly. That means that you only have assembly references available for inspection, and not project references. As far as the comiled assembly is concerned (and thus FxCop), there’s no such thing as a hint path for a reference.

      You’ll need to use a different technology to perform these types of validations.

      – Tatham

  4. How can i get the Procedure Modifier like ‘Overridable’? i want my rule to check if all the function have been declared ‘Overridable’

  5. Hello DEAR
    I have create a project in vs2010 BETA, now problem is that i have lost its database and vs2010 also.
    now i want to make this code run in vs2008 or vs2005.
    Can you help me please????????

    1. Hi Sam,

      I’m not sure what you’re trying to achieve here.

      If you don’t reference FxCopSdk.dll, Microsoft.Cci.dll and Microsoft.VisualStudio.CodeAnalysis.dll then what are you going to implement?

      It’s like asking if you can build an ASP.NET website without referencing System.Web.dll.

      – Tatham

  6. Fantastic post!

    I have created a new Rules Projekt for us and also included the rules.dll in it and in the path I’ve added a simple “\”…

    thanks very much

  7. Hi Tatham Oddie,

    Very helpful article.

    Currently i have created custom code analysis rules using Visual Studio 2010 and want to deploy them to different laptops / desktops.

    Please let me know how that can be achieved?

    Thanks,

    Vishal

  8. Awesome article. I am doing a series of posts on Code Analysis myself, from a business of software mindset. It is such an underutilized tool. Continue the great work Tatham.

  9. Hi Tatham,

    I have created a custom rule project and copied the custom rule DLL @ the location : C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop & also @ C:\Program Files\Microsoft FxCop 1.36\Rules

    Now whenever I build my application from the Visual Studio, the warnings and errors [as defined by the custom rules] are not getting invoked/showing up in the Warning section of the error log. But when I run the FxCop exe and set the target as the application, then these rules are getting called. Can you please help me in this regard. My whole aim is – if these custom rule DLLs are given to any other developer of my team, when they build they should be able to view the errors and warnings under the Warning section of the Error TAB.

    Could you please suggest.

    Thanks,
    Anirban

  10. Hi Tatham,

    I have created a custom rule project and copied the custom rule DLL @ the location : C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop & also @ C:\Program Files\Microsoft FxCop 1.36\Rules

    Now whenever I build my application from the Visual Studio, the warnings and errors [as defined by the custom rules] are not getting invoked/showing up in the Warning section of the error log. But when I run the FxCop exe and set the target as the application, then these rules are getting called. Can you please help me in this regard. My whole aim is – if these custom rule DLLs are given to any other developer of my team, when they build they should be able to view the errors and warnings under the Warning section of the Error TAB.

    Could you please suggest.

    Thanks,

    Anirban

    1. Hi Anirban,

      Are the default CA rules running in the build?

      I’d get them working first to ensure the CODE_ANALYSIS constant is set and that the rule set file is hooked up properly.

      – Tatham

      1. Yes the CODE_ANALYSIS is set correctly as I am able to get other Microsoft Warnings.

        Just to briefly jot down the steps I did:

        1)Created FxCop.Rules.dll [The code is same as the code you have provided]
        2)Copied this DLL @C:\Program Files\Microsoft FxCop 1.36\Rules

        3)Created an Application MyApplication.Sln. Created some dummy methods inside it
        4)Try to build from Visual Studio, do not see the custom warning message [CR1000]

        5)I run from FxCop.exe. I set the target to MyApplication.exe, I get to see the custom warning[CR1000].

        Thanks
        Anirban

  11. Hey thanks a lot, I really needed it for a quick start for Custom rules creation. :) god bless

  12. How to develop static code rule for:”Do not ‘shadow’ a name in an outer scope.”

    can you help me in developing an algorithm for this.

  13. Cracking Post!
    This post helped me to create custom code analysis rules.
    Thanks a lot.

  14. Hi Tatham,
    Great article!
    Does anybody have any idea about custom rule for measure “number methods in a class”?

    Much appreciated,
    Andrei.

  15. Hi Tatham,
    As mentioned above in the blog
    1. Created the CodeAnalysis dll, Rules.xml with sample code given in this blog for VS2010
    2. Referenced the dlls
    fxcopsdk.dll
    fxcopcommon.dll
    Microsoft.cci.dll
    Microsoft.Visualstudio.Codeanalysis.dll
    3. Created the test library project, ruleset
    4. Executed the test library project, its throwing error

    Error 2 CA0053 : Unable to load rule assembly ‘fxcopcommon.dll: The assembly contains no FxCop rules. ClassLibrary1
    Error 3 CA0053 : Unable to load rule assembly ‘fxcopsdk.dll’: The assembly contains no FxCop rules. ClassLibrary1
    Error 4 CA0053 : Unable to load rule assembly ‘microsoft.cci.dll’: The assembly contains no FxCop rules. ClassLibrary1
    Error 5 CA0053 : Unable to load rule assembly ‘microsoft.visualstudio.codeanalysis.common.dll’: The assembly contains no FxCop rules. ClassLibrary1

    I have checked the the above dlls path its coming from “C:\Program Files\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop”

    Its very urgent

  16. I’ve followed all the steps given by you correctly.But while running Code Analysis I’m getting an error saying “CA-0001 An unknown error occurred while running Code Analysis”.Please can you suggest me on how to solve this problem ? Is that due to VS 2010 installation problem ?

      1. I’ve created Custom rule set by following your steps given, It’s working fine for my app which located on my local machine, but i want it to be common for all those are working in my team. How to do this? please suggest me on this.

  17. Hi !
    Good article, I understand the principle, but I am not sure that I can make MY own rule.
    I would like to check if a method Trace() is called in each other methods.
    Can you please help me to do that ?
    Thanks

  18. I want to find, if the project contains datasources.aspx file or not? If this file not found, add a problem.

    I have tried the ProblemCollection Check(TypeNode type) but no luck yet.
    Can please helpme out on this?

  19. Thank you very much for this.
    One minor suggestion: I ran into the same problem as Vijay above. This could be avoided if you mentioned in point 2 to specify Copy Local = False when adding the three references.

Comments are closed.