Posts on Paul Hatcher

Locking manifests in Azure Container Registry

Mon, 20 May 2024 09:00:00 +0000

So a consequence of wanting to purge images from your container registry is sometimes you need to keep some of them.

One example is that in regulated industries you must be able to produce software that was deployed to production for audit/legal purposes, so if you are purging images that are over 3 months old you might have an issue.

You can update an image so that it can’t be overridden or deleted via the az acr repository update command e.g.

Purging manifests from Azure Container Registry

Sun, 19 May 2024 09:00:00 +0000

One housekeeping task if you are producing docker images is clearing your container registry of old images.

I’ve been particularly bad at this and had also made some mistakes such as publishing PR images and images produced via our auto-patching runs, leading to the ACR size being >1.2Tb (ouch!).

There are some features introduced by acr auto purge, but I found an issue since my images typically have three tags…

Build number produced by Azure DevOps, needed as release pipelines can’t pass the version
The git SHA1, easy tracing back to the source code rather than just the build
Version number produced by gitversion (also on the source as a tag)

When I tried to remove an image via a tag, it removed the tag but not the manifest due to the other tags referencing it. The documentation says that this should work, but it doesn’t appear to.

Wikiexport 0.4 Draft

Wed, 28 Feb 2024 17:00:00 +0000

I have a draft updated WikiExport to NET 6/8 since NET 3.1 is EOL

This release also has a couple of bug fixes

Have it as a beta release at the moment until a bit more operational testing has been done.

Wikiexport 0.3

Sun, 23 Jan 2022 17:00:00 +0000

Just pushed a new version of WikiExport with a couple of useful features

Change titleFormat to have explicit {project}/{title} macros
Support for .attachments in non-root folder - provided by @ricfre

The first is just a simplification since it’s more intentional to use named values rather than {0} etc, and we can also automatically determine projectInTitle from the titleFormat.

The second one allows support for code wikis and the image path not being in the root of the repository e.g. the repository may look a bit like this…

Project Documentation from Azure DevOps Wikis

Thu, 06 Jan 2022 09:00:00 +0000

One perennial issue when developing software is how to keep documentation up to date. For detailed documentation such as APIs this has become a lot easier in recent years with capabilities such as Swashbuckle being able to pick up annotations directly from the code.

The principal of keeping your documentation as close to the code as possible means that development teams are more likely to keep it up to date and relevant to the actual behaviour - presuming you are have a good definition of done and are performing code reviews 😉.

Moved Blog to Hugo

Fri, 31 Dec 2021 14:49:50 +0000

I need to resume my blog to talk about some projects I’ve been working on and I’ve migrated over to using Hugo as the blogging platform from my old MiniBlog based solution. The main reasons are ease of writing since the content typically has some code in it, and with Hugo I can write in Markdown. Went relatively easily since I didn’t have that many posts to fix but did find some that had gone missing 😄.

NCheck V4

Thu, 21 Nov 2019 19:30:00 +0000

I’ve released NCheck V4, largely tidy up and removal of legacy code and changed the framework to support net472, net48 and netstandard2.0.

The dictionary/list checking has been reworked as well so we can get per-element reporting e.g. if we look at the list test fixture we can see which elements are not in alignment,

We now get error messages like this…

SampleList.Children
Count: Expected:<3>. Actual:<2>
[1].Id: Expected:<2>. Actual:<4>

Useful when you have more than a couple of items in the list.

NCheck V3

Thu, 10 May 2018 19:30:00 +0000

NCheck V3, has just been released, not much change but it does now support net35, net452, net462 and netstandard1.5 and above.

OData Delta Patch Security

Fri, 05 Jan 2018 19:30:00 +0000

My main project is using OData a lot and one of the requirements was to ensure that certain properties should not be updatable via PATCH - this is to ensure the integrity of the object e.g. audit fields should not be changeable post-hoc.

There’s a couple of possibilities, depending on your use case…

You want to exclude the changes if they are supplied
You want to throw an error if non-editable fields are updated.

The way we approached this was to start with an attribute

Meerkat NameParser

Tue, 11 Apr 2017 19:30:00 +0000

I’ve released a little project Meerkat.NameParser which assists in splitting up personal names into their component parts.

This code has been around in one form or another since around 1988 when I wrote the first version in Informix 4GL as part of a project for a client. Subsequent versions were written in Clipper, Visual Basic and this one in C#.

It’s not perfect by any means but on projects I’ve used it on we were getting error rates of ~2 per 100K.

Dependency Injection For SpecFlow With Unity 4

Fri, 11 Nov 2016 12:56:00 +0000

One of my current projects is using SpecFlow for testing some complex security logic; there are lots of scenarios so the BDD style of testing is well suited to the problem. I wanted to use the same style of testing for some integration testing where the client is talking to my API but hit a problem of how/where to construct the container as even the client library uses dependency injection to construct itself.

Unauthenticated users in Meerkat.Security

Tue, 20 Sep 2016 14:00:00 +0000

Couple of changes released today, one to Meerkat.Security and the other to Meerkat.Caching.

The first was to introduce handling for grant/deny of actions for unauthenticated users. I’d missed this use case before as to date, the project was just used in a corporate Windows environment and so everyone was authenticated by the time we saw them.

As I’m now working on a project using Azure B2C Active Directory we are now back to the point where users need to authenticate, so you want to filter the actions they can see/perform before and after they authenticate, e.g. you want to allow unauthenticated users access to Home.Index and error pages but probably not much more, and as the principle of the project was to centralize the security checking, we need a way to allow unauthenticated users without using AllowAnonymous on the controller action.

Publishing a different branch to Azure WebSites

Fri, 09 Sep 2016 11:00:00 +0000

I’ve just started blogging again and decided to adopt MiniBlog, a minimal modern blog engine written by Mads Kristensen who also wrote BlogEngine.NET, as SubText has died.

It only has a small set of features and Mads intends to keep it like this, the concept is that you make your own fork on GitHub, adding the features you want and publish from there; this has a few consequences…

Your blog credentials are visible in your GitHub fork
Potentially your content is there as well
How to test out new features written by yourself and/or from other forks.

I decided on a three branch strategy

Securing WebAPI with HMAC - Protocol

Wed, 20 Apr 2016 14:00:00 +0000

I’m working on a project where we are using Azure B2C Active Directory to provide authentication so that we can avoid the security headaches associated with managing a password store for it.

Unfortunately, the current implementation doesn’t support .NET Windows applications or standalone services where no user-interface is present - it will eventually, but I can’t wait on the eventual delivery date for these two use-cases. I don’t want to re-introduce user name/password for this, as I’d introduce the same security issues that I was trying to avoid in the first place, but following a bit of research I decided the way forward was HMAC authentication.

NCheck V2.3

Fri, 29 May 2015 19:30:00 +0000

Updated NCheck, my test object comparison helper, to version 2.3 which…

Introduces a new technique to wire up conventions
Supports multi-threading e.g. test fixtures using NUnit Parallelizable

Customizing the CheckerFactory

The CheckerFactory has a number conventions which are used to automatically construct Checkers for each class; these conventions can be overridden by the developer if they don’t suit a particular scenario.

We support…

Type conventions: Applied to all instances of a particular type
Property conventions: Applied to properties which satisfy a function e.g. matching a particular name
Comparer conventions: Operates on a type or property convention and changes Value comparison from object.Equals to a specified comparer; useful for values such putting error bound on floating point comparison.

The suggested technique for this is to introduce an extension class to hold the factory methods

NCheck V2.2

Thu, 28 May 2015 19:30:00 +0000

Updated NCheck, my test object comparison helper, to version 2.2 which extends ICheckerFactory to support Compare<T> - this allows us to override checkers on a per-test basis which I’ve found useful on some of my current projects.

Customizing the CheckerFactory

NCheck V2

Sun, 29 Jun 2014 19:30:00 +0000

Just release V2 of NCheck, my test object comparison helper which tidies up a few things that simplify custom comparisons and excluding properties.

Reworked the ITypeCompareTargter/IPropertyCompareTargeter into a more generic IConvention<Source, Target>
Introduced custom comparers and use convention-based registration for them.
Retired Exclude, you can now use Compare(x => x.PropertyName).Ignore, unifies the syntax compared to other options.

NCheck: A object comparison helper

Wed, 16 Apr 2014 19:30:00 +0000

With unit testing one of the precepts is to check one thing in your test, but when you have objects this is difficult to express, since you want to check that the your business process has updated all of the expected properties (positive testing) and also that has not modified things that are not expected to have changed (negative testing).

This becomes even more problematic when you have an object graph, e.g. Order –> OrderLine –> Product where you have to navigate the graph, and reporting accurately where any errors is also a major issue as you need to track property names, collection indexes, etc. etc.

CsvReader now on GitHub

Sun, 16 Feb 2014 14:00:00 +0000

A number of years ago Sebastien Lorion wrote a very good CSV parser for .NET with the code and an accompanying article on Code Project - A Fast CSV Reader. I’d used this in a number of projects but recently I needed to make some changes to the code for my trading system project, and there wasn’t a public repository available; there’s a version on NuGet but it doesn’t point back to the source.

MVC Areas Anti-Pattern

Wed, 08 Sep 2010 22:35:54 +0000

I was working on the design of a small site where we have public display of information in one format and a set of administrative screens that allow authorised users to create and edit the information.

“I know”, I thought, “this is a perfect case for MVC Areas” – and boy was I wrong! I should have realised that the problems had started when all the routing started playing up..

Route Testing and MVC 2.0 Areas

Sun, 05 Sep 2010 18:12:31 +0000

I’ve been using the MvcContrib.TestHelper class for a while now to get rather nice fluent testing of the routes in my MVC applications.

A typical test class would look like this

public class CategoryRoutesFixture : RoutesFixture
{
    private const int categoryId = 34;

    private static string BaseUrl
    {
        get { return "~/admin/Category"; }
    }

    [Test]
    public void Index()
    {
        BaseUrl.ShouldMapTo&amp;lt;CategoryController&amp;gt;(x =&amp;gt; x.Index());
    }

    [Test]
    public void Create()
    {
        (BaseUrl + "/create").ShouldMapTo&amp;lt;CategoryController&amp;gt;(x =&amp;gt; x.Create());
    }

    [Test]
    public void Show()
    {
        (BaseUrl + "/" + categoryId).ShouldMapTo&amp;lt;CategoryController&amp;gt;(x =&amp;gt; x.Show(categoryId));
    }

    [Test]
    public void Edit()
    {
        (BaseUrl + string.Format("/{0}/edit", categoryId)).ShouldMapTo&amp;lt;CategoryController&amp;gt;(x =&amp;gt; x.Edit(categoryId));
    }
}

The base class just takes care of initialising the routes collection from the MvcApplication.

Algorithm Performance

Thu, 01 Jul 2010 18:20:24 +0000

I was chatting to a colleague recently who although he was a very good programmer, did not have a computer science/maths background – this was fine until he wanted to use bubble sort on a large(ish) (100k) number of records and I had to explain to him about how algorithms that take n² time are not your friend for big (or even little) n.

One of the most important decisions you can make when optimizing an application is the up-front choice of algorithm to use, performance tuning and optimization can make a difference but you are very lucky to achieve 2x-5x improvement, to be able to make 10x or 100x improvement in your application you need decent algorithm choice.

Automating Visual Studio Templates

Mon, 28 Jun 2010 21:17:01 +0000

I find Visual Studio Item Templates very useful, but a lot of teams I encounter haven’t really adopted them as they can be a pain to construct. Here’s the technique I use to manage the templates and make it painless to add new ones and modify existing templates if they need to change. What we do is put the templates into source control and then use a build process to automate the production and distribution of the zip files.

MVC 2 EditorForModel and DropDownList

Mon, 28 Jun 2010 19:44:29 +0000

Sometimes doing very basic things can be quite tricky in new technologies; case in point presenting a DropDownList (a ComboBox to those, including me, with a VB background) using the new EditorForModel syntax in ASP.NET MVC 2.

I had a little class in my MVP framework called IdLabel which as the name implied just held an Id and a Label so that I wasn’t forced to have a separate display and edit model for every trivial case, so I wanted to do the same sort of thing when writing my shiny new MVC apps. There’s a class called SelectListItem which fits the bill, but unfortunately a decision late in the MVC 2 Beta meant that by default, the object display/edit templates will not process anything other than basic types.

Debugging ASP.NET MVC 2 Source Code

Sun, 27 Jun 2010 14:53:42 +0000

I’m trying to write some templates for MVC 2 and one of the most useful things I’ve found is being able to step into the source code and see how it is actually interpreting what you’ve written.

It’s easy enough to acquire the source, but what you might not have done before is set up a symbol server. A symbol server is a location that the Windows and Visual Studio debugging tools can use to obtain pdb files, so that you can debug almost anything, including drivers and the operating system!

Four things you need to know about Recaptcha

Sat, 26 Jun 2010 10:02:40 +0000

Working on a project where we needed a (slight) proof of humanness, and we opted to use Recaptcha as it has a fairly simple integration with .NET.

Was fine until we wanted to use custom styling to tie in with the site which is when the fun started! A few hours later, after downloading the project’s sample app, I found some things out that I thought I’d share with you…

Subversion Upgrade and svn:externals changes

Tue, 23 Mar 2010 23:23:22 +0000

I’m just in the process of restructuring my source control environment, splitting the single subversion repository into multiple repositories.

Couple of reasons for this..

My repository is getting a bit big at 1.5Gb a big chunk of this are binary files produced by my continuous integration process, so I want to break this out to a dedicated repository.

I’d also like to put each client’s work into a separate repository, this makes it easier to archive off and/or remove it at the end of a project as there’s no way to delete files from a repository apart from dumping it and filtering it into a new one.

Quality Myths

Fri, 09 Oct 2009 13:44:31 +0000

Microsoft has published the results of some empirical studies about how development practices affect quality

Test Driven Development improves quality by 60 to 90 percent but takes 15 to 35 percent more ‘up front’ time. The time spent is compensated by savings in maintenance time later on.
Team & organization structure has a huge impact on quality. Although this is conventional wisdom the study publishes figures to prove this. The metrics used data such as how many engineers are involved in a project, how many times individual source files were modified.
Code coverage in tests must be used intelligently, and has less overall impact than other factors.

Another paper highlights the issues of not running the unit tests….

Windows 7

Sun, 04 Oct 2009 13:09:38 +0000

Decided to take the plunge on Windows 7, as a test I’ve installed in on my wife’s netbook (Samsung NC20) and if anything it’s more responsive than under Windows XP.

Next decision is whether to go the whole hog and install the 64-bit version; the machine is fairly new, a Dell Precision 690, and I like the idea of having >4Gb of RAM for some of the projects I’m working on when I might have three copies of VS 2008 open at the same time.

Swine Flu

Sat, 03 Oct 2009 13:15:48 +0000

Just recovering from flu at the moment and I’m an official Swine Flu statistic.

Not the nicest of bugs, couldn’t keep any food down until the anti-virals kicked in but the funniest thing about the whole episode is the screening questionnaire to see if you qualify for the Tamiflu.

First question is whether you are enquiring on behalf of yourself or someone else, on being told I that I was asking for myself the call centre staff (apologetically) asked

Project Structure

Thu, 27 Aug 2009 22:57:34 +0000

I try to keep the same project structure for all projects, this makes it easy when initiating new projects as you can automate the creation of the core project structure.

We have the usual subversion branches/tags/trunk structure and then within trunk we have

builds : Contains all continuous integration build files for the project.
code : All the code for the project, flat directory structure internally with one project per directory.
database: Database structure, produced using the Red Gate schema compare tool
docs: Documentation source and any other project documentation
lib: Holds the library files for the project, typically referenced via svn:externals but might really be here if only used by one project.
model: Location of any models, I generally use Enterprise Architect for UML modelling so the files go here.

I’ll explain about the builds directory a bit more in the post about continuous integration, but the reason it is here physically is that then keeps everything about the project in one location rather than some bits being held by the CI project.

NHibernate Search 2.0 on its way...

Wed, 05 Aug 2009 11:40:00 +0000

Finally had sometime to progress the port of Hibernate Search…

We are intending to be feature complete with Hibernate Search 3.0.1 GA and at the moment I would say that we have ported about 80% of the code.

I have also updated the libraries so that we are using Lucene 2.3.1 and NHibernate 2.1

Frictionless Development Environment

Mon, 03 Aug 2009 12:24:26 +0000

One of the things I’ve been working on over the last couple of years for myself and for clients, is making the development environment as frictionless as possible. By this I mean removing all the little annoyances that get in the way of producing good quality software.

This guiding idea has led me to one guiding principal: keep everything under source control!

For example, one classic problem I’ve encountered is that the build works on the developer’s workstation but not on the build server or vice versa. To solve this, we now put all of our tools, excluding Visual Studio, under source control, this minimizes the differences between the developers’ machines and build environment.

Lazy Programming

Sun, 02 Aug 2009 12:19:44 +0000

No this isn’t about how Haskell evaluates stuff, but how people design and write code.

There’s a mobile phone ad showing in the UK at the moment about how it’s the impatient people that drive towards new innovation and in software at least I strongly disagree.

On the contrary, lazy programmers rule. I’m a lazy programmer, I can’t bear writing the same code more than a couple of times and when I recognise the second instance I refactor it into a function. Why - basically its about how to keep the essential stuff in your head and good use of patterns.

My Development Environment

Tue, 28 Jul 2009 22:40:42 +0000

One of the reasons I started this blog was to write down some of the stuff I’ve learnt over the past couple of years on how to structure my development environment so that it’s easy to use and ties in nicely with things like continuous integration and automated testing.

So this is sort of a “head-up” of the topics I think I will be covering over the coming weeks..

Tools choice
Source control
Project structure
Binary dependencies
Continuous Integration

It won’t necessarily be in this order and I may throw some other stuff in if I find the inspiration

Welcome

Sun, 26 Jul 2009 22:36:42 +0000

Well I finally gave in and joined the blogging community.

This will be a general rant on technology, running a small software consultancy business and anything else that takes my fancy.

To give you all a bit of background I’ve been writing software for about 30 years now, having started on a mixture of home built (anyone remember the Tangerine MicroTan 65?) and early home computers including the Commodore Pet, Amstrad 1512, BBC Micro and Atari 1040ST.