How to make integration test for Azure Storage in C# – Part 3

So in the previous article part 2, we covered how to make a simple integration test of our Azure Blob Storage methods, and I assume the developers are as happy with the achievement as far as the office guys are.

But, this integration test we made has 2 drawbacks:

1. It requires fire-up Azurite docker and ensuring it is up and running each time you test, let’s say this is no problem for you.
2. When pushing code to GitHub and testing it via GitHub actions, how would Azurite docker startup?

So in this article, I will show how to solve this issue, both for the local development environment and GitHub actions.

The short answer for solving this challenge is by using Docker again, Yes that is correct, but this time, we will control Docker in C#.

So in our previous article part 2, we created a project AzureBlobStorageApp.UnitTests, we need to add 2 new dependencies to it:

<ItemGroup>
  ..
  ..
  <PackageReference Include="Docker.DotNet" Version="3.125.5" />
  <PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="6.0.0" />
  ..
  ..
</ItemGroup>

Docker.DotNet is a .net NuGet package for developing docker in C#, and we will use the UserSecrets package as explained in this article to our docker account, I will come to that later.

In our test project, let’s create a new class, and call it AzuriteContainer.cs that will consume azurite docker via C#. In this class we will define and set name, port, and other parameters, we will in addition also use our docker email address, username, and password to be able to pull images, otherwise, it will fail to pull the image:

public class AzuriteContainer
{
private const string AzuriteImg = "mcr.microsoft.com/azure-storage/azurite";
private const string AzuritePrefix = "azurite";

public static async Task<CreateContainerResponse> Start(DockerClient client, string? email = null, string? username = null, string? password = null)
{
    await client.Images.CreateImageAsync(
        new ImagesCreateParameters
        {
            FromImage = AzuriteImg,
            Tag = "latest",
        },
        new AuthConfig
        {
            Email = email,
            Username = username,
            Password = password
        },
        new Progress<JSONMessage>());

    var response = await client.Containers.CreateContainerAsync(new CreateContainerParameters()
    {
        Image = AzuriteImg,
        Name = $"{AzuritePrefix}-{Guid.NewGuid()}",

        ExposedPorts = new Dictionary<string, EmptyStruct>
        {
            {"10000", default(EmptyStruct)}
        },
        HostConfig = new HostConfig
        {
            PortBindings = new Dictionary<string, IList<PortBinding>>
            {
                {"10000", new List<PortBinding> {new PortBinding {HostPort = "10000"}}}
            },
            PublishAllPorts = true
        }
    });


    return response;
}

I have also created AzureBlobStorageIntegrationTests class for testing our Azure Blob Storage methods in docker, it has the same tests as AzureBlobStorageUnitTests but with extra docker stuff.

public class AzureBlobStorageIntegrationTests : IAsyncLifetime
{
    private const string Content = "Some Data inside file Content";

    private const string DefaultEndpointsProtocol = "http";
    private const string AccountName = "devstoreaccount1";
    private const string AccountKey = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
    private const string BlobEndpoint = "http://127.0.0.1:10000/devstoreaccount1";
    private const string ConnectionString = $"DefaultEndpointsProtocol={DefaultEndpointsProtocol};AccountName={AccountName};AccountKey={AccountKey};BlobEndpoint={BlobEndpoint};";

    private const string Container = "container-1";

    private AzureBlobStorage? _azureBlobStorage;
    IConfiguration Configuration { get; set; }

    private readonly DockerClient _client;
    private readonly Task<CreateContainerResponse> _response;
    private string _id = string.Empty;

    public AzureBlobStorageIntegrationTests()
    {
        var builder = new ConfigurationBuilder()
            .AddUserSecrets<AzureBlobStorageIntegrationTests>(true);

        Configuration = builder.Build();

        var email = Configuration["DockerEmail"];
        var username = Configuration["DockerUserName"];
        var password = Configuration["DockerPassword"];

        _client = new DockerClientConfiguration().CreateClient();
        _response = AzuriteContainer.Start(_client, email, username, password);
    }

    [Fact]
    public async Task AzureBlobStorageTest()
    {
        await _azureBlobStorage?.CreateTextFile("file.txt", Encoding.UTF8.GetBytes(Content))!;

        var readTextFile = await _azureBlobStorage.ReadTextFile("file.txt");
        Assert.Equal(Content, readTextFile);

        await _azureBlobStorage.DeleteTextFile("file.txt");
    }

    /// <summary>
    /// Initial first time setup
    /// </summary>
    /// <returns></returns>
    public async Task InitializeAsync()
    {
        if (string.IsNullOrEmpty(_id))
        {
            var resp = await _response;
            _id = resp.ID;
        }

        await _client.Containers.StartContainerAsync(_id, null);

        _azureBlobStorage = new AzureBlobStorage(ConnectionString, Container);
    }

    /// <summary>
    /// Teardown
    /// </summary>
    /// <returns></returns>
    async Task IAsyncLifetime.DisposeAsync()
    {
        var stopParams = new ContainerStopParameters()
        {
            WaitBeforeKillSeconds = 3
        };
        await _client.Containers.StopContainerAsync(_id, stopParams, CancellationToken.None);

        var removeParams = new ContainerRemoveParameters()
        {
            Force = true
        };
        await _client.Containers.RemoveContainerAsync(_id, removeParams, CancellationToken.None);
        _client.Dispose();
        _response.Dispose();
    }
}

So let’s start our test, this test will create a new docker instance on your Docker Desktop, testing the code and when it is done, it will dispose of the docker instance.

Now if all this work locally, the same code should also work on GitHub actions. I won’t come into how to create a GitHub repository and GitHub action, but you need to set up GitHub secrets that I covered in this article. When you have created a GitHub repository, set your secrets and push your code. You need to create a GitHub actions file that takes care of the building and testing of our code.

I have created a YAML file azuritedotnetcoretest.yml under .github/workflows/ folder in my repository.

azuritedotnetcoretest.yml

name: AzuriteDotNetCoreTest

on:
  push:
    paths:
      - AzuriteDotNetCoreTest/**

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 6.0.x
    - name: Restore dependencies
      run: dotnet restore
      working-directory: AzuriteDotNetCoreTest
    - name: Build
      run: dotnet build --no-restore
      working-directory: AzuriteDotNetCoreTest
    - name: Test
      env:
        DockerEmail: ${{ secrets.EMAIL }}
        DockerUserName: ${{ secrets.USERNAME }}
        DockerPassword: ${{ secrets.PASSWORD }}
      run: dotnet test --no-build --verbosity normal
      working-directory: AzuriteDotNetCoreTest

When you are done with it, commit it and push this workflow, see under GitHub Actions, one of the steps is testing as you can see in the image below.

Note: I have skipped (ignored) my old local tests methods from part 2 and let only AzureBlobStorageWithDockerAutomationUnitTests class run in GitHub tests, otherwise it will fail, and the reason for that is, that it can not fire up the docker instance automatically.

Here is the final result:

Conclusion

Congrats, you have now learned, to write Azure Storage Blob methods, testing them locally and testing them in GitHub actions as part of CI. This includes pulling and fire-up docker instances via C# as well.

Note, what I did is to demonstrate the concept, so the code is not necessarily production ready.

One last thing is left, if you need your test to hit the Azure test environment, so the best practice is to create a Managed Identity and make an Integration test for it in your code. But that part is not covered in this article, if you like to have an article about it, please write in the comment.

Graphic credit
The banner in this article is designed by my ❤️ 15 years son Jassin Fahmi 😀.