How to make integration test for Azure Storage in C# – Part 3

So in previous article part 2, we covered how to make simple integration test of our Azure Blob Storage methods, and I assume the developers are so happy for the achievement so far as the office guys.

But, but this integration test we made has 2 draw backs:

  1. It requires to fire-up Azurite docker and ensure it up and running each time you test, lets say this is no problem for you.
  2. When pushing code to GitHub and testing it via GitHub actions, how would Azurite docker starts up?

So in this article I will show how to solve this issue, both for local development environment and GitHub actions.

The short answer for solving this challenge, is by using Docker again, Yes that is correct, but this time, we will controller Docker in C#.

So in our previous article part 2, we created a project AzureBlobStorageApp.UnitTests, we need to add 2 new dependencies to it:

<ItemGroup>
  ..
  ..
  <PackageReference Include="Docker.DotNet" Version="3.125.5" />
  <PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="6.0.0" />
  ..
  ..
</ItemGroup>

Docker.DotNet is a .net NuGet package for developing docker in C#, and we will use UserSecrets package as explained in this article to our docker account, I will come to that later.

In our test project, lets create a new class, call it AzuriteContainer.cs that will consume azurite docker via C#. In this class we will define and set name, port and other parameters, we will in addition also use our docker email address, username and password to be able to pull images, otherwise it will fail pulling the image:

public class AzuriteContainer
{
private const string AzuriteImg = "mcr.microsoft.com/azure-storage/azurite";
private const string AzuritePrefix = "azurite";

public static async Task<CreateContainerResponse> Start(DockerClient client, string? email = null, string? username = null, string? password = null)
{
    await client.Images.CreateImageAsync(
        new ImagesCreateParameters
        {
            FromImage = AzuriteImg,
            Tag = "latest",
        },
        new AuthConfig
        {
            Email = email,
            Username = username,
            Password = password
        },
        new Progress<JSONMessage>());

    var response = await client.Containers.CreateContainerAsync(new CreateContainerParameters()
    {
        Image = AzuriteImg,
        Name = $"{AzuritePrefix}-{Guid.NewGuid()}",

        ExposedPorts = new Dictionary<string, EmptyStruct>
        {
            {"10000", default(EmptyStruct)}
        },
        HostConfig = new HostConfig
        {
            PortBindings = new Dictionary<string, IList<PortBinding>>
            {
                {"10000", new List<PortBinding> {new PortBinding {HostPort = "10000"}}}
            },
            PublishAllPorts = true
        }
    });


    return response;
}

I have also created AzureBlobStorageIntegrationTests class for testing our Azure Blob Storage methods in docker, it has the same tests as AzureBlobStorageUnitTests but with extra docker stuff.

public class AzureBlobStorageIntegrationTests : IAsyncLifetime
{
    private const string Content = "Some Data inside file Content";

    private const string DefaultEndpointsProtocol = "http";
    private const string AccountName = "devstoreaccount1";
    private const string AccountKey = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
    private const string BlobEndpoint = "http://127.0.0.1:10000/devstoreaccount1";
    private const string ConnectionString = $"DefaultEndpointsProtocol={DefaultEndpointsProtocol};AccountName={AccountName};AccountKey={AccountKey};BlobEndpoint={BlobEndpoint};";

    private const string Container = "container-1";

    private AzureBlobStorage? _azureBlobStorage;
    IConfiguration Configuration { get; set; }

    private readonly DockerClient _client;
    private readonly Task<CreateContainerResponse> _response;
    private string _id = string.Empty;

    public AzureBlobStorageIntegrationTests()
    {
        var builder = new ConfigurationBuilder()
            .AddUserSecrets<AzureBlobStorageIntegrationTests>(true);

        Configuration = builder.Build();

        var email = Configuration["DockerEmail"];
        var username = Configuration["DockerUserName"];
        var password = Configuration["DockerPassword"];

        _client = new DockerClientConfiguration().CreateClient();
        _response = AzuriteContainer.Start(_client, email, username, password);
    }

    [Fact]
    public async Task AzureBlobStorageTest()
    {
        await _azureBlobStorage?.CreateTextFile("file.txt", Encoding.UTF8.GetBytes(Content))!;

        var readTextFile = await _azureBlobStorage.ReadTextFile("file.txt");
        Assert.Equal(Content, readTextFile);

        await _azureBlobStorage.DeleteTextFile("file.txt");
    }

    /// <summary>
    /// Initial first time setup
    /// </summary>
    /// <returns></returns>
    public async Task InitializeAsync()
    {
        if (string.IsNullOrEmpty(_id))
        {
            var resp = await _response;
            _id = resp.ID;
        }

        await _client.Containers.StartContainerAsync(_id, null);

        _azureBlobStorage = new AzureBlobStorage(ConnectionString, Container);
    }

    /// <summary>
    /// Teardown
    /// </summary>
    /// <returns></returns>
    async Task IAsyncLifetime.DisposeAsync()
    {
        var stopParams = new ContainerStopParameters()
        {
            WaitBeforeKillSeconds = 3
        };
        await _client.Containers.StopContainerAsync(_id, stopParams, CancellationToken.None);

        var removeParams = new ContainerRemoveParameters()
        {
            Force = true
        };
        await _client.Containers.RemoveContainerAsync(_id, removeParams, CancellationToken.None);
        _client.Dispose();
        _response.Dispose();
    }
}

So lets start our test, this test will create a new docker instance on your Docker Desktop, testing the code and when it is done, it will dispose the docker instance.

Now if all this work locally, the same code should also work on GitHub actions. I wont come in how to create GitHub repository and GitHub action, but you need to set up GitHub secrets that I covered in this article. When you have created GitHub repository, set your secrets and push your code. You need to create a GitHub actions file that take care of building, testing our code.

I have created a yaml file azuritedotnetcoretest.yml under .github/workflows/ folder in my repository.

azuritedotnetcoretest.yml

name: AzuriteDotNetCoreTest

on:
  push:
    paths:
      - AzuriteDotNetCoreTest/**

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
    - name: Setup .NET
      uses: actions/[email protected]
      with:
        dotnet-version: 6.0.x
    - name: Restore dependencies
      run: dotnet restore
      working-directory: AzuriteDotNetCoreTest
    - name: Build
      run: dotnet build --no-restore
      working-directory: AzuriteDotNetCoreTest
    - name: Test
      env:
        DockerEmail: ${{ secrets.EMAIL }}
        DockerUserName: ${{ secrets.USERNAME }}
        DockerPassword: ${{ secrets.PASSWORD }}
      run: dotnet test --no-build --verbosity normal
      working-directory: AzuriteDotNetCoreTest

When you are done with it, commit it and push this workflow, see under Github Actions, one of the steps are testing as you can see in the image below.

Note: I have skipped (ignored) my old local tests methods from part 2 and let only AzureBlobStorageWithDockerAutomationUnitTests class run in GitHub tests, other wise it will fail, and the reason for that is, that it can not fire-up docker instance automatically.

Here is the final result:

GitHub test build

Conclusion

Congrats, you have now learned, to write Azure Storage Blob methods, testing them locally and testing them in GitHub actions as part of CI. This includes pull and fire-up docker instance via C# as well.

Note, what I did is to demonstrate the concept, so the code is not necessarily production ready.

One last thing is left, if you need your test to hit Azure test environment, so best practice is to create a Managed Identity and make Integration test for it in your code. But that part is not covered in this article, if you like to have an article about it, please write in the comment.

Graphic credit
The banner in this article is design by my ❤️ 15 years son Jassin Fahmi 😀.

Leave a Comment