Integration Testing with Entity Framework Core and SQL Server

Originally posted to: http://www.davepaquette.com/archive/2016/11/27/integration-testing-with-entity-framework-core-and-sql-server.aspx

Entity Framework Core makes it easy to write tests that execute against an in-memory store. Using an in-memory store is convenient since we don’t need to worry about setting up a relational database. It also ensures our unit tests run quickly so we aren’t left waiting hours for a large test suite to complete.

While Entity Framework Core’s in-memory store works great for many scenarios, there are some situations where it might be better to run our tests against a real relational database. Some examples include when loading entities using raw SQL or when using SQL Server specific features that can not be tested using the in-memory provider. In this case, the tests would be considered an integration test since we are no longer testing our Entity Framework context in isolation. We are testing how it will work in the real world when connected to SQL Server.

The Sample Project

For this example, I used the following simple model and DbContext classes.

public class Monster
{
public int Id { get; set; }
public string Name { get; set; }
public bool IsScary { get; set; }
public string Colour { get; set; }
}
public class MonsterContext : DbContext
{
public MonsterContext(DbContextOptions<MonsterContext> options)
: base(options)
{
}
public DbSet<Monster> Monsters { get; set; }
}

In an ASP.NET Core application, the context is configured to use SQL Server in the Startup.ConfigureServices method.

services.AddDbContext<MonsterContext>(options =>
{
options.UseSqlServer("DefaultConnection");
});

The DefaultConnection is defined in appsettings.json which is loaded at startup.

{
"ConnectionStrings": {
"DefaultConnection": "Server=(localdb)\\mssqllocaldb;Database=monsters_db;Trusted_Connection=True;MultipleActiveResultSets=true"
}
}

The MonsterContext is also configured to use Migrations which were initialized using the dotnet ef migrations add InitialCreate command. For more on Entity Framework Migrations, see the official tutorial.

As a simple example, I created a query class that loads scary monsters from the database using a SQL query instead of querying the Monsters DbSet directly.

public class ScaryMonstersQuery
{
private MonsterContext _context;
public ScaryMonstersQuery(MonsterContext context)
{
_context = context;
}
public IEnumerable<Monster> Execute()
{
return _context.Monsters
.FromSql("SELECT Id, Name, IsScary, Colour FROM Monsters WHERE IsScary = {0}", true);
}
}

To be clear, a better way to write this query is _context.Monster.Where(m => m.IsScary == true), but I wanted a simple example. I also wanted to use FromSql because it is inherently difficult to unit test. The FromSql method doesn’t work with the in-memory provider since it requires a relational database. It is also an extension method which means we can’t simply mock the context using a tool like Moq. We could of course create a wrapper service that calls the FromSql extension method and mock that service but this only shifts the problem. The wrapper approach would allow us to ensure that FromSql is called in the way we expect it to be called but it would not be able to ensure that the query will actually run successfully and return the expected results.

An integration test is a good option here since it will ensure that the query runs exactly as expected against a real SQL Server database.

The Test

I used xunit as the test framework in this example. In the constructor, which is the setup method for any tests in the class, I configure an instance of the MonsterContext connecting to a localdb instance using a database name containing a random guid. Using a guid in the database name ensures the database is unique for this test. Uniqueness is important when running tests in parallel because it ensures these tests won’t impact any other tests that aer currently running. After creating the context, a call to _context.Database.Migrate() creates a new database and applies any Entity Framework migrations that are defined for the MonsterContext.

public class SimpleIntegrationTest : IDisposable
{
MonsterContext _context;
public SimpleIntegrationTest()
{
var serviceProvider = new ServiceCollection()
.AddEntityFrameworkSqlServer()
.BuildServiceProvider();
var builder = new DbContextOptionsBuilder<MonsterContext>();
builder.UseSqlServer($"Server=(localdb)\\mssqllocaldb;Database=monsters_db_{Guid.NewGuid()};Trusted_Connection=True;MultipleActiveResultSets=true")
.UseInternalServiceProvider(serviceProvider);
_context = new MonsterContext(builder.Options);
_context.Database.Migrate();
}
[Fact]
public void QueryMonstersFromSqlTest()
{
//Add some monsters before querying
_context.Monsters.Add(new Monster { Name = "Dave", Colour = "Orange", IsScary = false });
_context.Monsters.Add(new Monster { Name = "Simon", Colour = "Blue", IsScary = false });
_context.Monsters.Add(new Monster { Name = "James", Colour = "Green", IsScary = false });
_context.Monsters.Add(new Monster { Name = "Imposter Monster", Colour = "Red", IsScary = true });
_context.SaveChanges();
//Execute the query
ScaryMonstersQuery query = new ScaryMonstersQuery(_context);
var scaryMonsters = query.Execute();
//Verify the results
Assert.Equal(1, scaryMonsters.Count());
var scaryMonster = scaryMonsters.First();
Assert.Equal("Imposter Monster", scaryMonster.Name);
Assert.Equal("Red", scaryMonster.Colour);
Assert.True(scaryMonster.IsScary);
}
public void Dispose()
{
_context.Database.EnsureDeleted();
}
}

The actual test itself happens in the QueryMonstersFromSqlTest method. I start by adding some sample data to the database. Next, I create and execute the ScaryMonstersQuery using the context that was created in the setup method. Finally, I verify the results, ensuring that the expected data is returned from the query.

The last step is the Dispose method which in xunit is the teardown for any tests in this class. We don’t want all these test databases hanging around forever so this is the place to delete the database that was created in the setup method. The database is deleted by calling _context.Database.EnsureDeleted().

Use with Caution

These tests are slow! The very simple example above takes 13 seconds to run on my laptop. My advice here is to use this sparingly and only when it really adds value for your project. If you end up with a large number of these integration tests, I would consider splitting the integration tests into a separate test suite and potentially running them on a different schedule than my unit test suite (e.g. Nightly instead of every commit).

The Code

You can browse or download the source on GitHub.

The Monsters Weekly - Episode 81 - Invoking View Components as Tag Helpers

A new feature in ASP.NET Core 1.1 allows us to invoke view components as tag helpers. Monster Dave shows us how to enable this new feature.

Related Episode - View Components

Creating a New View Engine in ASP.NET Core

Originally posted to: http://www.davepaquette.com/archive/2016/11/22/creating-a-new-view-engine-in-asp-net-core.aspx

Earlier in November, the ASP.NET Monsters had the opportunity to take part in the ASP.NET Core hackathon at the Microsoft MVP Summit. In past years, we have used the hackathon as an opportunity to spend some time working on GenFu. This year, we wanted to try something a little different.

The Crazy Idea

A few months ago, we had Taylor Mullen on The Monsters Weekly to chat about Razor in ASP.NET Core. At some point during that interview, it was pointed that MVC is designed in a way that a new view engine could easily be plugged into the framework. It was also noted that implementing a view engine is a really big job. This got us to thinking…what if we could find an existing view engine of some sort. How easy would it be to get actually put a new view engine in MVC?

And so, that was our goal for the hackathon. Find a way to replace Razor with an alternate view engine in a single day of hacking.

Finding a Replacement

We wanted to pick something that in no way resembled Razor. Simon suggested Pug (previously known as Jade), a popular view template engine used in Express. In terms of syntax, Pug is about as different from Razor as it possibly could be. Pug uses whitespace to indicate nesting of elements and does away with angle brackets all together. For example, the following template:

div
a(href='google.com') Google

would generate this HTML:

<div>
<a href="google.com">Google</a>
</div>

Calling Pug from ASP.NET Core

The first major hurdle for us was figuring out a way to compile pug templates from within an ASP.NET Core application. Pug is a JavaScript based template engine and we only had a single day to pull this off so a full port of the engine to C# was not feasible.

Our first thought was to use Edgejs to call Pug’s JavaScript compile function. Some quick prototyping showed us that this worked but Edgejs doesn’t have support for .NET Core. This lead us to explore the JavaScriptServices packages created by the ASP.NET Core team. Specifically the Node Services package which allows us to easily call out to a JavaScript module from within an ASP.NET Core application.

To our surpise, this not only worked, it was also easy! We created a very simple file called pugcompile.js.

var pug = require('pug');
module.exports = function (callback, viewPath, model) {
var pugCompiledFunction = pug.compileFile(viewPath);
callback(null, pugCompiledFunction(model));
};

Calling this JavaScript from C# is easy thanks to the Node Services package. Assuming model is the view model we want to bind to the template and mytemplate.pug is the name of the file containing the pug template:

var html = await _nodeServices.InvokeAsync<string>("pugcompile", "mytemplate.pug", model);

Now that we had proven this was possible, it was time to integrate this with MVC by creating a new MVC View Engine.

Creating the Pugzor View Engine

We decided to call our view engine Pugzor which is a combination of Pug and Razor. Of course, this doesn’t really make much sense since our view engine really has nothing to do with Razor but naming is hard and we thought we were being funny.

Keeping in mind our goal of implenting a view engine in a single day, we wanted to do this with the simplest way possible. After spending some time digging through the source code for MVC, we determined that we needed to implement the IViewEngine interface as well as implement a custom IView.

The IViewEngine is responsible for locating a view based on a ActionContext and a ViewName. When a controller returns a View, it is the IViewEngine‘s FindView method that is responsible for finding a view based on some convetions. The FindView method returns a ViewEngineResult which is a simple class containing a boolean Success property indicating whether or not a view was found and an IView View property containing the view if it was found.

/// <summary>
/// Defines the contract for a view engine.
/// </summary>
public interface IViewEngine
{
/// <summary>
/// Finds the view with the given <paramref name="viewName"/> using view locations and information from the
/// <paramref name="context"/>.
/// </summary>
/// <param name="context">The <see cref="ActionContext"/>.</param>
/// <param name="viewName">The name of the view.</param>
/// <param name="isMainPage">Determines if the page being found is the main page for an action.</param>
/// <returns>The <see cref="ViewEngineResult"/> of locating the view.</returns>
ViewEngineResult FindView(ActionContext context, string viewName, bool isMainPage);
/// <summary>
/// Gets the view with the given <paramref name="viewPath"/>, relative to <paramref name="executingFilePath"/>
/// unless <paramref name="viewPath"/> is already absolute.
/// </summary>
/// <param name="executingFilePath">The absolute path to the currently-executing view, if any.</param>
/// <param name="viewPath">The path to the view.</param>
/// <param name="isMainPage">Determines if the page being found is the main page for an action.</param>
/// <returns>The <see cref="ViewEngineResult"/> of locating the view.</returns>
ViewEngineResult GetView(string executingFilePath, string viewPath, bool isMainPage);
}

We decided to use the same view location conventions as Razor. That is, a view is located in Views/{ControllerName}/{ActionName}.pug.

Here is a simplified version of the FindView method for the PugzorViewEngine:

public ViewEngineResult FindView(
ActionContext actionContext,
string viewName,
bool isMainPage)
{
var controllerName = GetNormalizedRouteValue(actionContext, ControllerKey);
var checkedLocations = new List<string>();
foreach (var location in _options.ViewLocationFormats)
{
var view = string.Format(location, viewName, controllerName);
if(File.Exists(view))
return ViewEngineResult.Found("Default", new PugzorView(view, _nodeServices));
checkedLocations.Add(view);
}
return ViewEngineResult.NotFound(viewName, checkedLocations);
}

You can view the complete implentation on GitHub.

Next, we created a class called PugzorView which implements IView. The PugzorView takes in a path to a pug template and an instance of INodeServices. The MVC framework calls the IView‘s RenderAsync when it is wants the view to be rendered. In this method, we call out to pugcompile and then write the resulting HTML out to the view context.

public class PugzorView : IView
{
private string _path;
private INodeServices _nodeServices;
public PugzorView(string path, INodeServices nodeServices)
{
_path = path;
_nodeServices = nodeServices;
}
public string Path
{
get
{
return _path;
}
}
public async Task RenderAsync(ViewContext context)
{
var result = await _nodeServices.InvokeAsync<string>("./pugcompile", Path, context.ViewData.Model);
context.Writer.Write(result);
}
}

The only thing left was to configure MVC to use our new view engine. At first, we thought we could easy add a new view engine using the AddViewOptions extension method when adding MVC to the service collection.

services.AddMvc()
.AddViewOptions(options =>
{
options.ViewEngines.Add(new PugzorViewEngine(nodeServices));
});

This is where we got stuck. We can’t add a concrete instance of the PugzorViewEngine to the ViewEngines collection in the Startup.ConfigureServices method because the view engine needs to take part in dependency injection. The PugzorViewEngine has a dependency on INodeServices and we want that to be injected by ASP.NET Core’s dependency injection framework. Luckily, the all knowning Razor master Taylor Mullen was on hand to show us the right way to register our view engine.

The recommended approach for adding a view engine to MVC is to create a custom setup class that implements IConfigureOptions<MvcViewOptions>. The setup class takes in an instance of our IPugzorViewEngine via constructor injection. In the configure method, that view engine is added to the list of view engines in the MvcViewOptions.

public class PugzorMvcViewOptionsSetup : IConfigureOptions<MvcViewOptions>
{
private readonly IPugzorViewEngine _pugzorViewEngine;
/// <summary>
/// Initializes a new instance of <see cref="PugzorMvcViewOptionsSetup"/>.
/// </summary>
/// <param name="pugzorViewEngine">The <see cref="IPugzorViewEngine"/>.</param>
public PugzorMvcViewOptionsSetup(IPugzorViewEngine pugzorViewEngine)
{
if (pugzorViewEngine == null)
{
throw new ArgumentNullException(nameof(pugzorViewEngine));
}
_pugzorViewEngine = pugzorViewEngine;
}
/// <summary>
/// Configures <paramref name="options"/> to use <see cref="PugzorViewEngine"/>.
/// </summary>
/// <param name="options">The <see cref="MvcViewOptions"/> to configure.</param>
public void Configure(MvcViewOptions options)
{
if (options == null)
{
throw new ArgumentNullException(nameof(options));
}
options.ViewEngines.Add(_pugzorViewEngine);
}
}

Now all we need to do is register the setup class and view engine the Startup.ConfigureServices method.

services.AddTransient<IConfigureOptions<MvcViewOptions>, PugzorMvcViewOptionsSetup>();
services.AddSingleton<IPugzorViewEngine, PugzorViewEngine>();

Like magic, we now have a working view engine. Here’s a simple example:

Controllers/HomeController.cs

public IActionResult Index()
{
ViewData.Add("Title", "Welcome to Pugzor!");
ModelState.AddModelError("model", "An error has occurred");
return View(new { People = A.ListOf<Person>() });
}

Views/Home/Index.pug

block body
h2 Hello
p #{ViewData.title}
table(class='table')
thead
tr
th Name
th Title
th Age
tbody
each val in people
tr
td= val.firstName
td= val.title
td= val.age

Result

<h2>Hello</h2>
<p>Welcome to Pugzor! </p>
<table class="table">
<thead>
<tr>
<th>Name</th>
<th>Title</th>
<th>Age</th>
</tr>
</thead>
<tbody>
<tr><td>Laura</td><td>Mrs.</td><td>38</td></tr>
<tr><td>Gabriel</td><td>Mr. </td><td>62</td></tr>
<tr><td>Judi</td><td>Princess</td><td>44</td></tr>
<tr><td>Isaiah</td><td>Air Marshall</td><td>39</td></tr>
<tr><td>Amber</td><td>Miss.</td><td>69</td></tr>
<tr><td>Jeremy</td><td>Master</td><td>92</td></tr>
<tr><td>Makayla</td><td>Dr.</td><td>15</td></tr>
<tr><td>Sean</td><td>Mr. </td><td>5</td></tr>
<tr><td>Lillian</td><td>Mr. </td><td>3</td></tr>
<tr><td>Brandon</td><td>Doctor</td><td>88</td></tr>
<tr><td>Joel</td><td>Miss.</td><td>12</td></tr>
<tr><td>Madeline</td><td>General</td><td>67</td></tr>
<tr><td>Allison</td><td>Mr. </td><td>21</td></tr>
<tr><td>Brooke</td><td>Dr.</td><td>27</td></tr>
<tr><td>Jonathan</td><td>Air Marshall</td><td>63</td></tr>
<tr><td>Jack</td><td>Mrs.</td><td>7</td></tr>
<tr><td>Tristan</td><td>Doctor</td><td>46</td></tr>
<tr><td>Kandra</td><td>Doctor</td><td>47</td></tr>
<tr><td>Timothy</td><td>Ms.</td><td>83</td></tr>
<tr><td>Milissa</td><td>Dr.</td><td>68</td></tr>
<tr><td>Lekisha</td><td>Mrs.</td><td>40</td></tr>
<tr><td>Connor</td><td>Dr.</td><td>73</td></tr>
<tr><td>Danielle</td><td>Princess</td><td>27</td></tr>
<tr><td>Michelle</td><td>Miss.</td><td>22</td></tr>
<tr><td>Chloe</td><td>Princess</td><td>85</td></tr>
</tbody>
</table>

All the features of pug work as expected, including templage inheritance and inline JavaScript code. Take a look at our test website for some examples.

Packaging it all up

So we reached our goal of creating an alternate view engine for MVC in a single day. We had some time left so we thought we would try to take this one step further and create a NuGet package. There were some challenges here, specifically related to including the required node modules in the NuGet package. Simon is planning to write a separate blog post on that topic.

You can give it a try yourself. Add a reference to the pugzor.core NuGet package then call .AddPugzor() after .AddMvc() in the Startup.ConfigureServices method.

public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddMvc().AddPugzor();
}

Razor still works as the default but if no Razor view is found, the MVC framework will try using the PugzorViewEngine. If a matching pug template is found, that template will be rendered.

Pugzor

Wrapping it up

We had a blast working on this project. While this started out as a silly excercise, we sort of ended up with something that could be useful. We were really surprised at how easy it was to create a new view engine for MVC. We don’t expect that Pugzor will be wildly popular but since it works we thought we would put it out there and see what people think.

We have some open issues and some ideas for how to extend the PugzorViewEngine. Let us know what you think or jump in and contribute some code. We accept pull requests :-)

The Monsters Weekly - Episode 80 - ASP.NET 1.1 was Released...Time to Pump the Brakes?

ASP.NET Core 1.1, .NET Core 1.1, EF Core 1.1 and a multitude of other packages have all been released, but is now the time to adopt?

In this episode Monsters Dave and James talk about the release of ASP.NET 1.1 and how it will affect your projects. They mash on the tooling, upgrading a package and their experiences so far and hint at a few topics that are coming in future episodes.

For additional details on the release, be sure to check out these links:

Please , like and share, or comment and ask questions below.

The Monsters Weekly - Episode 79 - Code Labs with Jon Galloway

If you’re looking for content that will walk you through, step-by-step, as you transition to ASP.NET Core MVC, you’re in for a treat.

In this episode of the Monsters, Jon Gallaway joins us live on set at Channel 9 studios to talk about the .NET Core community, the Community Standups and, of course, a series of free labs that you can dive into to embrace your inner Core and learn more about the future of MVC.

Check out the labs yourself on GitHub:

Please , like and share, or comment and ask questions below.

The Monsters Weekly - Episode 78 - Azure Functions with Chris Anderson

Azure Functions have just gone "GA" and are a great way for you to truly leverage the promise of the cloud and it’s ability to scale. In this episode of the Monsters, we have a special guest from the Functions team to break down some of the basics and help us wrap our heads around the Azure offering. 

Join Chris Anderson and the ASP.NET Monsters at the Channel 9 studios as they chat about Azure Functions.

Check out these links:

C# Wildcard Variables

Originally posted to: http://blog.simontimms.com/2016/11/09/c-wildcardsdiscardsignororators/

There is some great discussion going on about including discard variables in C#, possibly even for the C# 7 timeframe. It is so new that the name for them is still up in the air. In Haskel it is called a wildcard. I think this is a great feature which is found in other languages but isn’t well known for people who haven’t done funcitonal programming. The C# language has been sneaking into being a bit more functional over the last few releases. There is support for lambdas and there has been a bunch of work on immutability. Let’s take a walk through how wildcards works.

Let’s say that we have a function which has a number of output paramaters:

void DoSomething(out List<T> list, out int size){}

Ugh, already I hate this method. I’ve never liked the out syntax because it is wordy. To use this function you would have to do

List<T> list = null;
int size = 0;
DoSomething(out list, out size);

There is some hope for that syntax in C# 7 with what I would have called inline declaration of out variables but is being called “out variables”. The syntax would look like

DoSomething(out List<T> list, out int size);

This is obviously much nicer and you can read a bit more about it at
https://blogs.msdn.microsoft.com/dotnet/2016/08/24/whats-new-in-csharp-7-0/

However in my code base perhaps I don’t care about the size parameter. As it stands right now you still need to declare some variable to hold the size even if it never gets used. For one variable this isn’t a huge pain. I’ve taken to using the underscore to denote that I don’t care about some variable.

DoSomething(out List<T> list, out int _);
//make use of list never reference the _ variable

The issue comes when I have some funciton which takes many parameters I don’t care about.

DoSomething(out List<T> list, out int _, out float __, out decimal ___);
//make use of list never reference the _ variables

This is a huge bit of uglyness because we can’t overload the _ variable so we need to create a bunch more variables. It is even more so ugly if we’re using tuples and a deconstructing declaration (also part of C# 7). Our funciton could be changed to look like

(List<T>, int, float, decimal) DoSomething() {}

This is now a function which returns a tuple containing everything we previously had as out prameters. Then you can break this tuple up using a deconstructing declaration.

(List<T> list, int size, float fidelity, decimal cost) = DoSomething();

This will break up the tuple into the fields you actually want. Except you don’t care about size, fidelity and cost. With a wildcard we can write this as

(List<T> list, int _, float _, decimal _) = DoSomething();

This beauty of this wildcard is that we can use the same wildcard for each field an not worry about them in the least.

I’m really hopeful that this feature will make it to the next release.

The Monsters Weekly - Episode 77 - Internationalization Part 2 - Request Localization

Are you ready for more Internationalization? Monster Dave talks about the Request Localization Middleware in ASP.NET Core and the three built in RequestCultureProviders.

Links:

Internationalization Part 1

ASP.NET Core Docs

The Monsters Weekly - Episode 76 - Curl

Postman is a great tool for exercising HTTP endpoints but sometimes you need a tool you can run in a script or you don’t have access to postman. In that case curl is your friend. In this episode Simon takes us through the basics of curl running on Ubuntu on Windows. 

 

You can read more about curl at 

https://curl.haxx.se/docs/manpage.html

 

The Monsters Weekly - Episode 75 - Internationalization Part 1 - Localizing Text

In today’s episode, Monster Dave takes us through the IStringLocalizer and IViewLocalizer services used to support text localization in an ASP.NET Core MVC application.

Links:https://docs.asp.net/en/latest/fundamentals/localization.html

Be sure to check out the second episode in internationalization too: find it here.