We have a code of conduct here https://confcodeofconduct.com/

Code

We’re going to use a couple of small sample projects during the day. They are on github at

• https://github.com/stimms/ClassStatus
• https://github.com/stimms/ClassStatusUpdater

Slides

There are some slides, some of them have secret commands in them. They are up on one drive here: https://1drv.ms/f/s!AmN_cJhXT0rP1yd-qMdkgcHgvKD0

Class Status

This serves a double purpose. It lets you update your status and it is the demo we’re building today. http://classstatus.aspnetmonsters.com/

After my recent misadventures attempting to use Noda Time with Entity Framework Core, I decided to see what it would take to use Dapper in a the same scenario.

A quick recap

In my app, I needed to model an Event that occurs on a particular date. It might be initially tempting to store the date of the event as a DateTime in UTC, but that’s not necessarily accurate unless the event happens to be held at the Royal Observatory Greenwich. I don’t want to deal with time at all, I’m only interested in the date the event is being held.

NodaTime provides a LocalDate type that is perfect for this scenario so I declared a LocalDate property named Date on my Event class.

public class Event
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
    public LocalDate Date {get; set;}
}

Querying using Dapper

I modified my app to query for the Event entities using Dapper:

var queryDate = new LocalDate(2019, 3, 26);
using (var connection = new SqlConnection(myConnectionString))
{
    await connection.OpenAsync();
    Events = await connection.QueryAsync<Event>(@"SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM [Events] AS[e]");
}

The app started up just fine, but gave me an error when I tried to query for events.

System.Data.DataException: Error parsing column 1 (Date=3/26/19 12:00:00 AM - DateTime) —> System.InvalidCastException: Invalid cast from ‘System.DateTime’ to ‘NodaTime.LocalDate’.

Likewise, if I attempted to query for events using a LocalDate parameter, I got another error:

var queryDate = new LocalDate(2019, 3, 26);
using (var connection = new SqlConnection("myConnectionString"))
{
    await connection.OpenAsync();

    Events = await connection.QueryAsync<Event>(@"SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM [Events] AS[e]
WHERE [e].[Date] = @Date", new { Date = queryDate });
}

NotSupportedException: The member Date of type NodaTime.LocalDate cannot be used as a parameter value

Fortunately, both these problems can be solved by implementing a simple TypeHandler.

Implementing a Custom Type Handler

Out of the box, Dapper already knows how to map to the standard .NET types like Int32, Int64, string and DateTime. The problem we are running into here is that Dapper doesn’t know anything about the LocalDate type. If you want to map to a type that Dapper doesn’t know about, you can implement a custom type handler. To implement a type handler, create a class that inherits from TypeHandler<T>, where T is the type that you want to map to. In your type handler class, implement the Parse and SetValue methods. These methods will be used by Dapper when mapping to and from properties that are of type T.

Here is an example of a type handler for LocalDate.

public class LocalDateTypeHandler : TypeHandler<LocalDate>
{
    public override LocalDate Parse(object value)
    {
        if (value is DateTime)
        {
            return LocalDate.FromDateTime((DateTime)value);
        }

        throw new DataException($"Unable to convert {value} to LocalDate");
    }

    public override void SetValue(IDbDataParameter parameter, LocalDate value)
    {
        parameter.Value = value.ToDateTimeUnspecified();
    }
}

Finally, you need to tell Dapper about your new custom type handler. To do that, register the type handler somewhere in your application’s startup class by calling Dapper.SqlMapper.AddTypeHandler.

Dapper.SqlMapper.AddTypeHandler(new LocalDateTypeHandler());

There’s a NuGet for that

As it turns out, someone has already created a helpful NuGet package containing TypeHandlers for many of the NodaTime types so you probably don’t need to write these yourself. Use the Dapper.NodaTime package instead.

Wrapping it up

TypeHandlers are a simple extension point that allows for Dapper to handle types that are not already handled by Dapper. You can write your own type handlers but you might also want to check if someone has already published a NuGet package that handles your types.

In today’s post, we explore a pattern to prevent multiple users (or processes) from accidentally overwriting each other’s change. Given our current implementation for updating the Aircraft record, there is potential for data loss if there are multiple active sessions are attempting to update the same Aircraft record at the same time. In the example shown below, Bob accidentally overwrites Jane’s changes without even knowing that Jane made changes to the same Aircraft record

The pattern we will use here is Optimistic Offline Lock, which is often also referred to as Optimistic Concurrency Control.

Modifying the Database and Entities

To implement this approach, we will use a rowversion column in SQL Server. Essentially, this is a column that automatically version stamps a row in a table. Any time a row is modified, the rowversion column will is automatically incremented for that row. We will start by adding the column to our Aircraft table.

ALTER TABLE Aircraft ADD RowVer rowversion

Next, we add a RowVer property to the Aircraft table. The property is a byte array. When we read the RowVer column from the database, we will get an array of 8 bytes.

public class Aircraft 
{
    public int Id { get; set; }
    public string Manufacturer {get; set;}
    public string Model {get; set;}
    public string RegistrationNumber {get; set;}
    public int FirstClassCapacity {get; set;}
    public int RegularClassCapacity {get; set;}
    public int CrewCapacity {get; set;}
    public DateTime ManufactureDate {get; set; }
    public int NumberOfEngines {get; set;}
    public int EmptyWeight {get; set;}
    public int MaxTakeoffWeight {get; set;}
    public byte[] RowVer { get; set; }
}   

Finally, we will modify the query used to load Aircraft entities so it returns the RowVer column. We don’t need to change any of the Dapper code here.

public async Task<Aircraft> Get(int id)
{ 
    Aircraft aircraft;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT 
Id
,Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight
,RowVer
FROM Aircraft WHERE Id = @Id";
        aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id});
    }
    return aircraft;
}

Adding the Concurrency Checks

Now that we have the row version loaded in to our model, we need to add the checks to ensure that one user doesn’t accidentally overwrite another users changes. To do this, we simply need to add the RowVer to the WHERE clause on the UPDATE statement. By adding this constraint to the WHERE clause, we we ensure that the updates will only be applied if the RowVer has not changed since this user originally loaded the Aircraft entity.

public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
  ,Model = @Model
  ,RegistrationNumber = @RegistrationNumber 
  ,FirstClassCapacity = @FirstClassCapacity
  ,RegularClassCapacity = @RegularClassCapacity
  ,CrewCapacity = @CrewCapacity
  ,ManufactureDate = @ManufactureDate
  ,NumberOfEngines = @NumberOfEngines
  ,EmptyWeight = @EmptyWeight
  ,MaxTakeoffWeight = @MaxTakeoffWeight
WHERE Id = @Id
      AND RowVer = @RowVer";
      
      await connection.ExecuteAsync(query, model);
    }

    return Ok();
}

So, the WHERE clause stops the update from happening, but how do we know if the update was applied successfully? We need to let the user know that the update was not applied due to a concurrency conflict. To do that, we add OUTPUT inserted.RowVer to the UPDATE statement. The effect of this is that the query will return the new value for the RowVer column if the update was applied. If not, it will return null.

public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    byte[] rowVersion;
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
  ,Model = @Model
  ,RegistrationNumber = @RegistrationNumber 
  ,FirstClassCapacity = @FirstClassCapacity
  ,RegularClassCapacity = @RegularClassCapacity
  ,CrewCapacity = @CrewCapacity
  ,ManufactureDate = @ManufactureDate
  ,NumberOfEngines = @NumberOfEngines
  ,EmptyWeight = @EmptyWeight
  ,MaxTakeoffWeight = @MaxTakeoffWeight
  OUTPUT inserted.RowVer
WHERE Id = @Id
      AND RowVer = @RowVer";
        rowVersion = await connection.ExecuteScalarAsync<byte[]>(query, model);
    }

    if (rowVersion == null) {
        throw new DBConcurrencyException("The entity you were trying to edit has changed. Reload the entity and try again."); 
    }
    return Ok(rowVersion);
}

Instead of calling ExecuteAsync, we call ExecuteScalarAsync<byte[]>. Then we can check if the returned value is null and raise a DBConcurrencyException if it is null. If it is not null, we can return the new RowVer value.

Wrapping it up

Using SQL Server’s rowversion column type makes it easy to implement optimistic concurrency checks in a .NET app that uses Dapper.

If you are building as REST api, you should really use the ETag header to represent the current RowVer for your entity. You can read more about this pattern here.

In today’s post, we explore a more complex scenario that involves executing multiple write operations. In order to ensure consistency at the database level, these operations should all succeed / fail together as a single transaction. In this example, we will be inserting a new ScheduledFlight entity along with an associated set of Flight entities.

As a quick reminder, a Flight represents a particular occurrence of a ScheduledFlight on a particular day. That is, it has a reference to the ScheduledFlight along with some properties indicating the scheduled arrival and departure times.

public class Flight 
{
    public int Id {get; set;}
    public int ScheduledFlightId {get; set;}
    public ScheduledFlight ScheduledFlight { get; set;}
    public DateTime Day {get; set;}
    public DateTime ScheduledDeparture {get; set;}
    public DateTime ScheduledArrival {get; set;}
}

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public int DepartureAirportId {get; set;}
    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public int ArrivalAirportId {get; set;}
    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

    public bool IsSundayFlight {get; set;}
    public bool IsMondayFlight {get; set;}
    // Some other properties
}

Inserting the ScheduledFlight

Inserting the ScheduledFlight and retrieving the database generated id is easy enough. We can use the same approach we used in the previous blog post.

// POST api/scheduledflight
[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int newScheduledFlightId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
        newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model);
    }
    return Ok(newScheduledFlightId);
}

According to the bosses at Air Paquette, whenever we create a new ScheduledFlight entity, we also want to generate the Flight entities for the next 12 months of that ScheduledFlight. We can add a method to the ScheduledFlight class to generate the flight entities.

NOTE: Let’s just ignore the obvious bugs related to timezones and to flights that take off and land on a different day.

public IEnumerable<Flight> GenerateFlights(DateTime startDate, DateTime endDate)
{
    var flights = new List<Flight>();
    var currentDate = startDate;

    while (currentDate <= endDate)
    {
        if (IsOnDayOfWeek(currentDate.DayOfWeek))
        {
            var departureTime = new DateTime(currentDate.Year, currentDate.Month, currentDate.Day, DepartureHour, DepartureMinute, 0);
            var arrivalTime = new DateTime(currentDate.Year, currentDate.Month, currentDate.Day, ArrivalHour, ArrivalMinute, 0);
            var flight = new Flight
            {
                ScheduledFlightId = Id,
                ScheduledDeparture = departureTime,
                ScheduledArrival = arrivalTime,
                Day = currentDate.Date
            };
            flights.Add(flight);
        }
        currentDate = currentDate.AddDays(1);
    }
    return flights;
}
public bool IsOnDayOfWeek(DayOfWeek dayOfWeek)
{
    return     (dayOfWeek == DayOfWeek.Sunday && IsSundayFlight)
            || (dayOfWeek == DayOfWeek.Monday && IsMondayFlight)
            || (dayOfWeek == DayOfWeek.Tuesday && IsTuesdayFlight)
            || (dayOfWeek == DayOfWeek.Wednesday && IsWednesdayFlight)
            || (dayOfWeek == DayOfWeek.Thursday && IsThursdayFlight)
            || (dayOfWeek == DayOfWeek.Friday && IsFridayFlight)
            || (dayOfWeek == DayOfWeek.Saturday && IsSaturdayFlight);
}

Now in the controller, we can add some logic to call the GenerateFlight method and then insert those Flight entities using Dapper.

// POST api/scheduledflight
[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int newScheduledFlightId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
        newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model);

        model.Id = newScheduledFlightId;
        var flights = model.GenerateFlights(DateTime.Now, DateTime.Now.AddMonths(12));

    var insertFlightsSql = @"INSERT INTO [dbo].[Flight]
    ([ScheduledFlightId]
    ,[Day]
    ,[ScheduledDeparture]
    ,[ActualDeparture]
    ,[ScheduledArrival]
    ,[ActualArrival])
VALUES
    (@ScheduledFlightId
    ,@Day
    ,@ScheduledDeparture
    ,@ActualDeparture
    ,@ScheduledArrival
    ,@ActualArrival)";

        await connection.ExecuteAsync(insertFlightsSql, flights);

    }
    return Ok(newScheduledFlightId);
}

Note that we passed in an IEnumerable<Flight> as the second argument to the ExecuteAsync method. This is a handy shortcut in Dapper for executing a query multiple times. Instead of writing a loop and calling ExecuteAsync for each flight entity, we can pass in a list of flights and Dapper will execute the query once for each item in the list.

Explicitly managing a transaction

So far, we have code that first inserts a ScheduledFlight, next generates a set of Flight entities and finally inserting all of those Flight entities. That’s the happy path, but what happens if something goes wrong along the way. Typically when we execute a set of related write operations (inserts, updates and deletes), we want those operations to all succeed or fail together. In the database world, we have transactions to help us with this.

The nice thing about using Dapper is that it uses standard .NET database connections and transactions. There is no need to re-invent the wheel here, we can simply use the transaction patterns that have been around in .NET since for nearly 2 decades now.

After opening the connection, we call connection.BeginTransaction() to start a new transaction. Whenever we call ExecuteAsync (or any other Dapper extension method), we need to pass in that transaction. At the end of all that work, we call transaction.Commit(). Finally, we wrap the logic in a try / catch block. If any exception is raised, we call transaction.Rollback() to ensure that none of those write operations are committed to the database.

[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int? newScheduledFlightId = null;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var transaction = connection.BeginTransaction();

        try
        {
            var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
            newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model, transaction);

            model.Id = newScheduledFlightId.Value;
            var flights = model.GenerateFlights(DateTime.Now, DateTime.Now.AddMonths(12));

            var insertFlightsSql = @"INSERT INTO [dbo].[Flight]
    ([ScheduledFlightId]
    ,[Day]
    ,[ScheduledDeparture]
    ,[ActualDeparture]
    ,[ScheduledArrival]
    ,[ActualArrival])
VALUES
    (@ScheduledFlightId
    ,@Day
    ,@ScheduledDeparture
    ,@ActualDeparture
    ,@ScheduledArrival
    ,@ActualArrival)";

            await connection.ExecuteAsync(insertFlightsSql, flights, transaction);
            transaction.Commit();
        }
        catch (Exception ex)
        { 
            //Log the exception (ex)
            try
            {
                transaction.Rollback();
            }
            catch (Exception ex2)
            {
                // Handle any errors that may have occurred
                // on the server that would cause the rollback to fail, such as
                // a closed connection.
                // Log the exception ex2
            }
            return StatusCode(500);
        }
    }
    return Ok(newScheduledFlightId);
}

Managing database transactions in .NET is a deep but well understood topic. We covered the basic pattern above and showed how Dapper can easily participate in a transaction. To learn more about managing database transactions in .NET, check out these docs:

• SqlConnection.BeginTransaction
• Transactions in ADO.NET

Wrapping it up

Using transactions with Dapper is fairly straight forward process. We just need to tell Dapper what transaction to use when executing queries. Now that we know how to use transactions, we can look at some more advanced scenarios like adding concurrency checks to update operations to ensure users aren’t overwriting each other’s changes.

In today’s post, we explore how easy it is to perform basic Insert, Update and Delete operations using the same Aircraft entity that we used in the first post in this series. Basically, instead of using Dapper’s QueryAsync extension method that we used to retrieve data, we will use the ExecuteAsync method.

As a quick reminder, here is the Aircraft class:

public class Aircraft 
{
    public int Id { get; set; }

    public string Manufacturer {get; set;}

    public string Model {get; set;}

    public string RegistrationNumber {get; set;}

    public int FirstClassCapacity {get; set;}

    public int RegularClassCapacity {get; set;}

    public int CrewCapacity {get; set;}

    public DateTime ManufactureDate {get; set; }

    public int NumberOfEngines {get; set;}
}   

NOTE: In these examples, I am ignoring some important aspects like validation. I want to focus specifically on the Dapper bits here but validation is really important. In a real-world scenario, you should be validating any data that is passed in to the server. I recommend using Fluent Validation.

Insert

Inserting a single new record is really easy. All we need to do is write an INSERT statement with parameters for each column that we want to set.

[HttpPost()]
public async Task<IActionResult> Post([FromBody] Aircraft model)
{
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
INSERT INTO Aircraft 
(Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight)
VALUES (@Manufacturer
,@Model
,@RegistrationNumber
,@FirstClassCapacity
,@RegularClassCapacity
,@CrewCapacity
,@ManufactureDate
,@NumberOfEngines
,@EmptyWeight
,@MaxTakeoffWeight)";
        await connection.ExecuteAsync(sqlStatement, model);
    }
    return Ok();
}

The version of the ExecuteAsync method we used here accepts two parameters: a string containing the SQL statement to execute and an object containing the parameter values to bind to the statement. In this case, it is an instance of the Aircraft class which has properties with names matching the parameters defined in the INSERT statement.

Our Aircraft table’s Id column is an auto-incremented identity column. That means the primary key is generated by the database when the row is inserted. We will likely need to pass that value back to whoever called the API so they know how to retrieve the newly inserted Aircraft.

An easy way to get the generated Id is to add SELECT CAST(SCOPE_IDENTITY() as int) after the INSERT statement. The SCOPE_IDENTITY() function returns the last identity value that was generated in any table in the current session and current scope.

Now, since the SQL statement we are executing will be returning a single value (the generated id), we need to call ExecuteScalarAsync<int>. The ExecuteScalarAsync method executes a SQL statement that returns a single value whereas the ExecuteAsync method executes a SQL statement that does not return a value.

[HttpPost()]
public async Task<IActionResult> Post([FromBody] Aircraft model)
{
    int newAircraftId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
INSERT INTO Aircraft 
(Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight)
VALUES (@Manufacturer
,@Model
,@RegistrationNumber
,@FirstClassCapacity
,@RegularClassCapacity
,@CrewCapacity
,@ManufactureDate
,@NumberOfEngines
,@EmptyWeight
,@MaxTakeoffWeight);

SELECT CAST(SCOPE_IDENTITY() as int)";
        newAircraftId = await connection.ExecuteScalarAsync<int>(sqlStatement, model);
    }
    return Ok(newAircraftId);
}

Update

Updating an existing entity is similar to inserting. All we need is a SQL statement containing an UPDATE statement that sets the appropriate columns. We also want to make sure we include a WHERE clause limiting the update only to the row with the specified Id.

Again, the parameters in the SQL statement match the names of the properties in our Aircraft class. All we need to do is call the ExecuteAsync method passing in the SQL statement and the Aircraft entity.

// PUT api/aircraft/id
[HttpPut("{id}")]
public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
,Model = @Model
,RegistrationNumber = @RegistrationNumber 
,FirstClassCapacity = @FirstClassCapacity
,RegularClassCapacity = @RegularClassCapacity
,CrewCapacity = @CrewCapacity
,ManufactureDate = @ManufactureDate
,NumberOfEngines = @NumberOfEngines
,EmptyWeight = @EmptyWeight
,MaxTakeoffWeight = @MaxTakeoffWeight
WHERE Id = @Id";
        await connection.ExecuteAsync(sqlStatement, model);
    }
    return Ok();
}

Delete

Deleting an entity is the easiest of the three operations since it only requires a single parameter: the unique Id to identify the entity being deleted. The SQL statement is a simple DELETE with a WHERE clause on the Id column. To execute the delete, call the ExecuteAsync method passing in the SQL statement and an anonymous object containing the Id to delete.

// DELETE api/aircraft/id
[HttpDelete("{id}")]
public async Task<IActionResult> Delete(int id)
{

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = "DELETE Aircraft WHERE Id = @Id";
        await connection.ExecuteAsync(sqlStatement, new {Id = id});
    }
    return Ok();
}

I really appreciate how simple delete is using Dapper. When using Entity Framework, delete requires you to first fetch the existing entity, then delete it. That requires 2 round trips to the database while the approach we used here only requires a single round trip.

Wrapping it up

Basic insert, update and delete operations are easy to implement using Dapper. Real world scenarios are often a little more complex and we will dig into some of those scenarios in future posts:

• Bulk inserts, updates and deletes
• Managing transactions
• Optimistic concurrency checks

In today’s post, we explore paging through large result sets. Paging is a common technique that is used when dealing with large results sets. Typically, it is not useful for an application to request millions of records at a time because there is no efficient way to deal with all those records in memory all at once. This is especially true when rendering data on a grid in a user interface. The screen can only display a limited number of records at a time so it is generally a bad use of system resources to hold everything in memory when only a small subset of those records can be displayed at any given time.

Source: AppStack Bootstrap Template

Modern versions of SQL Server support the OFFSET / FETCH clause to implement query paging.

In continuing with our airline theme, consider a Flight entity. A Flight represents a particular occurrence of a ScheduledFlight on a particular day. That is, it has a reference to the ScheduledFlight along with some properties indicating the scheduled arrival and departure times.

public class Flight 
{
    public int Id {get; set;}
    public int ScheduledFlightId {get; set;}
    public ScheduledFlight ScheduledFlight { get; set;}
    public DateTime Day {get; set;}
    public DateTime ScheduledDeparture {get; set;}
    public DateTime ScheduledArrival {get; set;}
}

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public int DepartureAirportId {get; set;}
    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public int ArrivalAirportId {get; set;}
    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

    public bool IsSundayFlight {get; set;}
    public bool IsMondayFlight {get; set;}
    // Some other properties
}

Writing the query

As we learned in a previous post, we can load the Flight entity along with it’s related ScheduledFlight entity using a technique called multi-mapping.

In this case, loading all the flights to or from a particular airport, we would use the following query.

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode

But this query could yield more results than we want to deal with at any given time. Using OFFSET/FETCH, we can ask for only a block of results at a time.

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY

Note that an ORDER BY clause is required when using OFFSET/FETCH.

Executing the Query

//GET api/flights
[HttpGet]
public async Task<IEnumerable<Flight>> Get(string airportCode, int page=1, int pageSize=10)
{
    IEnumerable<Flight> results;

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;
";

        results = await connection.QueryAsync<Flight, ScheduledFlight, Flight>(query,
          (f, sf) =>
              {
                  f.ScheduledFlight = sf;
                  return f;
              },
            new { AirportCode = airportCode,
                              Offset = (page - 1) * pageSize,
                              PageSize = pageSize }
            );        
    }

    return results;
}

Here we calculate the offset by based on the page and pageSize arguments that were passed in. This allows the caller of the API to request a particular number of rows and the starting point.

One step further

When dealing with paged result sets, it can be useful for the caller of the API to also know the total number of records. Without the total number of records, it would be difficult to know how many records are remaining which in turn makes it difficult to render a paging control, a progress bar or a scroll bar (depending on the use case).

A technique I like to use here is to have my API return a PagedResults<T> class that contains the list of items for the current page along with the total count.

public class PagedResults<T>
{
    public IEnumerable<T> Items { get; set; }
    public int TotalCount { get; set; }
}

To populate this using Dapper, we can add a second result set to the query. That second result set will simply be a count of all the records. Note that the same WHERE clause is used in both queries.

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;

SELECT COUNT(*)
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode

Now in our code that executes the query, we will the QueryMultipleAsync method to execute both SQL statements in a single round trip.

//GET api/flights
[HttpGet]
public async Task<PagedResults<Flight>> Get(string airportCode, int page=1, int pageSize=10)
{
    var results = new PagedResults<Flight>();

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;

SELECT COUNT(*)
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
";

        using (var multi = await connection.QueryMultipleAsync(query,
                    new { AirportCode = airportCode,
                          Offset = (page - 1) * pageSize,
                          PageSize = pageSize }))
        {
            results.Items = multi.Read<Flight, ScheduledFlight, Flight>((f, sf) =>
                {
                    f.ScheduledFlight = sf;
                    return f;
                }).ToList();

            results.TotalCount = multi.ReadFirst<int>();
        }
    }

    return results;
}
    

Wrapping it up

Paged result sets is an important technique when dealing with large amounts of data. When using a full ORM like Entity Framework, this is implemented easily using LINQ’s Skip and Take methods. It’s so easy in fact that it can look a little like magic. In reality, it is actually very simple to write your own queries to support paged result sets and execute those queries using Dapper.

The default MVC project templates already include jQuery, so you might use that. You’ll probably end up writing a lot of code if you go down that path. Chances are, you will want to use a JavaScript framework that offers two-way data binding between the elements in the DOM to a your model data.

I seems that for many people, Knockout.js is the default library to use in these scenarios. I won’t get into the specifics but I think that Knockout is a little dated and that there are better options these days. If you want to dig into some of the issues with Knockout, you can read Simon Timms’ rant on the subject.

Vue.js

My fellow ASP.NET Monster James Chambers recently strongly recommended I take a look at Vue.js. I had been meaning to give Vue a try for some time now and I finally had a chance to use it on a recent project. Let me tell you…I love it.

I love it for a whole bunch of reasons. The documentation is great! It is super easy to get drop in to your existing project and it doesn’t get in the way. For what I needed to do, it just did the job and allowed me to get on with my day. It is also designed to be “incrementally adoptable”, which means you can start out with just using the core view layer, then start pulling in other things like routing and state management if/when you need them.

A simple example

I won’t go into great detail about how to use Vue. If you want a full tutorial, head on over to the Vue docs. What I want to show here is just how simple it is to drop Vue into an existing ASP.NET MVC project and add a bit of client side functionality.

The simplest example I can think of is a set of cascading dropdowns. Let’s consider a form where a user is asked to enter their Country / Province. When the Country is selected, we would expect the Province dropdown to only display the valid Provinces/States for the selected Country. That probably involves a call to an HTTP endpoint that will return the list of valid values.

public class ProvinceLookupController : Controller
{
    public ActionResult Index(string countryCode)
    {
        var provinces = ProvinceLookupService.GetProvinces(countryCode);
        return Json(provinces, JsonRequestBehavior.AllowGet);
    }
}

Including Vue in your Razor (cshtml) view

The easiest way to include Vue on a particular Razor view is to link to the vue.js file from a CDN. You can add the following Script tag to your scripts section.

@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>
}

Be sure to check the docs to make sure you are referencing the latest version of Vue.

Binding data to the our View

Now that you have included the core Vue library, you can start using Vue to bind DOM elements to model data.

Start by defining a Vue object in JavaScript. You can add this in a new <script> tag in your scripts section.

@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>

    <script type="text/javascript">
        var app = new Vue({
            el: '#vueApp',
            data: {
                selectedCountryCode: null,
                countries: [
                    { code: 'ca', name: 'Canada' },
                    { code: 'us', name: 'United States' }
                ]            
            }
        });
    </script>
}

This Vue object targets the DOM element with id vueApp and contains some simple data. The currently selected country code and the list of countries.

Now, back in the HTML part of your csthml, wrap the form in a div that has an id="vueApp".

<div id="vueApp">
    <!-- your form -->
</div>

Next, bind the <select> element to the data in your Vue object. In Vue, data binding is done using a combination of custom attributes that start with v- and the double curly bracket (aka. Mustache) syntax for text.

<div class="form-group">
    @Html.LabelFor(model => model.CountryCode, new { @class = "control-label col-md-2" })
    <div class="col-md-10">
        <select id="CountryCode" name="CountryCode" class="form-control" 
                v-model="selectedCountryCode">
            <option v-for="country in countries" v-bind:value="country.code">
                {{ country.name }}
            </option>
        </select>
    </div>
</div>

Now, when you run the app, you should see a dropdown containing Canada and United States.

Adding functionality

Next, you will want to add some client side logic to get the list of valid provinces from the server whenever the selected country changes.

First, add an empty provinces array and a selectedProvinceCode property to the Vue object’s data.

Next, add a method called countryChanged to the Vue object. This method will call the ProvinceLookup action method on the server, passing in the selectedCountryCode as a parameter. Assign the response data to the provinces array.

var app = new Vue({
    el: '#vueApp',
    data: {
        selectedCountryCode: null,
        countries: [
            { code: 'ca', name: 'Canada' },
            { code: 'us', name: 'United States' }
        ],
        selectedProvinceCode: null,
        provinces: []
    },
    methods: {
        countryChanged: function () {
            $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                this.provinces = data;
            }.bind(this));
        }
    }
});

Here I used jQuery to make the call to the server. In the Vue community, Axios is a popular library for making HTTP requests.

Back in the HTML, bind the change event from the country select element to the countryChanged method using the v-on:change attribute.

<select id="CountryCode" name="CountryCode" class="form-control" 
        v-model="selectedCountryCode" v-on:change="countryChanged">
    <option v-for="country in countries" v-bind:value="country.code">
        {{country.name}}
    </option>
</select>

Now you can add a select element for the provinces.

<div class="form-group">
    @Html.LabelFor(model => model.ProvinceCode, new { @class = "control-label col-md-2" })
    <div class="col-md-10">
        <select id="ProvinceCode" name="ProvinceCode" class="form-control"
                v-model="selectedProvinceCode">
            <option v-for="province in provinces" v-bind:value="province.Code">
                {{province.Name}}
            </option>
        </select>
    </div>
</div>

Voila! You now have a working set of cascading dropdowns.

One last thing

You might want to disable the provinces dropdown whenever a request is being made to get the list of provinces for the selected country. You can do this by adding an isProvincesLoading property to the Vue object’s data, then setting that property in the countryChanged method.

var app = new Vue({
    el: '#vueApp',
    data: {
        selectedCountryCode: null,
        countries: [
            { code: 'ca', name: 'Canada' },
            { code: 'us', name: 'United States' }
        ],
        selectedProvinceCode: null,
        provinces: [],
        isProvincesLoading: false
    },
    methods: {
        countryChanged: function () {
            this.isProvincesLoading = true;
            $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                this.provinces = data;
                this.isProvincesLoading = false;
            }.bind(this));
        }
    }
});

In your HTML, bind the disabled attribute to the isProvincesLoading property.

<div class="form-group">
<select id="ProvinceCode" name="ProvinceCode" class="form-control"
        v-model="selectedProvinceCode"
        v-bind:disabled="isProvincesLoading">
    <option v-for="province in provinces" v-bind:value="province.Code">
        {{province.Name}}
    </option>
</select>

Putting it all together

Here is the entire cshtml file.

@{
    ViewBag.Title = "Location Settings";
}
@model Mvc5VueJsExample.Models.LocationSettingsModel

<h2>@ViewBag.Title.</h2>
<h3>@ViewBag.Message</h3>

<div id="vueApp">
    @using (Html.BeginForm("LocationSettings", "Home", FormMethod.Post, new { @class = "form" }))
    {
        <div class="form-group">
            @Html.LabelFor(model => model.CountryCode, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                <select id="CountryCode" name="CountryCode" class="form-control" 
                        v-model="selectedCountryCode" v-on:change="countryChanged">
                    <option v-for="country in countries" v-bind:value="country.code">
                        {{ country.name}}
                    </option>
                </select>
            </div>
        </div>

        <div class="form-group">
            @Html.LabelFor(model => model.ProvinceCode, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                <select id="ProvinceCode" name="ProvinceCode" class="form-control"
                        v-model="selectedProvinceCode"
                        v-bind:disabled="isProvincesLoading">
                    <option v-for="province in provinces" v-bind:value="province.Code">
                        {{province.Name}}
                    </option>
                </select>
            </div>
        </div>
        <button class="btn btn-primary" type="submit">Save</button>
    }

</div>
@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>

    <script type="text/javascript">
        var app = new Vue({
            el: '#vueApp',
            data: {
                selectedCountryCode: null,
                countries: [
                    { code: 'ca', name: 'Canada' },
                    { code: 'us', name: 'United States' }
                ],
                selectedProvinceCode: null,
                provinces: [],
                isProvincesLoading: false
            },
            methods: {
                countryChanged: function () {
                    this.isProvincesLoading = true;
                    $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                        this.provinces = data;
                        this.isProvincesLoading = false;
                    }.bind(this));
                }
            }
        });
    </script>
}

Wrapping it up

I hope this gives you a taste for how easy it is to work with Vue. My current thinking is that Vue should be the default choice for client side frameworks in existing ASP.NET MVC apps.

NOTE: This example uses ASP.NET MVC 5 to illustrate that Vue can be used with brownfield applications. It would be just as easy, if not easier, to use Vue in an ASP.NET Core project.

There were quite a few changes to the JavaScript interop APIs since we last published the package - all of them really good. However they were, as you get with alpha software, breaking changes. We now do a better of of serializing payloads too.

Usage

1) In your Blazor app, add the AspNetMonsters.Blazor.Geolocation NuGet package

Install-Package AspNetMonsters.Blazor.Geolocation -IncludePrerelease

1) In your Blazor app’s Startup.cs, register the ‘LocationService’.

public void ConfigureServices(IServiceCollection services)
{
    ...
    services.AddSingleton<LocationService>();
    ...
}

1) Now you can inject the LocationService into any Blazor page and use it like this:

@using AspNetMonsters.Blazor.Geolocation
@inject LocationService  LocationService
<h3>You are here</h3>
<div>
Lat: @location?.Latitude <br/>
Long: @location?.Longitude <br />
Accuracy: @location?.Accuracy <br />
</div>

@functions
{
    Location location;

    protected override async Task OnInitAsync()
    {
        location = await LocationService.GetLocationAsync();
    }
}

I wanted to install the site extension that enables Application Insights Monitoring of a live website. After digging into existing arm templates, I found the name of that extension is Microsoft.ApplicationInsights.AzureWebSites.

After searching for way too long, I eventually found PowerShell command I needed on a forum somewhere. I can’t find it again so I’m posting this here in hopes that it will be easier for others to find in the future.

Installing a site extension to an existing App Service Web App

New-AzureRmResource -ResourceType "Microsoft.Web/sites/siteextensions" -ResourceGroupName MyResourceGroup -Name "MyWebApp/SiteExtensionName" -ApiVersion "2018-02-01" -Force

For example, given a resource group named Test, a web app named testsite and a site extension named Microsoft.ApplicationInsights.AzureWebSites.

New-AzureRmResource -ResourceType "Microsoft.Web/sites/siteextensions" -ResourceGroupName "Test" -Name "testsite/Microsoft.ApplicationInsights.AzureWebSites" -ApiVersion "2018-02-01" -Force

Installing a site extension to a Web App Deployment Slot

The scenario I ran into was actually attempting to add this site extension to a deployment slot. When you create a deployment slot, it doesn’t copy over any existing site extensions, which is a problem because when you swap your new slot to production, your new production slot ends up losing the site extensions that were in the old production slot.

New-AzureRmResource -ResourceType "Microsoft.Web/sites/slots/siteextensions" -ResourceGroupName MyResourceGroup -Name "MyWebApp/SlotName/SiteExtensionName" -ApiVersion "2018-02-01" -Force

Using the same example as above and a slot named Staging:

New-AzureRmResource -ResourceType "Microsoft.Web/sites/slots/siteextensions" -ResourceGroupName "Test" -Name "testsite/Staging/Microsoft.ApplicationInsights.AzureWebSites" -ApiVersion "2018-02-01" -Force

Update: April 16, 2018 Something really cool happened in the comments. The amazing Phil Bolduc very kindly pointed out that the query I wrote was not optimal and as a result, my benchmarks were not showing the best results. He didn’t stop there, he also submitted a pull request to the sample repo so I could rerun my benchmarks. Great job Phil and thanks a ton for being constructive in the comments section! I have updated the post to include Phil’s superior query.

In today’s post, we look at another option for how to load Many-to-One relationships. In the last post, we used a technique called Multi-Mapping to load related Many-to-One entities. In that post, I had a theory that maybe this approach was not the most efficient method for loading related entities because it duplicated a lot of data.

To recap, we would like to load a list of ScheduledFlight entities. A ScheduleFlight has a departure Airport and an arrival Airport.

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

   //Other properties omitted for brevity 
}

public class Airport 
{
    public int Id {get; set;}
    public string Code {get; set;}
    public string City {get; set;}
    public string ProvinceState {get; set;}
    public string Country {get; set;}
}

Using Multiple Result Sets

In the previous post, we loaded the ScheduledFlight entities and all related Airport entities in a single query. In this example we will use 2 separate queries: One for the ScheduledFlight entities, one for the related arrival and departure Airport entities. These 2 queries will all be executed as a single sql command that returns multiple result sets.

SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
s.DepartureAirportId, s.ArrivalAirportId
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
    WHERE a1.Code = @FromCode
    
SELECT Airport.Id, Airport.Code, Airport.City, Airport.ProvinceState, Airport.Country
FROM Airport
  WHERE Airport.Id = @DepartureAirportId
    OR Airport.Id IN (SELECT s.ArrivalAirportId
    FROM ScheduledFlight s
WHERE s.DepartureAirportId = @DepartureAirportId)

Using Dapper’s QueryMultipleAsync method, we pass in 2 arguments: the query and the parameters for the query.

public async Task<IEnumerable<ScheduledFlight>> GetAlt(string from)
{
  IEnumerable<ScheduledFlight> scheduledFlights;
  using (var connection = new SqlConnection(_connectionString))
  {
  await connection.OpenAsync();
  var query = @"
SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
s.DepartureAirportId, s.ArrivalAirportId
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
    WHERE a1.Code = @FromCode
    
SELECT Airport.Id, Airport.Code, Airport.City, Airport.ProvinceState, Airport.Country
FROM Airport
WHERE Airport.Id = @DepartureAirportId
  OR Airport.Id IN (SELECT s.ArrivalAirportId
       FROM ScheduledFlight s
   WHERE s.DepartureAirportId = @DepartureAirportId)";

    using (var multi = await connection.QueryMultipleAsync(query, new{FromCode = from} ))
    {
        scheduledFlights = multi.Read<ScheduledFlight>();
        var airports = multi.Read<Airport>().ToDictionary(a => a.Id);
        foreach(var flight in scheduledFlights)
        {
            flight.ArrivalAirport = airports[flight.ArrivalAirportId];
            flight.DepartureAirport = airports[flight.DepartureAirportId];
        }

    }
  }
  return scheduledFlights;
}

The QueryMultipleAsync method returns a GridReader. The GridReader makes it very easy to map mutliple result sets to different objects using the Read<T> method. When you call the Read<T> method, it will read all the results from the next result set that was returned by the query. In our case, we call Read<ScheduledFlight> to read the first result set and map the results into a collection of ScheduledFlight entities. Next, we call Read<Airport> to read the second result set. We then call ToDictionary(a => a.Id) to populate those Airport entities into a dictionary. This is to make it easier to read the results when setting the ArrivalAirport and DepartureAirport properties for each ScheduledFlight.

Finally, we iterate through the scheduled flights and set the ArrivalAirport and DepartureAirport properties to the correct Airport entity.

The big difference between this approach and the previous approach is that we no longer have duplicate instances for Airport entities. For example, if the query returned 100 scheduled flights departing from Calgary (YYC), there would be a single instance of the Airport entity representing YYC, whereas the previous approach would have resulted in 100 separate instances of the Airport entity.

There is also less raw data returned by the query itself since the columns from the Airport table are not repeated in each row from the ScheduleFlight table.

Comparing Performance

I had a theory that the multi-mapping approach outlined in the previous blog post would be less efficient than the multiple result set approach outlined in this blog post, at least from a memory usage perspective. However, a theory is just theory until it is tested. I was curious and also wanted to make sure I wasn’t misleading anyone so I decided to test things out using Benchmark.NET. Using Benchmark.NET, I compared both methods using different sizes of data sets.

I won’t get into the details of Benchmark.NET. If you want to dig into it in more detail, visit the official site and read through the docs. For the purposes of this blog post, the following legend should suffice:

Mean      : Arithmetic mean of all measurements
Error     : Half of 99.9% confidence interval
StdDev    : Standard deviation of all measurements
Gen 0     : GC Generation 0 collects per 1k Operations
Gen 1     : GC Generation 1 collects per 1k Operations
Gen 2     : GC Generation 2 collects per 1k Operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)

10 ScheduledFlight records

As I suspected, the difference is minimal when dealing with small result sets. The results here are in microseconds so in both cases, executing the query and mapping the results takes less 1/2 a millisecond. The mutliple result sets approach takes a little longer, which I kind of expected because of the overhead of creating a dictionary and doing lookups into that dictionary when setting the ArrivalAirport and DepartureAirport properties. The difference is minimal and in a most real world scenarios, this won’t be noticable. What is interesting is that even with this small amount of data, we can see that there is ~1 more Gen 0 garbage collection happening per 1,000 operations. I suspect we will see this creep up as the amount of data increases.

100 ScheduledFlight records

When mapping 100 results, the multiple result sets query is already almost 25% faster. Keep in mind though that both cases are still completing in less than 1ms so this is very much still a micro optimization (pun intented). Either way, less than a millsecond to map 100 records is crazy fast.

1000 ScheduledFlight records

Here we go! Now the multiple result sets approach finally wins out, and you can see why. There are way more Gen 0 and Gen 1 garbage collections happening per 1,000 operations when using the multi-mapping approach. As a result, the multiple result sets approach is nearly twice as fast as the multi mapping approach.

10,000 ScheduledFlight records

One last test with 10,000 records shows a more substantial difference. The multiple result sets approach is a full 22ms faster!

Wrapping it up

I think that in most realistic scenarios, there is no discernable difference between the 2 approaches to loading many-to-one related entities. If you loading larger amounts of records into memory in a single query, then the multiple result sets approach will likely give you better performance. If you are dealing with < 100 records per query, then you likely won’t notice a difference. Keep in mind also that your results will vary depending on the specific data you are loading.

In today’s post, we will start our journey into more complex query scenarios by exploring how to load related entities. There are a few different scenarios to cover here. In this post we will be covering the Many-to-One scenario.

Continuing with our sample domain for the ever expanding Air Paquette airline, we will now look at loading a list of ScheduledFlight entities. A ScheduleFlight has a departure Airport and an arrival Airport.

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

   //Other properties omitted for brevity 
}

public class Airport 
{
    public int Id {get; set;}
    public string Code {get; set;}
    public string City {get; set;}
    public string ProvinceState {get; set;}
    public string Country {get; set;}
}

Side Note: Let’s ignore my poor representation of the arrival and departure times of the scheduled flights. In a future most we might look using Noda Time to properly represent these values.

Loading everything in a single query

Using Dapper, we can easily load a list of ScheduledFlight using a single query. First, we need to craft a query that returns all the columns for a ScheduledFlight, the departure Airport and the arrival Airport in a single row.

SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
       a1.Id, a1.Code, a1.City, a1.ProvinceState, a1.Country,
   a2.Id, a2.Code, a2.City, a2.ProvinceState, a2.Country
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
INNER JOIN Airport a2
ON s.ArrivalAirportId = a2.Id

We use the QueryAsync method to load a list of ScheduledFlight entities along with their related DepartureAirport and ArrivalAirport entities. The parameters we pass in are a little different from what we saw in our previous posts.

[HttpGet]
public async Task<IEnumerable<ScheduledFlight>> Get(string from)
{
    IEnumerable<ScheduledFlight> scheduledFlights;

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();

            var query = @"
SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
     a1.Id, a1.Code, a1.City, a1.ProvinceState, a1.Country,
     a2.Id, a2.Code, a2.City, a2.ProvinceState, a2.Country
FROM ScheduledFlight s
     INNER JOIN Airport a1
          ON s.DepartureAirportId = a1.Id
    INNER JOIN Airport a2
          ON s.ArrivalAirportId = a2.Id
WHERE a1.Code = @FromCode";

        scheduledFlights = 
            await connection.QueryAsync<ScheduledFlight, Airport, Airport, ScheduledFlight>(query,
                    (flight, departure, arrival ) => {
                        flight.DepartureAirport = departure;
                        flight.ArrivalAirport = arrival;
                        return flight;
                    },
                    new{FromCode = from} );
    }
    return scheduledFlights;
}

First, instead of a single type parameter <ScheduledFlight>, we need to provide a series of type parameters: <ScheduledFlight, Airport, Airport, ScheduledFlight>. The first 3 parameters specify the types that are contained in each row that the query returns. In this example, each row contains columns that will be mapped to ScheduledFlight and 2 Airports. The order matters here, and Dapper assumes that when it seems a column named Id then it is looking at columns for the next entity type. In the example below, the columns from Id to IsSaturdayFlight are mapped to a ScheduledFlight entity. The next 5 columns Id, Code, City, ProvinceState, Country are mapped to an Airport entity, and the last 5 columns are mapped to a second Airport entity. If you aren’t using Id, you can use the optional splitOn argument to specify the column names that Dapper should use to identity the start of each entity type.

What’s that last type parameter? Why do we need to specify ScheduledFlight again? Well, I’m glad you asked. The thing about Dapper is that it doesn’t actually know much about the structure of our entities so we need to tell it how to wire up the 3 entities that it just mapped from a row. That last ScheduledFlight type parameter is telling Dapper that ScheduledFlight is ultimately the entity we want to return from this query. It is important for the second argument that is passed to the QueryAsync method.

That second argument is a function that takes in the 3 entities that were mapped back from that row and returns and entity of the type that was specified as the last type parameter. In this case, we assign the first Airport to the flight’s DepartureAirport property and assign the second Airport to the flight’s ArrivalAiport parameter, then we return the flight that was passed in.

(flight, departure, arrival ) => {
    flight.DepartureAirport = departure;
    flight.ArrivalAirport = arrival;
    return flight;
}

The first argument argument passed to the QueryAsync method is the SQL query, and the third argument is an anonymous object containing any parameters for that query. Those arguments are really no different than the simple examples we saw in previous blog posts.

Wrapping it up

Dapper refers to this technique as Multi Mapping. I think it’s called that because we are mapping multiple entities from each row that the query returns. In a fully featured ORM like Entity Framework, we call this feature Eager Loading. It is an optimization technique that avoids the need for multiple queries in order to load an entity and it’s associated entities.

This approach is simple enough to use and it does reduce the number of round trips needed to load a set of entities. It does, however, come at a cost. Specifically, the results of the query end up causing some duplication of data. As you can see below, the data for the Calgary and Vancouver airports is repeated in each row.

This isn’t a huge problem if the result set only contains 3 rows but it can become problematic when dealing with large result sets. In addition to creating somewhat bloated result sets, Dapper will also create new instances of those related entities for each row in the result set. In the example above, we would end up with 3 instances of the Airport class representing YYC - Calgary and 3 instances of the Airport class representing YVR - Vancouver. Again, this isn’t necessarily a big problem when we have 3 rows in the result set but with larger result sets it could cause your application to use a lot more memory than necessary.

It is worth considering the cost associated with this approach. Given the added memory cost, this approach might be better suited to One-to-One associations rather than the Many-to-One example we talked about in this post. In the next post, we will explore an alternate approach that is more memory efficient but probably a little more costly on the CPU for the mapping.

Let’s just get this one out of the way early. Stored procedures are not my favorite way to get data from SQL Server but there was a time when they were extremely popular. They are still heavily used today and so this series would not be complete without covering how to use stored procedures with Dapper.

A Simple Example

Let’s imagine a simple stored procedure that allows us to query for Aircraft by model.

CREATE PROCEDURE GetAircraftByModel @Model NVARCHAR(255) AS
BEGIN
    SELECT 
       Id
      ,Manufacturer
      ,Model
      ,RegistrationNumber
      ,FirstClassCapacity
      ,RegularClassCapacity
      ,CrewCapacity
      ,ManufactureDate
      ,NumberOfEngines
      ,EmptyWeight
      ,MaxTakeoffWeight
    FROM Aircraft a
    WHERE a.Model = @Model
END

To execute this stored procedure and map the results to a collection of Aircraft objects, use the QueryAsync method almost exactly like we did in the last post.

//GET api/aircraft
[HttpGet]
public async Task<IEnumerable<Aircraft>> Get(string model)
{
    IEnumerable<Aircraft> aircraft;

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();

        aircraft = await connection.QueryAsync<Aircraft>("GetAircraftByModel",
                        new {Model = model}, 
                        commandType: CommandType.StoredProcedure);
    }
    return aircraft;
}

Instead of passing in the raw SQL statement, we simply pass in the name of the stored procedure. We also pass in an object that has properties for each of the stored procedures arguments, in this case new {Model = model} maps the model variable to the stored procedure’s @Model argument. Finally, we specify the commandType as CommandType.StoredProcedure.

Wrapping it up

That’s all there is to using stored procedures with Dapper. As much as I dislike using stored procedures in my applications, I often do have to call stored procedures to fetch data from legacy databases. When that situation comes up, Dapper is my tool of choice.

Stay tuned for the next installment in this Dapper series. Comment below if there is a specific topic you would like covered.

Today, we will start with the basics of loading a mapping and database table to a C# class.

What is Dapper?

Dapper calls itself a simple object mapper for .NET and is usually lumped into the category of micro ORM (Object Relational Mapper). When compared to a fully featured ORM like Entity Framework, Dapper lacks certain features like change-tracking, lazy loading and the ability to translate complex LINQ expressions to SQL queries. The fact that Dapper is missing these features is probably the single best thing about Dapper. While it might seem like you’re giving up a lot, you are also gaining a lot by dropping those types of features. Dapper is fast since it doesn’t do a lot of the magic that Entity Framework does under the covers. Since there is less magic, Dapper is also a lot easier to understand which can lead to lower maintenance costs and maybe even fewer bugs.

How does it work?

Throughout this series we will build on an example domain for an airline. All airlines need to manage a fleet of aircraft, so let’s start there. Imagine a database with a table named Aircraft and a C# class with property names that match the column names of the Aircraft table.

CREATE TABLE Aircraft
    (
        Id int not null IDENTITY(1,1) CONSTRAINT pk_Aircraft_Id PRIMARY KEY,
        Manufacturer nvarchar(255),
        Model nvarchar(255),
        RegistrationNumber nvarchar(50),
        FirstClassCapacity int,
        RegularClassCapacity int,
        CrewCapacity int,
        ManufactureDate date,
        NumberOfEngines int,
        EmptyWeight int,
        MaxTakeoffWeight int
    )

public class Aircraft 
{
    public int Id { get; set; }
    public string Manufacturer {get; set;}
    public string Model {get; set;}
    public string RegistrationNumber {get; set;}
    public int FirstClassCapacity {get; set;}
    public int RegularClassCapacity {get; set;}
    public int CrewCapacity {get; set;}
    public DateTime ManufactureDate {get; set; }
    public int NumberOfEngines {get; set;}
    public int EmptyWeight {get; set;}
    public int MaxTakeoffWeight {get; set;}
}

Installing Dapper

Dapper is available as a Nuget package. To use Dapper, all you need to do is add the Dapper package to your project.

.NET Core CLI: dotnet add package Dapper

Package Manager Console: Install-Package Dapper

Querying a single object

Dapper provides a set of extension methods for .NET’s IDbConnection interface. For our first task, we want to execute a query to return the data for a single row from the Aircraft table and place the results in an instance of the Aircraft class. This is easily accomplished using Dapper’s QuerySingleAsync method.

[HttpGet("{id}")]
public async Task<Aircraft> Get(int id)
{
  Aircraft aircraft;
  using (var connection = new SqlConnection(_connectionString))
  {
    await connection.OpenAsync();
    var query = @"
SELECT 
       Id
      ,Manufacturer
      ,Model
      ,RegistrationNumber
      ,FirstClassCapacity
      ,RegularClassCapacity
      ,CrewCapacity
      ,ManufactureDate
      ,NumberOfEngines
      ,EmptyWeight
      ,MaxTakeoffWeight
  FROM Aircraft WHERE Id = @Id";

    aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id});
  }
  return aircraft;
}

Before we can call Dapper’s QuerySingleASync method, we need an instance of an open SqlConnection. If you are an Entity Framework user, you might not be used to working directly with the SqlConnection class because Entity Framework generally manages connections for you. All we need to do is create a new SqlConnection, passing in the connection string, then call OpenAsync to open that connection. We wrap the connection in a using statement to ensure that connection.Dispose() is called when we are done with the connection. This is important because it ensures the connection is returned to the connection pool that is managed by .NET. If you forget to do this, you will quickly run into problems where your application is not able to connect to the database because the connection pool is starved. Check out the .NET Docs for more information on connection pooling.

We will use the following pattern throughout this series of blogs posts:

using(var connection = new SqlConnection(_connectionString))
{
  await connection.OpenAsync();
  //Do Dapper Things
}

As @Disman pointed out in the comments, it is not necessary to call connection.OpenAsync(). If the connection is not already opened, Dapper will call OpenAsync for you. Call me old fashioned but I think that whoever created the connection should be the one responsible for opening it, that’s why I like to open the connection before calling Dapper.

Let’s get back to our example. To query for a single Aircraft, we call the QuerySingleAsync method, specifying the Aircraft type parameter. The type parameter tells Dapper what class type to return. Dapper will take the results of the query that gets executed and map the column values to properties of the specified type. We also pass in two arguments. The first is the query that will return a single row based on a specified @Id parameter.

SELECT 
       Id
      ,Manufacturer
      ,Model
      ,RegistrationNumber
      ,FirstClassCapacity
      ,RegularClassCapacity
      ,CrewCapacity
      ,ManufactureDate
      ,NumberOfEngines
      ,EmptyWeight
      ,MaxTakeoffWeight
  FROM Aircraft WHERE Id = @Id

The next parameter is an anonymous class containing properties that will map to the parameters of the query.

new {Id = id}

Passing the parameters in this way ensures that our queries are not susceptible to SQL injection attacks.

That’s really all there is to it. As long as the column names and data types match the property of your class, Dapper takes care of executing the query, creating an instance of the Aircraft class and setting all the properties.

If the query doesn’t contain return any results, Dapper will throw an InvalidOperationException.

InvalidOperationException: Sequence contains no elements

If you prefer that Dapper returns null when there are no results, use the QuerySingleOrDefaultAsnyc method instead.

Querying a list of objects

Querying for a list of objects is just as easy as querying for a single object. Simply call the QueryAsync method as follows.

[HttpGet]
public async Task<IEnumerable<Aircraft>> Get()
{
  IEnumerable<Aircraft> aircraft;

  using (var connection = new SqlConnection(_connectionString))
  {
    await connection.OpenAsync();
    var query = @"
SELECT 
       Id
      ,Manufacturer
      ,Model
      ,RegistrationNumber
      ,FirstClassCapacity
      ,RegularClassCapacity
      ,CrewCapacity
      ,ManufactureDate
      ,NumberOfEngines
      ,EmptyWeight
      ,MaxTakeoffWeight
  FROM Aircraft";
    aircraft = await connection.QueryAsync<Aircraft>(query);
    }
  return aircraft;
}

In this case, the query did not contain any parameters. If it did, we would pass those parameters in as an argument to the QueryAsync method just like we did for the QuerySingleAsync method.

What’s next?

This is just the beginning of what I expect will be a long series of blog posts. You can follow along on this blog and you can track the sample code on GitHub.

Leave a comment below if there is a topic you would like me to cover.

I’ve recently been working on a team which has some downright amazing leadership on the testing side. As a result I’ve had to raise my testing game to a level I’ve not been at before. During the process the team developed some testing patters which might be useful to the general populace. Here they are:

1. Keep the number of test bed tests to a minimum.
2. Leverage Observable as a testing seam
3. Leverage monkey patching as a testing seam
4. No matter what anybody says e2e tests still need sleeps

I’m going to split this post into parts to keep the posts relatively short.

Test Bed

Angular 2 introduced the idea of the TestBed which is basically a way of testing out a component with a “real” DOM behind it. There is support for injecting services either real or mock into your component as well as binding your component’s model to the template. TestBed tests are the default type of test generated by the angular-cli when you create a new component. They are great and can be used to test a component much more thoroughly than testing with isolated tests alone.

The issue with them is that they tend to be quite slow to run. The interaction with the DOM and the setup of an entire dependency injection instance per test adds several hundred milliseconds for every TestBed test run. Just watching the test counter tick up in my command-line test reporter I can easily see when TestBed tests are encountered as the counter slows right now. The added time may not be huge in isolation but if we add 500ms (pretty conservative in my experience) per test on a collection of 1500 tests (pretty small project) then we’re talking twelve and a half minutes. Angular testing is already glacial so adding this coupled with the Karma runner’s inability to selectively run tests and you’re really in trouble.

Testing should be lightening fast because you want the feedback loop to be as tight as possible. That’s why I’m such a big fan of Live Unit Testing. My mantra is that you should be able to hold your breath during a test run without feeling uncomfortable (this makes former pearl divers well adapted to being Angular developers). Most of the functionality that we test on a component doesn’t need to be tested using a full featured TestBed. Any functions which mutate the state or call out to other services can be written without the need for the TestBed. Many of my components contain just two TestBed tests: one to check the component can be created and one to check it can be initted. These two test generally catch any typos in the template which is a big source of errors as the TypeScript compiler doesn’t catch things in there. In the init test you can also check that appropriate bindings are in place. It is faster to have a test which tests a bunch of properties at once than one test per property.

This being said there are still plenty of time when TestBed tests do come in useful, typically any time you’re building a complex user interface and want to validate that it works cross browsers. I’m certainly not saying don’t use TestBed at all but rather that its use should be limited and isolation tests should be favoured.

Let’s take a look at an example test which we can migrate away from the TestBed.

This component does some simple addition. Left side + right side = answer, unless the answer is less than 0 then ‘Value too small’:

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-math-component',
  templateUrl: './math-component.component.html',
  styleUrls: ['./math-component.component.css']
})

export class MathComponent implements OnInit {

  model: MathComponentModel = {
    left: 0,
    right: 0,
    answer: 0
  };

  constructor() { }

  ngOnInit() {
  }

  update() {
    this.model.answer = this.model.left + this.model.right;
    if (this.model.answer < 0) {
      this.model.answer = 'Value too small';
    }
  }
}

export class MathComponentModel {
  left: number;
  right: number;
  answer: number | string;
}

The template for it is equally simple

<p>
  <input [(ngModel)]="model.left" (change)="update()" type="number"/>
  <input [(ngModel)]="model.right" (change)="update()" type="number"/>=
  <input [(ngModel)]="model.answer" disabled>
</p>

A fully testbed test for this might look like

describe('MathComponent', () => {
  let component: MathComponent;
  let fixture: ComponentFixture<MathComponent>;

  beforeEach(async(() => {
    TestBed.configureTestingModule({
      declarations: [MathComponent],
      imports: [FormsModule]
    })
      .compileComponents();
  }));

  beforeEach(() => {
    fixture = TestBed.createComponent(MathComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  });

  it('should create', () => {
    expect(component).toBeTruthy();
  });

  it('should add up two numbers', fakeAsync(() => {
    const compiled = fixture.debugElement.nativeElement;
    compiled.querySelector('[data-autom=left]').value = '2';
    compiled.querySelector('[data-autom=left]').dispatchEvent(new Event('input'));

    compiled.querySelector('[data-autom=right]').value = '3';
    compiled.querySelector('[data-autom=right]').dispatchEvent(new Event('input'));

    component.update();
    fixture.detectChanges();
    tick();
    expect(compiled.querySelector('[data-autom=answer]').value).toBe('5');
  }));

  it('should set answer to "Value too small" if answer < 0', fakeAsync(() => {
    const compiled = fixture.debugElement.nativeElement;
    compiled.querySelector('[data-autom=left]').value = '2';
    compiled.querySelector('[data-autom=left]').dispatchEvent(new Event('input'));

    compiled.querySelector('[data-autom=right]').value = '-3';
    compiled.querySelector('[data-autom=right]').dispatchEvent(new Event('input'));

    component.update();
    fixture.detectChanges();
    tick();

    expect(compiled.querySelector('[data-autom=answer]').value).toBe('Value too small');
  }));
});

A couple of things to point out here: the first is that there is quite a bit of magic to interact with input boxes on the page. The second thing is that compiled component tests seem to be quite slow, doubly so if you haven’t made your modules highly granular. Much of the testing here could be handled by testing the model rather than the rendering. A testbed test is still needed to check the rendering once but after that we’re good with simpler tests.

describe('MathComponent bindings', () => {
  let component: MathComponent;
  let fixture: ComponentFixture<MathComponent>;

  beforeEach(async(() => {
    TestBed.configureTestingModule({
      declarations: [MathComponent],
      imports: [FormsModule]
    })
      .compileComponents();
  }));

  beforeEach(() => {
    fixture = TestBed.createComponent(MathComponent);
    component = fixture.componentInstance;
    fixture.detectChanges();
  });

  it('should init', () => {
    expect(component).toBeTruthy();
  });

  it('should create', fakeAsync(() => {
    component.model.answer = 5;
    component.model.answer = 5;
    component.model.left = 3;
    component.model.right = 2;

    fixture.detectChanges();
    tick();
    const compiled = fixture.debugElement.nativeElement;
    expect(compiled.querySelector('[data-autom=answer]').value).toBe(component.model.answer.toString());
    expect(compiled.querySelector('[data-autom=left]').value).toBe(component.model.left.toString());
    expect(compiled.querySelector('[data-autom=right]').value).toBe(component.model.right.toString());
  }));
});
describe('MathComponent', () => {
  it('should add up two numbers', () => {
    const component = new MathComponent();
    component.model.left = 1;
    component.model.right = 2;
    component.update();
    expect(component.model.answer).toBe(3);
  });

  it('should set answer to Value too small if answer < 0', () => {
    const component = new MathComponent();
    component.model.left = 1;
    component.model.right = -2;
    component.update();
    expect(component.model.answer).toBe('Value too small');
  });
});

The advantage here is that the tests are simpler and run faster. We also don’t have to worry about fiddling with fake async or ticks.

In the next article we’ll visit how we can use Observables, which are pretty popular in angular, as a seam to help write tests.

Resource-Based Authorization

Using the IAuthorizationService in ASP.NET Core, it is easy to implement an authorization strategy that depends not only on properties of the User but also depends on the resource being accessed. To learn how resource-based authorization works, take a look at the well written offical documentation.

Once you have defined your authorization handlers and setup any policies in Startup.ConfigureServices, applying resource-based authorization is a matter of calling one of two overloads of the AuthorizeAsync method on the IAuthorizationService.

Task<AuthorizationResult> AuthorizeAsync(ClaimsPrincipal user,
                          object resource,
                          string policyName);

Task<AuthorizationResult> AuthorizeAsync(ClaimsPrincipal user,
                          object resource,
                          IAuthorizationRequirement requirements);

One method takes in a policy name while the other takes in an IAuthorizationRequirement. The resulting AuthorizationResult has a Succeeded boolean that indicates whether or not the user meets the requirements for the specified policy. Using the IAuthorizationService in a controller is easy enough. Simply inject the service into the controller, call the method you want to call and then check the result.

 
public async Task<IActionResult> Edit(int id)
{
    var document = _documentContext.Find(documentId);

    var authorizationResult = await _authorizationService.AuthorizeAsync(User, Document, "EditDocument");

    if (authorizationResult.Succeeded)
    {
        return View(document);
    }    
    else
    {
        return new ChallengeResult();
    }
}

Using this approach, we can easily restrict which users can edit specific documents as defined by our EditDocument policy. For example, we might limit editing to only users who originally created the document.

Where things start to get a little ugly is if we want to render a UI element based on resource-based authorization. For example, we might only want to render the edit button for a document if the current user is actually authorized to edit that document. Out of the box, this would require us to inject the IAuthorizationService in the Razor view and use it like we did in the controller action. The approach works, but the Razor code will get ugly really fast.

Authorize Resource Tag Helper

Similar to the Authorize Tag Helper from the last blog post, this Authorize Resource Tag Helper will make it easy to show or hide blocks of HTML by evaluating authorization rules.

Resource-Based Policy Authorization

Let’s assume we have a named “EditDocument” that requires a user to be the original author of a Document in order to edit the document. With the authorize resource tag helper, specify the resource instance using the asp-authorize-resource attribute and the policy name using the asp-policy attribute. Here is an example where Model is an instance of a Document

<a href="#" asp-authorize-resource="Model" 
    asp-policy="EditDocument" class="glyphicon glyphicon-pencil"></a>

If the user meets the requirments for the “EditDocument” policy and the specified resource, then the block of HTML will be sent to the browser. If the requirements are not met, the tag helper will suppress the output of that block of HTML. The tag helper can be applied to any HTML element.

Resource-Based Requirement Authorization

Instead of specifying a policy name, authorization can be evaluated by specifying an instance of an IAuthorizationRequirement. When using requirements directly instead of policies, specify the requirement using the asp-requirement attribute.

<a href="#" asp-authorize-resource="document"
            asp-requirement="Operations.Delete" 
            class="glyphicon glyphicon-trash text-danger">                            
</a>

If the user meets Operations.Delete requirement for the specified resource, then the block of HTML will be sent to the browser. If the requirement is not met, the tag helper will suppress the output of that block of HTML. The tag helper can be applied to any HTML element.

Implementation Details

The authorize resource tag helper itself is fairly simple. The implementation will likely evolve after this blog post so you can check out the latest version here.

The tag helper needs an instance of the IHttpContextAccessor to get access to the current user and an instance of the IAuthorizationService. These are injected into the constructor. In the ProcessAsync method, either the specified Policy or the specified Requirement are passed in to the IAuthorizationService along with the resource.

[HtmlTargetElement(Attributes = "asp-authorize-resource,asp-policy")]
[HtmlTargetElement(Attributes = "asp-authorize-resource,asp-requirement")]
public class AuthorizeResourceTagHelper : TagHelper
{
    private readonly IAuthorizationService _authorizationService;
    private readonly IHttpContextAccessor _httpContextAccessor;

    public AuthorizeResourceTagHelper(IHttpContextAccessor httpContextAccessor, IAuthorizationService authorizationService)
    {
        _httpContextAccessor = httpContextAccessor;
        _authorizationService = authorizationService;
    }

    /// <summary>
    /// Gets or sets the policy name that determines access to the HTML block.
    /// </summary>
    [HtmlAttributeName("asp-policy")]
    public string Policy { get; set; }

    /// <summary>
    /// Gets or sets a comma delimited list of roles that are allowed to access the HTML  block.
    /// </summary>
    [HtmlAttributeName("asp-requirement")]
    public IAuthorizationRequirement Requirement { get; set; }


    /// <summary>
    /// Gets or sets the resource to be authorized against a particular policy
    /// </summary>
    [HtmlAttributeName("asp-authorize-resource")]
    public object Resource { get; set; }

    public override async Task ProcessAsync(TagHelperContext context, TagHelperOutput output)
    {
        if (Resource == null)
        {
            throw new ArgumentException("Resource cannot be null");                
        }
        if (string.IsNullOrWhiteSpace(Policy) && Requirement == null)
        {
            throw new ArgumentException("Either Policy or Requirement must be specified");
                
        }
        if (!string.IsNullOrWhiteSpace(Policy) && Requirement != null)
        {
            throw new ArgumentException("Policy and Requirement cannot be specified at the same time");
        }

        AuthorizationResult authorizeResult;

        if (!string.IsNullOrWhiteSpace(Policy))
        {
            authorizeResult = await _authorizationService.AuthorizeAsync(_httpContextAccessor.HttpContext.User, Resource, Policy);
        }
        else if (Requirement != null)
        {
            authorizeResult =
                await _authorizationService.AuthorizeAsync(_httpContextAccessor.HttpContext.User, Resource,
                    Requirement);
        }
        else
        {
            throw new ArgumentException("Either Policy or Requirement must be specified");
        }

        if (!authorizeResult.Succeeded)
        {
            output.SuppressOutput();
        }
    }

Note that either a policy or a requirement must be specified along with a resource, but you can’t specify both a policy AND a requirement. Most of the code in the ProcessAsync method is checking the argument values to make sure a valid combination was used.

Try it out

You can see the authorize resource tag helper in action on my tag helper samples site here. The sample site contains the examples listed in this blog post and also provides a way to log in as different users to test different scenarios.

The authorize resource tag helper is also available on NuGet so you can use it in your own ASP.NET Core application.

dotnet add package TagHelperSamples.Authorization

Let me know what you think. Would you like to see this tag helper including in the next release of ASP.NET Core?

NOTE: If you choose to use the authorize resource tag helper in your application, you should remember that hiding a section of HTML is not enough to fully secure your application. You also need to make sure that resource-based authorization is applied to any related controllers and action methods.

What’s Next?

There is one more authorization scenario related to supporting different authorization schemes that I hope to cover. Watch out for that in a future blog post. Also, this tag helper project is all open source so feel free to jump in on GitHub.

Follow @aspnetmonsters

Related Links:
Tag Helper Samples Reop - https://github.com/dpaquette/TagHelperSamples
GitHub Issue for MVC Core  - https://github.com/aspnet/Mvc/issues/3785
Blog Post Deep Dive - https://www.davepaquette.com/archive/2017/11/05/authorize-tag-helper.aspx

Follow @aspnetmonsters

In this blog post, we build an Authorize tag helper that makes it easy to control access to any block HTML in a Razor view.

Authorize Tag Helper

The basic idea of this tag helper is to provide similar functionality to the [Authorize] attribute and it’s associated action filter in ASP.NET Core MVC. The authorize tag helper will provide the same options as the [Authorize] attribute and the implementation will be based on the authorize filter. In the MVC framework, the [Authorize] attribute provides data such as the names of roles and policies while the authorize filter contains the logic to check for roles and policies as part of the request pipeline. Let’s walk through the most common scenarios.

Simple Authorization

In it’s simplest form, adding the [Authorize] attribute to a controller or action method will limit access to that controller or action method to users who are authenticated. That is, only users who are logged in will be able to access those controllers or action methods.

With the Authorize tag helper, we will implement a similar behaviour. Adding the asp-authorize attribute to any HTML element will ensure that only authenticated users can see that that block of HTML.

<div asp-authorize class="panel panel-default">
    <div class="panel-heading">Welcome !!</div>
    <div class="panel-body">
        If you're logged in, you can see this section
    </div>
</div>

If a user is not authenticated, the tag helper will suppress the output of that entire block of HTML. That section of HTML will not be sent to the browser.

Role Based Authorization

The [Authorize] attribute provides an option to specify the role that a user must belong to in order to access a controller or action method. For example, if a user must belong to the Admin role, we would add the [Authorize] attribute and specify the Roles property as follows:

[Authorize(Roles = "Admin")]
public class AdminController : Controller
{
  //Action methods here
}

The equivalent using the Authorize tag helper would be to add the asp-authorize attribute to an HTML element and then also add the asp-roles attribute specifying the require role.

<div asp-authorize asp-roles="Admin" class="panel panel-default">
    <div class="panel-heading">Admin Section</div>
    <div class="panel-body">
        Only admin users can see this section. Top secret admin things go here.
    </div>
</div>

You can also specify a comma separated list of roles, in which case the HTML would be rendered if the user was a member of any of the roles specified.

Policy Based Authorization

The [Authorize] attribe also provides an option to authorize users based on the requirements specified in a Policy. You can learn more about the specifics of this approach by reading the offical docs on Claims-Based Authorization and Custom-Policy Based Authorization. Policy based authorization is applied by specifying Policy property for the [Authorize] attribute as follows:

[Authorize(Policy = "Seniors")]
public class AdminController : Controller
{
  //action methods here
}

This assumes a policy named Seniors was defined at startup. For example:

services.AddAuthorization(o =>
    {
        o.AddPolicy("Seniors", p =>
        {
            p.RequireAssertion(context =>
            {
                return context.User.Claims
                      .Any(c => c.Type == "Age" && Int32.Parse(c.Value) >= 65);
            });
        });

    }
);

The equivalent using the Authorize tag helper would be to add the asp-authorize attribute to an HTML element and then also add the asp-policy attribute specifying the policy name.

<div asp-authorize asp-policy="Seniors" class="panel panel-default">
    <div class="panel-heading">Seniors Only</div>
    <div class="panel-body">
        Only users age 65 or older can see this section. Early bird dinner coupons go here. 
    </div>
</div>

Combining Role and Policy Based Authorization

You can combine the role based and policy based approaches by specifying both the asp-roles and asp-policy attributes. This has the effect of requiring that the user meets the requiremnts for both the role and the policy. For example, the following would require that the usere were both a member of the Admin role and meets the requirements defined in the Seniors policy.

<div asp-authorize asp-roles="Admin" asp-policy="Seniors" class="panel panel-default">
    <div class="panel-heading">Admin Seniors Only</div>
    <div class="panel-body">
        Only users who have both the Admin role AND are age 65 or older can see this section.
    </div>
</div>

Implementation Details

The Authorize tag helper itself is fairly simple. The implementation will likely evolve after this blog post so you can check out the latest version here.

The tag helper implements the IAuthorizeData interface. This is the interface implemented by the Authorize attribute in ASP.NET Core. In the ProcessAsync method, the properties of IAuthorizeData are used to create an effective policy that is then evaluated against the current HttpContext. If the policy does not succeed, then the output of the tag helper is supressed. Remember that supressing the output of a tag helper means that the HTML for that element, including it’s children, will be NOT sent to the client.

[HtmlTargetElement(Attributes = "asp-authorize")]
[HtmlTargetElement(Attributes = "asp-authorize,asp-policy")]
[HtmlTargetElement(Attributes = "asp-authorize,asp-roles")]
[HtmlTargetElement(Attributes = "asp-authorize,asp-authentication-schemes")]
public class AuthorizationPolicyTagHelper : TagHelper, IAuthorizeData
{
    private readonly IAuthorizationPolicyProvider _policyProvider;
    private readonly IPolicyEvaluator _policyEvaluator;
    private readonly IHttpContextAccessor _httpContextAccessor;

    public AuthorizationPolicyTagHelper(IHttpContextAccessor httpContextAccessor, IAuthorizationPolicyProvider policyProvider, IPolicyEvaluator policyEvaluator)
    {
        _httpContextAccessor = httpContextAccessor;
        _policyProvider = policyProvider;
        _policyEvaluator = policyEvaluator;
    }

    /// <summary>
    /// Gets or sets the policy name that determines access to the HTML block.
    /// </summary>
    [HtmlAttributeName("asp-policy")]
    public string Policy { get; set; }

    /// <summary>
    /// Gets or sets a comma delimited list of roles that are allowed to access the HTML  block.
    /// </summary>
    [HtmlAttributeName("asp-roles")]
    public string Roles { get; set; }

    /// <summary>
    /// Gets or sets a comma delimited list of schemes from which user information is constructed.
    /// </summary>
    [HtmlAttributeName("asp-authentication-schemes")]
    public string AuthenticationSchemes { get; set; }
    
    public override async Task ProcessAsync(TagHelperContext context, TagHelperOutput output)
    {
        var policy = await AuthorizationPolicy.CombineAsync(_policyProvider, new[] { this });

        var authenticateResult = await _policyEvaluator.AuthenticateAsync(policy, _httpContextAccessor.HttpContext);

        var authorizeResult = await _policyEvaluator.AuthorizeAsync(policy, authenticateResult, _httpContextAccessor.HttpContext, null);

        if (!authorizeResult.Succeeded)
        {
            output.SuppressOutput();
        }
    }
}

The code in the ProcessAsync method is based on the AuthorizeFilter from ASP.NET Core MVC.

Try it out

You can see the Authorize tag helper in action on my tag helper samples site here. The sample site contains the examples listed in this blog post and also provides a way to log in as different users to test different scenarios.

The Authorize tag helper is also available on NuGet so you can use it in your own ASP.NET Core application.

dotnet add package TagHelperSamples.Authorization

Let me know what you think. Would you like to see this tag helper included in the next release of ASP.NET Core?

What’s Next?

If you choose to use the Authorize tag helper in your application, you should remember that hiding a section of HTML is not enough to fully secure your application. You also need to make sure that authorization is applied to any related controllers and action methods. The Authorize tag helper is meant to be used in conjugtion with the [Authorize] attribute, not as a replacement for it.

There are a couple more scenarios I would like to go through and I will address those in a future post. One of those is supporting different Authorization Schemes and the other resource based authorization. Of course, this project is all open source so feel free to jump in on GitHub.

Related Links:
GitHub Repo - https://github.com/AspNetMonsters/EP109-EFDbContextPooling
Netling Load Testing Tool - https://github.com/hallatore/Netling
.NET Conf Video - What’s new in EF Core 2.0 - https://channel9.msdn.com/Events/dotnetConf/2017/T221

                 (Section on DbContext Pooling starts at 10:55s)

Follow @aspnetmonsters