Unit Testing in .NET Core with xUnit

In this post we see how to do unit testing in .NET Core with xUnit.

xUnit is a popular framework, with a lot of features and helpers to develop quickly our unit testings and implement them with a clean syntax; it’s very efficient and the result is that the tests execution is very fast.

In the previous post we implemented a  basic class with an in-memory DbContext, that we’ll use in the unit test.

The service

We can suppose that our sut is a service, like CategoryService.

The service has some methods to retrieve the category from the database with Entity Framework.

The service look like this:


public class CategoryService
{
readonly BlogContextFactory _contextFactory;

public CategoryService(BlogContextFactory contextFactory)
{
_contextFactory = contextFactory;
}

public IEnumerable<Category> GetCategories()
{
using (var db = _contextFactory.GetNewDbContext())
{
return db.Categories.OrderBy(c => c.Name).ToList();
}
}

public async Task<Category> AddCategory(Category category)
{
using (var db = _contextFactory.GetNewDbContext())
{
category.Id = Guid.NewGuid();

await db.Categories.AddAsync(category);
await db.SaveChangesAsync();

return category;
}
}

public async Task UpdateCategory(Category category)
{
using (var db = _contextFactory.GetNewDbContext())
{
db.Categories.Update(category);
await db.SaveChangesAsync();
}
}

public async Task DeleteCategory(Category category)
{
using (var db = _contextFactory.GetNewDbContext())
{
db.Categories.Remove(category);
await db.SaveChangesAsync();
}
}
}

Unit Test class

Now we can implement the Unit Test class; we can use the base class DatabaseFixture and pass the context factory configured with an in-memory DbContext to the service:


public class CategoryServiceTests : DatabaseFixture
{
readonly CategoryService _categoryService;

public CategoryServiceTests() : base("CategoryContext")
{
_categoryService = new CategoryService(ContextFactory);
}

[Fact]
public void should_return_categories_list()
{
var categories = _categoryService.GetCategories();
categories.Should().NotBeNullOrEmpty();
categories.Count().Should().BeGreaterThan(0);
}

[Fact]
public async Task should_add_new_category()
{
var category = new Category()
{
Id = new Guid("{1225FE5B-9C46-4BA3-9233-9337DDC5F478}"),
Name = "Test Category"
};

category = await _categoryService.AddCategory(category);

using (var db = ContextFactory.GetNewDbContext())
{
var newCategory = await db.FindAsync<Category>(category.Id);
newCategory.Should().NotBeNull();
}
}

[Fact]
public async Task should_edit_category()
{
using (var db = ContextFactory.GetNewDbContext())
{
var category = await db.FindAsync<Category>(new Guid("{BE7EB1E1-FB67-4FE1-A96E-4721D37AFE29}"));
category.Name = "category modified";
await _categoryService.UpdateCategory(category);

category = await db.FindAsync<Category>(new Guid("{BE7EB1E1-FB67-4FE1-A96E-4721D37AFE29}"));
category.Name.ShouldBeEquivalentTo("category modified");
}
}

[Fact]
public async Task should_delete_category()
{
var category = new Category() { Id = new Guid("{5D134CF2-EF37-4663-B3D2-64F2E3DBF3BE}"), Name = "Category1" };
await _categoryService.DeleteCategory(category);

using (var db = ContextFactory.GetNewDbContext())
{
category = await db.FindAsync<Category>(new Guid("{5D134CF2-EF37-4663-B3D2-64F2E3DBF3BE}"));
category.Should().BeNull();
}
}
}

The Fact attribute is the xUnit attribute used for the test methods.

As discussed above the ContextFactory is configured to returns an instance in-memory of the DbContext.

Now we can run the unit test by opening a command console, going to the UnitTest project folder and running the command:

dotnet test

The command will run the tests and will show us the results.

You can find the source code here.

 

Unit Testing in .NET Core with xUnit

In-memory DbContext with Entity Framework Core

Frequently, when we need to test a EF context from our unit test class, we need to implement a stub of the context manually, in order to have, for example, an in memory representation of the context.

In entity framework core this is useless because one of the options available to initialize a context is to use an in memory database.

With this option we can populate an in memory database that could be used, for test purposes.

Let’s go to take a look about the configurations that we need to do.

Context

The first thing is define a DbContext.

With Entity Framework Core, the things are quite different because the DbContext class has a new constructor that accept a parameter of type DbContextOptions; this parameter will be used to define for example the connection string of the context.

public class BlogContext : DbContext
{
public DbSet<Category> Categories { get; set; }
public DbSet<Tag> Tags { get; set; }
public DbSet<Post> Posts { get; set; }
public DbSet<Reply> Replies { get; set; }
public DbSet<PostCategory> PostCategories { get; set; }
public DbSet<PostTag> PostTags { get; set; }
public BlogContext(DbContextOptions<BlogContext> options) : base(options) { }
}

Context factory

Basically, what I want is a context factory that help me to get a new instance of a context with specific options.


public class BlogContextFactory
{
private readonly DbContextOptions<BlogContext> _options;

public BlogContextFactory(DbContextOptions<BlogContext> options)
{
_options = options;
}

public BlogContext GetNewDbContext()
{
return new BlogContext(_options);
}
}

The factory accept a DbContextOptions parameter in the constructor and returns a new instance of the BlogContext with these options.

Registration

Another important thing is the registration of the factory in a IoC container, that in my case is Autofac.

This is a mandatory stuff if I want to inject the factories in the services that depends on it:

var options = new DbContextOptionsBuilder<BlogContext>().UseSqlServer(_configuration.GetConnectionString("BlogContext")).Options;
builder.Register(b => new BlogContextFactory(options)).SingleInstance();

I’m using Autofac, so I register BlogContextFactory as single instance with specific options.

The first row is the most important; with the method UseSqlServer I’m saying that the db context will be configured with a sql server database, and obviously I’m giving the connections string as a parameter of the method.

In memory option

As well as I use this options to configure a db context with a sql database, I can do the same thing to configure an in memory database.


public abstract class DatabaseFixture : IDisposable
{
protected readonly BlogContextFactory ContextFactory;
readonly DbContextOptions<BlogContext> _options;

public DatabaseFixture(string databaseName)
{
_options = new DbContextOptionsBuilder<BlogContext>().UseInMemoryDatabase(databaseName).Options;
ContextFactory = new BlogContextFactory(_options);

var categories = new List<Category>()
{
new Category() { Id = new Guid("{5D134CF2-EF37-4663-B3D2-64F2E3DBF3BE}"), Name = "Category1" },
new Category() { Id = new Guid("{BE7EB1E1-FB67-4FE1-A96E-4721D37AFE29}"), Name = "Category2" },
new Category() { Id = new Guid("{BBB5D5DD-A746-4A87-9204-90EE87B5F560}"), Name = "Category3" }
};

var tags = new List<Tag>()
{
new Tag() { Id = new Guid("{859A4C65-F688-4F9F-8575-F389841665F8}"), Name = "Tag1" },
new Tag() { Id = new Guid("{A57D65BE-8A33-48E8-A3E4-3EF5D110A961}"), Name = "Tag2" },
new Tag() { Id = new Guid("{2D1AC63C-3D48-45A3-8F22-C9670A9D4F84}"), Name = "Tag3" }
};

var posts = new List<Post>()
{
new Post() { Id = new Guid("{7660E171-0776-451C-AD2C-1CA6F46EDB30}"), Title = "Post1", Content = "Content1", Published = true },
new Post() { Id = new Guid("{7221B957-E3F2-461F-A4F8-3D13699880A1}"), Title = "Post2", Content = "Content2", Published = false },
new Post() { Id = new Guid("{D2E934DB-7B64-42A7-953F-213EA771F596}"), Title = "Post3", Content = "Content3", Published = false }
};

var replies = new List<Reply>()
{
new Reply() { Id = new Guid("{9344DCAF-154E-4167-8EEF-0372896F5469}"), CreateUser = "user1", Content = "Content1", Published = true, Post = posts[0] },
new Reply() { Id = new Guid("{1DB75F77-A50E-4F90-83F8-711D3025EBA4}"), CreateUser = "user2", Content = "Content2", Published = true, Post = posts[0] },
new Reply() { Id = new Guid("{6772F1D7-E6D6-47DB-8724-E46353BFDFDD}"), CreateUser = "user3", Content = "Content3", Published = false, Post = posts[0] }
};

var postCategories = new List<PostCategory>()
{
new PostCategory() {
Id = new Guid("{267A0AE0-2E91-4FA5-B45C-361B8AAE635D}"),
Post = posts[0],
Category = categories[0] }
};

var postTags = new List<PostTag>()
{
new PostTag() {
Id = new Guid("{DA6CEA55-610F-4956-913F-D8F409014AC9}"),
Post = posts[0],
Tag = tags[0] }
};

using (var blogContext = ContextFactory.GetNewDbContext())
{
blogContext.Set<Category>().AddRange(categories);
blogContext.Set<Tag>().AddRange(tags);
blogContext.Set<Post>().AddRange(posts);
blogContext.Set<Reply>().AddRange(replies);
blogContext.Set<PostCategory>().AddRange(postCategories);
blogContext.Set<PostTag>().AddRange(postTags);
blogContext.SaveChanges();
}
}

public void Dispose()
{
using (var blogContext = ContextFactory.GetNewDbContext())
{
blogContext.RemoveRange(blogContext.Categories);
blogContext.RemoveRange(blogContext.Tags);
blogContext.RemoveRange(blogContext.Posts);
blogContext.RemoveRange(blogContext.Replies);
blogContext.RemoveRange(blogContext.PostCategories);
blogContext.RemoveRange(blogContext.PostTags);
blogContext.SaveChanges();
}
}
}

With the UseInMemoryDatabase method I can configure an instance of the db context with an in memory database.

The only things that I need to do is specify the name of the database (a string) and then populate the context with the entities collections.

After that, my in memory db context will be available for the using.

The source code of this topic is here.

In-memory DbContext with Entity Framework Core

Webpack with Angular2

In a previous post I spoke about the configurations to manage bundles in angular 2 with Systemjs.

For this purphose, the latest versions of Angular CLI uses webpack and we can use it as module bundler in the new angular 2/4 applications.

The reason of this choice is that webpack has a lot of plugins that they help us with a refined management of the modules bundling.

Entries

When we configure webpack, the first thing that we need to do is define the entries of the bundles.

Every bundle has the main entry point, that is the module where webpack start to build the bundle hierarchy.

I like to build two different bundle, the first one for the vendors libraries and the second one for the application modules.

The  main entry for the vendors is a typescript class that looks like this:


import '@angular/platform-browser';
import '@angular/platform-browser-dynamic';
import '@angular/core';
import '@angular/common';
import '@angular/http';
import '@angular/router';
import '@angular/animations';

import 'jquery';
import 'lodash';
import 'ng2-toastr';
import 'ng2-translate';
import 'reflect-metadata';
import 'zone.js';
import 'rxjs';

The second one is the main module for the app:


export { PostModule } from "./post.module";

Webpack is able to load these modules and build the dependencies of the childs descendans, and process them.

Loaders

Loaders are specific webpack tools that are able to process specific file types and to produce the right bundle outputs.

For example, in order to make webpack to be able to resolve the angular html templates or the router modules, or more simply to compile typescript files, specific loaders are needed.

Below you can see a bunch of loaders used in an Angular 2 application:


module: {
rules: [
{
test: /\.ts$/,
loaders: [
{
loader: 'awesome-typescript-loader',
options: { configFileName: 'tsconfig.json' }
}, 'angular2-template-loader', 'angular2-router-loader'
]
},
{
test: /\.html$/,
loader: 'html-loader'
},
{
test: /\.(png|jpe?g|gif|svg|woff|woff2|ttf|eot|ico)$/,
loader: 'file-loader?name=images/[name].[hash].[ext]'
},
{
test: /\.css$/,
exclude: helpers.root('src', 'app'),
loader: ExtractTextPlugin.extract({ fallbackLoader: 'style-loader', loader: 'css-loader?sourceMap' })
},
{
test: /\.css$/,
include: helpers.root('src', 'app'),
loader: 'raw-loader'
}
]
}

Plugins

These are utilities that we can help us in some additional tasks of the webpack script.

For example, we would like to share a common chunk and we would want that the chunk, if used from different module of the application, is not repeated in the bundle but is present only once.

CommonsChunkPlugin deal with this work:


plugins: [

.....

new webpack.optimize.CommonsChunkPlugin({
name: ['vendor']
}),

.....

]

Another requirement could be write the script references in the index.html file of the application.

HtmlWebpackPlugin can help us in this work:


plugins: [

.....

new HtmlWebpackPlugin({
filename: 'index.html',
template: 'src/index.html',
chunks: ['app', 'vendor']
}),

new HtmlWebpackPlugin({
filename: 'index.admin.html',
template: 'src/index.admin.html',
chunks: ['app.admin', 'vendor']
}),

.....
]

Output

In this section we can configure the file results of the webpack process.

We have to produce these files in the public folder of the application; the output section is look like this:


output: {
path: helpers.root('wwwroot'),
filename: 'js/[name].js',
chunkFilename: 'js/[id].chunk.js'
}

 

You can find the source code here.

 

Webpack with Angular2

Manage attachments chunks with ASP.NET Web Api

In the previous post I spoke about a custom MultipartFormData stream provider and how it can help us to manage some custom informations included in a request message.

In that example I generated chunks form a file and I sent those to a rest service (AKA Web API) with some additional informations that were then retrieved from the custom provider.

Now I want to use these informations to manage the upload session and merge all the chunks when received.

What I need to do is define the models involved in the process and the service that manage the chunks.

Models

We have to define two stuff, the first one is the model for the chunk:


public class ChunkMetadata
{
public string Filename { get; set; }
public int ChunkNumber { get; set; }

public ChunkMetadata(string filename, int chunkNumber)
{
Filename = filename;
ChunkNumber = chunkNumber;
}
}

The ChunkNumber property deserves an explanation; is the number associated to the chunk and will be useful to understand the correct order when we’ll have to merge all of them.

The second one is the model of the session, that is the bunch of the chunks that compose the file.

First of all we define the interface:


public interface IUploadSession
{
ConcurrentBag<ChunkMetadata> Chunks { get; set; }
string Filename { get; }
long Filesize { get; }
bool AddChunk(string filename, string chunkFileName, int chunkNumber, int totalChunks);
Task MergeChunks(string path);
}

The FileName and Filesize are closely tied to the session; we need AddChunk and MergeChunks methods as well.

We also need a thread safe collection for the chunks that compose the session, so we define a CuncurrentBag collection, that is the thread safe representation of the List.

Now we can implement the model:


public class UploadSession : IUploadSession
{
public string Filename { get; private set; }
public long Filesize { get; private set; }
private int _totalChunks;
private int _chunksUploaded;

public ConcurrentBag<ChunkMetadata> Chunks { get; set; }

public UploadSession()
{
Filesize = 0;
_chunksUploaded = 0;
Chunks = new ConcurrentBag<ChunkMetadata>();
}

public bool AddChunk(string filename, string chunkFileName, int chunkNumber, int totalChunks)
{
if (Filename == null)
{
Filename = filename;
_totalChunks = totalChunks;
}

var metadata = new ChunkMetadata(chunkFileName, chunkNumber);
Chunks.Add(metadata);

_chunksUploaded = Interlocked.Increment(ref _chunksUploaded);
return _chunksUploaded == _totalChunks;
}

public async Task MergeChunks(string path)
{
var filePath = path + Filename;

using (var mainFile = new FileStream(filePath, FileMode.Create))
{
foreach (var chunk in Chunks.OrderBy(c => c.ChunkNumber))
{
using (var chunkFile = new FileStream(chunk.Filename, FileMode.Open))
{
await chunkFile.CopyToAsync(mainFile);
Filesize += chunkFile.Length;
}
}
}

foreach (var chunk in Chunks)
{
File.Delete(chunk.Filename);
}
}
}

The implementation is quite simple.

The AddChunk method add the new chunk to the collection, then increment the _chunksUploaded property with the thread safe operation Interlocked.Increment; at the end, the method returns a bool that is true if all the chunks are received, otherwise false.

The MergeChunks method deal with the retrieve of all the chunks from the file system.

It gets the collection, order by the chunk number, read the bytes from the chunks and copy those to the main file stream.

After all, the chunks are deleted.

Service

The service will have an interface like this:


public interface IUploadService
{
Guid StartNewSession();
Task<bool> UploadChunk(HttpRequestMessage request);
}

In my mind, the StartNewSession method will instantiate a new Session object and assign a new correlation id that is the unique identifier of the session.

This is the implementation:


public class UploadService : IUploadService
{
private readonly Context _db = new Context();
private readonly string _path;
private readonly ConcurrentDictionary<string, UploadSession> _uploadSessions;

public UploadService(string path)
{
_path = path;
_uploadSessions = new ConcurrentDictionary<string, UploadSession>();
}

public async Task<bool> UploadChunk(HttpRequestMessage request)
{
var provider = new CustomMultipartFormDataStreamProvider(_path);
await request.Content.ReadAsMultipartAsync(provider);
provider.ExtractValues();

UploadSession uploadSession;
_uploadSessions.TryGetValue(provider.CorrelationId, out uploadSession);

if (uploadSession == null)
throw new ObjectNotFoundException();

var completed = uploadSession.AddChunk(provider.Filename, provider.ChunkFilename, provider.ChunkNumber, provider.TotalChunks);

if (completed)
{
await uploadSession.MergeChunks(_path);

var fileBlob = new FileBlob()
{
Id = Guid.NewGuid(),
Path = _path + uploadSession.Filename,
Name = uploadSession.Filename,
Size = uploadSession.Filesize
};

_db.FileBlobs.Add(fileBlob);
await _db.SaveChangesAsync();

return true;
}

return false;
}

public Guid StartNewSession()
{
var correlationId = Guid.NewGuid();
var session = new UploadSession();
_uploadSessions.TryAdd(correlationId.ToString(), session);

return correlationId;
}
}

In the StartNewSession method we use the thread safe method TryAdd to add a new session to the CuncurrentBag.

About the UploadChunk method, we seen the first part of the implementation in the previous post.

Once the metadata is retrieved from the request, we try to find the session object with a thread safe operation.

If we don’t find the object, of course we need to throw an exception because we expect that the related session exists.

If the session exists, we add the chunk to the session and we check the result of the operation.

If is the last chunk, we merge all of them and we can do a database operation if needed.

Controller

The implementation of the controller is very simple:


public class FileBlobsController : ApiController
{
private readonly IUploadService _fileBlobsService;
private readonly Context _db = new Context();

public FileBlobsController(IUploadService uploadService)
{
_fileBlobsService = uploadService;
}

[Route("api/fileblobs/getcorrelationid")]
[HttpGet]
public IHttpActionResult GetCorrelationId()
{
return Ok(_fileBlobsService.StartNewSession());
}

[HttpPost]
public async Task<IHttpActionResult> PostFileBlob()
{
if (!Request.Content.IsMimeMultipartContent())
throw new Exception();

var result = await _fileBlobsService.UploadChunk(Request);

return Ok(result);
}
}

You can find the source code here.

Manage attachments chunks with ASP.NET Web Api

Custom MultipartFormDataStreamProvider in C#

Frequently, when we manage multipart/form requests and we send them to the server, we might want to add some additional informations.

Perhaps we might want to split a big file in chunks and we might want to add some additional informations like the id of the upload session, the chunk number, the file name and the total chunks number that compose the file.

Suppose that we use for the client side Angularjs, the code of the controller is quite simple:


.....

public AddAttachment(event) {
let attachments = event.target.files;
if (attachments.length > 0) {
let file: File = attachments[0];

this.$http.get(this.url + "/GetCorrelationId").then((correlationId) => {
let chunks = this.SplitFile(file);

for (let i = 0; i < chunks.length; i++) {
let formData = new FormData();
formData.append("file", chunks[i], file.name);
formData.append("correlationId", correlationId.data);
formData.append("chunkNumber", i + 1);
formData.append("totalChunks", chunks.length);

this.$http.post(this.url, formData, { headers: { "Content-Type": undefined } }).then((result) => {
if(result.data) {
this.Load();
}
});
}
});
}
}

private SplitFile(file: File): Array<Blob> {
let chunks = Array<Blob>();
let size = file.size;
let chunkSize = 1024 * 1024 * 10;
let start = 0;
let end = chunkSize;

while (start < size) {
let chunk = file.slice(start, end);
chunks.push(chunk);
start = end;
end += chunkSize;
}

return chunks;
}

.....

The AddAttachment method is invoked by the view; once the file is retrieved, the split method generate the array of chunks.

Then, with the $http factory we send every single chunks to the server with additional metadata.

In order to read these datas from the server side, we need to implement a custom MultipartFormData stream provider.

The first step is define the interface of our provider:


public interface ICustomMultipartFormDataStreamProvider
{
string ChunkFilename { get; }
int ChunkNumber { get; }
string CorrelationId { get; }
string Filename { get; }
int TotalChunks { get; }
void ExtractValues();
}

The interface has the same properties sent by the client, and a method that deal with extract the values from the message.

Now we can proceed with the implementation:


public class CustomMultipartFormDataStreamProvider : MultipartFormDataStreamProvider, ICustomMultipartFormDataStreamProvider
{
public string Filename { get; private set; }
public string ChunkFilename { get; private set; }
public string CorrelationId { get; private set; }
public int ChunkNumber { get; private set; }
public int TotalChunks { get; private set; }

public CustomMultipartFormDataStreamProvider(string rootPath) : base(rootPath) { }

public CustomMultipartFormDataStreamProvider(string rootPath, int bufferSize) : base(rootPath, bufferSize) { }

public override Task ExecutePostProcessingAsync()
{
foreach (var file in Contents)
{
var parameters = file.Headers.ContentDisposition.Parameters;
var filename = ExtractParameter(parameters, "filename");
if (filename != null) Filename = filename.Value.Trim('\"');
}

return base.ExecutePostProcessingAsync();
}

public void ExtractValues()
{
var chunkFileName = FileData[0].LocalFileName;
var correlationId = FormData?.GetValues("correlationId");
var chunkNumber = FormData?.GetValues("chunkNumber");
var totalChunks = FormData?.GetValues("totalChunks");

if (string.IsNullOrEmpty(chunkFileName) || correlationId == null || chunkNumber == null || totalChunks == null)
throw new Exception("Missing values in UploadChunk session.");

ChunkFilename = chunkFileName;
CorrelationId = correlationId.First();
ChunkNumber = int.Parse(chunkNumber.First());
TotalChunks = int.Parse(totalChunks.First());
}

private NameValueHeaderValue ExtractParameter(ICollection<NameValueHeaderValue> parameters, string name)
{
return parameters.FirstOrDefault(p => p.Name.Equals(name, StringComparison.OrdinalIgnoreCase));
}
}

The class inherits from MultipartFormDataStreamProvider base class and implements our interface.

Two methods are implemented; the first one override ExecutePostProcessingAsync and in this method we retrieve the name of the main file.

The second one extract the custom parameters from the FormData; we retrieve also the chunk filename from the FileData object; this information is included as default information in the MultipartFormData message.

Now the informations are retrieved and we can use the custom provider in a service:


public async Task<bool> UploadChunk(HttpRequestMessage request)
{
var provider = new CustomMultipartFormDataStreamProvider(_path);
await request.Content.ReadAsMultipartAsync(provider);
provider.ExtractValues();

.....
}

The metadata will be available in the provider object.

You can find the source code here.

 

 

Custom MultipartFormDataStreamProvider in C#

Real-time search with ASP.NET and Elasticsearch

A common problem that we are faced when we have deployed our applications is improve the performance of a page or feature.

In my case for example I had a field where I could search and select a city, so the starting elements were a lot and the search was quite slow; I wanted a better user experience.

We can solve performance problems like these with the help of a cache or a full-text search.

I have chosen the last one and elastic search as full-text engine, so I’ll describe the steps that I followed to configure and use it in my application.

Installation

The first step is install the elastic search server, that you can download here.

Once installed we have to start it by executing the following executable:

<Installation path>\bin\elasticsearch.bat

This is the server log:

log

The server will take care to index the content that we will pass to it; in order to do that we need a client to use in our application; in my case the application was .NET and I used NEST.

NEST

As said above, NEST is an elastic search high level client for .NET applications.

The first step is install it in the application with nuget:

Install-package NEST

And in the package.config we’ll have:

package

Now we have all the necessary tools and we can develop the code for the search feature.

Client

We define a client class that has one responsability, that is setup the url and the default index of the client, and that instantiate it:


public class ElasticSearchClient
{
privatereadonlyIElasticClient _client;

publicElasticSearchClient(IElasticClient client)
{
_client = client;
}

publicElasticSearchClient(string uri, string indexName) : this(CreateElasticClient(uri, indexName)) {}

publicIElasticClientGetClient()
{
return_client;
}

privatestaticElasticClientCreateElasticClient(string uri, string indexName)
{
var node = newUri(uri);
var setting = newConnectionSettings(node);
setting.DefaultIndex(indexName);
returnnewElasticClient(setting);
}
}

Once instantiated, the class returns a new instance of the client; we can register it in the startup class of the application with autofac:


public partial class Startup
{
publicvoidConfiguration(IAppBuilder app)
{
var builder = newContainerBuilder();

builder.Register(c => newElasticSearchClient("http://localhost:9200", "cities"))
.AsSelf()
.SingleInstance();
...
}
}

Service base class

A service that uses an elasticsearch index should be able to do some basic operations, that concerns the logics of the full-text indexes.

We have to deal with the initialize a specific index, populate the index with the contents, obviously performs a search on the index with specific parameters.

So, we have to define an interface like this:


internal interface IElasticSearchService<T> where T : class
{
voidInit();
voidCheckIndex();
voidBulkInsert(List<T> objects);
IEnumerable<T> Search(string query);
}

I like to separate the init method, that create the index, from the checkindex method, that check if the index already exists.

Now we can implement the basic service:


public class ElasticSearchService<T> : IElasticSearchService<T> where T : class
{
protectedreadonlyContext Db = newContext();
protectedreadonlyElasticSearchClient ElasticSearchClient;
protectedreadonlystring IndexName;

publicElasticSearchService(ElasticSearchClient elasticSearchClient, string indexName)
{
ElasticSearchClient = elasticSearchClient;
IndexName = indexName;
}

publicvirtualvoidInit()
{
CheckIndex();
BulkInsert(Db.Set<T>().ToList());
}

publicvoidCheckIndex()
{
if (IndexExist()) return;
var response = CreateIndex();

if (!response.IsValid)
{
thrownewException(response.ServerError.ToString(), response.OriginalException);
}
}

publicvoidBulkInsert(List<T> objects)
{
var response = ElasticSearchClient.GetClient().IndexMany(objects, IndexName);
if (!response.IsValid)
{
thrownewException(response.ServerError.ToString(), response.OriginalException);
}
}

publicvirtualIEnumerable<T> Search(string query)
{
var results = ElasticSearchClient.GetClient().Search<T>(c => c.From(0).Size(10).Query(q => q.Prefix("_all", query)));

returnresults.Documents;
}

protectedvirtualIResponseCreateIndex()
{
var indexDescriptor = newCreateIndexDescriptor(IndexName).Mappings(ms => ms.Map<T>(m => m.AutoMap()));
returnElasticSearchClient.GetClient().CreateIndex(indexDescriptor);
}

protectedboolIndexExist()
{
returnElasticSearchClient.GetClient().IndexExists(IndexName).Exists;
}
}

The constructor accept the client and the index name.

We define a virtual init method, that check if the index exists and do a bulkinsert of a list of object; the method is virtual, we think that a derived service could override the method.

This bulkinsert method leverage the client to index the object list and the search method implements a basic search, that searchs in all the fields of the objects by using the special field _all, which contains the concatenate values of all fields.

The method returns the first 10 elements.

Createindex create a specific index with automap option, that infers the elasticsearch fields datatypes from the POCO object that we pass to it; it’s protected, so the derived class could use it.

IndexExists check if an index exists and it can be used from the derived class as well.

Service

Now we can implement a specific service, that inherits from ElasticSearchService class.

In this example I need to search in a list of cities and related districts, so I need to override the CreateIndex method like this:


public sealed class CitiesService : ElasticSearchService<City>
{
publicCitiesService(ElasticSearchClient elasticSearchClient, string indexName): base(elasticSearchClient, indexName) {}

protectedoverrideIResponseCreateIndex()
{
var indexDescriptor = newCreateIndexDescriptor(IndexName).Mappings(
ms => ms.Map<City>(m => m.AutoMap().Properties(ps =>
ps.Nested<District>(n => n
.Name(nn => nn.District)
.AutoMap()))));

returnElasticSearchClient.GetClient().CreateIndex(indexDescriptor);
}

publicoverrideIEnumerable<City> Search(string query)
{
var results = ElasticSearchClient.GetClient().Search<City>(c => c.From(0).Size(10).Query(q => q.Prefix(p => p.Name, query) || q.Term("district.name", query)));

returnresults.Documents.OrderBy(d => d.Name);
}
}

What I need to do is automap the city object and the district, that is a closely related entity of the city; so I have to map the District property as nested with the automap option as well.

Thus I will able to search for all the properties of the city and the district.

The other method that I override is the Search method; I search partially in the name of the city (Prefix) and the specific term in the district name (Term) and I returns the first 10 elements.

Now I have to register the service with autofac:


public partial class Startup
{
public void Configuration(IAppBuilder app)
{
var builder = newContainerBuilder();

builder.Register(c => newElasticSearchClient("http://localhost:9200", "cities"))
.AsSelf()
.SingleInstance();

builder.Register(c => newCitiesService(c.Resolve<ElasticSearchClient>(), "cities"))
.AsSelf()
.AsImplementedInterfaces()
.SingleInstance();

...
}
}

The last step is initialize the full text index of my service:


public partial class Startup
{
publicvoidConfiguration(IAppBuilder app)
{
var builder = newContainerBuilder();

builder.Register(c => newElasticSearchClient("http://localhost:9200", "cities"))
.AsSelf()
.SingleInstance();

builder.Register(c => newCitiesService(c.Resolve<ElasticSearchClient>(), "cities"))
.AsSelf()
.AsImplementedInterfaces()
.SingleInstance();

...

InitElasticSearchServices(containerBuilder);
}

privatestaticvoidInitElasticSearchServices(IContainer containerBuilder)
{
var citiesServices = containerBuilder.Resolve<CitiesService>();
citiesServices.Init();
}
}

I make a new instance of the service and call the Init method of the ElasticSearchService that we have seen above.

This method will create and populate the index.

Web API

Now I can use the service in my Web API, like this:


public class CitiesController : ApiController
{
privatereadonlyCitiesService _elasticSearchService;

publicCitiesController(CitiesService elasticSearchService)
{
_elasticSearchService = elasticSearchService;
}

// GET: api/Cities
publicIEnumerable<City> GetCities(string query)
{
return_elasticSearchService.Search(query);
}

protectedoverridevoidDispose(bool disposing)
{
base.Dispose(disposing);
}
}

You can find the source code of this topic here.

Real-time search with ASP.NET and Elasticsearch

Attachments management with Angular 2

A common issue that we faced in our applications is implement a component to allow the management of the attachment upload.

We need to insert a file input field in the page, grab the change event of the field, extract the file and send it to a service.

Recently I have needed to implement this functionality with Angular 2, so I’m going to explain what I have done.

Services

First of all I implement two different services, one for the file metadata and one for the blob object.

Based on a recent post, I use a base class WebApi and I define the service url:


import { Injectable } from "@angular/core";
import { Http } from "@angular/http";
import { Attachment } from "./attachment.model";
import { WebApi } from "../shared/webapi";

@Injectable()
export class AttachmentService extends WebApi<Attachment> {
constructor(public http: Http) {
super("/api/attachments", http);
}
}

The referenced service is a simple Restful service.

The second one is a service for the blob upload:


import { Injectable } from "@angular/core";
import { Http, Headers, RequestOptions, Response } from "@angular/http";
import { Observable } from "rxjs/Observable";
import { FileBlob } from "./fileBlob.model";
import { WebApi } from "../shared/webapi";

@Injectable()
export class FileBlobService extends WebApi<FileBlob> {
constructor(public http: Http) {
super("/api/fileBlobs", http);
}

public DownloadFile(id: string) {
window.open("api/fileBlobs/GetFileBlob?id=" + id, '_blank');
}

public PostFile(entity: File): Observable<File> {
let formData = new FormData();
formData.append(entity.name, entity);

return this.http.post(this.url, formData).map(this.extractData).catch(this.handleError);
}
}

The PostFile method compose a HTML FormData object with the content of the file and post it to a specific WebApi.

The DownloadFile method is simplier and call a service in a new window that returns the content of the file.

The server-side method is look like this:


public class FileBlobsController : ApiController
{
private readonly Context _db = new Context();

[ResponseType(typeof(Guid))]
public async Task<IHttpActionResult> PostFileBlob()
{
if (!Request.Content.IsMimeMultipartContent())
throw new Exception();

var provider = new MultipartMemoryStreamProvider();
await Request.Content.ReadAsMultipartAsync(provider);

HttpContent content = provider.Contents.First();
var fileName = content.Headers.ContentDisposition.FileName.Trim('\"');
var buffer = await content.ReadAsByteArrayAsync();

var fileBlob = new FileBlob()
{
Id = Guid.NewGuid(),
Name = fileName,
File = buffer
};

_db.FileBlobs.Add(fileBlob);
await _db.SaveChangesAsync();

return Ok(fileBlob.Id);
}
}

We have to use the MultiPartMemoryStreamProvider to retrieve the content of the file and store it in a specific table.

Component

We need two methods, the first one to download the existing attachment, the second one to add a new attachment:


import { Component, Input, Output, EventEmitter } from "@angular/core";
import { Constants } from "../shared/commons";
import { Attachment } from "./attachment.model";
import { FileBlobService } from "./fileBlob.service";
import { AlertService } from "../core/alert.service";

@Component({
moduleId: module.id,
selector: "attachment",
templateUrl: "attachment.component.html"
})

export class AttachmentComponent {
@Input() placeholder: string;
@Input() name: string;
@Input() validationEnabled: boolean;
@Input() attachment: Attachment;
@Output() onSaved = new EventEmitter<Attachment>();
public fileBlob: File;

constructor (private fileBlobService: FileBlobService, private alertService: AlertService) {}

public DownloadAttachment() {
this.fileBlobService.DownloadFile(this.attachment.IdFileBlob);
}

public AddAttachment(event) {
let attachments = event.target.files;
if (attachments.length > 0) {
let file:File = attachments[0];
this.fileBlobService.PostFile(file).subscribe(
(res) => {
let id: string = Constants.guidEmpty;

if (this.attachment != null)
id = this.attachment.Id;

this.attachment = {
Id: id,
IdFileBlob: res.toString(),
Name: file.name,
Size: file.size
};

this.onSaved.emit(this.attachment);
},
(error) => this.alertService.Error(error));
}
}

...
}

The AddAttachment method deserves an explanation; it accepts an event parameter, fired by the file input filed of the ui when a new attachment is selected.

The method retrieves the file from the event and pass it as a parameter to the PostFile method that we have seen above.

Once saved, an object with the file metadata is created and passed with the onSaved event to the parent component, that it deal with the object:


export class InvoiceDetailComponent {
...

public onAttachmentSaved(attachment: Attachment) {
this.attachment = attachment;
}
}

Module

We define a feature module like this:


import { NgModule } from "@angular/core";
import { HttpModule } from "@angular/http";

import { SharedModule } from "../shared/shared.module";
import { AttachmentComponent } from "./attachment.component";
import { AttachmentService } from "./attachment.service";
import { FileBlobService } from "./fileBlob.service";

let options: any = {
autoDismiss: true,
positionClass: 'toast-bottom-right',
};

@NgModule ({
imports: [
SharedModule,
HttpModule
],
exports: [
AttachmentComponent
],
declarations: [
AttachmentComponent
],
providers: [
AttachmentService,
FileBlobService
]
})

export class AttachmentModule {}

The module exports the component and provides the services discussed above.

View

We have to implement the view for the attachment module:

<div *ngIf="AttachmentIsNull()">
<label class="btn btn-primary" for="fileBlob">
<i class="fa fa-paperclip"></i> {{ "ATTACHINVOICE" | translate }}
<input id="fileBlob" type="file" [(ngModel)]="fileBlob" (change)="AddAttachment($event)" [required]="validationEnabled" style="display: none;" />
</label>
</div>
<div *ngIf="!AttachmentIsNull()">
<span *ngIf="attachment" (click)="DownloadAttachment()">{{attachment.Name}}</span>
<input type="button" class="btn btn-primary" value="Upload new" (click)="UploadNewAttachment()" />
</div>

In the view we have a file input field that bind the change event with the AddAttachment method.

The additional buttons allow us to clear the current attachment and upload a new one.

The last change is in the parent view:


<form #invoiceForm="ngForm">
<div class="form">
...
<div class="form-group">
<label for="attachment">{{ "ATTACHMENT" | translate }}</label>
<attachment placeholder="ATTACHMENT" name="attachment" [attachment]="attachment" (onSaved)="onAttachmentSaved($event)" [validationEnabled]="validationEnabled"></attachment>
</div>
</div>
</form>

We have added the attachment component in the view and we have binded the onSaved event, in order to retrieve the file metadata.

You can find the source code here.

 

 

 

 

Attachments management with Angular 2