GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND NET

GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND. NET WITH NEST A short introduction Oslo/NNUG Meetup Tomas Jansson 29/01/2014

THIS IS ME Tomas Jansson Manager & Group Lead. NET BEKK Oslo @Tomas. Jansson tomas. jansson@bekk. no github. com/mastoj blog. tomasjansson. com

TL; DR; https: //github. com/mastoj/Nest. Demo

AUDIENCE N 00 b Expert

BACKGROUND This is the data and we need this new application

THE MASTERPLAN

WHAT I WANT TO SHOW YOU IS. . . Elasticsearch is awesome Indexing using NEST Querying using NEST. . . not about advanced elasticsearch hosting

INSTALLATION Great news, install as a service added in 0. 90. 5 Powershell to the rescue

NEST Abstraction over Elasticsearch There is an low level abstraction as well called Raw. Elastic. Client

NEST Abstraction Fluent & Strongly over Elasticsearch typed

Functional C#

FUNC DEMO C: Devgit> scriptcs (ctrl-c or blank to exit) > Func<int, int> add = (x, y) => x + y; > add(1, 3) 4 Func executable

SIMPLE EXPRESSION DEMO > using System. Linq. Expressions; > Expression<Func<int, int>> add. Expr = (x, y) => x + y; > add. Expr(1, 3) Expression ”function description” (1, 1): error CS 1955: Non-invocable member 'add. Expr' cannot be used like a method. > var bin. Expr = add. Expr. Body as Binary. Expression; > Console. Write. Line(bin. Expr); (x + y) > var add 2 = add. Expr. Compile(); > add 2(3, 1); 4

MORE COMPLEX EXPRESSION DEMO > public class Some. Class { public string My. String { get; set; } } > Expression<Func<Some. Class, object>> prop. Expr = y => y. My. String + y. My. String; > var comp. Expr = prop. Expr. Compile(); > var obj = new Some. Class { My. String = "Hello world" }; > comp. Expr(obj) Hello world > var body = prop. Expr. Body as Binary. Expression; > Console. Write. Line(body); (y. My. String + y. My. String) > var left = body. Left as Member. Expression; > Console. Write. Line(left. Member. Name); My. String

MORE COMPLEX EXPRESSION DEMO > public class Some. Class { public string My. String { get; set; } } > Expression<Func<Some. Class, object>> prop. Expr = y => y. My. String + y. My. String; > var comp. Expr = prop. Expr. Compile(); > var obj = new Some. Class { My. String = "Hello world" }; > comp. Expr(obj) Hello world > var body = prop. Expr. Body as Binary. Expression; Enables us to translate from one domain to another in an ”easy” manner > Console. Write. Line(body); (y. My. String + y. My. String) > var left = body. Left as Member. Expression; > Console. Write. Line(left. Member. Name); My. String

Show me the code!

ELASTICSEARCH CONNECTION public class Elastic. Client. Wrapper : Elastic. Client { private static string _connection. String = Settings. Elastic. Search. Server; private static Connection. Settings _settings = new Connection. Settings(new Uri(_connection. String)) //http: //demoserver: 9200. Set. Default. Index(Settings. Alias) //"customer_product_mapping". Use. Pretty. Responses(); public Elastic. Client. Wrapper() : base(_settings) { } } //usage var client = new Elastic. Client. Wrapper();

MAPPING public class Product { public double Unit. Price { get; set; } public int Total. Quantity { get; set; } [Elastic. Property(Index = Field. Index. Option. not_analyzed)] public string Product. Name { get; set; } [Elastic. Property(Index = Field. Index. Option. not_analyzed)] public string Category. Name { get; set; } } public class Customer { public string Customer. ID { get; set; } public string Company. Name { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [Elastic. Property(Type = Field. Type. nested)] public Product[] Products { get; set; } }

MAPPING & INDEXING _client = new Elastic. Client. Wrapper(); Mapping created from attributes _client. Create. Index("index. Name", s => s. Add. Mapping<Customer>(m => m. Map. From. Attributes())); var customers = _customer. Repo. Get. Customers(); _client. Index. Many(customers, "index. Name"); Indexing will use the mapping for the specified index There is async versions of the methods

ALIAS _client = new Elastic. Client. Wrapper(); _client. Alias("index. Name", "alias. Name"); Alias Index_01

SWAPPING _client = new Elastic. Client. Wrapper(); _client. Swap("alias. Name", new [] { "Index_01" }, new [] { "Index_02" } ); 1. Create new index 2. Swap 3. Delete old index Alias Index_01 Index_02

MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES) public class Search. Model { private int? _number. To. Take; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? Number. To. Take { get { return _number. To. Take. Has. Value ? _number. To. Take. Value : 25; } set { _number. To. Take = value; } } }

QUERYING Elasticsearch NEST { _client. Search<Customer>(sd => sd. Query. String(Input. Query)); "query": { "query_string": { "query": "tomas" } } }

FUZZY Elasticsearch NEST { _client. Search<Customer>(sd => sd. Query(q => q. Fuzzy(fd => fd. On. Field("_all"). Min. Similarity(0. 6). Prefix. Length(1). Value(Input. Query)))); "query": { "fuzzy": { "_all": { "min_similarity": 0. 6, "prefix_length": 1, "value": "tomas" } } Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»

FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH { "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } } }, { "fuzzy": { "_all": { "boost": 0. 1, "min_similarity": 0. 6, "prefix_length": 1, "value": "tomas" } } }] } } }

FUZZY IMPROVED (USING BOOL QUERY) - NEST _client. Search<Customer>(sd => sd. Query(q => q. Bool(b => b. Should(new Func<Query. Descriptor<Customer>, Base. Query>[] { _ => _. Match(m => m. On. Field("_all"). Query. String(Input. Query)), _ => _. Fuzzy(fd => fd. On. Field("_all"). Min. Similarity(0. 6). Prefix. Length(1). Value(Input. Query). Boost(0. 1)) }))));

HIGHLIGHT RESULT - ELASTICSEARCH { "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" ], "fields": { "company. Name": { "fragment_size": 100, "number_of_fragments": 1 } }

HIGHLIGHT RESULT - NEST _client. Search<Customer>(sd => sd. Query( /* See previous example */ ). Highlight(h => h. Pre. Tags("<span class='highlight'>"). Post. Tags("</span>"). On. Fields(new Action<Highlight. Field. Descriptor<Customer>>[] { _ => _. On. Field(c => c. Company. Name). Number. Of. Fragments(1). Fragment. Size(100) })));

FACETS - ELASTICSEARCH { "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products. product. Name": { "nested": "products", "terms": { "field": "products. product. Name", "size": 1000 } }, "products. category. Name": { "nested": "products", "terms": { "field": "products. category. Name", "size": 1000 } }, "country": { "terms": { "field": "country", "size": 1000 } }

FACETS - NEST _client. Search<Customer>(sd => sd. Query( /* See previous example */ ). Highlight( /* See previous example */ ). Facet. Term(f => f. Nested(c => c. Products). On. Field(c => c. Products[0]. Product. Name). Size(1000)). Facet. Term(f => f. Nested(c => c. Products). On. Field(c => c. Products[0]. Category. Name). Size(1000)). Facet. Term(f => f. On. Field(c => c. Country). Size(1000)));

http: //go-gaga-over-testing. blogspot. no/2011/09/solution-to-warning-in-quality-center. html

FILTERS - ELASTICSEARCH { "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ { "terms": { "country": ["usa"] } }, { "nested": { "query": { "terms": { "products. category. Name": ["Condiments", "Seafood"] } }, "path": "products" } }, { "nested": { "query": { "terms": { "products. product. Name": ["Chai"] } }, "path": "products" } } ] } }, "facets": { /* See previous example */}, "highlight": { /* See previous example */ } }

FILTERS – NEST – PART 1, THE CUSTOMERS FILTER private static Base. Filter Add. Customer. Filter(IEnumerable<string> items, Expression<Func<Customer, object>> prop. Expr) { return Filter<Customer>. Terms(prop. Expr, items. To. Array()); }

FILTERS – NEST – PART 1, THE PRODUCTS FILTER private static Base. Filter Add. Products. Filter(IEnumerable<string> items, Expression<Func<Customer, object>> prop. Expr) { return Filter<Customer>. Nested(sel => sel. Path(c => c. Products). Query(q => q. Terms(prop. Expr, items. To. Array()))); }

FILTERS – NEST – PART 1, THE MAGIC DICTIONARY public Dictionary<string, Func<IEnumerable<string>, Base. Filter>> Filter. Desc = new Dictionary<string, Func<IEnumerable<string>, Base. Filter>>() { {"products. product. Name", ps => Add. Products. Filter(ps, c => c. Products[0]. Product. Name)}, {"products. category. Name", cs => Add. Products. Filter(cs, c => c. Products[0]. Category. Name)}, {"country", cs => Add. Customer. Filter(cs, c => c. Country)} };

FILTERS – NEST – PART 1, ALL THE HELPERS private static Base. Filter Add. Customer. Filter(IEnumerable<string> items, Expression<Func<Customer, object>> prop. Expr) { return Filter<Customer>. Terms(prop. Expr, items. To. Array()); } private static Base. Filter Add. Products. Filter(IEnumerable<string> items, Expression<Func<Customer, object>> prop. Expr) { return Filter<Customer>. Nested(sel => sel. Path(c => c. Products). Query(q => q. Terms(prop. Expr, items. To. Array()))); } public Dictionary<string, Func<IEnumerable<string>, Base. Filter>> Filter. Desc = new Dictionary<string, Func<IEnumerable<string>, Base. Filter>>() { {"products. product. Name", ps => Add. Products. Filter(ps, c => c. Products[0]. Product. Name)}, {"products. category. Name", cs => Add. Products. Filter(cs, c => c. Products[0]. Category. Name)}, {"country", cs => Add. Customer. Filter(cs, c => c. Country)} };

FILTERS – NEST – PART 2, THE QUERY _client. Search<Customer>(sd => sd. Query(q => q. Filtered(fq => { fq. Query(qs => { if (!string. Is. Null. Or. Empty(Input. Query)) { qs. Bool( /* See previous example */ )); } else { qs. Match. All(); } return qs; }); if (Input. Filter. Count > 0) { var filters = Input. Filter. Select(_ => Filter. Desc[_. Key](_. Value)). To. Array(); fq. Filter(fs => fs. Bool(bf => bf. Must(filters))); } })). Highlight( /* See previous example */ ). Facet. Term( /* See previous example */ );

SUMMARY Elasticsearch NEST Easy installation Strongly typed client Awesome search engine Fluent Abstraction over Elasticsearch

RESOURCES Demo code: https: //github. com/mastoj/Nest. Demo Nest documentation: http: //nest. azurewebsites. net/ Nest source code: https: //github. com/Mpdreamz/NEST Slideshare: http: //www. slideshare. net/mastoj/getting-started-with-elasticsearch-and-net Sense (great tool to query elastic search in the browser): https: //github. com/bleskes/sense

Questions?

Thank you! @Tomas. Jansson