
By Sematext Blog | Article Rating: |
|
June 12, 2017 08:02 AM EDT |

Logsene, our hosted ELK stack now part of Sematext Cloud, exposes the Elasticsearch API. As a Logsene user you may or may not be aware of this. If you have experience with Elasticsearch though, if you’re aware of its rich ecosystem of tools, you know how valuable this is. Besides using the Logsene UI, Kibana, or Grafana to search your logs or build charts and dashboards, you can also query all your data via Logsene’s Elasticsearch API. You can search your logs, filter them, run aggregation queries, use scan and scroll to retrieve large ranges of your data, etc.
Index Templates
When you send your logs to a Logsene app their structure (their mapping in Elasticsearch parlance) is automatically detected. Log event fields are created and their types are deduced from the received data. The mapping and the index template are also created. Under the hood Logsene maintains a sequence of indices for each Logsene app. Every so often a new index is added and its mapping is based on the index template for that Logsene app. If you are familiar with how Elasticsearch works you may have already created custom Elasticsearch index templates for your Logsene apps. This is useful when you want to change the structure of your index, change the type of a field, and so on. Until recently, after changing your Logsene app’s index template you’d have to wait until next index rotation in your Logsene app for this change to apply. You couldn’t force index rotation to happen right there and then. Bummer!
We’re happy to report that this annoying behaviour is now history! No more waiting! From now, when you push a new template to Logsene it will be applied immediately. Also, all templates are automatically verified, in real-time, in addition to verification that Elasticsearch does. Logsene checks all field definitions and, if any issues are found, reports them in the response. Valid templates are applied immediately, immediately visible, and ready for new data! We hope you like this!
How do I Create Custom Index Templates?
This is easy. We’ll borrow from the Custom Elasticsearch Index Templates in Logsene:
curl -XPUT 'logsene-receiver.sematext.com/_template/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee_MyTemplate' -d '{ "template" : "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee*", "order" : 21, "mappings" : { "message" : { "properties" : { "message" : { "type" : "string" }, "tags" : { "type" : "string", "analyzer" : "my_own_lowercase" }, "nick" : { "type" : "string", "analyzer" : "my_own_lowercase" } } } } }'
Please see Custom Elasticsearch Index Templates in Logsene for a few more important details.
What About The Mappings?
Logsene will let you set your index mappings, too. Note that you will need to know the name of the underlying index in order to change the mapping. You can get the index name by searching your Logsene app via the API and looking for index names in the response. That said, we suggest you simply set the template instead of the mapping. Setting the template provides you a superset of functionality. It doesn’t change the mapping of the current index, but instead creates the mapping/template for your future indices and immediately creates a new index for you.
Looking for a Hosted ELK as a Service? Try Sematext Cloud!
Get actionable insights while using the first All-in-One Monitoring and Logging solution.
What’s Next?
In the next Logsene release you will be able to modify your mapping via a UI wizard that will make index mapping changes as easy as click-click-click. You’ll be able to add or remove fields, change their types or properties, etc. The wizard will also warn you about any incompatible changes you try to make and let you optionally reindex (or delete) historical data. More on that soon!
Read the original blog entry...
Published June 12, 2017
Copyright © 2017 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Sematext Blog
Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.
![]() Apr. 24, 2018 01:15 AM EDT Reads: 16,151 |
By Yeshim Deniz Apr. 24, 2018 01:00 AM EDT Reads: 2,435 |
By SmartBear Blog ![]() Apr. 24, 2018 01:00 AM EDT Reads: 2,003 |
By Liz McMillan ![]() Apr. 24, 2018 12:00 AM EDT Reads: 8,112 |
By Pat Romanski ![]() Apr. 23, 2018 11:30 PM EDT Reads: 1,978 |
By Pat Romanski Apr. 23, 2018 11:15 PM EDT Reads: 2,817 |
By Elizabeth White ![]() Apr. 23, 2018 11:00 PM EDT Reads: 1,134 |
By Elizabeth White ![]() Apr. 23, 2018 11:00 PM EDT Reads: 6,433 |
By Liz McMillan ![]() Apr. 23, 2018 10:45 PM EDT Reads: 6,998 |
By Pat Romanski ![]() Apr. 23, 2018 10:45 PM EDT Reads: 7,184 |
By Liz McMillan ![]() Apr. 23, 2018 10:30 PM EDT Reads: 17,449 |
By Yeshim Deniz ![]() Apr. 23, 2018 09:45 PM EDT Reads: 6,865 |
By Liz McMillan ![]() Apr. 23, 2018 09:45 PM EDT Reads: 22,657 |
By Elizabeth White ![]() Apr. 23, 2018 08:45 PM EDT Reads: 9,238 |
By Yeshim Deniz Apr. 23, 2018 08:00 PM EDT Reads: 2,748 |