Welcome!

Blog Feed Post

Securing Elasticsearch and Kibana with Search Guard for free

Note: This is a guest post by Jochen Kressin, the CTO of floragunn GmbH, the makers of Search Guard, an open-source X-Pack Security alternative.

In this article, we show you how to secure Elasticsearch and Kibana for free using the Community edition of Search Guard. We start with a vanilla Elasticsearch and Kibana setup, install and configure Search Guard for Elasticsearch, and use the Search Guard Kibana plugin to add session management capabilities to Kibana.

Prerequisites

As a prerequisite, install the latest Elasticsearch and Kibana version. At the time of writing, this is 5.4.0.

Installing Elasticsearch

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html

Installing Kibana:

https://www.elastic.co/guide/en/kibana/current/setup.html

Installing Search Guard

Search Guard is an Elasticsearch plugin, and can thus be installed by using the standard plugin installation procedure of Elasticsearch.

First, figure out which version of Search Guard you need for your Elasticsearch installation. Due to compatibility checks in Elasticsearch, the version of Search Guard must match the version of Elasticsearch exactly. So if you run Elasticsearch 5.4.0, you need to install Search Guard 5.4.0. Other versions will not work. If in doubt, you can always refer to our version matrix, which lists all available versions of Search Guard. In this tutorial, we use Elasticsearch and Kibana 5.4.0.

First, stop all running nodes, change to the installation directory of Elasticsearch and install Search Guard:

bin/elasticsearch-plugin install -b com.floragunn:search-guard-5:5.4.0-12

The plugin will be downloaded from Maven and installed automatically. Once the installation is complete, you will find a new folder plugins/search-guard-5 in your Elasticsearch installation directory.

Configuring TLS

Search Guard makes heavy use of TLS to encrypt and secure the complete traffic in your cluster. In order for TLS to work, you need to generate and install TLS certificates on all nodes. This makes sure that no one can sniff or tamper with your data, and that only trusted nodes can join the cluster.

There are several ways of generating the necessary certificates, but for this quickstart tutorial we will use the demo installation script that ships with Search Guard. For other ways of generating certificates, please refer to the section “Configuring Search Guard SSL” of the official Search Guard documentation.

In order to execute the demo installation script, change to the directory

<ES installation directory>/plugins/search-guard-5/tools

and execute the script

install_demo_configuration.sh

Depending on which platform you are on, you may need to set execution permissions first. The script will generate the TLS certificates and add the corresponding configuration settings to the elasticsearch.yml file. Once this is finished, you can start your nodes again.

Initializing Search Guard

After your nodes are up, you need to initialize Search Guard. In this tutorial, we will use the sample configuration files that ship with Search Guard. They already contain the correct users, roles and permissions for the Kibana integration.

The configuration files are located in this directory:

<ES installation directory>/plugins/search-guard-5/sgconfig

In order to upload this configuration to Search Guard, you can either use the sgadmin command line tool, or the REST management API. When using the sgadmin command line tool, you need to specify various parameters, such as the path to your certificates or the name of your cluster. You can read all about sgadmin in the official documentation.

For our quick start tutorial, we will use the demo sgadmin script, which already contains all required parameters. Change to:

<ES installation directory>/plugins/search-guard-5/tools

and execute the script

sgadmin_demo.sh

As with the installation script, you might need to set execution permissions first. After the script has been executed, Search Guard is initialized and your cluster is fully secured by TLS.

You can verify that everything works as expected by visiting the following URL with a browser:

https://localhost:9200/_searchguard/authinfo

Since we are using self-signed certificates, ignore the browser warning and accept the Root certificate that Search Guard is using. The warning message varies from browser to browser. Note that because TLS is now enabled on the REST layer, you have to use https when talking to Elasticsearch. Connecting with unsecured http will not work anymore.

When prompted for a username and password, you can use admin/admin. This user has full access to the cluster and all indices. You should see information about the currently logged in user in JSON format.

Congratulations, Search Guard is now installed and configured properly.

If you want to learn more about the structure and contents of the configuration files, refer to chapter Search Guard configuration of the documentation. After you make changes to the configuration, simply execute sgadmin_demo.sh again for the changes to take effect.

Adding Kibana to the mix

As with Elasticsearch, Kibana does not provide any security or session management out of the box. As the first step, we will configure Kibana so that it is able to talk to our Search Guard secured cluster. In the second step, we will install the Search Guard Kibana plugin which adds session management capabilities.

Configuring Kibana

For Kibana to work with a Search Guard secured cluster, we need to make some adjustments in the kibana.yml configuration file. First, change the default Elasticsearch URL from http to https:

elasticsearch.url: "https://localhost:9200"

And since we are using self-signed certificates in this tutorial, we need to tell Kibana to accept them explicitly by disabling certificate verification:

elasticsearch.ssl.verificationMode: none

This is of course not recommended for production. In production, you would rather install and configure your Root CA in Kibana, but that’s out of the scope of this tutorial.

Kibana is using a service user under the hood to manage the Kibana index, check the cluster health etc. This service user has to be present in Search Guard, and it needs to have the correct permissions to execute all required actions. The demo configuration already contains such a service user. The username and password of this user is “kibanaserver”. To tell Kibana to use these credentials when talking to Elasticsearch, add the following lines to kibana.yml:

elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"

These are the minimal changes you need to perform. You can now start Kibana again and check that the connection to Elasticsearch works correctly.

Installing the Kibana plugin

If you now try to access Kibana, you will see a Basic Authentication dialog popping up. This works, but we’d like to have a proper login dialogue and the possibility to logout again, without closing the browser. This is where the Kibana plugin comes into play.

As with Search Guard, you need to install the plugin that matches your Kibana version exactly. In our case, this is version 5.4.0-2. You can find all available versions on the GitHub release page:

https://github.com/floragunncom/search-guard-kibana-plugin/releases

Stop Kibana and download the zip file of the plugin to a directory on your machine:

https://github.com/floragunncom/search-guard-kibana-plugin/releases/download/v5.4.0/searchguard-kibana-5.4.0-2.zip

After that, install it like any other Kibana plugin: Change to the Kibana installation directory and execute:

bin/kibana-plugin install file:///path/to/searchguard-kibana-5.4.0-2.zip

This will install the plugin, and run the Kibana optimizer afterwards. This may take a couple of minutes. After that, restart Kibana and access it via

http://localhost:5601

You will be redirected to the login page and are prompted to provide username and password.

https://sematext.com/wp-content/uploads/2017/05/image1-237x300.png 237w" sizes="(max-width: 704px) 100vw, 704px" />Use the demo Kibana user to login, the username and password is “kibanaro”. This user is mapped to the role sg_kibana and has access to all indices.

That’s it. Both Elasticsearch and Kibana are secured with Search Guard.

Where to go next?

Deep-dive into Search Guard by reading the official documentation:

https://github.com/floragunncom/search-guard-docs

Learn more about integrating Kibana and Search Guard:

https://github.com/floragunncom/search-guard-docs/blob/master/kibana.md

Add Kibana multi tenancy to separate dashboards and visualizations

https://github.com/floragunncom/search-guard-docs/blob/master/multitenancy.md

Ask questions on the Search Guard Google Group:

https://groups.google.com/forum/#!forum/search-guard

Read about our fair, flexible and affordable enterprise license model:

https://floragunn.com/searchguard/searchguard-license-support/

Follow us on Twitter to stay up-to-date with new releases and features:

https://twitter.com/searchguard

 

Read the original blog entry...

More Stories By Sematext Blog

Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.

Latest Stories
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its ...
Microsoft Azure Container Services can be used for container deployment in a variety of ways including support for Orchestrators like Kubernetes, Docker Swarm and Mesos. However, the abstraction for app development that support application self-healing, scaling and so on may not be at the right level. Helm and Draft makes this a lot easier. In this primarily demo-driven session at @DevOpsSummit at 21st Cloud Expo, Raghavan "Rags" Srinivas, a Cloud Solutions Architect/Evangelist at Microsoft, wi...
Containers are rapidly finding their way into enterprise data centers, but change is difficult. How do enterprises transform their architecture with technologies like containers without losing the reliable components of their current solutions? In his session at @DevOpsSummit at 21st Cloud Expo, Tony Campbell, Director, Educational Services at CoreOS, will explore the challenges organizations are facing today as they move to containers and go over how Kubernetes applications can deploy with lega...
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: imple...
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, will discuss how by using...
Today most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes significant work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reducti...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
SYS-CON Events announced today that Yuasa System will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Yuasa System is introducing a multi-purpose endurance testing system for flexible displays, OLED devices, flexible substrates, flat cables, and films in smartphones, wearables, automobiles, and healthcare.
SYS-CON Events announced today that CAST Software will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CAST was founded more than 25 years ago to make the invisible visible. Built around the idea that even the best analytics on the market still leave blind spots for technical teams looking to deliver better software and prevent outages, CAST provides the software intelligence that matter ...
SYS-CON Events announced today that Daiya Industry will exhibit at the Japanese Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ruby Development Inc. builds new services in short period of time and provides a continuous support of those services based on Ruby on Rails. For more information, please visit https://github.com/RubyDevInc.
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
As businesses evolve, they need technology that is simple to help them succeed today and flexible enough to help them build for tomorrow. Chrome is fit for the workplace of the future — providing a secure, consistent user experience across a range of devices that can be used anywhere. In her session at 21st Cloud Expo, Vidya Nagarajan, a Senior Product Manager at Google, will take a look at various options as to how ChromeOS can be leveraged to interact with people on the devices, and formats th...