Welcome!

Blog Feed Post

Log Management for IBM Bluemix and Cloud Foundry

Enterprises without DevOps teams and culture typically experience long applications deployment cycles. Once developers make a new release the operations team needs to deploy it to one of the existing or new servers, and it often takes a long time (up to several weeks is not uncommon) to get a new application deployed to production. This process may include hardware purchases, configuration, and troubleshooting sessions with the development team. Especially the troubleshooting portion might take a while, with the typical e-mail ping-pong with commands for the administrators, sysadmins sending log files to the developers… Modern digital businesses don’t operate that way any more.  Instead, modern DevOps teams often utilize PaaS to boost productivity, without even thinking of server resources, ansible/chef/puppet scripts and related stuff. They push their application releases with a few commands to “the cloud” – typically a platform as a service (PaaS) like IBM Bluemix! Once deployments get simplified in such a way, the next logical step is to disrupt the e-mail ping-pong for troubleshooting by introducing centralized log management. Logs are the most important source of information for application troubleshooting. Central log management is crucial for providing real-time log access to relevant teammates.  

In this post, we’ll show how to do this by deploying a Node.js app to IBM Bluemix and using Logsene for Log Management.


Let’s start with the deployment of a Node.js app on IBM Bluemix. First, we’ll clone a git repository.

> git clone hello-world-app
> cd hello-world-app

Next, because IBM Bluemix is based on Cloud Foundry, we’ll use a few commands to get access to IBM Bluemix:

> cf api https://api.ng.bluemix.net> cf login

When prompted for username and password enter your IBM ID (email) and password. The final step is to push the app in the current directory to IBM Bluemix:

> cf push app1

This takes a while and the command prints the deployment status to the console. Once your app starts successfully, it will be available via the displayed URL (orgSpace-appName), e.g.

Showing health and status for app hello-world-seti123 in org ORG1 / space seti321 as [email protected]…
OK
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: hello-world-seti123.mybluemix.net
last uploaded: Tue Dec 13 16:02:11 UTC 2016
stack: cflinuxfs2
buildpack: SDK for Node.js(TM) (ibm-node.js-4.6.2, buildpack-v3.9-20161128-1327)
     state     since                    cpu    memory         disk           details
#0   running   2016-12-13 05:03:27 PM   0.0%   114.9M of 1G   117.8M of 1G

Access the app via curl or the browser and check the most recent application logs in the IBM Bluemix user interface.

image02https://i0.wp.com/sematext.com/wp-content/uploads/2017/01/image02.png?re... 300w, https://i0.wp.com/sematext.com/wp-content/uploads/2017/01/image02.png?re... 768w, https://i0.wp.com/sematext.com/wp-content/uploads/2017/01/image02.png?re... 1024w, https://i0.wp.com/sematext.com/wp-content/uploads/2017/01/image02.png?w=... 1296w" sizes="(max-width: 648px) 100vw, 648px" data-recalc-dims="1" />

While at first you may think Bluemix provides log management service and wonder why you’d need to ship your logs from Bluemix to a Log Management SaaS, you will quickly realize that IBM Bluemix shows only the most recent logs. That, of course, is a no-go for production, where we need logs for at least the last few days, several weeks, or even archived for a longer period.

The Cloud Foundry documentation shows the option to forward logs via so called log drains from any Cloud Foundry app to centralized Log Management tools like Logsene. Here you can choose the retention time, save search queries, define alerts on logs, or visualize them in Kibana! The setup for this is done with another small set of  Cloud Foundry commands:

  1. Create a Logsene Application to obtain the Logsene Application token
  2. Create the log drain to your Logsene App (note the Logsene token in the URL):
    cf cups logsene-log-drain -l https://logsene-cf-receiver.sematext.com/1a4a748f-33c3-4e73-9e25-f3f0c82ff0f2
  3. Bind the log drain to your app. You could optionally bind multiple apps to the same log drain:
    cf bind-service YOUR-APP-NAME logsene-log-drain
  4. Re-stage the app to pick up the newly bound service:
    cf restage YOUR-APP-NAME

A few seconds later your logs will start arriving to Logsene and you will be able to search and visualize them in Logsene’s built-in Kibana:image01https://i1.wp.com/sematext.com/wp-content/uploads/2017/01/image01.png?re... 300w, https://i1.wp.com/sematext.com/wp-content/uploads/2017/01/image01.png?re... 768w, https://i1.wp.com/sematext.com/wp-content/uploads/2017/01/image01.png?re... 1024w, https://i1.wp.com/sematext.com/wp-content/uploads/2017/01/image01.png?w=... 1296w" sizes="(max-width: 648px) 100vw, 648px" data-recalc-dims="1" />

IBM Bluemix logs in Kibana

image00https://i2.wp.com/sematext.com/wp-content/uploads/2017/01/image00.png?re... 300w, https://i2.wp.com/sematext.com/wp-content/uploads/2017/01/image00.png?re... 768w" sizes="(max-width: 648px) 100vw, 648px" data-recalc-dims="1" />

Alert definition for logs in Logsene

Your logs are safe for the configured retention time, and if you need to archive your logs to Amazon S3, Logsene can archive your logs to S3 for you, too. Just another little time safer for modern DevOps teams!

SIGN UP – FREE TRIAL

Read the original blog entry...

More Stories By Sematext Blog

Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.

Latest Stories
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists looked at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deliver...
"We do one of the best file systems in the world. We learned how to deal with Big Data many years ago and we implemented this knowledge into our software," explained Jakub Ratajczak, Business Development Manager at MooseFS, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Tintri focuses on the Ops side of the DevOps, which basically is pushing more and more of the accessibility of the infrastructure to the developers and trying to get behind the scenes," explained Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Cloud applications are seeing a deluge of requests to support the exploding advanced analytics market. “Open analytics” is the emerging strategy to deliver that data through an open data access layer, in the cloud, to be directly consumed by external analytics tools and popular programming languages. An increasing number of data engineers and data scientists use a variety of platforms and advanced analytics languages such as SAS, R, Python and Java, as well as frameworks such as Hadoop and Spark...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
Both SaaS vendors and SaaS buyers are going “all-in” to hyperscale IaaS platforms such as AWS, which is disrupting the SaaS value proposition. Why should the enterprise SaaS consumer pay for the SaaS service if their data is resident in adjacent AWS S3 buckets? If both SaaS sellers and buyers are using the same cloud tools, automation and pay-per-transaction model offered by IaaS platforms, then why not host the “shrink-wrapped” software in the customers’ cloud? Further, serverless computing, cl...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), provided an overview of various initiatives to certify the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldwide re...