Welcome!

Related Topics: FinTech Journal, @DevOpsSummit

FinTech Journal: Article

Log-level Usage Reports | @DevOpsSummit #DevOps #Docker #Monitoring

The reports can give you an intelligent breakdown of your log volumes by application, server, process

What You Can Learn From Log-level Usage Reports

Separating the signal from the noise is one of the biggest challenges when dealing with machine-generated log data today and has generally required deep technical expertise. However once you find that signal it can be massively useful and can help you make business decisions with a big impact. Today Logentries is announcing log level usage reports, which is one more way we are striving to do the hard work so you don't have to!

log-level-usage-reports

At first glance, log-level usage reports might seem quite rudimentary insofar as they are weekly reports of the log volumes you send to Logentries. However, the reports can give you an intelligent breakdown of your log volumes by application, server, process as well as at the individual log level.The usage reports are designed to provide IT and Dev Ops teams with a weekly snapshot of their system activity broken down at a very fine-grained log level giving the ability to see interesting trends in system behavior. In fact, so many of our customers have been asking our newLogentries Data Insights team to run these reports that we've decided to make them widely available. The reports are part of a new service being produced by this team comprised of PhDs and applied research scientists in distributed systems,  and they are supported by Logentries proprietary machine learning and community analytics technology.

Over the past few months we have  used these reports to help a large number of our customers to investigate spikes in system load, potential issues as well as to understand business trends and how they relate to their systems - and might I add with phenomenal results -  based on some really simple analysis. I suppose it doesn't need to be complex to be valuable...

But don't take my word for it...according to Rich Archbold, director of Ops at Intercom

"The new Logentries Log Analytics Reporting service has provided us with insight into our environment that we would never have been able to get before. We now have deep, log-level visibility into application activity, trends, and potential issues on a weekly basis delivered right into our inbox!"

Specifically the usage reports will provide the following:

  • Total number of logs and log sets
  • Total log volume across all logs
  • Top 20 logs by volume
  • Daily breakdown of log volume per log
  • List of inactive logs

Screen Shot 2015-05-07 at 9.55.22 AM

Sample Usage Report
The new reports can be especially useful for:

  • Logging cost analysis: When your log volumes increase significantly you usually want to know why. Looking at your total log volumes will generally not give enough insight to allow you to figure out what has caused the increase or from what part of your system the increase is coming from. However by getting a breakdown of your top 20 logs by volume you can immediately see the log or logs responsible, allowing you to take the next step in determining as to whether this was a result of increased system load or whether this was related to a system issue or rogue process.
  • System load analysis: Log volumes are a great indicator for understanding system load. As load increases so too do your log volumes. Understanding what log(s) are resulting in the increased load will give you better visibility into what part of your system is actually seeing the increased workloads.
  • Troubleshooting: The most common cause of a sudden spike in log volumes is generally not sudden increased  load on a system, but unfortunately is more often a result of a misconfiguration or application bug that starts to produce cascading errors. A sudden spike from a specific log is often an indicator of issues emanating from a particular application component. Similarly if logs suddenly become inactive, it may signify something has gone horribly wrong.
  • Infrastructure usage analysis: When your cloud bills start to grow, it is common for CFOs and IT leads to sit down weekly/monthly to understand what is driving this. This has been notoriously difficult in the past and has resulted in the rise of companies like cloudability and cloudtechhealth for example. Looking at your cloud costs breakdown along side log level usage data can help you correlate and understand some of the drivers behind these costs, especially if there has been significant change in your cloud cost structure.

To sign up for your weekly log level usage report simply go here and provide your details.


10-ways-to-lead-with-analytics

Sign-up for a free 30 Day Trial

Get set up in minutes, and gain insights in seconds

Start Free Trial Setup a Demo

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

Latest Stories
Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo Silicon Valley Call for Papers is now open.
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and valu...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Automation is enabling enterprises to design, deploy, and manage more complex, hybrid cloud environments. Yet the people who manage these environments must be trained in and understanding these environments better than ever before. A new era of analytics and cognitive computing is adding intelligence, but also more complexity, to these cloud environments. How smart is your cloud? How smart should it be? In this power panel at 20th Cloud Expo, moderated by Conference Chair Roger Strukhoff, pane...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
SYS-CON Events announced today that Tintri, Inc, a leading provider of enterprise cloud infrastructure, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Tintri offers an enterprise cloud platform built with public cloud-like web services and RESTful APIs. Organizations use Tintri all-flash storage with scale-out and automation as a foundation for their own clouds – to build agile development environments...
SYS-CON Events announced today that DivvyCloud will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. DivvyCloud software enables organizations to achieve their cloud computing goals by simplifying and automating security, compliance and cost optimization of public and private cloud infrastructure. Using DivvyCloud, customers can leverage programmatic Bots to identify and remediate common cloud problems in rea...
SYS-CON Events announced today that Carbonite will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Carbonite protects your entire IT footprint with the right level of protection for each workload, ensuring lower costs and dependable solutions with DoubleTake and Evault.
SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...