Blog Feed Post

Using Dynatrace AI based Monitoring to Ensure Blockchain Performance

The explosion of Bitcoin and cryptocurrency has been a proving ground for Blockchain transactions.  Blockchain is a constantly growing list of records called blocks, which are connected and secured using cryptography (1).  Blockchains provide permanent verifiable means of recording transactions between two parties.   Serious actors in Finance, eCommerce and IT are massively investing in and building out Blockchain frameworks.  Blockchain by its very nature requires tremendous processing power and network conversations with actors exchanging blocks.  There has not been a lot of research into the performance impact adding Blockchain transactions into ecommerce applications will have.  Blockchain adoption raises several operational concerns:

  1. It is unclear how poor performance will impact end users or applications relying on a Blockchain transaction.
  2. As Blockchain becomes part of the fabric of cloud based, new stack ecommerce/financial applications the onus on how Blockchain behaves and performs will not only fall on to those providing the core Blockchain processing, but any actor integrating it into their applications or exchanges.
  3. The explosion of complexity around cloud and container based approaches these Blockchain applications will be based on will require new discipline and insight to ensure that they are adequately monitored from a performance perspective.

That last item is where most organizations will struggle.  Blockchain use at web scale will happen in ridiculously complex environments which will include elastically scaling compute resources which may only exist temporarily.  The scale of these applications will require a new monitoring paradigm as it will be impractical and in many cases impossible for humans to monitor the health of these Blockchain components.

Monitoring Blockchains

Most literature which can be found on monitoring Blockchain focuses on very specific metrics regarding the health of the chain. Here is a view from an IBM Bluemix UI.

Above you can see metrics like Total Blocks, Recent Instantiations, Invocations, etc. (2)

Below is an example of some Blockchain activity from https://Blockchain.info/charts

Above you can see that in this example Blockchain data is being aggregated into Block Details, Mining Information, Network Activity and Wallet Activity.  This is clearly pertinent information about the Blockchain, however:

  1. What happens if there is an infrastructure issue with your Blockchain processing?
  2. What happens if there is a network issue between Blockchain actors?
  3. How do the above metrics allow an organization to diagnose that a performance issue is occurring with their Blockchain?
  4. How do the above metrics allow an organization to understand the impact to end users or service calls depending on a Blockchain?

Monitoring Blockchains requires visibility into the entire technology stack.  In addition, monitoring Blockchains requires monitoring every transaction.  You can’t skip requests or sample/throttle data when monitoring Blockchains and blindly trust that the application, services, process, network or infrastructure layers are always providing 100% availability and optimal performance.

Automating Performance Monitoring for Blockchains

Here is an example of how to monitor a simple Blockchain based on Ethereum (3).  Ethereum (https://www.ethereum.org) is an open-source Blockchain technology.  In this case it is deployed as GOlang (a Google programming language popular in new stack deployments).

In this example, we are using Dynatrace to automatically discover the GO processes which were created when we spun up Ethereum in the Amazon Cloud using EC2 (AWS).

The Blockchain is being computed in the geth process.

As Blockchains get deployed into containers and micro-services in cloud environments, Dynatrace automatically discovers these processes.  Dynatrace will automatically pull in the log files from the geth process (more on this in a bit).

Blockchains are about communicating between two parties, below is how Dynatrace automatically discovers requests being made to potential externally held Blockchains.  Calls to Blockchains can be executed in a variety of ways, one of which is via JSON based REST calls.

The screenshot above shows how external request to Blockchains can be monitored.  Below is an example of how network traffic comming in to and out of our cloud instance hosting the Blockchain is discovered and monitored.

By including Dynatrace as part of your container/cloud deployments you can automate the discovery of your Blockchain services.  This is as simple as including Dynatrace in your buildpacks, as your Blockchain compute resources are created they will be automatically instrumented and monitored.

Deterministic AI to the Rescue

The sample Blockchain I showed above is an overly simple example.  Blockchains will exist in highly complex environments executing thousands of transactions per minute.

These environments will challenge even the most mature IT organization to monitor these business-critical transactions.  This is where deterministic Artificial Intelligence monitoring, using specific machine learning algorithms will be able to auto-discover, auto-baseline 100% of your traffic in a full stack fashion (from End User to Application to Blockchain to Infrastructure) and be able to determine the impact of performance events as well as the root cause.

Deterministic AI is the only approach to effectively monitor these environments. Cognitive AI will take too long to understand all of the entities involved, and by the time it could have understood the relationships the environment will have changed.  Monitoring Blockchain containers running in elastic cloud environments which will be constantly changing and evolving requires a deterministic AI approach.  This is what Dynatrace provides, an inherent understanding of all the compute entities involved in operating a Blockchain based application.  Understanding the entities, relationships and dependencies is only part of what needs to be done.  Dynatrace employs a variety of machine learning algorithms to baseline, compare time frames, create multi-dimensional views of data to then determine and rank issues which inevitably occur in the complex environments.

What this provides organizations is the ability to build self-healing Blockchain applications.  For example, if a Blockchain process is having a memory issue, Dynatrace can detect that state and be used to trigger a Chef, Puppet or Ansible script to adjust the memory settings automatically to maintain that Blockchain processing.

Tailor Your Blockchain Monitoring

While the Dynatrace AI will automatically discover and baseline your Blockchain processes, there may be situations where there are details locked away in your Blockchain logs that you want to be aware of.

Below is an example of how Dynatrace is able to monitor Blockchain log files.  You can create rules to look for important strings within your Blockchain log files and automatically trigger external events based on those events/strings.

This works automatically and doesn’t require expensive, complicated third party log analysis tools pushing data into third party data aggregation tools which would then have to push out to external automation scripts.

In addition to monitoring Blockchain log files, Dynatrace can be used to insert additional metadata, what we call tags, over your Blockchain compute resources.  These tags could be used to pull things like the Chaincode ID, Chaincode Version or Peer ID.  This will allow your teams to create separate visualizations, reports, alerts, etc…  for only those Blockchains they are working with.

Blockchain is Too Important Not to Monitor, Try it for Yourself!

The example we showed above is a highly-simplified example, your Blockchain applications will exist in much more complicated environments.  We invite organizations to try Dynatrace in your Blockchain enabled environments. Here is a Free Trial of Dynatrace which you can download and install into your environments where you are running your Blockchain processes.  Dynatrace can easily install on to Hosts directly or you can include Dynatrace in your container/cloud build-packs with nominal effort.

Go ahead and give this a try with your Blockchains, please post your experience below, we would love to get your feedback.

  1. https://en.wikipedia.org/wiki/Blockchain
  2. https://console.bluemix.net/docs/services/blockchain/howto/monitor_network.html
  3. https://blockgeeks.com/two-node-setup-of-a-private-ethereum/

The post Using Dynatrace AI based Monitoring to Ensure Blockchain Performance appeared first on Dynatrace blog – monitoring redefined.

Read the original blog entry...

More Stories By APM Blog

APM: It’s all about application performance, scalability, and architecture: best practices, lifecycle and DevOps, mobile and web, enterprise, user experience

Latest Stories
So the dumpster is on fire. Again. The site's down. Your boss's face is an ever-deepening purple. And you begin debating whether you should join the #incident channel or call an ambulance to deal with his impending stroke. Yes, we know this is a developer's fault. There's plenty of time for blame later. Postmortems have a macabre name because they were once intended to be Viking-like funerals for someone's job. But we're civilized now. Sort of. So we call them post-incident reviews. Fires are ne...
The digital transformation is real! To adapt, IT professionals need to transform their own skillset to become more multi-dimensional by gaining both depth and breadth of a wide variety of knowledge and competencies. Historically, while IT has been built on a foundation of specialty (or "I" shaped) silos, the DevOps principle of "shifting left" is opening up opportunities for developers, operational staff, security and others to grow their skills portfolio, advance their careers and become "T"-sh...
This session will provide an introduction to Cloud driven quality and transformation and highlight the key features that comprise it. A perspective on the cloud transformation lifecycle, transformation levers, and transformation framework will be shared. At Cognizant, we have developed a transformation strategy to enable the migration of business critical workloads to cloud environments. The strategy encompasses a set of transformation levers across the cloud transformation lifecycle to enhance ...
Authorization of web applications developed in the cloud is a fundamental problem for security, yet companies often build solutions from scratch, which is error prone and impedes time to market. This talk shows developers how they can (instead) build on-top of community-owned projects and frameworks for better security.Whether you build software for enterprises, mobile, or internal microservices, security is important. Standards like SAML, OIDC, and SPIFFE help you solve identity and authenticat...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
Mike is managing director in Deloitte Consulting LLP's Cloud practice, responsible for helping clients implement cloud strategy and architecture to drive digital transformation. Beyond his technology experience, Mike brings an insightful understanding of how to address the organizational change, process improvement, and talent management challenges associated with digital transformation. Mike brings more than 30 years of experience in software development and architecture to his role. Most recen...
Having been in the web hosting industry since 2002, dhosting has gained a great deal of experience while working on a wide range of projects. This experience has enabled the company to develop our amazing new product, which they are now excited to present! Among dHosting's greatest achievements, they can include the development of their own hosting panel, the building of their fully redundant server system, and the creation of dhHosting's unique product, Dynamic Edge.
Your job is mostly boring. Many of the IT operations tasks you perform on a day-to-day basis are repetitive and dull. Utilizing automation can improve your work life, automating away the drudgery and embracing the passion for technology that got you started in the first place. In this presentation, I'll talk about what automation is, and how to approach implementing it in the context of IT Operations. Ned will discuss keys to success in the long term and include practical real-world examples. Ge...
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
CloudEXPO New York 2018, colocated with DevOpsSUMMIT and DXWorldEXPO New York 2018 will be held November 12-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI and Machine Learning to one location.
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments t...