Blog Feed Post

Using Dynatrace AI based Monitoring to Ensure Blockchain Performance

The explosion of Bitcoin and cryptocurrency has been a proving ground for Blockchain transactions.  Blockchain is a constantly growing list of records called blocks, which are connected and secured using cryptography (1).  Blockchains provide permanent verifiable means of recording transactions between two parties.   Serious actors in Finance, eCommerce and IT are massively investing in and building out Blockchain frameworks.  Blockchain by its very nature requires tremendous processing power and network conversations with actors exchanging blocks.  There has not been a lot of research into the performance impact adding Blockchain transactions into ecommerce applications will have.  Blockchain adoption raises several operational concerns:

  1. It is unclear how poor performance will impact end users or applications relying on a Blockchain transaction.
  2. As Blockchain becomes part of the fabric of cloud based, new stack ecommerce/financial applications the onus on how Blockchain behaves and performs will not only fall on to those providing the core Blockchain processing, but any actor integrating it into their applications or exchanges.
  3. The explosion of complexity around cloud and container based approaches these Blockchain applications will be based on will require new discipline and insight to ensure that they are adequately monitored from a performance perspective.

That last item is where most organizations will struggle.  Blockchain use at web scale will happen in ridiculously complex environments which will include elastically scaling compute resources which may only exist temporarily.  The scale of these applications will require a new monitoring paradigm as it will be impractical and in many cases impossible for humans to monitor the health of these Blockchain components.

Monitoring Blockchains

Most literature which can be found on monitoring Blockchain focuses on very specific metrics regarding the health of the chain. Here is a view from an IBM Bluemix UI.

Above you can see metrics like Total Blocks, Recent Instantiations, Invocations, etc. (2)

Below is an example of some Blockchain activity from https://Blockchain.info/charts

Above you can see that in this example Blockchain data is being aggregated into Block Details, Mining Information, Network Activity and Wallet Activity.  This is clearly pertinent information about the Blockchain, however:

  1. What happens if there is an infrastructure issue with your Blockchain processing?
  2. What happens if there is a network issue between Blockchain actors?
  3. How do the above metrics allow an organization to diagnose that a performance issue is occurring with their Blockchain?
  4. How do the above metrics allow an organization to understand the impact to end users or service calls depending on a Blockchain?

Monitoring Blockchains requires visibility into the entire technology stack.  In addition, monitoring Blockchains requires monitoring every transaction.  You can’t skip requests or sample/throttle data when monitoring Blockchains and blindly trust that the application, services, process, network or infrastructure layers are always providing 100% availability and optimal performance.

Automating Performance Monitoring for Blockchains

Here is an example of how to monitor a simple Blockchain based on Ethereum (3).  Ethereum (https://www.ethereum.org) is an open-source Blockchain technology.  In this case it is deployed as GOlang (a Google programming language popular in new stack deployments).

In this example, we are using Dynatrace to automatically discover the GO processes which were created when we spun up Ethereum in the Amazon Cloud using EC2 (AWS).

The Blockchain is being computed in the geth process.

As Blockchains get deployed into containers and micro-services in cloud environments, Dynatrace automatically discovers these processes.  Dynatrace will automatically pull in the log files from the geth process (more on this in a bit).

Blockchains are about communicating between two parties, below is how Dynatrace automatically discovers requests being made to potential externally held Blockchains.  Calls to Blockchains can be executed in a variety of ways, one of which is via JSON based REST calls.

The screenshot above shows how external request to Blockchains can be monitored.  Below is an example of how network traffic comming in to and out of our cloud instance hosting the Blockchain is discovered and monitored.

By including Dynatrace as part of your container/cloud deployments you can automate the discovery of your Blockchain services.  This is as simple as including Dynatrace in your buildpacks, as your Blockchain compute resources are created they will be automatically instrumented and monitored.

Deterministic AI to the Rescue

The sample Blockchain I showed above is an overly simple example.  Blockchains will exist in highly complex environments executing thousands of transactions per minute.

These environments will challenge even the most mature IT organization to monitor these business-critical transactions.  This is where deterministic Artificial Intelligence monitoring, using specific machine learning algorithms will be able to auto-discover, auto-baseline 100% of your traffic in a full stack fashion (from End User to Application to Blockchain to Infrastructure) and be able to determine the impact of performance events as well as the root cause.

Deterministic AI is the only approach to effectively monitor these environments. Cognitive AI will take too long to understand all of the entities involved, and by the time it could have understood the relationships the environment will have changed.  Monitoring Blockchain containers running in elastic cloud environments which will be constantly changing and evolving requires a deterministic AI approach.  This is what Dynatrace provides, an inherent understanding of all the compute entities involved in operating a Blockchain based application.  Understanding the entities, relationships and dependencies is only part of what needs to be done.  Dynatrace employs a variety of machine learning algorithms to baseline, compare time frames, create multi-dimensional views of data to then determine and rank issues which inevitably occur in the complex environments.

What this provides organizations is the ability to build self-healing Blockchain applications.  For example, if a Blockchain process is having a memory issue, Dynatrace can detect that state and be used to trigger a Chef, Puppet or Ansible script to adjust the memory settings automatically to maintain that Blockchain processing.

Tailor Your Blockchain Monitoring

While the Dynatrace AI will automatically discover and baseline your Blockchain processes, there may be situations where there are details locked away in your Blockchain logs that you want to be aware of.

Below is an example of how Dynatrace is able to monitor Blockchain log files.  You can create rules to look for important strings within your Blockchain log files and automatically trigger external events based on those events/strings.

This works automatically and doesn’t require expensive, complicated third party log analysis tools pushing data into third party data aggregation tools which would then have to push out to external automation scripts.

In addition to monitoring Blockchain log files, Dynatrace can be used to insert additional metadata, what we call tags, over your Blockchain compute resources.  These tags could be used to pull things like the Chaincode ID, Chaincode Version or Peer ID.  This will allow your teams to create separate visualizations, reports, alerts, etc…  for only those Blockchains they are working with.

Blockchain is Too Important Not to Monitor, Try it for Yourself!

The example we showed above is a highly-simplified example, your Blockchain applications will exist in much more complicated environments.  We invite organizations to try Dynatrace in your Blockchain enabled environments. Here is a Free Trial of Dynatrace which you can download and install into your environments where you are running your Blockchain processes.  Dynatrace can easily install on to Hosts directly or you can include Dynatrace in your container/cloud build-packs with nominal effort.

Go ahead and give this a try with your Blockchains, please post your experience below, we would love to get your feedback.

  1. https://en.wikipedia.org/wiki/Blockchain
  2. https://console.bluemix.net/docs/services/blockchain/howto/monitor_network.html
  3. https://blockgeeks.com/two-node-setup-of-a-private-ethereum/

The post Using Dynatrace AI based Monitoring to Ensure Blockchain Performance appeared first on Dynatrace blog – monitoring redefined.

Read the original blog entry...

More Stories By APM Blog

APM: It’s all about application performance, scalability, and architecture: best practices, lifecycle and DevOps, mobile and web, enterprise, user experience

Latest Stories
DX World EXPO, LLC, a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
"ZeroStack is a startup in Silicon Valley. We're solving a very interesting problem around bringing public cloud convenience with private cloud control for enterprises and mid-size companies," explained Kamesh Pemmaraju, VP of Product Management at ZeroStack, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...