Welcome!

Blog Feed Post

Creating an Autoconfiguring IBM UrbanCode Deploy Agent in Skytap

Skytap recently announced the availability of the Skytap Automation Pack for IBM UrbanCode Deploy. You can read about it in this Skytap blog post from earlier this year.

You'll recall that this plugin adds the following functionality into the IBM UrbanCode Deploy Application Process web UI to create and manipulate templates and configurations in Skytap:

  • Skytap Cloud Authentication
  • Add Configuration to Project
  • Add Template to Project
  • Change Configuration State
  • Connect to Network in another Configuration (ICNR)
  • Connect to VPN Tunnel
  • Create Configuration from Template
  • Create Published Service
  • Create Published URL
  • Create Template from Configuration
  • Delete Configuration
  • Get VM ID
  • List Published URL for Configuration
  • List VM Published Service
  • Merge Template into Configuration
  • Set VM User Data

In this blog post we’ll explore how to use some of the advanced capabilities available in Skytap Cloud along with the functions available in the Skytap Automation Pack for IBM UrbanCode Deploy. This will enable us to create a Linux autoconfiguring IBM UrbanCode Deploy Agent and save it as a reusable Skytap Environment Template.

Prerequisite Software

As we begin this exercise, the following software must already be installed on a CentOS 6.4 VM in a Skytap Environment:

  • The IBM UrbanCode Deploy Agent software
  • The “curl” command package to enable interaction with the Skytap API
  • Install into /usr/local/bin, “jq”, a a command line JSON processor to enable parsing Skytap Environment metadata returned from the Skytap API.  This application is available from http://stedolan.github.io/jq

Managing Self-Registration of IBM UrbanCode Deploy Agents

When the IBM UrbanCode Deploy Agent service starts on a compute engine, it seeks to register itself with an IBM UrbanCode Deploy Server. As part of this registration process, each agent that is registered with an IBM UrbanCode Deploy Server must have a unique name by which it will be referenced by the IBM UrbanCode Deploy Server. Both the server to which the agent registers itself, as well as the name it will be assigned upon registration, are specified on the IBM UrbanCode Deploy Agent in the configuration file located at < IBM UrbanCode Deploy Agent Install Path >/conf/agent/installed.properties. In our case, this path is as follows:

/opt/ibm-ucd/agent/conf/agent/installed .properties

The default contents of this configuration file are as follows (items in italics are set during the agent install process):

IBM\ UrbanCode\ Deploy/java.home=/usr/java/jdk1.7.0_51/jre

locked/agent.brokerUrl=failover\:(ah3\://localhost\:7918)

locked/agent.home=/opt/ibm-ucd/agent

locked/agent.http.proxy.host=

locked/agent.http.proxy.port=

locked/agent.id=< randomized id string >

locked/agent.jms.remote.host=localhost

locked/agent.jms.remote.port=7918

locked/agent.keystore=../conf/ibm-ucd.keystore

locked/agent.keystore.pwd=pbe{< randomized key string >}

locked/agent.mutual_auth=false

locked/agent.name=< hostname >

Proper operation of an IBM UrbanCode Deploy agent depends on the following prerequisites:

  1. Each IBM UrbanCode Deploy agent connecting to a common IBM UrbanCode Deploy server must have a unique agent name.  This agent name is set in the “locked/agent.name=…” parameter in the installed.properties file you see above.

  2. The IBM UrbanCode Deploy server to which an agent is to connect must be specified in the “locked/agent.jms.remote.host=…” parameter in the installed.properties file you see above.

In order to ensure proper operation of our Skytap IBM UrbanCode Depoy Autoconfiguring Agent, we will set these parameters during the initial boot of the agent VM. To enable this operation we will alter the installed.properties files as follows:

locked/agent.brokerUrl=failover\:(ah3\://xxxUCD_SERVERxxx\:7918)

locked/agent.jms.remote.host=xxxUCD_SERVERxxx

locked/agent.jms.remote.port=7918

locked/agent.http.proxy.host=

locked/agent.http.proxy.port=

IBM\ UrbanCode\ Deploy/java.home=/usr/java/jdk1.7.0_51/jre

locked/agent.home=/opt/ibm-ucd/agent

locked/agent.mutual_auth=false

locked/agent.keystore=../conf/ibm-ucd.keystore

locked/agent.keystore.pwd=pbe{some-unique-random-string}

locked/agent.name=xxxINSTALLED_AGENT_NAMExxx

The entries in bold above will be changed upon boot of the Skytap VM to the appropriate value:

xxxUCD_SERVERxxx will be changed to the hostname of the IBM Urbancode Deploy Server to which the Skytap IBM UrbanCode Depoy Autoconfiguring Agent is to connect

xxxINSTALLED_AGENT_NAMExxx will be changed to a unique name to allow multiple Skytap IBM UrbanCode Depoy Autoconfiguring Agents to register with a common IBM UrbanCode Deploy server. Note that in the Skytap environment, using the hostname as the agent name does not guarantee uniqueness because Skytap Environments that are created from the same Skytap Template will have the same hostname.

We will save this modified “installed.properties” file as our reference file, “installed.properties.orig” for use in the Skytap IBM UrbanCode Depoy Autoconfiguring Agent startup script later in this article.

Configuring Skytap User Data

We will use the Skytap VM User Data to specify values to be used for the “xxxUCD_SERVERxxx” and “xxxINSTALLED_AGENT_NAMExxx” fields in the Skytap IBM UrbanCode Depoy Autoconfiguring Agent installed.properties file:

It the above screenshot, we see a value of “Skytap-Agent” assigned to a “UCD-Group” variable. We will use this UCD-Group variable as a mnemonic identifier string to prepend to the unique ID assigned to the Skytap IBM UrbanCode Depoy Autoconfiguring Agent that is registered with the IBM UrbanCode Deploy Server. In practice, we could preconfigure several different autoconfiguring agent Skytap Templates to serve as database servers, application servers, etc. Use of this mnemonic string allows us to easily discern the role of the agent once it is registered with the IBM UrbanCode Deploy Server. 

In the above screenshot, we also see a value of “10.0.0.1” assigned to the “UCD-Server” variable. This value is the IP address of the IBM UrbanCode Deploy Server.

You can read more about the Skytap VM User Data feature here.

The IBM UrbanCode Deploy Agent Boot-time Startup Script

The script to startup the Skytap IBM UrbanCode Depoy Autoconfiguring Agent process at system boot is located in the file < IBM UrbanCode Deploy Agent Install Path >/bin/init/ibm-ucdagent. In our case this file is located as follows:

/opt/ibm-ucd/agent/bin/init/ibm-ucdagent

Recall that we modified the installed.properties file to parameterize the values required to dynamically configure the agent and saved this modified file as “installed.properties.orig” for use in the Skytap IBM UrbanCode Depoy Autoconfiguring Agent startup script. Prior to starting the Skytap IBM UrbanCode Depoy Autoconfiguring Agent we must alter the startup script to use the Skytap REST API to retrieve from the Skytap VM User Data, the IBM UrbanCode Deploy Server designation and the UCD-Group mnemonic string to use in creating a unique name for the Skytap IBM UrbanCode Depoy Autoconfiguring Agent. 

Creating the Skytap IBM UrbanCode Deploy Autoconfiguring Agent Name

Since the Skytap IBM UrbanCode Depoy Autoconfiguring Agent name must be unique, we can use the Skytap VM ID, which is guaranteed to be unique in the Skytap environment, for this purpose. This VM ID, along with the UCD-Group mnemonic string specified in the VM User Metadata are both accessible via the Skytap REST API. By concatenating these values together in the form - we can construct a name that is both descriptive and unique.

Modifying the Skytap IBM UrbanCode Deploy Autoconfiguring Agent Startup Script

The modifications we make to the Skytap IBM UrbanCode Depoy Autoconfiguring Agent startup script are made in the start() routine of the script. These modifications as well as annotations in the comments are seen below.

start() {

  touch $agent_pid

  echo -n "Starting ibm-ucdagent: "

# -- Skytap Autoconfig Section Begin ------------------------------------------

#

# This pause is necessary to allow initial boot-time system processes time to start up

#

  sleep 15

#

#Set the home directory of the “installed.properties file

#

  PROPFILE_HOME="$AGENT_HOME/conf/agent"

#

# Save any existing installed.properties file with a time-date stamp for future reference, if required

#

  mv -f ${PROPFILE_HOME}/installed.properties ${PROPFILE_HOME}/installed.properties.`date +%F-%T`

#

# create a fresh copy of the installed.properties file from a parameterized installed.properties file 

#  (saved in installed.properties.orig)

  cp ${PROPFILE_HOME}/installed.properties.orig ${PROPFILE_HOME}/installed.properties

#

# remove any previous intermediate ConfigData that might be left from a previous run of this script

#

  rm -f /tmp/ConfigData_*

#

# issue a call to the Skytap API to get the Skytap Configuration metadata for this VM and save it in 

#  in an intermediate file

#

  curl -s -X GET "http://gw/skytap" -H "Content-Type: application/json" -H "Accept: application/json"  >/tmp/ConfigData_$$

#

# use the jq command line JSON parser along with some sed pattern matching and text substitution

#  to extract the VM ID from the metadata retrieved from the SkytapAPI

#

vmID=`cat /tmp/ConfigData_$$ | /usr/local/bin/jq .id | sed s/\"//g`

#

# remove any previous intermediate UserData file that might be left from a previous run of this script

#

rm -f /tmp/UserData_*

#

# use the jq command line JSON parser along with some sed pattern matching and text substitution

#   to extract the UserData field from the VM Metadata

#

cat /tmp/ConfigData_* | /usr/local/bin/jq .user_data | sed s/\\\\r\\\\n/\"\\n\"/g > /tmp/UserData_$$

#

# Now use some grep, awk and sed pattern matching and text parsing to extract the UCD-Group and 

#    UCD-Server designations from the VM User Data

#

ucdGroup=`grep UCD-Group /tmp/UserData_$$ | awk -F= '{print $2}' | sed s/\"//g`

ucdServer=`grep UCD-Server /tmp/UserData_$$ | awk -F= '{print $2}' | sed s/\"//g`

#

# echo what we found to the boot log

#

echo "Installed Agent Name: ${ucdGroup}-${vmID}"

echo "UCD Server: ${ucdServer}"

#

# Now use sed pattern matching and substitution to modify the installed.properties file

#

# first set the Agent Name and save to an intermediate file

#

  sed s/xxxINSTALLED_AGENT_NAMExxx/`echo ${ucdGroup}-${vmID}`/ < ${PROPFILE_HOME}/installed.properties > ${PROPFILE_HOME}/new.installed.properties

#

# next use that intermediate file as input and set the IBM UrbanCode Deploy Server to which

#  the agent is to connect and save to another intermediate file

#

  sed s/xxxUCD_SERVERxxx/`echo ${ucdServer}`/ < ${PROPFILE_HOME}/new.installed.properties > ${PROPFILE_HOME}/newer.installed.properties

#

#  move the modified file into place overwriting the existing installed.properties file (Note the –f

#   argument to suppress any warning or user prompt)

#

  mv -f ${PROPFILE_HOME}/newer.installed.properties ${PROPFILE_HOME}/installed.properties

# -- Skytap Autoconfig Section End ------------------------------------------

# AMD -- Made lock location variable.

  if [ -f $agent_lock ]; then

    if [ -f $agent_pid ]; then

The steps in the above script that create intermediate files could be combined, and the intermediate files reduced or eliminated, but these files are useful for debugging or further study of the process and have been left in for purposes of this writeup.

Notable steps in the script include the use of the curl command to interact with the Skytap API to retrieve information about the currently running Skytap VM. In particular, note that the running Skytap VM can get information about itself via the Skytap API  without requiring authentication.

Enable the Boot-time Startup of the IBM UrbanCode Deploy Agent

To enable the Skytap IBM UrbanCode Depoy Autoconfiguring Agent startup script to run at boot time, add it to the /etc/init.d directory as follows:

ln –s /opt/ibm-ucd/agent/bin/init/ibm-ucdagent /etc/init.d

For our CentOS system, we also need to issue the appropriate chkconfig command to enable the script to run at system boot:

chkconfig –-add ibm-ucdagent

Different procedures may be required for alternate Linux implementations.

Testing the Skytap IBM UrbanCode Deploy Autoconfiguring Agent

Now that we have put all the pieces in place, we can test our Skytap IBM UrbanCode Depoy Autoconfiguring Agent. To do this, we must first shutdown our Autoconfiguring Agent Skytap Environment. Do this by navigating to the Configuration Details page that contains the Skytap IBM UrbanCode Depoy Autoconfiguring Agent and click the “Shutdown” button on the Agent VM:

When the agent VM has shutdown, we will open a browser window to the UrbanCode Server and navigate to the Resources->Agents tab:

In the above screenshot we can see that there is already one agent named “host1.skytap.example” running, the one hosted on our IBM UrbanCode Deploy Server VM.

Now we start our Skytap IBM UrbanCode Depoy Autoconfiguring Agent VM by clicking the “Run” button on the Agent VM:

After the VM has started, we will see the VM Status is “running”:

Reviewing the IBM UrbanCode Deploy Server web interface, we should see the Autoconfiguring Agent appear in the “Agents” tab:

To enable reuse of this IBM UrbanCode Skytap Autoconfiguring Agent we will first stop the VM:

Once it has stopped, we can press the “Save as Template” button in the Skytap Configuration Details page:

Complete the Template Creation dialog:

When the template has been created, we will see the following screen:

The template will appear in the template listing page for “My” templates and available for reuse:

In my next blog post, I’ll discuss how to use the Skytap Automation Pack for IBM UrbanCode Deploy to instantiate, configure and use the Skytap IBM UrbanCode Depoy Autoconfiguring Agent Template in a complex IBM UrbanCode Deploy process.

Summary

We’ve explored an advanced scenario where we created a Skytap Environment containing an IBM UrbanCode Deploy Agent that automatically configures itself at boot-time for successful registration and use by and IBM UrbanCode Deploy Server. 

Install the software prerequisites on a VM in a Skytap Environment

Modify the Skytap IBM UrbanCode Depoy Autoconfiguring Agent “installed.properties” file to parameterize the IBM UrbanCode Deploy Server to which the agent is to connect and establish a unique agent name

Modify the Skytap User Data associated with the Skytap IBM UrbanCode Depoy Autoconfiguring Agent to specify the IBM UrbanCode Deploy Server to which the agent process is to connect and to specify an identifier string to prepend to the agent name.

Modify the Skytap IBM UrbanCode Depoy Autoconfiguring Agent startup script to modify the parameterized “installed.properties” file prior to starting the agent process

Save the modified Skytap Environment as a Skytap Template to allow reuse.

The principles demonstrated in this exercise and the additional Skytap actions provided by the Skytap Cloud Automation Pack for IBM UrbanCode Deploy can easily be expanded further into even more complex scenarios to satisfy even the most intricate and complex deployment requirements.

The next blog post will expand on this topic to demonstrate how to use the Skytap Automation Pack for IBM UrbanCode Deploy in conjuction with the Skytap IBM UrbanCode Depoy Autoconfiguring Agent Template we created in this discussion to dynamically provision a complex deployment environment.

Please contact Skytap (1-888-759-8278  or http://www.skytap.com) if you want more information about Skytap or the Skytap Cloud Automation Pack for IBM UrbanCode Deploy.

Read the original blog entry...

More Stories By Skytap Blog

Author: Noel Wurst is the managing content editor at Skytap. Skytap provides SaaS-based dev/test environments to the enterprise. Skytap solution removes the inefficiencies and constraints that companies have within their software development lifecycle. As a result, customers release better software faster. In this blog, we publish engaging, thought provoking stories that revolve around agile enterprise applications and cloud-based development and testing.

Latest Stories
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
SYS-CON Events announced today that Synametrics Technologies will exhibit at SYS-CON's 22nd International Cloud Expo®, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Synametrics Technologies is a privately held company based in Plainsboro, New Jersey that has been providing solutions for the developer community since 1997. Based on the success of its initial product offerings such as WinSQL, Xeams, SynaMan and Syncrify, Synametrics continues to create and hone inn...
As many know, the first generation of Cloud Management Platform (CMP) solutions were designed for managing virtual infrastructure (IaaS) and traditional applications. But that's no longer enough to satisfy evolving and complex business requirements. In his session at 21st Cloud Expo, Scott Davis, Embotics CTO, explored how next-generation CMPs ensure organizations can manage cloud-native and microservice-based application architectures, while also facilitating agile DevOps methodology. He expla...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
DevOps promotes continuous improvement through a culture of collaboration. But in real terms, how do you: Integrate activities across diverse teams and services? Make objective decisions with system-wide visibility? Use feedback loops to enable learning and improvement? With technology insights and real-world examples, in his general session at @DevOpsSummit, at 21st Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, explored how leading organizations use data-driven DevOps to close th...
Continuous Delivery makes it possible to exploit findings of cognitive psychology and neuroscience to increase the productivity and happiness of our teams. In his session at 22nd Cloud Expo | DXWorld Expo, Daniel Jones, CTO of EngineerBetter, will answer: How can we improve willpower and decrease technical debt? Is the present bias real? How can we turn it to our advantage? Can you increase a team’s effective IQ? How do DevOps & Product Teams increase empathy, and what impact does empath...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, discussed how given the magnitude of today's application ...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, examined the regulations and provided insight on how it affects technology, challenges the established rules and will usher in new levels of diligence arou...
You know you need the cloud, but you're hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You're looking at private cloud solutions based on hyperconverged infrastructure, but you're concerned with the limits inherent in those technologies. What do you do?
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
Sanjeev Sharma Joins June 5-7, 2018 @DevOpsSummit at @Cloud Expo New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, discussed the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, application p...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
The 22nd International Cloud Expo | 1st DXWorld Expo has announced that its Call for Papers is open. Cloud Expo | DXWorld Expo, to be held June 5-7, 2018, at the Javits Center in New York, NY, brings together Cloud Computing, Digital Transformation, Big Data, Internet of Things, DevOps, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...