|By Greg Schulz||
|February 11, 2017 03:19 PM EST||
The following are some of the more popular viewed posts from StorageIOblog.com during 2016.
Big Files and Lots of Little File Processing and Benchmarking with Vdbench - Need to test, validate, compare, contrast or simply apply workload to file systems, NAS or other file-based access? Want the flexibility and simplicity to software define your benchmark workload to meet various needs? For example, millions of small files or thousands of large 5GB, 10GB, 15GB (or larger) files with various read, write size and access patterns spanning a single directory, or many with various depths? Do you want the flexibility for different platforms including Windows, *NIX, bare metal, container, virtual or cloud without a bulk tool using simple scripts that produce lots of insightful results? Then you will want to check this post out.
Breaking the VMware ESXi 5.5 ACPI boot loop on Lenovo TD350 - Ever have a VMware host server go into a boot loop and purple screen of death (PSD) then displaying a message about ACPI or similar? After spending time searching and applying many filters to sift through the noise of false positive matches, finally found the simple fix (e.g. a BIOS setting) to break the VMware ESXi vSphere boot loop, or at least on a Lenovo server.
Cloud conversations: AWS EBS, Glacier and S3 overview (Part I) - This is one of the perennial favorites that while new features have been added with others extended, the post series still provides a good overview, primer or refresher of various Amazon Web Services (AWS) services including how they work. Interesting in learning more about Microsoft and Azure, then check out this, this, this and this.
Cloud Conversations: AWS EFS Elastic File System (Cloud NAS) - This is a companion to the above AWS as well as other cloud post series that looks at AWS Elastic File System. Note that other cloud service providers have also added NAS file access support, some are intra (e.g. inside AWS cloud), others are inter-cloud (e.g. inside and outside cloud) such as Azure (can work with external Windows Servers using SMB3). Even OpenStack has added NAS file with Manila folders and Ceph with CephFS among others. So when some people tell you that NAS and file access are dead particular for cloud, remind them of the increasing number of services and software stacks that are adding new services to allow their solution to be compatible with existing environments or applications.
Collecting Transaction Per Minute from SQL Server and HammerDB - If you have used the free tool HammerDB (e.g. Hammora) for driving database workloads, simulations or benchmarks you should recall that the resulting statistics are rather lacking. Sure there is a nice GUI chart that shows current executing transactions per second (TPS) along with some very simple counters in the log. However compared to some other tools such as sysbench, Quest Benchmark Factory and YCSB among others, the Hammer metrics are rather lacking. In this post I show how you can collect some more metrics from SQL Server if you have to use HammerDB. View more server storage I/O performance benchmark and monitoring tools resources here.
Gaining Server Storage I/O Insight into Microsoft Windows Server 2016 - Microsoft released into general availability Windows Server 2016 and this post looks at some of the new features along with functionality including Storage Spaces Direct (S2D), Storage Replica (SR) as well as other enhancements. With these new and enhanced features Windows Servers increase their interoperability with Azure, as well as supporting aggregated hyper-converged infrastructure (HCI), disaggregated converged (CI) as well as traditional workloads along with Hyper-V (and containers). One of the other new enhancements in Windows Server 2016 which now uses ReFS (Reliable File System) as its default file system that you can read more about here. RIP Windows SIS (Single Instance Storage), or at least in Server 2016 With Windows Server 2016 Microsoft removed single instance storage replacing with new capabilities that you can read more about in the this post.
Garbage data in, garbage information out, big data or big garbage? There is a classic IT expression of garbage data in results in garbage data (or information out) in that your algorithms and data structures (which equals programs e.g. Niklaus Wirth) are only as good as the data they work on. What this means then is that if there is a large amount of big data then there can also be a big garbage in and garbage out problem unless addressed.
Hard product vs. soft product - Hard product refers to something such as hardware, software or a service resource that is obtained and then joined with other resources in a particular way to create a soft product. Not to be confused with software, the soft product is the result or how resources get defined that give some ability or benefit. Think of a soft product as for how airlines can use the same airplane, serve the same coca cola, have same seats, yet their soft product is the service experience of how those are delivered, as well as how you find and buy or use them. Another way of thinking about it is hard products are the ingredients for a recipe, the recipe defines how those ingredients result in some food dish.
Part II: How many IOPS can a HDD, HHDD or SSD do with VMware? - This is part of a multi-post series looking at how many IOPs (or bandwidth) various HDD and SSDs can do handling different workloads. Of course, your results will vary with configuration settings, tools among other considerations. However, some of the older rules of thumb (RUT) about RPM and other considerations for HDDs have changed and continue to do so. As an example of how HDDs continue to evolve check out this popular post from the 2016 list Which Enterprise HDDs to use for a Content Server Platform.
Part II: What I did with Lenovo TS140 in my Server and Storage I/O Review - This is a popular post series of some things I have done with a Lenovo TS140 including defining with various software as well as hardware. This is a great price performer value system that several years ago after testing one Lenovo sent me, I returned that to Lenovo and bought several of them to join my other systems.
Server and Storage I/O Benchmarking and Performance Resources - This is a collection of various server, storage I/O and networking hardware, software as well as services tools, techniques as well as tips for benchmarking, comparing, simulation, testing, gaining insight across cloud, virtual, container and legacy resources. Server and Storage I/O Benchmark Tools: Microsoft Diskspd (Part I) - This is one of the tools found on the server, storage I/O benchmarking and performance resources page. Diskspd is a tool developed by Microsoft as an alternative to using Iometer, vdbench, fio.exe, SQLIO among many others, plus, it is on github.
The NVM (Non Volatile Memory) and NVMe Place - Interesting and adoption in nand flash, nvram, 3D XPoint among other SSD and Non-volatile Memory (NVM) continues. Another popular post that you can find at thenvmeplace.com is this NVMe overview and primer – Part I. There is a growing interest, awareness and deployment adoption around NVM Express (NVMe) the new protocol for accessing NVMs and SSDs. Some of the common conversations and questions I encounter is confusion between NVM and NVMe, too which the answer is one (the former) are the media or devices, the other is the access method alternative to using AHCI/SATA or SCSI (e.g. SAS, iSCSI, FCP, SRP) among others.
VMware VVOLs and storage I/O fundamentals (Part 1) - VMware Virtual Volumes (VVOL) continue to gain adoption and this post is part of an overview and primer. If you want to go deeper into VVOL as well as see some adoption insights check out Eric Sieberts post here over at vsphere-land.com
Welcome to the Object Storage Center page - This is a micro site that has a primer and overview of cloud as well as object storage along with an expanding list of links to various resources, tips, technologies, tools, trends and industry activity.
Where To Learn More
www.storageio.com particular if you have not been there for awhile to check out the new streamlined look and navigation to various content including Server StorageIO update newsletters (free subscription) among other resources.
What this all means and wrapping up
Some of the popular posts for 2016 are perennial favorites and based on experience will probably appear on the 2017 list. However there are also several new posts that appeared in 2016 that I suspect will also appear on the 2017 version of the above list, along with new content from 2017. Thank you to all of you who frequent StorageIOblog.com as well as StorageIO.com along with our various micro sites including server storage I/O performance and benchmarking resources, thenvmeplace.com, thessdplace.com, cloud and objectstoragecenter.com, data protection diaries among others. Also thank you for viewing various partner venues and syndicates with extra ones appearing throughout 2017. Watch for more content in the coming weeks, months and throughout 2017 on software defined data infrastructures (SDDI) along with server, storage I/O, networking, hardware, software, cloud, container, data protection and related topics, trends, technologies, tools and tips.
Again, thank you
Ok, nuff said, for now.
Greg Schulz - Microsoft MVP Cloud and Data Center Management, vSAN and VMware vExpert. Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2017 Server StorageIO(R) and UnlimitedIO All Rights Reserved
Information technology (IT) advances are transforming the way we innovate in business, thereby disrupting the old guard and their predictable status-quo. It’s creating global market turbulence. Industries are converging, and new opportunities and threats are emerging, like never before. So, how are savvy chief information officers (CIOs) leading this transition? Back in 2015, the IBM Institute for Business Value conducted a market study that included the findings from over 1,800 CIO interviews ...
Mar. 26, 2017 12:30 AM EDT Reads: 5,147
Virtualization over the past years has become a key strategy for IT to acquire multi-tenancy, increase utilization, develop elasticity and improve security. And virtual machines (VMs) are quickly becoming a main vehicle for developing and deploying applications. The introduction of containers seems to be bringing another and perhaps overlapped solution for achieving the same above-mentioned benefits. Are a container and a virtual machine fundamentally the same or different? And how? Is one techn...
Mar. 26, 2017 12:30 AM EDT Reads: 2,869
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Mar. 26, 2017 12:30 AM EDT Reads: 1,828
What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, will discuss the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information,
Mar. 26, 2017 12:15 AM EDT Reads: 707
As businesses adopt functionalities in cloud computing, it’s imperative that IT operations consistently ensure cloud systems work correctly – all of the time, and to their best capabilities. In his session at @BigDataExpo, Bernd Harzog, CEO and founder of OpsDataStore, will present an industry answer to the common question, “Are you running IT operations as efficiently and as cost effectively as you need to?” He will expound on the industry issues he frequently came up against as an analyst, and...
Mar. 26, 2017 12:00 AM EDT Reads: 4,103
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor - all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
Mar. 26, 2017 12:00 AM EDT Reads: 1,688
ChatOps is an emerging topic that has led to the wide availability of integrations between group chat and various other tools/platforms. Currently, HipChat is an extremely powerful collaboration platform due to the various ChatOps integrations that are available. However, DevOps automation can involve orchestration and complex workflows. In his session at @DevOpsSummit at 20th Cloud Expo, Himanshu Chhetri, CTO at Addteq, will cover practical examples and use cases such as self-provisioning infra...
Mar. 25, 2017 11:15 PM EDT Reads: 2,912
The financial services market is one of the most data-driven industries in the world, yet it’s bogged down by legacy CPU technologies that simply can’t keep up with the task of querying and visualizing billions of records. In his session at 20th Cloud Expo, Jared Parker, Director of Financial Services at Kinetica, will discuss how the advent of advanced in-database analytics on the GPU makes it possible to run sophisticated data science workloads on the same database that is housing the rich inf...
Mar. 25, 2017 10:45 PM EDT Reads: 3,614
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
Mar. 25, 2017 09:45 PM EDT Reads: 3,555
Apache Hadoop is emerging as a distributed platform for handling large and fast incoming streams of data. Predictive maintenance, supply chain optimization, and Internet-of-Things analysis are examples where Hadoop provides the scalable storage, processing, and analytics platform to gain meaningful insights from granular data that is typically only valuable from a large-scale, aggregate view. One architecture useful for capturing and analyzing streaming data is the Lambda Architecture, represent...
Mar. 25, 2017 08:45 PM EDT Reads: 5,969
My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
Mar. 25, 2017 08:45 PM EDT Reads: 2,758
Things are changing so quickly in IoT that it would take a wizard to predict which ecosystem will gain the most traction. In order for IoT to reach its potential, smart devices must be able to work together. Today, there are a slew of interoperability standards being promoted by big names to make this happen: HomeKit, Brillo and Alljoyn. In his session at @ThingsExpo, Adam Justice, vice president and general manager of Grid Connect, will review what happens when smart devices don’t work togethe...
Mar. 25, 2017 06:15 PM EDT Reads: 2,562
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
Mar. 25, 2017 05:15 PM EDT Reads: 1,930
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), will provide an overview of various initiatives to certifiy the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldw...
Mar. 25, 2017 04:00 PM EDT Reads: 508
Adding public cloud resources to an existing application can be a daunting process. The tools that you currently use to manage the software and hardware outside the cloud aren’t always the best tools to efficiently grow into the cloud. All of the major configuration management tools have cloud orchestration plugins that can be leveraged, but there are also cloud-native tools that can dramatically improve the efficiency of managing your application lifecycle.
Mar. 25, 2017 04:00 PM EDT Reads: 2,831