Welcome!

Related Topics: @BigDataExpo, Java IoT, Microservices Expo, Containers Expo Blog, @CloudExpo, SDN Journal

@BigDataExpo: Article

Consolidating Big Data

How to make your data center more cost-effective while improving performance

Cloud computing has opened the doors to a vast array of online services. With the emergence of new cloud technologies, both public and private companies are seeing increases in performance gains, elasticity and convenience. However, maintaining a competitive advantage has become increasingly difficult. Service providers are taking a closer look at their data storage infrastructure for ways to improve performance and cut costs.

If the status quo remains, maintaining low-cost cloud services will become increasingly difficult. Service providers will incur higher costs, while consumers become burdened with storage capacity restrictions. Such obstacles are influencing service providers to find new ways to scale cost-effectively and increase performance in the data center.

Cost-Benefit Analysis
In response to the increase of online account activity, service providers are consolidating their data centers to a centralized environment. By doing so, they are able to cut costs while increasing efficiency, allowing data to be accessible from any location. Centralizing equipment enables providers the ability to deliver enhanced Internet connections, performance and reliability.

However, with these added benefits also come disadvantages. For instance, scalability becomes more expensive and difficult to achieve. Improving efficiency within a centralized data center requires the purchase of additional high-performance, specialized equipment, which increases costs and energy consumption, challenging endeavors to control at scale. In an economy where cost-cutting is becoming a necessity for large and small enterprises alike, these added expenses are unacceptable.

Characteristics of the Cloud
Solving performance problems, like data bottlenecks, is a growing concern for cloud providers who must oversee significantly more users and accompanying performance demands, than do enterprises. Although the average user of an enterprise system requires elevated performance, these systems generally manage fewer users who are able to access their data directly through the network. Moreover, enterprise system users are accessing, saving and sending comparatively relatively small files that require less storage capacity and performance.

Outside the internal enterprise network, however, it's a different story. Cloud systems are simultaneously being accessed by a multitude of users across the Internet, which itself becomes a performance bottleneck. The average cloud user stores relatively larger files than the average enterprise user placing greater strains on data center resources. The cloud provider's storage system not only has to scale to each user, but must also sustain performance across all users as well.

Best Practices
In response to growing storage demands, cloud providers are faced with profound business implications. Service providers need to scale quickly in order to meet the booming demand for more data storage. The following best practices can help optimize data center ROI in a period of significant IT cutbacks:

  • Opt for commodity components when possible: Low-energy hardware makes good business sense. Commodity hardware is not only cost-effective, but also energy-efficient, which significantly reduces both setup and operating costs in one move.
  • Seek out a distributed storage system: Distributed storage presents the best way to build at scale even though the data center trend has been moving toward centralization. Increased performance at the software level counterbalances the performance advantage of a centralized data storage approach.
  • Avoid bottlenecks: A single point of entry can easily lead to a performance bottleneck. Adding caches to relieve the bottleneck, as most data center infrastructures currently do, quickly adds cost and complexity to a system. On the other hand, a horizontally scalable system that distributes data among all nodes delivers a high level of redundancy.

Moving Forward
Currently, Big Data storage consists mainly of high performance, vertically scaled storage systems. Since these infrastructures can only scale to a single petabyte and are costly, they are not a sustainable solution. Moving to a horizontally scaled data storage model that distributes data evenly onto energy-efficient hardware can reduce costs and increase performance in the cloud. With these insights, cloud service providers can take the necessary steps to improve the efficiency, scalability and performance of their data storage centers.

More Stories By Stefan Bernbo

Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, he has designed and built numerous enterprise scale data storage solutions designed to be cost effective for storing huge data sets. From 2004 to 2010 Stefan worked within this field for Storegate, the wide-reaching Internet based storage solution for consumer and business markets, with the highest possible availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile and fixed network operators.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Latest Stories
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive ad...
WebRTC services have already permeated corporate communications in the form of videoconferencing solutions. However, WebRTC has the potential of going beyond and catalyzing a new class of services providing more than calls with capabilities such as mass-scale real-time media broadcasting, enriched and augmented video, person-to-machine and machine-to-machine communications. In his session at @ThingsExpo, Luis Lopez, CEO of Kurento, introduced the technologies required for implementing these idea...
Enterprise networks are complex. Moreover, they were designed and deployed to meet a specific set of business requirements at a specific point in time. But, the adoption of cloud services, new business applications and intensifying security policies, among other factors, require IT organizations to continuously deploy configuration changes. Therefore, enterprises are looking for better ways to automate the management of their networks while still leveraging existing capabilities, optimizing perf...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addres...
The WebRTC Summit New York, to be held June 6-8, 2017, at the Javits Center in New York City, NY, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 20th International Cloud Expo and @ThingsExpo. WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web co...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
While not quite mainstream yet, WebRTC is starting to gain ground with Carriers, Enterprises and Independent Software Vendors (ISV’s) alike. WebRTC makes it easy for developers to add audio and video communications into their applications by using Web browsers as their platform. But like any market, every customer engagement has unique requirements, as well as constraints. And of course, one size does not fit all. In her session at WebRTC Summit, Dr. Natasha Tamaskar, Vice President, Head of C...
The idea behind this session is my blog post - 5 Logstash Alternatives - which is unfortunately too short to do the presented log shippers justice. In his session at @DevOpsSummit at 20th Cloud Expo, Radu Gheorghe, Software Engineer at Sematext Group, will talk more about the things that matter: kinds of buffers, protocols, ways of parsing, correlating and de-duplicating messages, as well as supported inputs and outputs. And of course performance. All this should let you know which log shipper...
"We are an all-flash array storage provider but our focus has been on VM-aware storage specifically for virtualized applications," stated Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, discussed the impact of technology on identity. Sho...
SYS-CON Events announced today that Catchpoint Systems, Inc., a provider of innovative web and infrastructure monitoring solutions, has been named “Silver Sponsor” of SYS-CON's DevOps Summit at 18th Cloud Expo New York, which will take place June 7-9, 2016, at the Javits Center in New York City, NY. Catchpoint is a leading Digital Performance Analytics company that provides unparalleled insight into customer-critical services to help consistently deliver an amazing customer experience. Designed ...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
With all the incredible momentum behind the Internet of Things (IoT) industry, it is easy to forget that not a single CEO wakes up and wonders if “my IoT is broken.” What they wonder is if they are making the right decisions to do all they can to increase revenue, decrease costs, and improve customer experience – effectively the same challenges they have always had in growing their business. The exciting thing about the IoT industry is now these decisions can be better, faster, and smarter. Now ...
"Splunk basically takes machine data and we make it usable, valuable and accessible for everyone. The way that plays in DevOps is - we need to make data-driven decisions to delivering applications," explained Andi Mann, Chief Technology Advocate at Splunk and @DevOpsSummit Conference Chair, in this SYS-CON.tv interview at @DevOpsSummit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.