|By Trevor Parsons||
|August 12, 2014 01:01 PM EDT||
I was very interested to see that Manning Publications have just announced the Early Access release of a new book called Unified Log Management. I looked through the table of contents and it was what I expected - a roll your own log management cookbook with a list of technologies that might help (Kinesis, Redshift, Kafka etc.). Presumably it will also include Google DataFlow by the time the book ships. So it's only really for those super-talented all-rounders with a lot of time on their hands or companies with the budget to hire a team of software developers and data scientists. Not really for the very busy DevOps or IT Ops crew with systems to keep available and the need for powerful yet easy to use tools to help them distill the hosepipe of data being generated.
What really peaked my interest was the blog post by the book's author, Alex Dean, here. It turns out the "Unified" bit in the title was influenced by Jay Kreps article on the log. In my opinion, this is one of those articles that comes around once a generation! I don't think it's overstating the case to say that this captures all the most important lessons about big data, Internet of Things, massively scaled-out services in the cloud, growth hacking etc. It's up there with the 1988 paper that introduced the data warehouse concept and it does capture the future of enterprise architecture in a narrative that is refreshingly philosophical and erudite (at least for a tech blog post).
- Capture every event that happens in your business - particularly every customer interaction
- Store it multiple times because this stuff is gold
- Pick the right tools to view this data from the perspectives that make sense for your business
- Need better data - feed more into the log
- Need a new perspective - add a new tool
The perspectives are visualizations of the output data. The tools are getting more specialized every day... time-series databases for metrics, complex event processing engines for security threats, distributed graph databases for social interactions, search indexing etc. If you know what you are looking for then use real-time pattern matching. If you don't know what you are looking for but want to be told when things start to look different then look for anomalies.
The log, according to Kreps, is at the heart of all these data transformations.
Stream data into the system as fast as your customers can generate it. On the other end you get insights out of the system. And in the middle you painstakingly, hand-craft your log transformation algorithms (with your team of superstar coders and data scientists).
We are busy working on new perspectives and visualizations. New tools and transformations. New insights and new alerts. The book is not out until Spring 2015 so we'll have a ton more features out before then.
Seeing as we started talking about unified logs - how about this for unified?
What are the new priorities for the connected business? First: businesses need to think differently about the types of connections they will need to make – these span well beyond the traditional app to app into more modern forms of integration including SaaS integrations, mobile integrations, APIs, device integration and Big Data integration. It’s important these are unified together vs. doing them all piecemeal. Second, these types of connections need to be simple to design, adapt and configure...
Sep. 28, 2016 11:49 AM EDT
Is your aging software platform suffering from technical debt while the market changes and demands new solutions at a faster clip? It’s a bold move, but you might consider walking away from your core platform and starting fresh. ReadyTalk did exactly that. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue and over a decade of audio conferencing product development to start an innovati...
Sep. 28, 2016 11:45 AM EDT Reads: 2,060
Creating replica copies to tolerate a certain number of failures is easy, but very expensive at cloud-scale. Conventional RAID has lower overhead, but it is limited in the number of failures it can tolerate. And the management is like herding cats (overseeing capacity, rebuilds, migrations, and degraded performance). Download Slide Deck: ▸ Here In his general session at 18th Cloud Expo, Scott Cleland, Senior Director of Product Marketing for the HGST Cloud Infrastructure Business Unit, discusse...
Sep. 28, 2016 11:45 AM EDT Reads: 2,376
“We're a global managed hosting provider. Our core customer set is a U.S.-based customer that is looking to go global,” explained Adam Rogers, Managing Director at ANEXIA, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Sep. 28, 2016 11:45 AM EDT Reads: 3,056
Sep. 28, 2016 11:30 AM EDT Reads: 2,837
Sep. 28, 2016 11:21 AM EDT
Sep. 28, 2016 11:00 AM EDT Reads: 1,095
Sep. 28, 2016 11:00 AM EDT Reads: 366
Sep. 28, 2016 10:45 AM EDT Reads: 4,101
Sep. 28, 2016 10:45 AM EDT Reads: 241
Sep. 28, 2016 10:45 AM EDT Reads: 3,524
Sep. 28, 2016 10:30 AM EDT Reads: 2,936
Sep. 28, 2016 10:30 AM EDT Reads: 4,541
Sep. 28, 2016 10:15 AM EDT Reads: 3,249
Sep. 28, 2016 10:00 AM EDT Reads: 1,088