Key Capacity Metrics - Top 5 Key Capacity Management Concerns for UNIX/Linux (11 of 12)

What key capacity metrics should you be monitoring for your Big Data environment? My list includes some key CPU, Memory,  File System and I/O metrics which can give you a good understanding of how well your systems are performing and whether any potential capacity issues can be identified.

•      Standard CPU metrics

o  utilization, system/user breakdown

•      Memory

o  Usage, Paging, Swapping

•      User/Process breakdown – define workloads

•      File System

o  Size

o  Number of Files

o  Number of Blocks

o  Ratios

o  User breakdown

•      I/O

o  Response time

o  Read/Writes

o  Service times

o  Utilization

By capturing the user/process breakdown on your UNIX/Linux systems, we can start to define workloads and couple that with the predicted business usage to produce both baseline and predictive analytical models. 

Some of the following key questions can then be answered:

•      What is the business usage/growth forecast for next 3, 6, 12 months?

•      Will our existing infrastructure be able to cope?

•      If not what will be required?

•      Are any storage devices likely to experience a capacity issue within the next 3,6,12 months?

•      Are any servers or storage devices experiencing any performance issues and what is the likely root cause?

This is not an exhaustive list, but it does provide information on the key capacity metrics you should be monitoring for your Big Data environment. 

In my final blog on Wednesday I'll be looking at CPU breakdown and summarizing. In the meantime sign up to our Community for access to some of our great Capacity Management resources such as white papers and on-demand webinars http://www.metron-athene.com/_resources/published-papers/login.asp

Jamie Baker
Principal Consultant

What should we be monitoring? - Top 5 Key Capacity Management Concerns for UNIX/Linux (10 of 12)

Following on from my previous blog on Big Data this is relatively new technology and therefore knowledge around performance tuning is immature.  Our instinct tells us that we monitor the systems as a Cluster, how much CPU and Memory is being used with the local storage being monitored both individually and as one aggregated piece of storage.  Metrics such as I/O response times, file system capacity and usage are important, to name a few.

What are the challenges?

Big Data Capacity Challenges

So with Big Data technology being relatively new with limited knowledge, our challenges are:

•      Working with the business to predict usage - so we can produce accurate representations of future system and storage usage.  This is normally quite a challenge for more established systems and applications so it we have to bear in mind that getting this information and validating it will not be easy.

•      New technology - limited knowledge around performance tuning

•      Very dynamic environment - which provides the challenge to be able to configure, monitor, and track any service changes to be able to provide effective Capacity Management for Big Data.

•      Multiple tuning options - that can greatly affect the utilization/performance of systems

What key capacity metrics should you be monitoring for your Big Data environment?

Find out in my next blog and ask us about our Unix Capacity & Performance Essentials Workshop.

http://www.metron-athene.com/services/online-workshops/index.html

Jamie Baker

Principal Consultant