Thursday, May 29, 2014

Cloud backup solution (AWS, Rackspace, Azure, Softlayer, HP, etc.)


  As a consultant I receives many queries regarding the data security on cloud and also the durability of data on cloud. So I am writing this post assuming your basic knowledge of cloud demon. 
Everyone knows that cloud object storage is highly durable 99.99999999%,But what about the block storage (AWS EBS, Softlayer virtualDrive, etc.)?

   The failure rate of block storage on cloud is 0.1% to 0.4% every year. It is very important to have a backup of cloud block storage or entire cloud instance (VM) to either on cloud storage or on-premise.

  There are many ways we can take backup of cloud instance, and each type of cloud backup has its own pros and cons.

    1) Application Level Backup 
    2) VM level backup of cloud instance 
    3) OS Level Backup of cloud instance
    3) File level backup of cloud instance
        
    The backup solution type selection depends on the application, data and user requirement in terms of RTO(Recovery time objective).

    Lets discuss each in detail,
    1) Application Level Backup
          Application level backup is always seems to be as Da Vinci's demon. Each type of application backup are different. Let's consider AWS as common cloud terminology for simplicity
           If we consider Database as Application then we can use native tools for backup and we can store it to S3 object storage. 3rd party tools (eg. Cloudberry) also can be very helpful for database backup to cloud storage. 
           There are also tools available to take backup of database to on-premise local storage but as i mentioned these demons will not allow you to achieve low RTO of  database or application.

   2) Cloud Instance backup (VM Level):
            These kind of backup can provide acceptable RTO for production environment. every cloud service provider provides snapshot level backup. these snapshot will be stored in cloud object storage (Extra cost per GB per month). In order to take the backup of cloud instance as FULL backup optionally you can use Import/Export feature of cloud service provider. 
               
     3)OS Level Backup of cloud instance:
            OS level backup requires 3rd party agent to be installed in cloud instance (eg. Cloudberry). you can redirect cloud instance data to you local storage or cloud storage using these tools. The RTO of this kind of backup's restore will be higher than VM level backup type.

     4) File Level Backup of Cloud instance:
             File level backup always requires require 3rd party backup software to automate backup task as well as perform encryption and compression to files. The RTO of file level backup depends on types of file. (eg. if you are restoring only document files then RTO will be very less but if you are restoring Database files then RTO will increase because these database files later need to be restored to database engine also)

   
After considering the type of backup next step is to decide weather to take backup in same cloud storage or other cloud storage or take backup locally 

     The quest of backup storage options depends on where do you want to restore your production machine in case of failure.
   ->  If you want to restore the data to same cloud instance i would prefer to select same cloud backup storage.
   -> If you want to restore the data to other cloud service provider infrastructure then select specific cloud service provider object storage(It can also work as cloud migration strategy)
   -> If you want to restore entire server to on premise as virtual machine or physical host then select on-premise storage.
     
     Above discussion was more generic without any specific tools and solution. after selecting the proper type of backup solution we can select available tools for the same.

   

Tuesday, May 6, 2014

Horizon 6 dos and don'ts

 Lets catch up with Horizon 6 best practice. there are lots of article i have found on VDI best practice but it seems like everyday the era of vdi technology is changing and now vdi is also moving to public cloud infrastructure(desktone, AWS etc.). Many of our existing customers were asking for the best practice for vdi deployment which can give optimal performance.

  I was chatting with some of vmware technical support team guyz regarding current view issues and they have mentioned that most of the customers are facing issues because of wrong sizing or incorrect design in their VDI environment. You may find bellow table generalized for other vdi solutions too. feel free to comment for any specific feature's best practice.
    
Tag
Recommendation
Core, scaling, front-end
acceleration
Do: Consider front-end web acceleration solutions for high-volume
infrastructures supporting large numbers of clients.

Core, vCenter

Do: Implement vCenter according to its requirements; back up the database
regularly and have provisions in place in case of an outage. vCenter is very
important.

Core, Active Directory,
DNS, DHCP, Load
Balancing
Do: Consider front-end web acceleration solutions for high-volume
infrastructures supporting large numbers of clients.

Net, VMkernel 10GbE or 8 Gbps FC for storage, uplinks,
dedicated storage net
Do: Always use dedicated (non-shared) VMkernel ports with redundant
10GbE or 8Gbps FC uplinks for connecting each vSphere server to storage.
(network recommendation changes in each scenario but above recommendation will give best performance in any scenario)
Net, Connection Servers,
Load Balancing

Do: Implement more than one Horizon View Connection Server VM to
provide high availability of the desktop VM logon services

Net, uplink (NIC)
redundancy

Do: Assign multiple physical uplinks (network interfaces) to each VMkernel
port and VM Port Group to avoid an outage related to a network interface
card failure in the server.

Net, VLAN tagging

Do: Consider using VLAN tags for VMkernel and VM Port Groups to reflect
VLANs defined within your network infrastructure.

Net, infrastructure (e.g.
switch, etc.) redundancy
Do: Deploy a fully redundant networking infrastructure to reduce the
likelihood of an outage resulting from a failed networking switch or other
component.

Net, VLANs, management

Do: Consider using VLANs to simplify the management of the physical
network.

Net, utilization and
threshold monitoring

Do: Monitor Ethernet port and switch utilization to help reduce the
possibility that an overloaded switch causes performance problems.

Net, storage network,
Back plane bandwidth of switch
Do: Use high-speed  switches between the ESXi hosts and storage which minimum supports back plane bandwidth equals to no. of populated switch port.

Net, Dedicated VMkernel
ports and uplinks for
storage
Do not: Create VMkernel ports for other purposes (e.g. Fault Tolerance
Logging), or VM Port Groups on the same vSwitch (and its uplinks)
designated for storage.

Net, VLANs, bottlenecks,
proper use/design

Do Not: Use VLANs to simplify the network at the expense of creating
network bottlenecks.

Net, 1 GbE, VMstore,
bottlenecks

Do Not: Use 1GbE Ethernet switches for the connections between the
vSphere hosts and storage if VDI is loaded with IO and latency intensive applications

Server, testing, scaling,
carefully consider last
minute changes

Do not: Make significant last-minute changes or substitutions to the
successfully tested server equipment and desktop VM configurations
without considering the potential impact to performance and user
acceptance.

Server, vSphere Host,
CPU cores

Do: Always ensure that the vSphere hosts have the adequate number of
cores and CPU performance required by the desktop VMs.

Server, vSphere Host,
installed memory (RAM)

Do: Always ensure that the vSphere hosts have the adequate amount of
memory (RAM) to run the desktop VMs and their applications efficiently.

Server, user acceptance
testing, end-to-end
(endpoint) performance
Do: Always perform end-to-end user acceptance testing as early as possible
in a deployment cycle to exercise the servers, as well as the entire
environment from the servers to remote users and their endpoint devices
(e.g. smart phones, PCs, laptops, tablets, etc.).

Server, parent images,
preparation

Do: Test and verify the design of the proposed parent VM(s) to determine
the resources they will need, such as virtual CPUs, memory assignments,
storage IOPS, etc.

Server, parent images,
testing, validation
Do: Test and validate the proposed VM images with users and incorporate
their feedback as much as possible before starting a full pilot process.

storage
best practices

Do: Always start with the latest Best practices for deploying storage in VMware vSphere environments.
 Bandwidth sizing
Do: Alwaws calculate datacenter internet bandwidth requirement if users are accessing VDI remotely here is useful link for protocol required bandwidth calculation link from Andri's blog Protocol bandwidth calculator

    feel free to comment if you guyz knows any other tips too.. till then ADIOS AMIGOS...
    

Saturday, May 3, 2014

HPC (Grid computing) provisioning on cloud!!! Why?

        Ability to adapt pay per use model has been an goal of computing for years and has moved into the cloud ecosystem lately. It is now coming to fruition for enterprise scale. IBM has announced the IBM Platform Computing Cloud Service on SoftLayer. This marries the best of dynamic high performance computing capabilities of IBM Platform Computing software with the infrastructure as a service (IaaS) capabilities of SoftLayer.


What does this mean to you?

• Already running an IBM high performance computing (HPC) cluster in your organization? Now you can burst out to the cloud to add more capacity to work through more complicated problems without dealing with the investment related to servers.
• You don’t have an HPC cluster but want to apply these capabilities to your business? Now you can do this on the cloud in SoftLayer.
• Got a complex big data analytics need? Now you can leverage the high performance computing capability to spin up MapReduce clusters to solve support your analytical needs.

One key point here is that these types of applications tend to be designed to solve a specific problem. You know, the grid computing type of stuff, which is cool if you’re trying to crack the mysteries of the Higgs Boson particle. What if you’re just trying to scale a web application?



You know, application architects who are building applications based on typical multi-tier patterns for retailers, banks, and more. Or what if you’re simply building an app with an application programming interface (API)?
There are several ways these types of apps can breathe on SoftLayer today:

• Applications can be set to autoscale using the RightScale SaaS.
• You could follow Netflix’s lead and leverage their OSS tooling. IBM has worked to marry the best of SoftLayer and Netflix tools with the know-how of IBM to create another instance of a cloud that breathes.
• You could use clustering and scaling capabilities available in most application servers.

What’s the bottom line?

Know your lab and your monster

Creating a living breathing creature is not easy, whether you’re a mad scientist or just a plain old developer. The devil is in the details. High performance computing, dynamic scaling and autoscaling all can mean different things to different people.


Three things to consider


• What is your application architecture? Is it an HPC/Grid computing style application? Is it a multi-tier application that manages state? Both scale differently.

• Are you all in the cloud or looking to burst from an existing data center? Are you looking to have an app that breathes only in the cloud, or between your data center and the cloud?

• How dynamically do you need your cloud to breathe? Keep it simple where possible. Yes, it is cool to have an application that can scale immediately based on traffic. However, you should ask yourself if you really need this. Sometimes, the simple capabilities we already have may be enough.

I’ll explore these architectural decisions and things to consider in upcoming posts.

It’s not all magic, but there are implications and considerations for your systems to live in a cloud that breathes. Are you ready?

What's new in VMware Horizon 6



            In many previous discussion with customers i have seen lots of questions regarding hosted share desktop specially SMBs. I believe that citrix is (or was) market leader in end user computing because they were able to deliver hosted share desktop and app virtualization to SMB sector and when those SMBs grows they will definitely implemented xendesktop for VDI as scalable architecture.

          Recently 4 months back i am hearing many new announcement from vmware in end user computing. In those announcement cool stuff like airwatch, vsan and horizon 6 was big attraction for many customers. Now VMware has hosted share desktop which is a big competition for Citrix.


VMware Horizon 6 has five major enhancements:
  • Cloud Pod Architecture. 
  • Remote Desktop Session Host (RDS) Hosted Apps 
  • Virtual SAN 
  • Application Catalog 
  • vCops for View 6 

Cloud Pod Architecture

Enable Horizon deployments across multiple datacenters New data layer replication across all Horizon Connection Servers (such as pool configurations and user entitlements).
  • Support single namespace for end users with a global URL 
  • Global entitlement layer to assign and manage desktops and users 
  • Scale Horizon View deployments to multiple datacenter above 10.000 desktops 
  • Support Active/Active and DR use case of Horizon deployments 
  • Support geo-reaming users 
Maximums of the Cloud Pod Architecture
DescriptionNumber
Number of sites (datacenters)2
Number of Pods4
Number of users/desktops (sessions)20.000

RDS Hosted Apps Windows 2008 and 2012 Microsoft Remote Desktop Services Hosts are supported
Seamless local look, feel and interaction for users
Works with Windows and Non-Windows devices such as Windows XP, Windows 7 and Windows 8 desktops, laptops and thin clients, iOS and Android tablets and Mac OSX. A client for Linux will be available soon

Virtual SAN for Horizon View Desktops


  • Application Catalog 
  • XenApp integration in the Application catalog 
  • ThinApp package delivery on any Windows desktop 
  • Office 365 and non-SAML web apps 
  • Improved resource management & categorization 
  • Seamless integration with Horizon View 

vCops for View 6

  • Horizon 6 support 
  • 25K concurrent users per instance 
  • Single integrated console for all vCOPs support environments (desktop, server etc) 
  • Application & In Guest Metrics. Drill down to the process level for key resource consumption per user and application 

Licenses

  • Horizon View Standard Edition: Delivers simple, high-performance VDI-based virtual desktops with a great user experience 
  • Horizon Advanced Edition: Offers the lowest cost solution for virtual desktop and application management, optimized storage with VMware Virtual SAN, image management and a unified workspace that supports hosted desktops and applications. 
Here is an overview picture of the Horizon 6 architecture:









In version 5 Horizon View supports 10000 desktops in a Pod. If you need to to have more or than 10000 desktops or needed to span a datacenter another Pod was needed. With the Cloud Pod Architecture the following improvements has been made :





Benefits:

Maximums of the Cloud Pod Architecture
The Horizon 6 View infrastructure servers supports Windows Server 2012 R2 as Operating System.
Prior Horizon 6 VMware only offers a VDI desktops. With RDS Hosted Apps in Horizon 6, VMware offers access to applications and full desktops running on Microsoft Remote Desktop Services Hosts with the PCoIP and Blast protocol. 

The RDS apps are available to the Horizon View broker.

Benefits


In the View clients the full desktops and RDSH applications looks as follows:





In Horizon View 5.3.1 support for VSAN was added. It was available as separate product. Now in Horizon 6 VSAN is added for free in the Horizon 6 Advanced and Enterprise Edition!





The Application Catalog offers a unified workspace for applications. One portal to for all applications (Local ThinApps, Citrix XenApp, SAAS and Remote Apps) from different devices.





The Application Catalog key themes are:

The application catalog has multi-forest Active Directory support and can be easily customized by changing logos, login prompt, application launchers, backgrounds etc.





vCenter Operations Manager for View 6 has the following new improvements:




Airwatch product integration with horizon is already available and I think very soon we will be seeing vcops integration for airwatch product family.




vSphere performance issue: Make your future self proud as admin

Baselining is, like cable management, a skill often downplayed, overlooked, or otherwise ignored—but is absolutely essential to any virtual environment. Consider this scenario:

On Monday, you arrive to a slew of emails: " I think my blaa-blaa app is not working correctly. It is super-slow." You think about it, do a quick check, and I mean quick, and go get your coffee. Then, you return to your desk and your boss walks over, and says, "I've been hearing a lot of people talking about their problems with blaa-blaa app. What's going on?"

Now, if you haven't been practicing baselining, you won't be able to give anything other than a subjective answer: "Well, they think it is running slowly, but I don't think it is running slowly." There is no empirical data to back up your statement; it is your word against theirs.

Baselining is the answer to the problem of subjectivity in the computing experience. It provides you, as a systems administrator, the ability to 
1) know what good performance looks like,
 2) determine, by delta comparison, when things are not functioning properly and performing below designed expectations, and, 
3) help to correct any erroneous experiences—whether you are wrong, and the app is performing poorly, or perhaps something else is going wrong, on the user's end.

So how does baselining work, and what is the proper way to perform it, to achieve this desired outcome? I'm glad you asked!

In my experience, many people baseline a system before Go Live—that is, without any users on it. Now, that is important. However, a true production baseline must be performed under normal user workload and resource consumption, otherwise how will you know if anything is not operating as expected?
Picture
A few things to remember; call them the "gotchas" of performance baselining.
  • Utilization is a measure of resource consumption, not user workload.
  • Latency is a measure of resource transmission, not user experience.

Too often our designs are based on static calculations: I know each VDI session will consume 500MHz of processor and produce .25Mbps of LAN traffic. Scale out. We create a modular "pod" if you will to figure out what we need for hardware, etc. And this is a great practice, and is used by virtually everyone when it comes to sizing.

The problem enters, most often, when VMware admins and other Sysadmins take what is meant to be a sizing guideline and turn it into a baseline—a function for which it was never intended. The sizing guideline is for purchasing and initial configuration; it is a place to start, whereas a baseline is really supposed to be representative of the end-state at go-live under normal operating conditions.

So how do you baseline? Well, you use the same tools, you just use them properly. You can use simulated user loads, but in my experience users are less predictable than we think. It is best, if possible, to perform your final baseline under normal user workload conditions, if possible over several days. And you should be able to summarize it as such—keep it simple for your users and boss's boss:

VDI baseline: On an average Monday, we have
  • 127 users logged in, with a peak login hours at 7:45–8:30am.
  • 58% CPU utilization per host, 78% during peak login
  • 62% RAM utilization per host, 68% during peak login
  • 12% network utilization per host, 35% during peak login
  • 2.3ms average latency per datastore, 15.8ms during peak login

Make sure that you have a baseline not just for servers in general, but for specific workloads, too. In this case above, I used VDI as an example.

Make sense? So avoid the trap of laziness, and do the hard work of optimizing and recording a baseline, and your future self will thank you greatly for it. And by the way, a tool like VMware vSphere Operations Manager or NetApp OnCommand Performance Manager can assist you greatly along the way.