Table of Contents
The purpose of this guide is to describe the following topics related to the RightScale Cloud Appliance (RCA-V) for vSphere.
The RightScale Cloud Appliance (RCA-V) for vSphere is a stateless web service that presents a compute cloud API on top of of VMware vSphere environment. Its purpose is to enable the RightScale Cloud Management Platform to manage a vSphere installation (or designated sub-portion) as if it were an on-premise cloud environment (resource pool) similar to public cloud IaaS offerings such as Amazon Web Services or Google Cloud Platform, or private clouds such as OpenStack and CloudStack.
The appliance is a minimal layer on top of vSphere that serves three main purposes:
Note: Since the appliance is designed to be stateless, you can run multiple appliances in parallel for fault tolerance. In such cases, both appliances must be able to communicate with each other.
The RightScale Cloud Appliance for vSphere sits between the RightScale multi-cloud gateway and the vSphere API implemented by a vCenter server in a VMware vSphere installation. The multi-cloud gateway is part of RightScale’s SaaS service and is operated by RightScale. Whereas the RCA-V is a dedicated virtual machine (VM) that's deployed from the vCenter console as a dedicated appliance within the vSphere environment. It's important that the RCA-V is operated in close proximity to the vCenter server(s) because it requires low latency access to RCA-V API endpoint.
Note: The RCA-V is distributed in OVA format.
RightScale to RCA-V
Once the RCA-V is deployed, it establishes an outbound connection over HTTPS (using websockets) with the RightScale MultiCloud Gateway. Therefore, the RCA-V must be allowed to make outbound connections to the specified IP address range used by the RightScale gateway. Make sure the appropriate network firewall permissions are properly configured before the RCA-V is deployed. See Prepare vSphere environment for RightScale Connectivity.
RCA-V to vCenter
The RCA-V does not require administrator credentials but only requires a non-admin user with a minimum set of privileges required by the RightScale platform to access the vSphere API on the vCenter server(s) over HTTPS. For more details, see vCenter Access Requirements.
The RCA-V is designed to leverage the existing vCenter API as much as possible in order to simplify communication between the RightScale platform and the vSphere installation. For example, in order to place a VM onto a host, the implemented solution is to leverage vSphere’s DRS (distributed resource scheduler) instead of encoding a custom algorithm into the appliance. However, the RCA-V does partition available resources within a vSphere installation in order to enable multi-tenancy, which is one of the core features of a cloud (IaaS) environment.
Tenants are defined in the appliance and consist of a specific datacenter:cluster combination(s), where each tenant is associated with a single RightScale account, so multiple RightScale accounts can be granted access to a vSphere installation. Each tenant has a designated partition of the vSphere installation (i.e. cloud-like resources such as VMs, volumes, etc. that can be provisioned for on-demand usage.) When a resource is provisioned, such as a running VM, it can only belong to a single tenant. Since multiple tenants can be configured to leverage the same underlying hosts, tenants can be configured to expose all running instances for a resource pool, which means that a RightScale account associated with tenant 'A' may see VMs that were provisioned to another RightScale account tied to tenant 'B'. Of course, VMs can only be controlled from the account to which they were provisioned. The visibility of all provisioned (running) resources is a configurable option which you may want to enable in order to provide more transparency of known VMs in your vSphere cloud.
The RightScale Cloud Appliance for vSphere is designed to be compatible with other tools, such as VMware vSphere client, that access the same vCenter installation. The appliance tracks resources using the IDs exposed in the vSphere API and tries to pay minimal attention to the configuration of each resource. In general, as long as the ID of a resource (such as a template, VM, datacenter, datastore, or network) does not change the appliance will be able to manage each resource as expected. Since resources are identified by their IDs, the names of each resource can be changed. Although it's not possible to change the allocated amount of cpu or memory size to an allocated (running) VM through the RightScale Cloud Management Dashboard/API, or in the RCA-V user interface, a network administrator can manipulate the underlying hardware using VMware tools. The RCA-V always tries to stay synchronized with its vSphere environment so that the names of all resource types are consistent between the vSphere client application and RCA-V admin console, which makes it easier to identify and match known resources in both user interfaces.
It's important to remember that changes to the underlying vSphere environment may affect how certain resources are managed via RightScale. For example, the removal of vSphere resources such as datacenters, clusters, or networks may cause affected resources to be left in an unknown or inaccurate state. Before you make any major changes to a vSphere environment, please consult RightScale support to ensure that all changes are handled appropriately in order to minimize any negative impacts to end users.
The RCA-V also discovers any new resources that are created using other third party tools. For example, VMs launched via other tools are discovered as instances and exposed to the appropriate tenant in accordance with the defined multi-tenancy policy.
The appliance is also intended to deal properly with vSphere automation, in particular with vMotion. A VM that is vMotioned to a different host does not appear any differently through RightScale, however more testing is required in order to ascertain the limits of compatibility. (Note: At this stage, the appliance does not initiate vMotion.)
The API implemented by the RCA-V is similar in nature to that of many public clouds, including EC2, GCE, or OpenStack. As such, its REST API has calls to list images, datacenters, and instances, as well as launch an instance from an image or terminate an instance. These API calls are implemented in the appliance on top of the VMware vSphere SOAP API, which is lower-level than a cloud API. For example, launching an instance requires a few dozen vSphere API calls and a number of decisions to be made.
An important characteristic of the appliance is that it's more of a policy layer than an automation or functionality layer because vSphere already contains automation to do just about anything a cloud needs to do, but it lacks a standardization/policy layer that makes it possible to give end-users control over their set of resources. For example, vSphere has automation to place new VMs onto the most appropriate host. However, what it actually does can be set in many places: in the VM template, in the VM, in the clone/launch call, or in various DRS settings. What the appliance does is to let the cloud administrator decide on which VM placement policy is to be used and then enforce that for all VMs. Similarly the administrator can decide which form of provisioning is to be used, e.g. linked-clone or not, thin, thick-eager, or thick-lazy.
The implementation of the appliance is designed to be configurable by the customer/administrator and customizable by RightScale professional services. For this purpose the implementation is divided into two parts: the core of the appliance with a set of procedures and a customizable policy framework. The policy framework provides a number of options for each policy that can be set by the administrator and, in addition, allows custom policies to be inserted into the code by RightScale professional services. As an example, when selecting which datastore a new instance should be placed into the options available include requesting a recommendation from DRS, selecting a fixed datastore, or selecting the datastore with the most free space. If a different policy is desired it can typically be implemented and inserted into the appliance. Ideally, such custom policies can be written in a generic manner and be included as standard into future appliance releases.
The current release supports the following resource types:
|RightScale Resource||vSphere Correspondence||Purpose|
|Cloud Accounts||None - implemented in the appliance||Separate tenants within a vCenter implementation|
|Datacenters / Zones||Cluster within a datacenter||Fault tolerance separation (datacenters are intended to fail independently)|
|Images||VM Templates||Immutable images from which instances are launched|
|Instance Types||Specific VM settings for cpu, memory, etc.||Standardized virtual machine configurations|
|Instances||VMs||Running virtual servers|
|Subnets||Networks||Allows instances to be connected to a network|
|IP Addresses||None||Support static and dynamic IP address assignments|
|SSH Keys||None||Enables users to SSH into instances|
The following resource types are expected to be supported in future releases:
|RightScale Resource||vSphere Correspondence||Purpose|
|Volumes||VMDKs||Allows block storage devices to be dynamically mounted on instances.|
|Snapshots||Snapshots||Allows volumes to be backed-up under automation control.|
|Network Interfaces||Network Virtual Devices||Allows instances to be connected to multiple subnets.|
|Misc. Network Concepts||NSX||May allow networks and network configurations to be managed directly.|
As a reference, the vSphere object hierarchy is as follows:
The RightScale Cloud Appliance for vSphere (RCA-V) exposes what RightScale refers to as a private cloud, where a cloud is a set of resources and an administrative domain. The appliance may be configured to only expose a subset (and not all) of the visible vSphere resources.
The appliance partitions the resources accessible through vSphere into non-overlapping tenants that are called cloud accounts in RightScale. It is imperative that, with a few exceptions, the resources accessible through one tenant be disjoint from the resources accessible through all other tenants. The exceptions are: images can be visible to all tenants (more in the images subsection), networks can be shared across tenants, hosts and datastores are not visible to tenants and can be shared appropriately.
Tenants (cloud accounts) are specified in the appliance configuration where, for each tenant, the datacenter & cluster combinations that the tenant has access to are listed. The appliance then exposes each datacenter & cluster combo as a RightScale datacenter/zone (vSphere uses the term “datacenter” for something different than RightScale does). For each combo a configuration switch determines whether a tenant sees all VMs, or only its own VMs, or its own VMs plus other “unowned” VMs.
This enables typical configurations where a cluster is dedicated to a particular purpose, such as production, or QA and can be exposed as one tenant. It also allows for a cluster to be shared by multiple tenants. In all cases, however, for a given cluster each VM must only be seen by one tenant, so only one tenant can see all VMs or all unowned VMs and if there is a tenant that can see all VMs then it's impossible to configure tenants that see their own VMs too.
The following figure shows a vSphere configuration with two datacenters and two clusters of ESXi hosts in each datacenter. It shows the appliance configured for 5 tenants. Tenant 1 has dedicated access to cluster 1. Tenant 2 has dedicated access to clusters 2 and 3. These will be exposed as two availability zones in the RightScale system. Tenants 3, 4, and 5 share access to cluster 4. While they share the resources in the cluster each tenant will only see the VMs it launched. In clusters 1, 2, and 3 the tenant will see all VMs, including ones launched outside of RightScale. In cluster 4 one of the tenants can be selected to see such VMs, the other tenants will not.
Each cluster in vCenter is exposed as one datacenter/zone in the RCA-V and RightScale Cloud Management Dashboard/API.
In the initial implementation of the appliance VM templates are mapped to read-only images (i.e. There is no way to create or modify images from the appliance). You can only create or modify VM templates using VMware tools and not with the RCA-V. Although it may be possible to map vApp templates, OVFs, or even raw VMDKs to images in the future and support the management of those resources using the appliance, this type of functionality is currently not supported.
In the initial implementation of the appliance, VM templates appear as visible images for all tenants that have access to the datastore in which those templates are stored. In the future, it may be possible to configure tenant-specific images, but that type of functionality is currently not supported.
One issue with VM templates is that they can be converted to a VM using VMware tools and then converted back into templates (i.e. images). However, confusion and problems will occur if those VMs are never converted back into templates because the appliance will still show both the original template and the launched VM even though the template cannot be used to launch another VM, which most users who are familiar with the RightScale platform would expect to be supported. Confusion and errors will result if templates are converted to VMs but never converted back (into templates).
The appliance follows the typical IaaS cloud model where instance types are defined by their cpu and memory capacity. Instance types are defined when you set up and configure the RCA-V.
Note: An "unknown" type is used for mapping all discovered instances that do not map to another instance type.
Virtual machines (that are not templates) in vSphere are mapped 1-1 to instances in the appliance. Powered-off VMs are mapped to stopped instances. The instance type of a VM that was launched through the appliance always reflects the instance type (as it was defined when the VM was launched). Any changes that are made to the VMware tools to the VM’s cpu/memory allocation after the VM is launched will not be reflected in the appliance, although such changes are technically allowed.
Note: If an instance is launched outside of the appliance, its displayed instance type will either match an existing instance type (if a match can be found) or be displayed as a special "custom" type (if a match cannot be found).
When an instance is terminated through RightScale the VM is powered off and then deleted from the datastore (i.e. The VM permanently vanishes from the system!) In the future, support for stopping/starting an instance, which is essentially equivalent to powering off/on a VM in vSphere, will be supported in the RightScale platform.
SSH keys are public SSH keys that can be created in the scope of a cloud tenant for the purpose of logging into servers. The recommended approach is not to use these cloud SSH keys described here but to use RightScale’s “managed login” feature, which allows each end-user to have a unique and private SSH key that is automatically loaded on all instances (assuming the user has the 'server_login' user role permission in the RightScale account). However, the managed login keys are only loaded onto the instance after the RightScale RightLink agent starts and establishes communication with RightScale, which can become problematic and difficult to troubleshoot if instances do not start-up correctly with RightLink properly installed and started. However, this condition typically only occurs during the development of new images. When you're creating new images for your vSphere environment, it's useful to use your cloud SSH keys available because you can use them to SSH into a problematic machine when RightLink is not working as expected. In such cases, you may want to make sure the cloud SSH key is installed early in the boot process by an init.d script.
Note: Currently, there is no mechanism for supporting Windows RDP passwords.
The appliance maps vCenter networks into subnets (and each tenant ends up with a single default network per datacenter in the RightScale platform). The networks are read-only from the appliance perspective and their only purpose is to select and determine which public and/or private network an instance will be connected to at launch time.
The appliance implements support for assigning IP addresses to instances, specifically, to assign an IP address to the network interface on an instance. It does not currently deal with any upstream devices that may NAT external IP addresses to instances or otherwise load balance across instances.
The appliance offers 3 options for managing an instance’s IP address(es):
DHCP based Assignment - The images need to contain code in order to set the IP address. (Typically, DHCP is used for this purpose but no restrictions are enforced at this time.)
Dynamic IP Assignment - Instances are assigned an IP address (at launch time) from a “dynamic” IP address pool configured using RCA-V Admin UI, which will essentially yield the same result as a DHCP service (when DHCP is not available).
Specific IP Assignment - Instances are assigned a specific IP address (at launch time) from a “static” IP address pool. IP assignments are configured by the user launching the VM.
For option 2, the configuration of the appliance can include the specification of an IP range for each network. The multicloud gateway then selects an unused IP address for each newly launched instance. Such dynamic IP address ranges are not communicated back to the RightScale platform (beyond the gateway) and are not visible to users. However, the specific IP address assigned to an instance is reported back assuming that it is running the VMware tools.
For option 3, the configuration of each tenant and network combination can include an IP range specification. This static IP range is reported back to the RightScale platform as a set of assignable IP addresses, which has a similar functionality to Amazon's Elastic IPs in EC2. The user can then choose an IP address for this set when creating a server or launching an instance. However, initially such IP address assignments are immutable for the lifetime of an instance. Option 3 can be combined with options 1 or 2 giving the user the choice of specifying an explicit IP address from the provided pool or letting a dynamic IP address be assigned automatically.
In all cases the result of assigning an IP address is to set the IP address on the virtual network interface and sending out ARP announcements to let routers establish the appropriate IP to MAC address mapping (assuming that the routers are properly configured). It is also entirely possible for instances to change the IP address of their interface. In such cases, the appliance does not interfere with changes to IP assignments and will simply report any changes back to the RightScale platform so that they are accurately reflected in the RightScale Dashboard.
An important detail is that the appliance uses the IP address reported by the VMware tools to report the actual IP address of an instance back to the RightScale platform. Thus, if the VMware tools are not installed on an instance it is impossible for RightScale to detect the actual IP address.
© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.