RightScale Cloud Appliance (RCA-V) for vSphere: Troubleshooting Guide

1. Log in to the vSphere client.
2. Look at the RightScale Adapter VM and verify that it's running. It should have an IP address assigned to it and you should be able to ping it from behind your network firewall.

1. Log in to the vSphere client.
2. Go to Edit Settings > Options > vApp Options > Properties
3. Make sure all fields are properly configured. In the Hardware tab, make sure your network adapter is in the correct network and that you're pinging the appliance from a device on the same network.

This could be due to couple of reasons:

• IP address may not be reachable. (See the issue in the row above.)
• The Admin backend process may not be up -- If you suspect this to be the case, you can either restart the appliance (which will restart all processes) or you can SSH into the appliance, switch to the root user ($ sudo su) and then run the following command: # service vscale-admin restart

There are several reasons why the status may be red instead of green:

A connectivity problem with the WebSocket tunnel between the RCA-V and RightScale platform could be a result of issues related to the wstuncli (tunnel client on the appliance) or wstunsrv (RightScale's tunnel server). Run the following UNIX command from a computer with Internet connectivity to make a request to the cloud's gateway controller.

$ curl -kvu 'rightscale:<your_password>' https://wstunnel10-1.rightscale.com/_token/<your_token>/gw/v1/acct/<your_tenant_name>/instance-types

If everything is working as expected you should get a JSON response with the defined instance types for the vSphere cloud. However, if you don't get a response or receive a connection error, it's probably a problem with wstunsrv (RightScale's tunnel server). Please inform RightScale immediately.

If there is an issue with wstuncli (tunnel client on the appliance), you will likely see the following response (see below) and need to restart wstuncli.

* Adding handle: conn: 0x7f83c3003a00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f83c3003a00) send_pipe: 1, recv_pipe: 0
* About to connect() to wstunnel.test.rightscale.com port 443 (#0)
*   Trying 50.112.73.62...
* Connected to wstunnel.test.rightscale.com (50.112.73.62) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
* Server certificate: *.test.rightscale.com
* Server certificate: Go Daddy Secure Certification Authority
* Server certificate: Go Daddy Class 2 Certification Authority
* Server certificate: http://www.valicert.com/
* Server auth using Basic with user 'rightscale'
> GET /_token/your_token/gw/v1/acct/ut-tenant1/instance-types HTTP/1.1
> Authorization: Basic cmlnaHRzY2FsZTp2c2NhbGUyMDEzQEA=
> User-Agent: curl/7.30.0
> Host: wstunnel.test.rightscale.com
> Accept: */*

There are two different ways to restart wstuncli.

1. (Recommended) Log in to the RCA-V console. Go to Connectivity > RightScale Platform > RightScale Connection (wstunnel). Click the Restart button.
2. SSH in to the RCA-V instance (as 'rightscale') and switch to the 'root' user (# sudo su) and run the following UNIX command: # service wstuncli restart

If it looks like you are still not seeing your images and/or instances in the RightScale Cloud Management Dashboard even after you've manually re-queried the cloud controller (see previous row) and you have not received an error message, you may have connectivity problems (between your vSphere cloud environment and RightScale) or an incorrect cloud token.

Check Connectivity

1. Log in to the RCA-V console. Go to Connectivity > RightScale Platform > Overview. 
2. Make sure all statuses are green.

Check Cloud Redez-vous Token

1. Use a command prompt from a computer with Internet connectivity and run the following command:

$ curl -kvu 'rightscale:<your_password>' https://wstunnel10-1.rightscale.com/_token/<your_token>/gw/v1/acct/<your_tenant_name>/instance-types

• If you get a successful response, it means the cloud's gateway is functioning as expected but RightScale doesn't have the correct information to connect to it. In such cases, you must make sure the cloud's gateway endpoint URL (that your RightScale account uses to connect to the vSphere cloud) is correct. For example, maybe the cloud's Rendez-vous token was changed in the RCA-V console after the cloud was registered with RightScale. In such cases, you must update the cloud's Endpoint URL in your RightScale account to match what is used in the vSphere's RCA-V. Log in to the RightScale Dashboard and go to Settings > Administered Clouds > Your_Cloud > Edit tab. Change the Endpoint URL accordingly. You may need to contact the person who manages the RCA-V in order to retrieve the correct cloud token.
• If the problem still persists, you must remove the cloud from your RightScale account and add it again. Note: Any running instances will not be terminated when you remove (disassociate) the cloud from your RightScale account. (See Delete a Private Cloud Account from RightScale.) Once the cloud has been successfully re-added to the account with the correct Gateway URL, you will see the running instances once again in the RightScale Dashboard.

1. Log in to the RCA-V console and go to Cloud Appliance Mgmt > System. 
2. In the Logs card you can see all of the RCA-V's log files. The key log files are:

vscale.log

vscale-admin.log

wstuncli.log

If you happen to login to the appliance directly (using SSH), you will find the log files in /var/log directory

There are several reasons that might cause a VM (instance) to become stranded in a 'booting' state.

• RightLink is not properly installed on the VM that was launched. Make sure you launch VMs with images that install RightLink (e.g. RightImage).
• The RightLink agent fails to connect to the RightScale platform to "phone home" after the instance is running.​ Perhaps the VM does not have an outbound route to the Internet so that it can connect to RightScale. Maybe DNS is not set up properly or the public DNS server (such as 8.8.8.8) is not reachable.
• A script in the ServerTemplate's boot phase did not get executed or handled correctly.

Why is the Cloud Controller status red (and not green)?

When you check the status of the VMware vSphere cloud (Settings > Account Settings > Clouds) and the status of the Cloud Controller is red (as shown below), it means that the RightScale platform cannot properly connect to the RCA-V and/or related vSphere cloud. 

There could be several issues that could cause this type of problem.

• RCA-V is not running
• RCA-V is not properly installed on the server. You may need to launch a new RCA-V server.
• The tenant (that the RightScale account was assigned) was deleted.
• vSphere environment no longer exists
• The RCA-V appliance cannot connect to the vSphere cloud's API endpoint.
  • Perhaps the network firewall permissions were changed
  • Maybe the 'RightScale' user was accidentally deleted inside the vSphere Client application or the permissions associated with the 'RightScale' role were changed and no longer satisfy the minimum set of requirements.

  • Perhaps the VMware vSphere cloud was deregistered and is no longer accessible via the RightScale platform.

 See the RightScale Cloud Appliance for vSphere (RCA-V): Installation Guide.

When you check the status of the VMware vSphere cloud (Settings > Account Settings > Clouds) and the status of the Cloud Account is red (as shown below), it means that there is a problem with your cloud's subscription.

• Invalid Cloud Subscription/Credentials. Please log a RightScale Support ticket or email support@rightscale.com.