We have had customers who did not want to use AWS S3 for passing files to the worker code. This can be easily accomplished by creating one (1) dummy input file, downloading it once to your worker code, and configuring the worker daemon to ignore re-downloading the same file. In your work unit, you would also include any data necessary for your worker to retrieve its data (e.g. ftp path, URL, database query, etc.). Then, inside your worker code, your code logic would retrieve the data that it needs by the method you have chosen.
Although you can write several classes on a Worker Server Array to handle different kinds of work and for a Worker Server Array to monitor more than one input queue, it is best practice to segment out disparate work logic to different servers, which allows you to achieve scaling based on the work performed. Otherwise, you may have difficulty identifying which class is taking up a lot of server resources.
The way RightScale Grid handles errors is configurable. You can configure the system to queue error messages in the same queue as the output messages. This may be useful if the same program handles both success and failure conditions.
The best practice for creating a Grid Computing System is to create JobConsumer programs that poll the output and error queues. Do not rely on the queue.size() function to determine the amount of work in the queues as SQS returns an approximate value for the # of messages in the queue that varies plus or minus around the true value.
When the sqs_item_age elasticity function is selected, your date strings in your programs that create work_units should use the format string "%Y-%M-%D %H:%m:%s %Z". We have noticed that some machines generate the string GMT while others generate UTC for a substitution for the %Z. Make sure that this generated suffix is the one used in the regex expression when defining the elasticity function.
Configuration Files and worker classes are loaded into memory. Changing their corresponding files directly on the target machine will not have the intended effects. Make changes to these items in the ServerTemplate/Scripts and then relaunch your servers.