HQueue, or Houdini Queue, is a distributed job scheduling system.
HQueue, or Houdini Queue, is a distributed job scheduling system. It consists of an HQueue server and a farm of processing nodes, or clients. The server accepts job submissions from users and assigns them to available clients.
The HQueue Simulation output driver submits dynamic simulation jobs to HQueue for distribution on the farm. The simulation is performed by the node specified in the Output Driver parameter. This is typically a Geometry ROP.
Parameters
| Render | Submits a simulation job to the HQueue system. If the submission is successful, then a dialog pops-up with the job’s id. The id can be used to track the simulation progress on HQueue’s web-based user interface. |
General
| Job Name | The name of the job. Set this to <default> or set blank if you want the default job name to be used. | ||||||||||
| Output Driver | The render node used to generate the output files. For most cases, this would be a path to a Geometry ROP. | ||||||||||
| HQueue Server | The machine name and listen port of the HQueue server. | ||||||||||
| Target HFS | The path to the Houdini distribution that will execute the simulation on the HQueue farm. The distribution must be installed in a location that is accessible by every machine on the farm. You can use the $HQROOT and $HQCLIENTARCH variables to help specify the folder path (seeHQueue Variables). | ||||||||||
| Job Priority | The job’s priority. Jobs with higher priorities are scheduled and processed before jobs with lower priorities. 0 is the lowest priority. | ||||||||||
| |||||||||||
| Target HIP | When Simulate Target HIP File is chosen, set this parameter’s value to the location of the .hip file to be loaded and simulated on the farm. The .hip file must be saved somewhere on the shared drive, or any other location that is accessible by every machine on the farm. You can use the $HQROOT variable to help specify the .hip file path (see HQueue Variables). | ||||||||||
| Project Path | When Copy Project Files to Shared Folder and Simulate is chosen, set this parameter’s value to the location on the shared folder where you want to copy your .hip file and its dependencies to. The location must be accessible by all machines on the farm. You can use the$HQROOT variable to help specify the .hip file path (see HQueue Variables). | ||||||||||
| Automatically Save HIP File | When checked, Houdini automatically saves the current .hip file before submitting a job to HQueue. This guarantees that the farm simulates with the latest changes made to your scene. This parameter does not apply whenSimulate Target HIP File is chosen. | ||||||||||
| Partitioning Type | Choose how the simulation is to be partitioned and distributed on the farm.
| ||||||||||
| Slice Divisions | Describe how the simulation should be divided in the x, y and z-directions. Each slice created by the divisions requires a dedicated machine on the HQueue farm. For example, if the divisions are set to 2×2×2, then there would be 8 slices and the simulation would require a total of 8 machines. Having more divisions allows for more work to be parallelized and generally improves simulation times. However, too many divisions can potentially slow down the simulation since more data must be passed between the slices. Note that the number of slices dictates the number of generated geometry files. For instance, a simulation with 2 slices will create 2 sets of geometry files. So if 10 frames are to be simulated, then a total of 20 geometry files will be created, each file named with the slice number that it belongs to. This parameter only applies if thePartitioning Type is set to Volume Slices. | ||||||||||
| Number of Slices | The total number of slices in the particle fluid simulation. HQueue will assign a client machine for each slice. This parameter only applies if thePartitioning Type is set to Particle Slices. | ||||||||||
| Controls Node | The DOPs control node that assists HQueue in distributing the simulation. The node’s Tracker Address, Tracker Port and Slice Divisions parameters are automatically set by HQueue when the scene is processed on the farm. This parameter does not apply if thePartitioning Type is set to None orClusters. | ||||||||||
| Cluster Node | The Cluster node that specifies that clustering to be used to determine the distributed jobs. This parameter only applies if thePartitioning Type is set to Clusters. |
Email Options
| Send Status Emails | Enable this parameter if you want status emails for the specified events to be sent out. |
Note
The following parameters only apply when the Send Status Emails parameter is turned on.
| Email Address | The list of emails that will be sent status updates. Use commas to separate the entries. |
| On Start | Enable this parameter to have emails sent when the job starts being processed by a client. |
| On Success | Enable this parameter to have emails sent when the job is successfully completed. |
| On Failure | Enable this parameter to have emails sent when the job has failed, been canceled, or abandoned. |
| On Pause | Enable this parameter to have emails sent when the job is paused. |
| On Resume | Enable this parameter to have emails sent when the job is resumed. |
| On Reschedule | Enable this parameter to have emails sent when a job is rescheduled. |
| On Priority Change | Enable this parameter to have emails sent when a job’s priority is changed. |
Advanced
| Assign To | Set this parameter if you want HQueue to assign the simulation job to a specific list of clients or client groups.
| ||||||
| Clients | The list of client machines that HQueue can assign the simulation jobs to. Use commas to separate the machine names. This parameter only applies if Listed Clients is chosen in the Assign Toparameter. | ||||||
| Select Clients | Click to pop-up a dialog with a list of client machines registered on the HQueue farm. Accepting the selection in the dialog automatically populates the Clientsparameter with the selected clients. | ||||||
| Client Groups | The list of client groups that HQueue can assign the simulation jobs to. Use commas to separate the group names. Specifically, HQueue assigns simulation jobs to only the machines that are members of at least one of the groups in the list. This parameter only applies if Clients from Listed Groups is chosen in theAssign To parameter. | ||||||
| Select Client Groups | Click to pop-up a dialog with a list of client groups registered on the HQueue farm. Accepting the selection in the dialog automatically populates the Client Groups parameter with the selected groups. | ||||||
| Use Dedicated Machine for Tracker | The tracker process coordinates the communication between the machines simulating the slices. It does very little work aside from receiving and sending messages to the other machines. By default, HQueue allows other simulation or render jobs to be assigned to the machine running the tracker. Check this parameter to tell HQueue not to assign other jobs to the tracker machine. You may want to do this if you have several slices and you are worried that the communication from the client machines to the tracker will slow the tracker machine down. This parameter does not apply when thePartitioning Type parameter is set toNone or Clusters. | ||||||
| Enable Performance Monitor Logging | When checked, HQueue turns on performance monitor logging using the hscript perfmon command for the jobs doing the simulations. The log is merged with the job’s command output log and can be downloaded from the job view page on the HQueue web interface. | ||||||
| Set Number of CPUs per Job | Check this parameter if you want to set the maximum number of CPUs used for each simulation job. | ||||||
| CPUs per Job | The maximum number of CPUs that will be used by a simulation job. If the number exceeds a client machine’s number of CPUs, then the client machine will not be assigned to any simulation jobs. | ||||||
| Environment | A list of variable names and values that should be defined on the client machine when running the job. The variables appear as shell environment variables. | ||||||
| Create Directories | A list of output directories that should be created on the farm before any files are rendered out. For example, if your simulation generates a geometry files in$HIP/geo, then you will want to add this to the list. Note This parameter does nothing if the output directories already exist. |
HQueue Variables
HQueue defines a set of variables which appear in the environment of a running job. You can use these variables to refer to common file paths, machine settings and server configuration values.
| HQROOT | Evaluates to the root directory of the HQueue shared file system. The default HQueue installation sets this variable to /mnt/hq on Linux systems, H:/ on Windows systems, and/Volumes/HQShared on Mac OSX systems. Note that the shared file system can be configured to reside in a different location so check with your HQueue administrator for details. |
| HQCLIENTARCH | Evaluates to the architecture and operating system of the client machine assigned to execute the simulation job. For example, if the HQueue server assigned a job to a Linux, 64-bit machine on the farm, then $HQCLIENTARCHis set to ‘linux-x86_64’ in the environment. This variable is useful for when you want to refer to paths that depend on the client machine’s platform. For example,/mnt/hq/houdini_distros/hfs.$HQCLIENTARCHcould refer to the Houdini distribution to use for a simulation job. Specifically, it tells HQueue to use/mnt/hq/houdini_distros/hfs.linux-i686when running the job on a 32-bit Linux machine, and to use/mnt/hq/houdini_distros/hfs.linux-x86_64when running the job on a 64-bit Linux machine. |
