Queued Work Stream
|
en construction |
under construction |
Matières
Work Streams
What is a work stream
A work stream is a series of "jobs" having a similar resource profile. In order not to overtax the system job scheduler with a myriad of relatively "small" work items, said items are inserted into "pseudo queues" and processed by a "master job".
A user's work stream(s) will be found in directory $HOME/.job_queues. This directory in turn contains subdirectories, one for each "pseudo queue".
More than one master job can go "fishing" into a "pseudo queue".
The main characteristics of a work stream are
- a name (arbitrary)
- a set of pseudo queues (may be used to implement some sort of priority scheme)
- a computing surface (number of nodes)
- a duration (number of hours, days, weeks...)
- a maximum idle time (if a stream is using a large number of nodes, its maximum idle time should be very short)
How do i insert work into a work queue
The ord_soumet utility is used to insert work into a "pseudo queue". The syntax is almost the same as for submitting a job to the system's batch scheduler. The "-q pseudo_queue_name@" parameter to ord_soumet is used to indicate that instead of being submitted directly, the piece of work (job) should rather be inserted into the "pseudo_queue_name" work queue.
How do i start a master job for a work stream
By submitting a master job with the (to come) u.run_work_stream command
