JobFolder Module

Classes and functions pertaining to job-management.

The pylada.jobfolder provides tools for high-throughput calculations. It is centered around an object - the job-folder - which organizes calculations within a tree of folders, much as one would manually organize calculations within a tree of directories. Each folder can be executable, e.g. there is something to compute there, or non-executable. And each folder can further hold any number of sub-folders. Furthermore, classes are provided which make it easy to manipulate the parameters for the calculations in an executable folder, as well as within all subfolders. Finally, a similar infrastructure is provided to collect the computational results across all executable sub-folders.

See also

Organized high-throughput calculations: job-folders

• JobFolder class
• Parameter mass manipulation
• Result mass extraction
• MassExtract base classes
• Forwading dictionary

pylada.jobfolder.load(path='jobfolder.dict', timeout=None)

Unpickles a job-folder from file.

Parameters:

• path (str) – Filename of a pickled job-folder.
• timeout (int) – How long to wait when trying to acquire lock on file. Defaults to forever.

Returns:

Returns a JobFolder object.

This method first acquire an exclusive lock on the file before reading. This way not two processes can read/write to this file while using this function.

pylada.jobfolder.save(jobfolder, path='jobfolder.dict', overwrite=False, timeout=None)

Pickles a job-folder to file.

Parameters:

• jobfolder (JobFolder) – A job-dictionary to pickle.
• path (str) – filename of file to which to save pickle. overwritten. If None then saves to “pickled_jobfolder”
• timeout (int) – How long to wait when trying to acquire lock on file. Defaults to forever.
• overwrite (bool) – if True, then overwrites file.

This method first acquire an exclusive lock on the file before writing (see pylada.misc.open_exclusive()). This way not two processes can read/write to this file while using this function.