Jobdict class

Submodule declaring the folder folders class.

class pylada.jobfolder.jobfolder.JobFolder

Bases: object

High-throughput folder class.

Means to organize any calculations in folders and subfolders. A folder is executable is the functional attribute is not None. The attribute should be set to a pickleable calleable. The parameters for the calls should be inserted in the params attribute. Sub-folders can be added using the __div__() and __setitem__() methods. The latter offers the ability to access and set subfolders at any point within the tree of folders from any subfolder. The executable subfolders can also be iterated in a manner similar to a job-dictionary. Finally, a folder can be executed via the compute() method.

__init__()

children

A dictionary holding instances of subfolders further down the tree.

params

A dictionary holding the parameters for the calculation in this particular folder, if any. The parameters will be passed on to the functional as **params.

parent

Reference to the JobFolder instance which holds this one, i.e. the parent directory. If None, then this folder-dictionary is the root.

Note

This back-reference creates reference cycles. However, we cannot use a weak-reference here, unless we are OK with risking the loss of a parent folder when only a reference to the subfolder is held by the user.

functional

Returns current functional.

The functional is implemented as a property to make sure that it is either None or a pickleable callable. The functional is deepcopied from the input. In other words, this functional stored in the folderdictionary is no longuer the one given on input – it is not a reference to the input. This parameter can never be truly deleted.

>>> del folder.functional 

is equivalent to:

>>> folder.functional = None

Note

To store a reference to a global functional, one could do folder._functional = functional instead. However, modifying the input functional will affect the stored functional and vice-versa.

name

Returns the name of this dictionary as an absolute path.

is_executable

True if functional is not None.

untagged_folders

Returns a string with only untagged folders.

is_tagged

True if current folder is tagged.

In practice, this is used to turn a folder on (untagged) or off (tagged). The meaning of tagged is not enforced, so it could be used for other purposes.

nbfolders

Returns the number of folders in sub-tree.

subfolders = <unbound method JobFolder.subfolders>

root

Returns root dictionary.

__iter__()

Iterator over keys.

iteritems(prefix='')

Iterates over executable sub-folders.

Iterates over all executable subfolders. A subfolder is executable if it holds a functional to execute.

Parameters:prefix (str) – Prefix to add to the name of this folder. Convenient when iterating over a folder folders with the intention of executing the folders it contains. Returns:yields (directory, folder):

• name of this folder, prefixed with prefix.
• folder is an executable Folderdict.

itervalues()

Iterates over all executable sub-folders.

iterkeys()

Iterates over names of all executable subfolders.

items()

List of all folders.

values()

List of all executable sub-folders.

keys()

List of names of all executable sub-folders.

subfolders() → [str, str,...]

Sorted keys of the folders directly under this one.

update(JobFolder) → None

Updates folder and tree with other.

Parameters:

• other – JobFolder dictionary from which to update.
• merge (bool) – If false (default), then actual folders in other completely overwrite actual folders in self. If False, then params in self is updated with params in other if either one is an executable folder. If other is an executable folder, then functional in self is overwritten. If other is not an executable folder, then functional in self is not replaced.

Updates the dictionaries of parameters and sub-folders. Actual folders in other (eg with self.is_executable==True) will completely overwrite those in self. if items in other are found in self, unless merge is set to true. This function is recurrent: subfolders are also updated.

compute(**kwargs)

Executes the functional in this particular folder.

If this particular folder of the folder folders is not executable (e.g. self.functional is None), then None is returned.

If, on the other hand, this folder contains a real functional, then the latter is called taking the parameters stored in the folder as keyword arguments. Futhermore, additional keyword arguments passed to this method are passed on the functional, possibly overriding those stored in the folder. The return from the functional is returned by this method: In practice the call is as follows:

>>> return self.functional(**self.params.copy().update(kwargs))

tag() → None

Tags this folder.

untag() → None

Untags this folder.

__getattr__(name)

Returns folder parameter.

Folder parameters stored in Jobdict.params can also be accessed via the . operator.

__setattr__(name, value)

Sets folder parameter.

Folder parameters stored in Jobdict.params can also be accessed via the . operator.

__div__(name)

Adds a folderdictionary to the tree.

Any path can be given as input. This is akin to doing mkdir -p. The newly created folder folders is returned.

__getitem__(index)

Returns folder description from the dictionary.

If the folder does not exist, will create it.

__setitem__(name, value)

Sets folder/subfolder description in the dictionary.

If the folder does not exist, will create it. A deepcopy of value is inserted, rather than a simple shallow ref.

__contains__(index)

Returns true if index a branch in the folder folders.