A researcher runs the Python program wrapUpload.py to upload results to the server. The program takes as input either ...
- A directory tree, in which case command line parameters control which subdirectories are accepted for uploading.
- A list of specific directories to upload.
The wrapUpload program creates a unique identifier for this upload, called the wrapId. See the SQL database overview for more information on the wrapId.
Then wrapUpload makes a list of all files to be uploaded. These must be in directories that passed the filters above and is restricted to the files:
- metadata: user-specified metadata like name and publication DOI.
- INCAR: Main list of parameters.
- KPOINTS: Specify the k-point grid.
- POSCAR: Specify atom positions.
- POTCAR: Specify atomic pseudopotentials.
- OUTCAR: General log and results in human-readable format.
- vasprun.xml: All results in XML format.
Then wrapUpload makes a single JSON file, wrapId.json, containing metadata such as the list of directories. Then wrapUpload makes a single compressed tar file, wrapId.tgz, containing all the files. Finally wrapUpload uses scp to upload three files:
- wrapId.json: General statistics and information
- wrapId.tgz: Compendium of all files to be archived.
- wrapId.zzflag: Zero-length flag.
The wrapId.zzflag file gets uploaded last. The server process wrapReceive doesn’t start processing the files until receiving the flag file, thereby preventing the server from starting to process partly-received data.
Locates model runs, checks and extracts dir contents, and uses tar and scp to send the data to the server running wrapReceive.
Command line parameters:
| Parameter | Type | Description |
|---|---|---|
| -bugLev | integer | Debug level. Normally 0. |
| -readType | string | If ‘outcar’, read the OUTCAR files. Else if ‘xml’, read the vasprun.xml files. |
| -requireInput | boolean | no/yes: do we require that the files INCAR, KPOINTS, POSCAR exist. |
| -requireIcsd | boolean | no/yes: do we require that the file path names contain ICSD info. See notes below. |
| -keepList | string | File containing the absolute paths of the dirs to be uploaded. Still topDir must be specified, and all paths in keepList must start with the specified topDir. If keepList is specified, keepPatterns and omitPatterns must not be specified. If none of keepList, keepPatterns, or omitPattens are specified, all dirs below topDir containing a metadata file will be archived. |
| -keepPatterns | string | Comma separated list of regular expressions matching the relative paths of those directories to be kept. If specified, keepList must not be specified. |
| -omitPatterns | string | Comma separated list of regular expressions matching the relative paths of those directories to be omitted. If specified, keepList must not be specified. |
| -topDir | string | Top of dir tree to upload. |
| -workDir | string | Work dir |
If requireIcsd is true, getIcsdMap() must be able to extract ICSD info from the file names. File names must be like:
.../icsd_083665/icsd_083665.cif/ls-anti-ferro-7/relax_cellshape/1
^^^^^^ ^^^^^^^^ ^ ^^^^^^^^^^^^^^^ ^
icsdNum magType magNum relaxType relaxNum
Locates model runs, checks and extracts dir contents, and uses tar and scp to send the data to the server running wrapReceive.
If keepList is specified, creates keepAbsPaths = unique absolute paths and calls iterateDirs().
Otherwise calls searchDirs() to recursively search the directory tree starting at topDir.
Parameters:
Returns
Recursive: locates model runs, checks dir contents, and appends names to lists of dirs and files.
Parameters:
Returns
For each path in keepAbsPaths, checks dir contents, and appends names to lists of dirs and files.
Parameters:
Returns
Prepares to archive a single directory, and appends names to lists of dirs and files.
Parameters:
Returns
Creates a map of ICSD info based on the dir name.
Example parsing inDir:
.../icsd_083665/icsd_083665.cif/ls-anti-ferro-7/relax_cellshape/1
^^^^^^ ^^^^^^^^ ^ ^^^^^^^^^^^^^^^ ^
icsdNum magType magNum relaxType relaxNum
Resulting map:
icsdMap = {
'icsdNum' : 83665,
'magType' : 'lsaf',
'magNum' : 7,
'relaxType' : 'rc',
'relaxNum' : 1,
}
Parameters:
Returns
(No longer used): Reads and saves the header sections from a POTCAR file.
Saves every section starting with ‘PAW_PBE’ to the following line ‘Description’.
Parameters:
Returns
Parses a metadata file and returns a corresponding map.
Parameters:
Returns
Map of metadata values. The map structure is:
Key |
Value Type |
Source description |
|---|---|---|
firstName |
str |
Researcher first name |
lastName |
str |
Researcher last name |
spacegroup |
str |
international space group number |
parents |
str[] |
comma separated list of sha1sum(s) of the vasprun.xml files of previous runs, if any |
publications |
str[] |
comma separated list of DOIs without the leading http:// |
standards |
str[] |
comma separated list of standardized keywords |
keywords |
str[] |
comma separated list of any keywords |
notes |
str |
textual notes |
Insures that fname exists and has length > 0.
Parameters:
Returns
Raises:
Insures that fname exists. It may have length == 0.
Parameters:
Returns
Raises:
Calls the executable indicated by args and waits for completion.
Parameters:
Returns
Raises:
Finds the number of files having name == tag in the tree at dir. Yes, Python has os.walk, but this is better.
Parameters:
Returns
Formats a uui (wrapId).
A wrapId looks like:
@2013.08.13@12.58.22.735311@someUser@home.someUser.redmesa.old.td.testlada.2013.04.06.Fe.O@
Parameters:
Returns
Parses a uui (wrapId).
The input string may have subdirs info after the initial wrapId.
Parameters:
Returns
Prints a map.
Parameters:
Returns