wrapReceive.py

The server runs a single Python program: wrapReceive.py. Every few seconds wrapReceive checks for files sent into the directory /data/incoming by the client process wrapUpload.

In particular, wrapReceive checks for files having the format wrapId.zzflag. See the SQL database overview for more information on the wrapId.

If wrapReceive finds a file there having the format wrapId.zzflag, ...

• It checks that the three files are present:
  • wrapId.json: General statistics and information
  • wrapId.tgz: Compendium of all files to be archived.
  • wrapId.zzflag: Zero-length flag.
• It creates a directory /data/arch/wrapId, and moves moves the three files from /data/incoming to /data/arch/wrapId.
• Within directory /data/arch/wrapId ...
• It untars wrapId.tgz to subdirectory vdir
• It calls fillDbVasp.py, passing the directory /data/arch/wrapId.

---

nrelmat.wrapReceive.main()

This is the receiver for data uploaded by wrapUpload.py.

It updates the model and contrib tables. The function is determined by the -func parameter; see below.

This function calls fillDbVasp.py and augmentDb.py.

Command line parameters:

Parameter	Type	Description
-bugLev	integer	Debug level. Normally 0.
-func	string	Function. See below.
-useCommit	boolean	false/true: do we commit changes to the DB.
-allowExc	boolean	false/true: continue after error.
-inDir	string	Input dir for uploaded files.
-archDir	string	Dir used for work and archiving.
-logFile	string	Log file name.
-inSpec	string	JSON file containing parameters. See below.
-pollSec	int	poll interval in seconds

Values for the -func Parameter:

readIncoming

Every few seconds list the files in inDir. For each file name matching wrapId.zzflag, call function gatherArchive() to process the three files: wrapId.json, wrapId.tgz, and wrapId.zzflag. Since program wrapUpload always writes the flag file last, the other two should already be present.

redoArch

Re-process all the subDirs found in archDir by calling function processTree() for each subDir. This is useful when someone changes the database tables, for example by adding an new column. Then one can use the following to recreate the tables with the new column.

fillDbVasp.py -func createTableModel -deleteTable true
fillDbVasp.py -func createTableContrib -deleteTable true
wrapReceive.py -func redoArch

inSpec File Parameters:

Parameter	Description
dbhost	Database hostname.
dbport	Database port number.
dbuser	Database user name.
dbpswd	Database password.
dbname	Database database name.
dbschema	Database schema name.
dbtablemodel	Database name of the “model” table.
dbtablecontrib	Database name of the “contrib” table.

inSpec file example::

{
  "dbhost"         : "scctest",
  "dbport"         : "6432",
  "dbuser"         : "x",
  "dbpswd"         : "x",
  "dbname"         : "cidlada",
  "dbschema"       : "satom",
  "dbtablemodel"   : "model",
  "dbtablecontrib" : "contrib"
}

nrelmat.wrapReceive.gatherArchive(bugLev, useCommit, allowExc, inDirPath, archDirPath, wrapId, inSpec)

Moves inDirPath/wrapId.* to archDir and adds the info to the database.

Moves inDirPath/wrapId{.json,.tgz,.zzflag} to the new dir archDir/wrapId. Untars the .tgz file. Then calls function processTree() to add the info to the database.

Parameters:

• bugLev (int): Debug level. Normally 0.

• useCommit (bool): do we commit changes to the DB.

• allowExc (bool): do we continue after error.

• inDirPath (str): Absolute path of the command line parm inDir.

• archDirPath (str): Absolute path of the command line parm archDir.

• wrapId (str): The wrapId extracted from the current filename.

• inSpec (str): Name of JSON file containing DB parameters.

  See description at main().

Returns

• None

nrelmat.wrapReceive.processTree(bugLev, useCommit, allowExc, subDir, wrapId, inSpec)

Calls fillDbVasp to add info to the database, and augmentDb to fill additional DB columns.

Parameters:

• bugLev (int): Debug level. Normally 0.

• useCommit (bool): do we commit changes to the DB.

• allowExc (bool): do we continue after error.

• wrapId (str): The wrapId extracted from the current filename.

• subDir (str): archDirPath/wrapId

• inSpec (str): Name of JSON file containing DB parameters.

  See description at main().

Returns

• None

nrelmat.wrapReceive.checkDupProcs()

Tests if a process with the same program name as ours is already running, and if so, quits.

Parameters:

• None

Returns

• None

Raises

• Exception (via throwerr) if another process has the same name.

nrelmat.wrapReceive.throwerr(msg)

Prints an error message and raises Exception.

Parameters:

• msg (str): Error message.

Returns

• (Never returns)

Raises

• Exception