Package Ganga :: Package Lib :: Package LCG :: Module GridSandboxCache :: Class GridSandboxCache
[hide private]
[frames] | no frames]

Class GridSandboxCache

source code

                 object --+        
                          |        
   GPIDev.Base.Objects.Node --+    
                              |    
GPIDev.Base.Objects.GangaObject --+
                                  |
                                 GridSandboxCache

Helper class for upladong/downloading/deleting sandbox files on a grid cache.


Author: Hurng-Chun Lee

Contact: hurngchunlee@gmail.com

Nested Classes [hide private]

Inherited from GPIDev.Base.Objects.GangaObject: __metaclass__

Instance Methods [hide private]
 
__init__(self)
x.__init__(...) initializes x; see help(type(x)) for signature
source code
 
upload(self, files=[], opts='')
Uploads multiple files to a remote grid storage.
source code
 
download(self, files=[], dest_dir=None, opts='')
Downloads multiple files from remote grid storages to a local directory.
source code
 
delete(self, files=[], opts='')
Deletes multiple files from remote grid storages.
source code
 
cleanup(self, opts='')
Cleans up the uploaded files.
source code
 
get_cached_files(self, opts='')
Gets the indexes of the uploaded files on the grid.
source code
 
list_cached_files(self, loop=True, opts='')
Lists the uploaded files.
source code
 
impl_upload(self, files=[], opts='')
Uploads multiple files to a remote grid storage.
source code
 
impl_download(self, files=[], dest_dir=None, opts='')
Downloads multiple files from remote grid storages to a local directory.
source code
 
impl_delete(self, files=[], opts='')
Deletes multiple files from remote grid storages.
source code
 
impl_bookkeepUploadedFiles(self, files=[], append=True, opts='')
basic implementation for bookkeeping the uploaded files.
source code
 
impl_getUploadedFiles(self, opts='')
basic implementation for getting the previously uploaded files from the job repository.
source code
 
__get_file_index_objects__(self, files=[])
Gets file index object according to the given file list
source code
 
__get_unique_fname__(self)
gets an unique filename
source code
 
__cmd_retry_loop__(self, shell, cmd, maxRetry=3)
Executing system command with retry feature
source code

Inherited from GPIDev.Base.Objects.GangaObject: __construct__, __deepcopy__, __getstate__, __setstate__, _attribute_filter__set__, _auto__init__, accept, getJobObject

Inherited from GPIDev.Base.Objects.Node: __copy__, __eq__, __ne__, clone, copyFrom, printSummaryTree, printTree

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Class Methods [hide private]
Class Variables [hide private]
  _schema = Schema(Version(1, 1), {'vo': SimpleItem(defvalue= 'd...
  _category = 'GridSandboxCache'
  _name = 'GridSandboxCache'
  _exportmethods = ['upload', 'download', 'delete', 'get_cached_...
  logger = getLogger()

Inherited from GPIDev.Base.Objects.Node (private): _index_cache, _parent

Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self)
(Constructor)

source code 

x.__init__(...) initializes x; see help(type(x)) for signature

Overrides: object.__init__
(inherited documentation)

upload(self, files=[], opts='')

source code 

Uploads multiple files to a remote grid storage.

@param files is a list of local files to be uploaded to the grid.
       The elemement can be a file path or a File object.

@return True if files are successfully uploaded; otherwise it returns False

download(self, files=[], dest_dir=None, opts='')

source code 

Downloads multiple files from remote grid storages to 
a local directory.

If the file is successfully downloaded, the local file path would be:

    - os.path.join(dest_dir, os.path.basename(local_fname_n)

@param files is a list of files to be downloaded from the grid.
       The data format of it should be:
       - [index_grid_file_1, index_grid_file_2, ...]

@param dest_dir is a local destination directory to store the downloaded files.

@return True if files are successfully downloaded; otherwise it returns False

delete(self, files=[], opts='')

source code 

Deletes multiple files from remote grid storages.

@param files is a list of files to be deleted from the grid.
       The data format of it should be:
       - [index_grid_file_1, index_grid_file_2, ...]

@return True if files are successfully deleted; otherwise it returns False

cleanup(self, opts='')

source code 

Cleans up the uploaded files.

@return True if all grid files listed in the index file are successfully deleted.

get_cached_files(self, opts='')

source code 

Gets the indexes of the uploaded files on the grid. 

@return the dictionary indexing the uploaded files on the grid.
        The key of the dictionary should be the main index (e.g. GUID) of the grid files.

list_cached_files(self, loop=True, opts='')

source code 

Lists the uploaded files.

if loop = True, it prints also the uploaded files associated with subjobs.

impl_upload(self, files=[], opts='')

source code 

Uploads multiple files to a remote grid storage.

Parameters:
  • files, is, a, list, of, files, in, URL, format, (i.e., file - //...)

    @return a list of successfully uploaded files represented by GridFileIndex objects

impl_download(self, files=[], dest_dir=None, opts='')

source code 

Downloads multiple files from remote grid storages to a local directory.

@param files is a list of files represented by GridFileIndex objects @param dest_dir is a local destination directory to store the downloaded files.

@return a list of successfully downloaded files represented by GridFileIndex objects

impl_delete(self, files=[], opts='')

source code 

Deletes multiple files from remote grid storages.

@param files is a list of files represented by GridFileIndex objects @return a list of successfully deleted files represented by GridFileIndex objects

impl_bookkeepUploadedFiles(self, files=[], append=True, opts='')

source code 

basic implementation for bookkeeping the uploaded files. It simply keeps the GridFileIndex objects in the job repository.

@param files is a list of files represented by GridFileIndex objects @return True if files are successfully logged in the local index file

impl_getUploadedFiles(self, opts='')

source code 

basic implementation for getting the previously uploaded files from the job repository.

@return a list of files represented by GridFileIndex objects

__get_file_index_objects__(self, files=[])

source code 

Gets file index object according to the given file list

  • try to get the GridFileIndex object from the local index file.

@param files is a list of file indexes @return a list of files represented by GridFileIndex objects


Class Variable Details [hide private]

_schema

Value:
Schema(Version(1, 1), {'vo': SimpleItem(defvalue= 'dteam', hidden= 1, \
copyable= 0, doc= 'the Grid virtual organization'), 'middleware': Simp\
leItem(defvalue= 'EDG', hidden= 1, copyable= 1, doc= 'the LCG middlewa\
re type'), 'protocol': SimpleItem(defvalue= '', copyable= 1, doc= 'fil\
e transfer protocol'), 'max_try': SimpleItem(defvalue= 1, doc= 'max. n\
umber of tries in case of failures'), 'timeout': SimpleItem(defvalue= \
180, copyable= 0, hidden= 1, doc= 'transfer timeout in seconds'), 'upl\
oaded_files': ComponentItem('GridFileIndex', defvalue= [], sequence= 1\
...

_exportmethods

Value:
['upload', 'download', 'delete', 'get_cached_files', 'list_cached_file\
s', 'cleanup']