StorageService (Coming soon)

A StorageService connection allows a COOL standalone server/workers to store and retrieve data on an external data storage, for example, an NFS, an HDFS cluster, or AWS S3 service. A worker will attempt to pull the data from the storage service if it is not found locally. Data generated by COOL can be persisted to the storage service to increase fault tolerance and provide access to other workers or other systems. The StorageService works at the granularity of files locally.

A sample implementation for the HDFS connection can be found under the hdfs-extensions.

Extension

Developers are welcome to provide connections to other types of storage services by implementing the StorageService interface.