SHARE requirements

This chapter gives an overview of the requirements you need to take into account for a good performance of CLOUDFLOW SHARE.

CLOUDFLOW SHARE is a technology to share and synchronize information between separate and independent CLOUDFLOW servers across the globe. See CLOUDFLOW SHARE for more information.

If your system does not meet these requirements, a good performance of CLOUDFLOW SHARE is not guaranteed and possible issues and problems will arise.

Network

These are the network requirements.
  • The bidirectional interconnection speed between the different sites needs to be at least 10 Mbit/s (megabit per second).
  • The network latency needs to be lower than 1 second. You can test this by pinging.
  • The bandwidth depends on the amount of data you want to transmit. This can be calculated as by estimating the amount of data that will be interchanged between the sites during an acceptable time interval that is needed for the transmission of the data from one site to another (for example 1 hour).

    Example

    In 1 hour, 20 job folders are exchanged. The folders contain of 200 MB of data, which makes a total of 4 GB. this needs 10 Mbit/s, dedicated for CLOUDFLOW SHARE.
  • A direct and reliable connection between the CF servers needs to exist.

File store

  • The average consistent indexing speed of the complete File store you want to use on CLOUDFLOW SHARE needs to be lower than 5 minutes. If it consistently takes longer than 5 minutes to index the File store, all CLOUDFLOW SHARE transactions will be slowed down.
    What can you do to improve the indexing speed?
    • Split up the File store to sub-directories.
    • Decrease the volume of files that are on the File store.
    • Speed up the storage (for example move the storage to CLOUDFLOW or vice versa).
    Or you can try a combination of the above measures.
  • The file system needs to read and manage the metadata (such as modification date) correctly.
  • The resource forks and alternate data streams need to be manages identically on both sites.
  • The operating system and storage system need to be identical on both sites.

CLOUDFLOW SHARE Setup

  • Make sure to work transactional and to restrict the activity on both sites on the same time of the synchronized CLOUDFLOW SHARE folders. Limit the concurrent file transactions on these folders. A high concurrent folder activity and too many file transactions imply many decisions that CLOUDFLOW SHARE needs to take, which may cause conflicts and unexpected results.

    You can use CLOUDFLOW nodes to set up and manage a transactional workflow.

    Example of a transactional workflow

    This is an example of a transactional workflow with check-out and check-in.

    Setup
    • A central CLOUDFLOW system where all production files are stored.
    • Two production sites.
    • Two File stores on each site:
      • RO (Read Only) with read permissions only and where the operators save the files when they are finished. This File store is synchronized with the central CLOUDFLOW site through a synchronization mapping.
      • RW (Read and Write) with read and write permissions and where the operators work on the files.

    Workflow

    When an operator wants to work on a file, the operator checks out the file to the RW File store in the production site. When ready, the operator copies the file to the RO File store in the same production site. This File store is synchronized with the central CLOUDFLOW site through a synchronization mapping.

    Workflow nodes in CLOUDFLOW
  • Check out
    • Check Out: starts the check-out workflow.
    • Add Sync Spec Mapping: makes sure the RO File store is synchronized with the central COUDFLOW.
    • Wait For sync: makes sure the workflows waits until the files are synchronized.
    • Copy Folder: copies the files from the RO File store to the RW File store so the operators have access to the files and can work on them.
    • Notify: script that sends out a notification of the action.
    Check in
    • Check In: starts the check-in workflow.
    • Copy Folder: copies the files from the RW File store to the RO File store.
    • Wait For sync: makes sure the workflows waits until the files are synchronized.
    • Notify: script that sends out a notification of the action.
  • The maximum amount of files in a mapping should be 500. The maximum amount of mappings in a sync spec should be 25. This makes a maximum total of 12.500 files in a syncspec. It is also important to keep the amount of mappings and the amount of syncspecs balanced. Treat a sync spec as a job folder, with subfolders (TBA).
  • Disable obsolete sync specs.

Things not to do

Avoid the following:
  • Full File server syncing between sites.
  • Full active WIP (Work in Progress) folder syncing.
  • Overlap of mappings.