ECIDA

The EcidaModule package is a Python module designed to facilitate deployment and communication between components in a distributed system. This package provides various functionalities including Kafka message handling, environment variable checking, and directory management.

How to Use

  1. Importing the package

    To use the EcidaModule in your Python program, first, you will need to import the EcidaModule class from the package:

      from Ecida import EcidaModule

  2. Create an ECiDA Module

    Define a module with this signature

      def create_module() -> EcidaModule:
    M = EcidaModule("MODULE_NAME", "MODULE_VERSION")
    # ADD MODULE INPUTS/OUTPUTS HERE
    ...
return M

  3. Add inputs and outputs

    Use the following syntax for adding input and output to module. You can add as many I/O as required.

      # ADD STREAMING INPUT WITH 
M.add_input("INPUT_NAME", "string")
# ADD STREAMING OUTPUT 
M.add_output("OUTPUT_NAME", "string")
# ADD INPUT DIRECTORY FOR FILE USE 
M.add_input_directory("INPUT_DIRECTORY_NAME")
# ADD OUTPUT DIRECTORY FOR FILE USE 
M.add_output_directory("OUTPUT_DIRECTORY_NAME")
# ADD INPUT DIRECTORY PRELOADING FROM GIT FOR FILE USE
M.add_input_from_git("INPUT_DIRECTORY_NAME", "GIT_REPOSITORY", "DIRECTORY/PATH/IN/GIT")

  4. Entry Point

    Use the following boiler-plate as the entry-point to your code.

      if __name__ == "__main__":
    M = create_module()
    M.initialize()
    main(M)

    and for the main function use the following signature:

      def main(M: EcidaModule):    
    #YOUR LOGIC COMES HERE

Example python file

An example template for the modules.

  from Ecida import EcidaModule

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


def create_module() -> EcidaModule:
    M = EcidaModule("MODULE_NAME", "MODULE_VERSION")
    # ADD MODULE INPUTS/OUTPUTS HERE

    return M

def main(M: EcidaModule):    
    print(f"START MODULE {M.name}:{M.version}")
    # LOGIC COMES HERE
      


if __name__ == "__main__":
    M = create_module()
    M.initialize()
    main(M)

NOTES

A python file can contain no more than one module, since the module creation happens through using create_module() function with exact same signature.

Naming restrictions

Please take account of the following restrictions when it comes to defining modules. These restrictions hold all variables inside the create_module, such as input names, output names, type hints and module names.

Both input names and output names should be unique.

Wrong:

  
M.add_input("data-in", "string")
M.add_input("data-in", "string")

Correct:

  
M.add_input("data-in", "string")
M.add_input("data-in2", "string")

Strings used inside the module definition, such as module name, IO names and types should be in “pure string format”.

That is, strings used in the module definition cannot use concatenation or formatted “f-strings”:

Wrong:

  
M.add_input(f"data-in-{id}", "string")

Correct:

  
M.add_input("data-in", "string")

Variable strings cannot be imported from other files. Constants in the same module file can be used.

Wrong:

  
from constants import INPUT_NAME 
M.add_input(INPUT_NAME, "string")

Correct:

  
INPUT_NAME = "in"

M.add_input(INPUT_NAME, "string")

Strings cannot contain capital letters or dots.

Wrong:

  
M.add_input("InputEndpoint.1", "string")

Correct:

  
M.add_input("input-endpoint-1", "string")
Additionally, input names, output names and type hints cannot contain underscores.

Wrong:

  
M.add_input("input-endpoint_1", "string")

Correct:

  
M.add_input("input-endpoint-1", "string")