Troubleshooting
Troubleshooting
This troubleshooting page helps you fix issues when running ECiDA.
Unable to view logs, links or metrics
It may take a while before modules are fully loaded, if it takes longer than 5 minutes until modules logs are displayed, something may be wrong with your module.
Possible causes
1. Invalid module
Make sure that the module name, version, endpoints, and alias inside the pipeline respects the current specified naming restrictions.
2. Module has not finished building yet
Make sure the module is built successfully. Inside your module sources repository, check the status of the latest commit:
The icon next to the commit name shows the current status:
- Yellow: process is pending
- Blue: module is currently building
- Red: module failed building
- Green: building completed
If the status is Red or Green, please check the build logs by doing the following:
- Click the aforementioned icon
- Click
build_push
inside ‘containers’. This shows the logs of the module building stage.
Module building stuck at “pending”
The CICD process for building modules is run via GitLab runners. At rare points, there may not be enough runners available to run the process immediately.
If the building process does not transition from pending, please restart the building process:
- Click the pending icon to open the CICD pipeline window
- For the specific job that is pending, click the restart button:
Module restarts when finished
As of now, the logic behind modules is designed for stream-processing, and thus are ‘meant’ to always be running. Therefore, when a modules stop, ECiDA restarts it.
We are working on a solution that allows modules to complete properly. Until then, add an empty while True
loop at the end of your module code:
Example:
import time
if __name__ == "__main__":
M = create_module()
M.initialize()
main(M)
while True:
time.sleep(3600)
A running pipeline does not use the updated module
If module logic is updated, the pipeline needs to be restarted. there are two variants to this issue.
1: Module logic was updated, but the version stays the same
Remove the pipeline from deployment, and redeploy it when it is removed.
If you changed the module definition, e.g. changed input, output or config names do the following:
- Remove the changed module from the pipeline
- Re-add module via the Library: it is listed separately
- Add the links to the new module exactly like the old module
- Save the pipeline and update the deployment
If this step is skipped, the old pipeline may still hold old names and is therefore malformed. This will cause it to get stuck while deploying.
2: Module logic was updated, and saved as a new version
Edit the pipeline such that the new module version is used:
- Add the new module via the Library: it is listed separately
- Add the links to the new module exactly like the old module
- Remove the old module from the pipeline
- Save the pipeline and update the deployment
Pipeline does not deploy after module has been changed
See A running pipeline does not use the updated module.
Module doesn’t start
Make sure that the module source file, apart from the create_module
and main
methods, also contains the if-clause
for running said methods.
Add the following snipped to the end of your source file:
if __name__ == "__main__":
M = create_module()
M.initialize()
main(M)
Module file was deleted, CICD pipeline finished, but module is still visible in the app.
Deleting a module file does not delete it from ECiDA yet for the source control management and not breaking the active pipelines that are using these modules. We are working on a solution for safe deletion.