State machine for single-node jobs
The AWS Step Functions workflow is defined as a json file in ~/cloud-imaging/step_functions/state_machine.json
. The file does not need to be modified. To create the workflow for the first time, run the following command from a terminal within the ~/cloud-imaging
directory. This step requires the ARN of the Step Functions service role. The ARN can be found in the AWS console -> Services
-> IAM
-> Roles
-> StepFunctionsLambdaRole
. Copy the ARN and use it in the following command:
aws stepfunctions create-state-machine --name LSRTM-SGD \
--role-arn step_functions_service_role_arn \
--definition file://step_functions/state_machine.json
To upload an updated version of the workflow, first find the ARN of the state machine that we created with the above command. Go to the AWS console -> Step Functions
-> LSRTM-SGD
and copy the ARN. Then run the following command using this ARN:
aws stepfunctions update-state-machine \
--state-machine-arn state_machine_arn \
--definition file://step_functions/state_machine.json
State machine for multi-node jobs
To set up the state-machine for the multi-node workflow, follow the same steps as above. The multi-node workflow is defined in ~/cloud-imaging/step_functions/state_machine_multinode.json
. Obtain the ARN of the StepFunctionsLambdaRole
(as described above) and upload the workflow as follows:
aws stepfunctions create-state-machine --name LSRTM-SGD-MultiNode \
--role-arn step_functions_service_role_arn \
--definition file://step_functions/state_machine_multinode.json