Processing Flight in a Workflow

After a new flight is created, there are seven steps to process a flight from raw images to phenotypic values. Some steps require user interaction through web interface, and other can be automatically processed in the server side.

Workflow overview

1. Upload images: Upload all images in a flight through web or transfer to storage.
2. Extract image information: The image information is extracted by cluster and visualized to show flight path and raw images.
3. Add ground control point: Ground control points are manually added into raw images.
4. Stitch ortho-mosaic: All raw images are stitched together using all information above and specified parameters.
5. Generate pyramid retile: All ortho-mosaic and vegetative index are retiled for online visualization.
6. Segment plots: Plots are created using drawing tools and copied from flights in the same field. The reverse calculation is used to segment undistorted images.
7. Extract phenotypic values: Phenotypic values are extracted using pre-defined functions and applied for each vegetative index.
8. Archive: All flight data in the hard drive is archived into a tar file, transfered into long-term storage (e.g. tape) and deleted from the active storage to save space.

Workflow status

Five statuses are used to represent a workflow from wait to finish.

1. Wait for previous workflows to finish.
2. Ready to start to this workflow.
3. Schedule to run on the cluster.
4. Processing on the cluster.
5. Finish to process all required tasks.

Workflow notification and logs

Email will send to all users with write permission on the flight when a workflow is finished and failed. A single email will send if there are multiple sub-tasks in each workflow (e.g. pyramid retile is running for each layer as a sub-task). User can disable email notification at the user setting.