Data Conversion & Automation

these requests can be used to upload datasets, run conversions and download datasets prerequisites mapping commands must be pre configured in ryze using the dataset mapper plugin you must enable a data transformation add on for the standard/study enable the data transformation add on by doing a test conversion in your standard/study in ryze the following diagram shows an example flow for a conversion process upload dataset data for an asset group in a container endpoint post https //ryze formedix com/fws/api/repository/container/containeraid/version/assetgroup/assetgroupaid/upload multipart post, for example host ryze formedix com content type multipart/form data; boundary= xxx authorization bearer 12345d c123 1ce0 9893 0123456c83b \ xxx content disposition form data; name="file"; filename=" /ae1 zip" (data) \ xxx content disposition form data; name="json" {"fileformat" "csv"} \ xxx this request can be used to upload data for datasets in a given asset group use containeraid and version to specify the container, and assetgroupaid to specify the asset group if the related metadata for the dataset asset does not exist, it is automatically created if the related metadata for the dataset asset does exist, the existing metadata is updated fileformat is optional, and specifies the format for the uploaded dataset files valid formats are sas transport v5 (xpt), dataset xml (dsxml) and comma delimited (csv) utf 8 encoding must be used in csv files if not specified, this will default to "csv" the file uploaded must be a zipped archive of one or more dataset data files in the requested format this request is sent as a https //www w3 org/tr/html401/interact/forms html#h 17 13 4 2 request if the file format is unspecified, the first part of the multipart post should be a json payload the uuid returned can be used to retrieve the status of the upload see also https //docs formedix com/api/data conversion and automation#m6nbk example response { "uuid" "01234567 89ab cdef 0123 456789abcdef" } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid assetgroup aid an invalid administeredid was specified as the assetgroupaid 400 dataset cleanup in progress when attempting to upload data on a container that is being cleaned up 404 unknown container unknown containeraid / version pair 404 unknown assetgroup unknown assetgroupaid within the containeraid / version pair 404 invalid addons no addons available to be consumed 404 duplicate files the zip file contains multiple datasets of the same name see also docid\ tqpncp50s5muu3bfnmk9w upload dataset data for a named asset group in a container post https //ryze formedix com/fws/api/repository/container/containeraid/version/upload/name multipart post, for example host ryze formedix com content type multipart/form data; boundary= xxx authorization bearer 12345d c123 1ce0 9893 0123456c83b \ xxx content disposition form data; name="file"; filename=" /csvfile zip" (data) \ xxx content disposition form data; name="json" {"fileformat" "csv"} \ xxx this request can be used to upload data for datasets in an asset group with a specified name use containeraid and version to specify the container, and name to specify the asset group if the related metadata for the dataset asset(s) does not exist, it is automatically created if the related metadata for the dataset asset(s) does exist, the existing metadata is updated ‘fileformat’ is optional, and specifies the format for the uploaded dataset files valid formats are sas transport v5 (xpt), dataset xml (dsxml) and comma delimited (csv) utf 8 encoding must be used in csv files if not specified, this will default to "csv" the file uploaded must be a zipped archive of one or more dataset data files in the requested format this request is sent as a https //www w3 org/tr/html401/interact/forms html#h 17 13 4 2 request if the file format is unspecified, the first part of the multipart post should be a json payload the file uploaded must be a zipped archive of one or more dataset data files in the requested format the uuid returned can be used to retrieve the status of the upload see also https //docs formedix com/api/data conversion and automation#m6nbk example response { "uuid" "01234567 89ab cdef 0123 456789abcdef" } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 dataset cleanup in progress when attempting to upload data on a container that is being cleaned up 404 unknown container unknown containeraid / version pair 404 invalid addons no addons available to be consumed 404 duplicate files the zip file contains multiple datasets of the same name see also docid\ tqpncp50s5muu3bfnmk9w retrieve upload status for dataset data in a container this request can be used to retrieve the status of a previously uploaded dataset data request containeraid and version specify the container, and uuid specifies the upload request to track if the container specified does not exist, or there is no matching upload request within this container for the given uuid, then an appropriate error is returned the status returned is one of successful , failed or inprogress endpoint get https //ryze formedix com/fws/api/repository/container/containeraid/version/upload/status/uuid example response { "uuid" "01234567 89ab cdef 0123 456789abcdef", "containeraid" "01234567 89ab cdef 0123 456789abcdef", "containerversion" 0, "status" "successful", "message" "" } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid uuid an invalid uuid was specified as the uuid 404 unknown container unknown containeraid / version pair 404 unknown upload unknown upload request uuid see also docid\ tqpncp50s5muu3bfnmk9w retrieve a list of integrations for a container this request can be used to return a list of configured data integrations with remote systems for the container with the specified containeraid and version to determine if an integration is running, check the requested and inprogress flags requested = true the integration is requested requested = false, inprogress = true the integration is being processed requested = false, inprogress = false the integration has completed endpoint get https //ryze formedix com/fws/api/repository/container/containeraid/version/integration example response \[ { "uuid" "01234567 89ab cdef 0123 456789abcdef", "system" "", "name" "", "inprogress" false, "requested" true }, ] errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 404 unknown container unknown containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w request that an integration be queued for execution put https //ryze formedix com/fws/api/repository/container/containeraid/version/integration/integrationuuid this request triggers the integration with the given uuid (defined within the container) with the specified containeraid and version this is a non blocking call, it queues the integration for execution at the first available future point, and returns a success once the integration is queued if the specified integration is already queued, then an already queued error is returned see also, https //app archbee com/docs/3v0gzyeyw xiey1zyz jn/ujpmztqiqtzf61zukkibt#hbidy example response { } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid uuid an invalid administeredid was specified as the integrationuuid 400 already queued the specified integrationuuid is already in the queue to be executed 400 remote integration error issue returned by the integration service a context property may also be provided with a more specific error for example, if authentication details for rave web services are incorrect (e g incorrect or expired password), the context will be “streamed rest call failed with 401” 404 unknown container unknown containeraid / version pair 404 unknown integration could not find specified integrationuuid for given containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w request a data conversion for an asset group endpoint put https //ryze formedix com/fws/api/repository/container/containeraid/version/assetgroup/assetgroupaid/convert request that the dataset data for an asset group is generated using the rules defined in the dataset mapper plugin in ryze conversions can be run at asset group (study) level or run on a specific dataset within an asset group similar to the ryze user interface the type of conversion that is run is reflected in the response to the ‘retrieve the list of conversion tasks for a container’ request datasets are generated for the specified containeraid , version and assetgroupaid asynchronously and are made available in ryze when completed the conversion task uuid is returned and the caller may repeatedly poll requests to monitor the status of the conversion if your company has a p21e integration enabled, then the p21 validation is run automatically after the conversion task example response { "uuid" "01234567 89ab cdef 0123 456789abcdef" } see also, /#retrieve the list of conversion tasks for a container errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid assetgroup aid an invalid administeredid was specified as the assetgroupaid 400 conversion for asset group in progress a conversion for the specified asset group is already running 404 unknown container unknown containeraid / version pair 404 unknown assetgroup could not find specified assetgroupaid for given containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w request a data conversion for a dataset put https //ryze formedix com/fws/api/repository/container/containeraid/version/assetgroup/assetgroupaid/convert/datasetaid request that the dataset data for a specific dataset within an asset group is generated using the rules defined in the dataset mapper plugin in ryze conversions can be run at asset group (study) level or focused on a specific dataset within an asset group similar to the ryze user interface the type of conversion that is run is reflected in the response to the ‘retrieve the list of conversion tasks for a container’ request the dataset data with the given datasetaid is generated for the specified containeraid , version and assetgroupaid asynchronously and is made available in ryze when completed the conversion task uuid is returned, and the caller may poll requesting the conversion task to monitor the status of the conversion @ example response { "uuid" "01234567 89ab cdef 0123 456789abcdef" } see also, /#retrieve the list of conversion tasks for a container errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid assetgroup aid an invalid administeredid was specified as the assetgroupaid 400 invalid asset aid an invalid administeredid was specified as the datasetaid 400 conversion for asset group in progress a conversion for the specified asset group is already running 400 conversion for dataset in progress a conversion for the specified dataset is already running 404 unknown container unknown containeraid / version pair 404 unknown assetgroup could not find specified assetgroupaid for given containeraid / version pair 404 unknown dataset could not find specified datasetaid for the given containeraid / version / assetgroupaid triple see also docid\ tqpncp50s5muu3bfnmk9w cancel in progress data conversion request delete https //ryze formedix com/fws/api/repository/container/containeraid/version/conversion/conversiontaskuuid request that the specified conversion task be cancelled example response 200 ok if successful see also, /#retrieve the list of conversion tasks for a container errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid uuid an invalid uuid was specified as the conversiontaskuuid 400 conversiontask not in progress conversion task is not in progress 404 unknown container unknown containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w retrieve the list of conversion tasks for a container request get https //ryze formedix com/fws/api/repository/container/containeraid/version/conversion this request can be used to return the list of conversion tasks that have been performed within the container with the specified containeraid and version in the system conversions can be run at asset group (study) level or focused on a specific dataset within an asset group the type of conversion that is run is reflected in the response this request can be used to return the list of conversion tasks that have been performed within the container with the specified datasetaid (when running a conversion at dataset level), containeraid and version in the system up to 250 of the most recent records are retrieved and returned response when running the conversion task at asset group level \[ { "uuid" "c82dfd61 2920 47b3 b5a7 f8192fc9b6a7", "requested" "20220817t103834 461z", "completed" "20220817t103835 388z", "status" "successful", "conversiontype" "assetgroup", "assetgroup" { "aid" "1861ad74 da57 40ff 8537 78348f8ba19e" }, "latest" false } ] response when running the conversion task at asset level \[ { "uuid" "e1700e1f cdf3 4bee a4a1 d1e9a2d04c92", "requested" "20230516t141706 552z", "queued" "20230516t141708 956z", "started" "20230516t141708 972z", "completed" "20230516t141709 747z", "status" "successful", "conversiontype" "asset", "assetgroup" { "aid" "54c513ae 5b88 421b bd5a e60bdf165d55" }, "asset" { "aid" "d2c30dce fd0f 462c 89ff b36036233452", "name" "ae" }, "latest" false } ] the response fields when running the conversion task at asset group level can include the following values key value uuid generated from the 'request a data conversion ' request requested time in iso 8601 format queued time in iso 8601 format started time in iso 8601 format completed time in iso 8601 format status successful , failed , preparing , queued or in progress conversiontype asset or assetgroup assetgroup aid asset aid and name latest true or false responses when running the conversion task at dataset level include additional details, including the administeredid (aid) and name the last conversion task to be run is indicated with "latest" true this is not necessarily the last record returned in the response errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 404 unknown container unknown container aid/version pair see also docid\ tqpncp50s5muu3bfnmk9w retrieve the list of conversion issues for a conversion task request get https //ryze formedix com/fws/api/repository/container/containeraid/version/conversion/ conversiontaskuuid/issues this request returns the list of issues that occurred during a conversion task run the conversion task is identified by the conversiontaskuuid within the container with the specified containeraid and version example response \[ { "sourcedatasetaid" "01234567 89ab cdef 0123 456789abcdef", "sourcedataset" "", "sourcerow" "", "sourcevariable" "", "targetdatasetaid" "01234567 89ab cdef 0123 456789abcdef", "targetdataset" "", "targetrow" "", "targetvariable" "", "targetvalue" "", "ruleid" "", "message" "" }, ] errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid uuid an invalid uuid was specified as the containertaskuuid 404 unknown container unknown containeraid / version pair 404 unknown conversiontask could not find specified conversiontaskuuid for given containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w request download of data for an asset group request put https //ryze formedix com/fws/api/repository/container/containeraid/version/assetgroup/assetgroupaid/data { "datetime" "ccyymmddthhmmssz", "fileformat" "xpt", "encoding" "utf 8" } request for the dataset data present for the given containeraid , version and assetgroupaid to be archived and made available to download this request returns a uuid if the specified asset group has data if the asset group has no data, then the response from the docid 6jx3 efy93 mdseidxbji request is 404 unknown file a datetime value is optional but the datetime parameter must be included in the payload if you send an empty string "" it defaults to the current utc datetime fileformat is optional, and specifies the format to use for the archived dataset files valid formats are sas transport v5 (xpt), dataset xml (dsxml) and comma delimited (csv) if not specified, this defaults to "csv" a unique identifier for the future prepared download is returned to the caller, which can then subsequently be used to request download of the prepared file the returned uuid is used in the file retrieval call as the fileuuid encoding is an optional string property you can specify either latin or utf 8 encoding if no encoding is specified, it defaults to latin if invalid encoding is encountered, it returns an invalid encoding error see also docid 6jx3 efy93 mdseidxbji example response { "uuid" "01234567 89ab cdef 0123 456789abcdef" } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 400 invalid assetgroup aid an invalid administeredid was specified as the assetgroupaid 400 invalid datetime datetime specified was not understood if present, should be an iso8601 string as detailed above 400 invalid fileformat the fileformat specified was not recognised valid file formats are as detailed above 400 invalid encoding an invalid encoding type is entered in the encoding field 400 no data for download no files were able to be prepared for download for this request 404 unknown container unknown containeraid / version pair 404 unknown assetgroup could not find specified assetgroupaid for given containeraid / version pair 404 unknown dataset could not find any file for the specified assetgroupaid for given containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w request deletion of all dataset data for a version of a container request delete https //ryze formedix com/fws/api/repository/container/containeraid/version/purgedata this request removes the following dataset data any issues reported during conversion of dataset data metadata associated with the dataset data that was generated pre existing non generated metadata remains in the system conversion task and repository logs for the removed datasets also remain in the system example response { } errors http status error code 400 invalid container aid an invalid administeredid was specified as the containeraid 400 invalid container version a non numeric value was specified as the version 404 unknown container unknown containeraid / version pair see also docid\ tqpncp50s5muu3bfnmk9w