Data Movement APIs
  • Updated on 07 Apr 2020
  • 2 minutes to read
  • Contributors
  • Print
  • Share
  • Dark
    Light

Data Movement APIs

  • Print
  • Share
  • Dark
    Light

Igneous provides a RESTful API for managing data movement between Igneous and network attached storage (NAS). Customers may implement applications that leverage the data movement API to ingest data from remote devices, to tier data between hot (NAS) and cold (Igneous) storage tiers, or for any backup and archive use cases. The API exposes tasks that manage data movement with parameters for identifying the source and destination.

Copy Data from NAS to Igneous

This API is used to copy data from Primary NAS onto the Igneous appliance data store.
The body of the POST request will include in JSON format:

  • Source ‑ The URL of the source export such as nfs://nas.example.com/data
  • Destination ‑ The Igneous target bucket name in the format igneous:///bucket‑name
  • Path ‑ The full path to copy from including the export (i.e., /data/dir1/dir2) that must be rooted in the export directory
  • Params
    • CopyOnly ‑ A boolean value of true or false
      The CopyOnly parameter has the following effects on a copy from NAS to Igneous:
      • False ‑ This mode behaves like an incremental backup. Files that were found by earlier copy tasks but have since been deleted on the NAS source will be marked for deletion on Igneous. The files marked for deletion will not be recovered by the API copy task from Igneous to NAS. Your Igneous Customer Success team at customersuccess@igneous.io can apply a retention policy to the bucket that will periodically scan for files deleted beyond the retention period to remove from the system.
      • True ‑ In contrast this mode never marks files for deletion on Igneous even if they’re deleted from the source NAS. All files in the target Igneous bucket will therefore be copied to the NAS target when using the API including files that may have been deleted from the source NAS.

After sending the request the JSON response will include a task ID, state and additional messages.

POST /x/igneous/v1/tasks

Example using CURL

curl --insecure -X POST \
https://<IP>:443/x/igneous/v1/tasks \
-H 'Authorization: 2WXRXD92X2PRWZH57V9Z' \
-H 'Content-Type: application/json' \
-d '{
"Source": "nfs://10.111.1.100/data",
"Destination": "igneous:///data-copy",
"Path": "",
"Params": {
"CopyOnly": true
}
}'

Example output

{
"Tasks": [
{
"ID": "QVdRTW43dFdyR0plZkx6cmU2ZGE=",
"State": "pending",
"Message": ""
}
]
}

Copy data from Igneous to NAS

This API is used to copy data from Igneous to Primary NAS. Files that have been marked for deletion due to using the "CopyOnly : False" option will not be copied. The full path of the data will be copied to the
destination, so for example if the path of the source was /data/dir1/dir2 then it will be restored to the
target export as /<target export>/data/dir1/dir2.
● Source ‑ The Igneous source bucket name in the format igneous:///bucket‑name
● Destination ‑ The URL of the source export such as nfs://nas.example.com/data
● Path ‑ The full path to copy from such as /data/dir1/dir2 (the full path will be restored on the
target export). If empty, the entire volume will be restored.

POST /x/igneous/v1/tasks

Example using CURL

curl --insecure -X POST \
https://<IP>:443/x/igneous/v1/tasks \
-H 'Authorization: 2WXRXD92X2PRWZH57V9Z' \
-H 'Content-Type: application/json' \
-d '{
"Source": "igneous:///data-copy",
"Destination": "nfs://10.111.1.101/target",
"Path": "",
}'
Was this article helpful?