Cloud Passage Customers,
We will soon be changing our default retention period for historical scan data. Server events and scan details older than 90 days will no longer be stored for most customers. Summarized scan data and Halo portal auditing events will continue to be stored for 2 years. (Your site administrator will receive an email before this happens.)
For accounts needing to retain any data beyond the above data retention periods, we have created a script that you may use and modify to extract this data from Halo to your own offline storage:
Usage: archiveScanData.rb [flag]
where flag can be one of:
--auth=<file> Read auth info from <file>
--starting=<when> Only get status for scans after when (ISO-8601 format)
--ending=<when> Only get status for scans before when (ISO-8601 format)
--base=<url> Override base URL (normally https://portal.cloudpassage.com/)
--localca Use local CA file (needed on Windows)
--detailsfiles Write details about each scan's results to a set of files
--threads=<num> Set number of threads to use downloading scan results
Your auth file should contain a single line, in the form of:
<api key>|<secret key>
If you do not supply a start or end date, then the script will pull all available scan information from your Halo account.
You will need to use the --detailsfiles flag to record output to local disk. Data will be saved to the details directory in your current working directory.
On Windows machines, OAuth2 can't find the SSL cert repository, so you will need to supply the --localca argument on a Windows host. This tells the script to look for a local copy of the CA (certificate authority) file in "./certs/cacert.pem". (This is a known issue with ruby; see https://gist.github.com/fnichol/867550 for more information.)
Since the script makes individual API calls for each scan, this may take a long time to run, especially if you are downloading a lot of data. To improve performance:
- run the script during off hours
- download in batches, e.g., pull a month or a week's worth of data in each run
- use the --threads option
Please contact firstname.lastname@example.org if you have any questions.