Scripted Migration Utility
New in Release 6.1: Learn more about how end-users can mark content they would like to have migrated.
As of Release 5.6.0, it is possible to Migrate Content using our Export/Import Migration Scripts. This enables System Admins to setup a content migration pipeline wherein, at a specified time each day (setup via something like cron), all content is migrated from a staging environment to a production environment.
The Scripted Migration process includes two main stages:
-
Content Export via
insightdump.py
-
Content Import via
insightload.py
This article details how to move a Category and all included elements from one server to another. The process of exporting individual Objects or Elements is essentially the same. See this article:
Migration Capabilities
Note: Root privileges are required to run Migration.
MI Elements and Objects |
Scripted Migration |
Details |
Metrics |
can be migrated |
BEFORE MIGRATION, on the new instance make sure to:
|
Reports |
can be migrated |
|
External Reports |
can be migrated |
|
Datasets/Users Maps |
can be migrated |
|
Categories |
can be migrated |
|
Folders | can be migrated (6.2.1 and beyond) | |
Dimensiosn |
NOT SUPPORTED |
ALL OBJECTS AND ENTITIES that cannot be migrated directly have to be rebuilt on the new instance. |
Plugin Data Sources |
NOT SUPPORTED |
Key Migration Dependencies
Migration Dependencies | |
---|---|
Element/Object IDs |
|
Data Sources |
|
Technical/Business Owners |
|
1. Exporting Content
Exporting content involves creating a .json file with information on all migrated Elements/Objects that can later be uploaded to a different server.
- The insightdump.py script is used to export a Category.
- All associated Documents can be exported in an archive as separate files.
To initiate export, run the following command:
sudo /opt/mi/.python/bin/python /opt/mi/generator/insightdump.py -c 86 -f /<directory>/<archive name>.tar.gz -a
where:
- /opt/mi/.python/bin/python is a Python Interpreter (installed during the installation of the MI application)
-
/opt/mi/generator/insightdump.py is a path to the
insightdump.py
script - -c 86 is a Category parameter followed by Category ID
-
-f parameter allowing Users to create a
.json
file or a.tar.gz
archive - <directory> where the dump file will be created
- <archive name>.tar.gz is a user-defined archive name
-
-a is an archive parameter
- archived export allows for migration of Documents that are associated with the migrated Elements/Objects
-
-a parameter is optional; if Document files do not need to be migrated, a single
.json
file can be exported
To export a Category:
- Run
insightdump.py
with the desired parameters - Check the response for errors
- In output, review export details
- [Optionally] verify that the Category was saved to the
.json
file ortar.gz
archive with the name you specified - [Optionally] view the contents of the exported
tar.gz
archive- In an archive, Documents are stored by Number IDs (This IDs are archive-relevant only)
- The actual Document names are stored inside a
.json
file
1.1. Optional Parameters for export
The list of arguments to use at export | |
---|---|
-h, --help |
Show help message and exit |
-f FILENAME, --file FILENAME |
Output file name |
-c CATEGORY_IDS, --category CATEGORY_IDS |
Category ID to dump |
-e ELEMENT_IDS, --element ELEMENT_IDS |
Element IDs to dump (comma-separated) |
-d DATASET_IDS, --dataset DATASET_IDS |
Dataset IDs to dump (comma-separated) |
--exclude-access-maps |
Do not import any dependent access maps. |
-a, --archive |
Make tar.gz archive with related documents |
2. Importing content
Importing content involves uploading it to the required server.
- The insightload.py script is used to import Elements and Objects.
PREREQUISITES:
- Before running the import script, copy the saved
.tar.gz
archive or.json file
to the server where your content needs to be imported.
To initiate import, run the following command:
sudo /opt/mi/.python/bin/python /opt/mi/generator/insightload.py -f /<directory>/<archive name>.tar.gz -b ~/.
where:
- /opt/mi/.python/bin/python is a Python Interpreter (installed during the installation of the MI application)
-
/opt/mi/generator/insightload.py is a path to the
insightdump.py
script that dumps data -
-f parameter allowing Users to specify a
.json
file or a.tar.gz
archive that will be uploaded to the new server - <directory> from which the upload will be run
- <archive name>.tar.gz is a user-defined archive name
- -b is a backup parameter
To import a Category:
- Run
insightload.py
- Check the response for errors
- In output, see export details
2.1. Optional Parameters for import
The list of arguments to use at import | |
---|---|
-h, --help |
Show this help message and exit |
-m, --match |
Load matching elements |
-b BACKUP, --backup BACKUP |
Backup elements before replacing |
-f FILENAME, --file FILENAME |
File to load elements from |
-n, --no-preserve |
Do not preserve IDs |
-s, --strict |
Delete elements in the target category if they are not in the source category |
--preserve-access-map |
Do not overwrite access map configuration |
3. Verify Migration Results
Upon successful Migration, all migrated content will be accessible from the UI.
4. If Migration runs with errors
In case Migration runs with errors:
- Verify that all the Migration Prerequisites have been met (For details, check the Script Response and Output).
- Having eliminated the cause of errors, rerun the upload script to update Migration results.