Scripted Migration Utility
It is possible to Migrate Content using our Export/Import Migration Scripts that enable System Admins to setup a content migration pipeline. The article shows an example of how to migrate content from one server to another.
The Scripted Migration process includes two main stages:
-
Content Export via
run.py export -
Content Import via
run.py import
This article details how to move a Category and all included elements from one server to another.
- The process of exporting individual Objects or Elements is essentially the same. See Set Migration for Categories, Folders, Elements, Datasets/User Maps or Portal Pages via their Editors
Migration Capabilities
Note: Root privileges are required to run Migration.
|
MI Elements and Objects |
Scripted Migration |
Details |
| Metrics |
can be migrated |
BEFORE MIGRATION, on the new instance make sure to:
|
| Reports |
can be migrated |
|
| External Reports |
can be migrated |
|
| Datasets/Users Maps |
can be migrated |
|
| Categories |
can be migrated |
|
| Folders | can be migrated | |
| Portal Pages |
can be migrated |
|
| Bursts |
can be migrated |
|
| Digest Templates |
can be migrated |
|
| Custom Fields |
can be migrated |
|
| Dimensions |
NOT SUPPORTED |
ALL OBJECTS AND ENTITIES that cannot be migrated directly have to be rebuilt on the new instance. |
| Plugin Data Sources |
NOT SUPPORTED |
| Migration Dependencies | |
|---|---|
| Element/Object IDs |
|
| Data Sources |
|
| Technical/Business Owners |
|
1. Exporting Content
Exporting content involves creating a .tar.gz file with information on all migrated Elements/Objects that can later be uploaded to a different server.
All associated Documents/Previews/Attachaments/JS Templates can be exported in an archive as separate files.
To initiate export, run the following command:
sudo python3.9 /opt/mi/migration-tool/run.py export -c 86 -f /<directory>/<archive name>.tar.gzwhere:
-
python3.9is a Python Interpreter (installed during the installation of the MI application) -
/opt/mi/migration-tool/run.pyis a migration script -
exportis a command which should be performed -
-c 86is a Category parameter followed by Category ID -
-fparameter specifies the name for the output archive -
<directory>where the dump file will be created -
<archive name>.tar.gzis a user-defined archive name
To export a Category (as an example):
- Run
run.py exportwith the desired parameters - The response of the script will contain an integer, which is the
migration_export_logtable ID. The table along withmigration_export_log_entitiycontains details about the export.- NOTE: In case of fatal errors, the response will return an error text message. This may happen when the program is not able to connect to the DB or some system migration tables were absent (or corrupted)
- Optionally, verify that the Category was saved to the
tar.gzarchive with the name you specified - Optionally, view the contents of the exported
tar.gzarchive- In an archive, Documents are stored by Number IDs
- The actual Document names are stored inside a
.jsonfile
| The list of arguments to use at export | |
|---|---|
| -h, --help | Show help message and exit |
| -e ELEMENTS [ELEMENTS ...], --elements ELEMENTS [ELEMENTS ...] | IDs of elements |
| --bursts BURSTS [BURSTS ...] | Burst IDs to dump (space-separated) |
| -d DATASETS [DATASETS ...], --datasets DATASETS [DATASETS ...] | IDs of Datasets |
| -f FILE_NAME, --file_name FILE_NAME | Output file name |
| --exclude-access-maps | Do not export any dependent access maps |
| --folders FOLDERS [FOLDERS ...] | Folder IDs to dump (space-separated) |
| --group-sharing | Include sharing user groups for migrated objects |
| -c CATEGORIES [CATEGORIES ...], --categories CATEGORIES [CATEGORIES ...] | Category ID to dump (space-separated) |
| -p PORTAL_PAGES [PORTAL_PAGES ...], --portal-pages PORTAL_PAGES [PORTAL_PAGES ...] | Portal Page IDs to dump (space-separated) |
| --digest-templates DIGEST_TEMPLATES [DIGEST_TEMPLATES ...] | Digest template IDs to dump (space-separated) |
| --custom-fields CUSTOM_FIELDS [CUSTOM_FIELDS ...] | Custom field IDs to dump (comma-separated) |
| --include-folders | If an Element is included in Folder, but that Folder has not been explicitly selected for export, this flag will automatically include the Folder along with the Element. |
| -s, --scheduled | Include scheduled migration |
| --skip_external_report | Do not dump External Report |
| --skip_documents | Do include dump documents in the outcome archive |
| --skip_other_external_content |
Do not dump Other External Content |
2. Importing Сontent
Importing content involves uploading it to the required server.
- The
run.py importscript is used to import Elements and Objects.
PREREQUISITES:
- Before running the import script, copy the saved
.tar.gzarchive to the server where your content needs to be imported.
To initiate import, run the following command:
sudo python3.9 /opt/mi/migration-tool/run.py import -f /<directory>/<archive name>.tar.gz -b ~/.where:
-
python3.9is a Python Interpreter (installed during the installation of the MI application) -
/opt/mi/migration-tool/run.pyis a migration script -
-fparameter allowing Users to specify a .tar.gz archive that will be uploaded to the new server -
<directory>from which the upload will be run -
<archive name>.tar.gzis a user-defined archive name -
-bis a backup parameter
To import a Category (as an example):
- Run
run.py import - The response of the script will contain an integer, which is the
migration_import_logtable ID. The table along with themigration_import_log_entitiyandmigration_import_operationtables contains details about the import.
2.1. Optional Parameters for Import
| The list of arguments to use at import | |
|---|---|
| -h, --help | Show this help message and exit |
| -f FILE_NAME, --file_name FILE_NAME | File to load elements from |
| --test_output | Output as Python dict (for development purposes only) |
| -b BACKUP_DIR, --backup-dir BACKUP_DIR | Directory to back up changed records to |
| --preserve-access-map | Do not overwrite access map configuration |
| --burst_create_only | Create new Bursts only. Do not update existed Bursts |
| --new_entity_topics_only | Create topic only for new entities (Elements, Datasets, Portal Pages, Bursts). Skip topics for existing records |
| --use_category_owner | For new Elements/Datasets, Owner are set to Category owners. For existing records, Owners are not updated |
| --reset_certification | For new Elements/Datasets, certified attribute is set to 'N' (reset_certification, last_certified_time as well). For existing records, certified attribute is not updated |
| -s, --strict | Delete Elements/Topics in the target Category if they are not in the source Category |
| -m, --match | Load matching Elements |
| -r, --relax | Keeping data on target system |
| -n, --no-preserve |
Do not preserve IDs |
3. Rolling Back
[6.3.0] The rollback command allows users to cancel the import operation. To able to do this, it is required to specify the ID of import operation (migration_import_log ID).
python3.9 run.py rollback 111The rollback operation uses migration_import_log_entity and migration_import_operation tables to perform the revert operation. For example:
- insert –> delete
- update –> update to the state previous to the import
- delete –> restore
The rollback operation works with the Objects that were involved in import and brings them back to the state in which they were before being imported.
