import data reference
yb-voyager import data
The following page describes the usage of the following import commands:
- import data (and import data to target)
- import data status
- get data-migration-report
- import data to source
- import data to source-replica
import data
For offline migration, Import the data to the YugabyteDB database.
For live migration (with fall-forward and fall-back), the import data command is an alias of import data to target which imports the data from the data directory to the target database, and starts ingesting the new changes captured by export data to the target database.
Syntax
Syntax for import data is as follows:
Usage: yb-voyager import data [ <arguments> ... ]
Syntax for import data to target is as follows:
Usage: yb-voyager import data to target [ <arguments> ... ]
Arguments
The valid arguments for import data are described in the following table:
| Argument | Description/valid options |
|---|---|
| --batch-size | Size of batches in the number of rows generated for ingestion during import data. Default: 20000 rows Example: yb-voyager import data ... --batch-size 20000 |
| --disable-pb | Use this argument to disable progress bar or statistics during data import. Default: false Accepted parameters: true, false, yes, no, 0, 1 |
| --enable‑upsert | Enable UPSERT mode on target tables while importing data. Note: Ensure that tables on the target YugabyteDB database do not have secondary indexes. If a table has secondary indexes, setting this flag to true may lead to corruption of the indexes. Default: false Usage for disabling the mode: yb-voyager import data ... --enable-upsert falseAccepted parameters: true, false, yes, no, 0, 1 |
| --table-list | Comma-separated list of names of source database tables whose data is to be imported. Table names can also be glob patterns containing wildcard characters, such as an asterisk (*) (matches zero or more characters) or question mark (?) (matches one character). To use a glob pattern for table names, enclose the list in single quotes (''). For example, --table-list '"Products", order*'.For example, --table-list '"Products", order*'.This argument is unsupported for live migration. |
| --exclude‑table‑list | Comma-separated list of names of source database tables for which data needs to be excluded during import. Table names follow the same convention as --table-list. This argument is unsupported for live migration. |
| --table-list-file‑path | Path of the file containing the list of names of source database tables (comma separated or line separated) to import. Table names use the same convention as --table-list. |
| ‑‑exclude‑table‑list‑file‑path | Path of the file containing the list of names of source database tables (comma separated or line separated) to exclude while importing data of those exported tables. Table names follow the same convention as --table-list. |
| ‑‑enable‑adaptive‑parallelism | Adapt parallelism based on the resource usage (CPU, memory) of the target YugabyteDB cluster. Default: true Accepted parameters: true, false, yes, no, 0, 1 |
| --adaptive-parallelism-max | Number of maximum parallel jobs to use while importing data when adaptive parallelism is enabled. By default, voyager tries to determine the total number of cores N and use N/2 as the maximum parallel jobs. |
| -e, --export-dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file. |
| -h, --help | Command line help. |
| --parallel-jobs | Number of parallel jobs to use while importing data. Depending on the YugabyteDB database configuration, the value of --parallel-jobs should be tweaked such that at most 50% of target cores are utilised. Default: If yb-voyager can determine the total number of cores N in the YugabyteDB database cluster, it uses N/2 as the default for import data and N/4 for import data to target. Otherwise, it defaults to twice the number of nodes in the cluster. |
| --send-diagnostics | Enable or disable sending diagnostics information to Yugabyte. Default: true Accepted parameters: true, false, yes, no, 0, 1 |
| --start-clean | Starts a fresh import with data files present in the data directory.If the target YugabyteDB database has non-empty tables, you are prompted to continue the import without truncating those tables; if you go ahead without truncating, then yb-voyager starts ingesting the data present in the data files in upsert mode. Note that for cases where a table doesn't have a primary key, duplicate data may be inserted. You can avoid duplication by excluding the table using --exclude-table-list, or by truncating those tables manually before using the start-clean flag. Default: false Accepted parameters: true, false, yes, no, 0, 1 |
| --target-db-host | Domain name or IP address of the machine on which target database server is running. Default: "127.0.0.1" |
| --target-db-name | Target database name. Default: yugabyte |
| ‑‑target‑db‑password |
Password to connect to the target YugabyteDB database. Alternatively, you can also specify the password by setting the environment variable TARGET_DB_PASSWORD. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes. |
| --target-db-port | Port number of the target database server. Default: 5433 |
| --target-db-schema | Schema name of the target database. MySQL and Oracle migrations only. |
| --target-db-user | Username of the target database. |
| --target-endpoints | Comma-separated list of node's endpoint to use for parallel import of data Default: Use all the nodes in the cluster. For example: "host1:port1,host2:port2" or "host1,host2". Note: use-public-ip flag will be ignored if this is used. |
| --use-public-ip | Use the node public IP addresses to distribute --parallel-jobs uniformly on data import. Default: false Note that you may need to configure the database with public IP addresses by setting server-broadcast-addresses. Example: yb-voyager import data ... --use-public-ip trueAccepted parameters: true, false, yes, no, 0, 1 |
| --target-ssl-cert | Path to a file containing the certificate which is part of the SSL <cert,key> pair. |
| --target-ssl-key | Path to a file containing the key which is part of the SSL <cert,key> pair. |
| --target-ssl-crl | Path to a file containing the SSL certificate revocation list (CRL). |
| --target-ssl-mode | Specify the SSL mode for the target database as one of disable, allow, prefer (default), require, verify-ca, or verify-full. |
| --target-ssl-root-cert | Path to a file containing SSL certificate authority (CA) certificate(s). |
| -y, --yes | Answer yes to all prompts during the export schema operation. Default: false |
Example
Offline migration
An example for offline migration is as follows:
yb-voyager import data --export-dir /dir/export-dir \
--target-db-host 127.0.0.1 \
--target-db-user ybvoyager \
--target-db-password 'password' \
--target-db-name target_db \
--target-db-schema target_schema \
--parallel-jobs 12
Live migration
An example for all live migration scenarios is as follows:
yb-voyager import data to target --export-dir /dir/export-dir \
--target-db-host 127.0.0.1 \
--target-db-user ybvoyager \
--target-db-password 'password' \
--target-db-name target_db \
--target-db-schema target_schema \
--parallel-jobs 12
import data status
For offline migration, get the status report of an ongoing or completed data import operation. The report contains migration status of tables, number of rows or bytes imported, and percentage completion.
Syntax
Usage: yb-voyager import data status [ <arguments> ... ]
Arguments
The valid arguments for import data status are described in the following table:
| Argument | Description/valid options |
|---|---|
| -e, ‑‑export‑dir | Path to the export directory. This directory is a workspace used to keep the exported schema, data, state, and logs. |
| -h, --help | Command line help. |
| -y, --yes | Answer yes to all prompts during the import data operation. Default: false |
Example
yb-voyager import data status --export-dir /dir/export-dir
get data-migration-report
Note that this command is applicable for Live migrations only.
Provides a consolidated report of data migration per table among all the databases involved in the live migration. The report includes the number of rows exported, the number of rows imported, change events exported and imported (INSERTS, UPDATES, and DELETES), and the final row count on the database.
Syntax
Usage: yb-voyager get data-migration-report [ <arguments> ... ]
Arguments
The valid arguments for get data-migration-report are described in the following table:
| Argument | Description/valid options |
|---|---|
| -e, ‑‑export‑dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file. |
| -h, --help | Command line help. |
| ‑‑source‑db‑password | Password to connect to the source database. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. Alternatively, you can also specify the password by setting the environment variable SOURCE_DB_PASSWORD. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose it in single quotes. |
| ‑‑source‑replica‑db‑password | Password to connect to the source-replica database. Alternatively, you can also specify the password by setting the environment variable SOURCE_REPLICA_DB_PASSWORD. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose it in single quotes. |
| --target-db-password | Password to connect to the target YugabyteDB database. Alternatively, you can also specify the password by setting the environment variable TARGET_DB_PASSWORD. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes. |
Example
yb-voyager get data-migration-report --export-dir /dir/export-dir
import data to source
Note that this command is applicable for Live migrations only.
Imports data to the source database, and streams new changes from the YugabyteDB database to the source database.
Syntax
Usage: yb-voyager import data to source [ <arguments> ... ]
Arguments
The valid arguments for import data to source are described in the following table:
| Argument | Description/valid options |
|---|---|
| -e, --export-dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file. |
| -h, --help | Command line help for import data to source. |
| --parallel-jobs | Number of parallel jobs to use while importing data. Default: 16(Oracle) |
| --send-diagnostics | Enable or disable sending diagnostics information to Yugabyte. Default: true Accepted parameters: true, false, yes, no, 0, 1 |
| ‑‑source‑db‑password | Password to connect to the source database. |
| --start-clean | Starts a fresh import with exported data files present in the export-dir/data directory. If any table on YugabyteDB database is non-empty, it prompts whether you want to continue the import without truncating those tables; if you go ahead without truncating, then yb-voyager starts ingesting the data present in the data files with upsert mode. Note that for the cases where a table doesn't have a primary key, this may lead to insertion of duplicate data. To avoid this, exclude the table using the --exclude-file-list or truncate those tables manually before using the start-clean flag. Default: false Accepted parameters: true, false, yes, no, 0, 1 |
| -y, --yes | Answer yes to all prompts during the migration. Default: false |
Example
yb-voyager import data to source --export-dir /dir/export-dir \
--source-db-password 'password'
import data to source-replica
Note that this command is applicable for Live migrations only.
Imports data to the source-replica database, and streams new changes from the YugabyteDB database to the source-replica database.
Syntax
Usage: yb-voyager import data to source-replica [ <arguments> ... ]
Arguments
The valid arguments for import data to source-replica are described in the following table:
| Argument | Description/valid options |
|---|---|
| --batch-size | Size of batches in the number of rows generated for ingestion when you import data to source-replica database. Default: 10000000 (Oracle) or 100000 (PostgreSQL) |
| --disable-pb | Use this argument to disable progress bar or statistics during data import. Default: false Accepted parameters: true, false, yes, no, 0, 1 |
| -e, ‑‑export‑dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file. |
| ‑‑source‑replica‑db‑host | Domain name or IP address of the machine on which source-replica database server is running. Default: 127.0.0.1 |
| ‑‑source‑replica‑db‑name | Name of the database in the source-replica database on which import needs to be done. |
| ‑‑source‑replica‑db‑password | Password to connect to the source-replica database. Alternatively, you can also specify the password by setting the environment variable SOURCE_REPLICA_DB_PASSWORD. If you don't provide a password via the CLI or environment variable during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes. |
| --source-replica-db-port | Port number of the source-replica database server. Default: 1521 (Oracle) |
| --source-replica-db-schema | Schema name of the source-replica database. |
| --source-replica-db-sid | Oracle System Identifier (SID) that you wish to use while importing data to Oracle instances. Oracle migrations only. |
| --source-replica-db-user | Username to connect to the source-replica database. |
| --source-replica-ssl-cert | Source-replica database SSL Certificate path. |
| --source-replica-ssl-crl | Source-replica database SSL Root Certificate Revocation List (CRL). |
| --source-replica-ssl-key | Source-replica database SSL key path. |
| --source-replica-ssl-mode | One of disable, allow, prefer(default), require, verify-ca, or verify-full. |
| --source-replica-ssl-root-cert | Source-replica database SSL Root Certificate path. |
| -h, --help | Command line help for import data to source-replica. |
| --oracle-home | Path to set $ORACLE_HOME environment variable. tnsnames.ora is found in $ORACLE_HOME/network/admin. Oracle migrations only. |
| --oracle-tns-alias | TNS (Transparent Network Substrate) alias configured to establish a secure connection with the server. Oracle migrations only. |
| --parallel-jobs | The number of parallel batches issued to the source-replica database. Default: 16 (Oracle); or, if yb-voyager can determine the total number of cores N, then N/2, otherwise 8 (PostgreSQL) |
| --start-clean | Starts a fresh import with data files present in the data directory.If the target YugabyteDB database has any non-empty tables, you are prompted to continue the import without truncating those tables; if you proceed without truncating, then yb-voyager starts ingesting the data present in the data files in non-upsert mode. Note that for cases where a table doesn't have a primary key, duplicate data may be inserted. You can avoid duplication by excluding the table using --exclude-table-list, or by truncating those tables manually before using the start-clean flag. Default: false Accepted parameters: true, false, yes, no, 0, 1 |
| --send-diagnostics | Enable or disable sending diagnostics information to Yugabyte. Default: true Accepted parameters: true, false, yes, no, 0, 1 |
| -y, --yes | Answer yes to all prompts during the migration. Default: false |
Example
yb-voyager import data to source-replica --export-dir /dir/export-dir \
--source-replica-db-host 127.0.0.1 \
--source-replica-db-user ybvoyager \
--source-replica-db-password 'password' \
--source-replica-db-name source_replica_db \
--source-replica-db-schema source_replica_schema \
--parallel-jobs 12