21: Product Overview

Owned by Michael Litner
Oct 20, 2024
10 min read

The Data Virtualization Module lets you create and manage virtual databases (VDBs) that are used by data consumer users (e.g. testers, developers, and data analysts). Before data consumer users can use VDBs, it is necessary for the admin-user to do the following operations:

  1. Add Source Host: define the connection to the server that hosts the source databases from which VDBs are created.

  2. Add Test Data Environment: define access to the source database that resides on the source host and create a golden image (GI).

  3. Add Target Host: define the connection to the server that hosts the VDBs.

  4. Add DB Home: define access to the target database that resides on the target host from which VDBs are created.

  5. Create (master) VDB: create and start a VDB on a golden image snapshot.

  6. Assign the VDB to the data consumer users.

Source Host

A source host is a server that hosts the source databases. To access the source databases, it is necessary to configure the connection to the source host. The following operations can be done on a source host object:

Only an admin-user can do operations on a source host object.

  1. Add a source host, i.e. configure the connection to the source host.

  2. See info about a source host.

  3. Remove a source host.

For more information, see 21: Oracle Source Management.

For more information, see 21: PostgreSQL Source Management.

For more information, see 21: MS-SQL Source Management.

Source Database

A source database can be a production database, a Data Guard Standby database, or a source database replica (for a LIVE GI) that resides on a source host.

Source databases are created and managed with the test data environment object.

Golden Image

A golden image is a full synchronized copy of the source database files. A golden image is used to create VDBs. Golden images reside on the Accelario server.

Golden images are always created on the primary storage pool.

There are several types of golden images:

  • LIVE GI – a golden image that is created and synchronized from a source DB replica. Accelario Replication is usually used to create the source DB replica. The LIVE GI is continuously synchronized.

 

  • RMAN GI – a golden image that is created and synchronized using Oracle Recovery Manager (RMAN) incremental forever backup mechanism. RMAN incremental backups can be automatically scheduled or a GI Refresh operation is used to do a manual backup.

 

  • RMAN STANDBY GI – a golden image that is created and synchronized using RMAN incremental forever backup mechanism from a Data Guard standby source database. RMAN incremental backups can be automatically scheduled or a GI Refresh operation is used to do a manual backup.

  • RDS AWS RMAN GI
    The RDS AWS RMAN GI is exclusively for Oracle databases hosted on Amazon RDS. It combines the features of RMAN with optimizations for AWS cloud infrastructure.

  • Native SQL Server Backup GI - provide automatic initial load and refresh using the customer Native SQL Backup files. The initial load and subsequent restores are scheduled automatically by the system.

  • AWS RDS MSSQL GI

    Provided for MS SQL Server databases running on Amazon RDS.

 

Golden Image Snapshots

GI snapshot (GI snap) – a read/write point-in-time copy of the golden image. It uses minimum storage space because the unchanged data is accessed directly from the golden image. A GI snap can be created manually or can be scheduled.

Managing GI snapshots for a RMAN GI and a RMAN Standby GI

  • At the end of a successful refresh operation, a snapshot is automatically added.

  • A refresh operation is done each time a scheduled snapshot is added. Before the snapshot is added, the system automatically does an RMAN incremental backup operation.

Test Data Environment

A test data environment includes the access definition to one source database and a GI. The following operations can be done on a test data environment object.

  1. Create test data environment – a wizard to make a test data environment:

    1. Select the golden image type

    2. Select the source host

    3. Configure the GI snapshot policy – activate snapshots, schedule a start point and the interval to make snapshots, plus set the snapshot retention policy

    4. Define access to a source database

    5. For a RMAN GI and a RMAN Standby GI, select the Refresh immediately checkbox option to make an initial full copy of the source database files to the golden image and activate it (recommended)

  2. Modify test data environment parameters.

  3. See test data environment source database info.

  4. Remove test data environment.

5. Refresh GI.

For more information, see 21: Oracle Initial Setup | To create a test data environment:

For more information, see 21: PostgreSQL Initial Setup | To create a test data environment:

For more information, see 21: MS-SQL Initial Setup | To create a test data environment:

GI Management

After a golden image is made and started with the Create Test Data Environment operation, the golden image is shown in the VDB management work area. The following operations can be done on the three types of GI objects:

  1. Add or remove a snapshot manually.

  2. Create a VDB on a GI point-in-time snapshot.

  3. Modify the GI parameters.

Oracle

Additional operations available to do for a RMAN GI and RMAN Standby GI:

  1. Refresh GI – executing an RMAN incremental recovery. After a successful refresh, a new snapshot is added.

  2. Remove GI object – only a GI that does not have configured VDBs can be removed.

  3. Activate GI - when creating an RMAN GI without selecting Refresh immediately the GI is in deactivated mode. Use this operation to activate it.

Additional operations available to do for a LIVE GI:

  1. Activate LIVE GI – when the LIVE GI is created it cannot be activated automatically. It is only possible to do this after an initial manual copy of the source database is made by the system administrator. Only after the Oracle database files are moved and the source database is up is it possible to activate the LIVE GI.

  2. Deactivate GI - before a GI object can be removed it is necessary to remove its first snapshot. Only after the first snapshot is successfully removed is it possible to deactivate the GI.

PostgreSQL

Additional operations available to do for a LIVE GI:

  1. Activate LIVE GI – when the LIVE GI is created it cannot be activated automatically. It is only possible to do this after an initial manual copy of the source database is made by the system administrator. After the PostgreSQL database files are moved and the source database is up is it possible to activate the LIVE GI.

  2. Deactivate GI - before a GI object can be removed it is necessary to remove its first snapshot. Only after the first snapshot is successfully removed is it possible to deactivate the GI.

MS-SQL

Additional operations available to do for a Native SQL Server Backup GI:

  1. Activate Native SQL Server Backup GI – when the Native SQL Server Backup is created it cannot be activated automatically. It is only possible to do this after an initial manual copy of the source database is made by the system administrator. After the MS-SQL database files are moved and the source database is up is it possible to activate the Native SQL Server Backup.

  2. Deactivate GI - before a GI object can be removed it is necessary to remove its first snapshot. Only after the first snapshot is successfully removed is it possible to deactivate the GI.

Additional operations available to do for a LIVE GI:

  1. Activate LIVE GI – when the LIVE GI is created it cannot be activated automatically. It is only possible to do this after an initial manual copy of the source database is made by the system administrator. After the MS-SQL database files are moved and the source database is up is it possible to activate the LIVE GI.

  2. Deactivate GI - before a GI object can be removed it is necessary to remove its first snapshot. Only after the first snapshot is successfully removed is it possible to deactivate the GI.

Target Host

A target host is a server that hosts the VDBs that are used by the data consumer users. The following operations can be done on a target host:

  1. Add a target host - define access to the target host.

  2. See info about a target host.

  3. Remove a target host.

For more information, see 21: Oracle Target Management.

For more information, see 21: PostgreSQL Target Management.

For more information, see 21: MS-SQL Target Management.

DB Home

A DB Home is the target database that is used to initiate VDB instances. A target database is defined by its DB Home. The following operations can be done on a DB Home object:

  1. Add a DB Home - set the parameters to access to the target database.

  2. See info about a target host.

  3. Remove a target host.

For more information, see 21: Oracle Initial Setup | To add a DB Home:

For more information, see 21: PostgreSQL Initial Setup | To add a DB Home:

For more information, see 21: MS-SQL Initial Setup | To add a DB Home:

Managing VDBs

VDBs are the primary function to use the Data Virtualization Module. Because VDBs use snapshots as their database file containers they use minimal space and can be created quickly (usually in a few minutes).

The following operations can be done on a VDB object:

  1. Create VDB – a VDB can be created either on a golden image snapshot (we refer to it as master VDB) or on a VDB snapshot (we refer to it as child VDB). When a VDB is created, it is necessary to set the following parameters:

    1. Start Immediately – after the VDB is created it will be started automatically and can be used by data consumer users

    2. Start as RAC – start the VDB as a RAC instance

    3. Target DB Home – select the target database that the VDB will executed from by selecting the required DB Home object

    4. Snapshot Policy – activate snapshots, schedule a start point and the interval to make snapshots, plus set the snapshot retention policy

    5. Advanced Parameters – ability to define Oracle specific parameters and pre/post scripts

  2. Start/Stop VDB – a VDB can be easily started or stopped.

  3. Recover VDB to any point-in-time snapshot – a VDB can be rolled forward/backward to an existing point-in-time snapshot. This operation can be executed only on a stopped VDB. When rolling backwards, all snapshots after the specified snapshot is deleted.

  4. Recover to first point-in-time snapshot – used to roll backward to the original VDB point-in-time (i.e. the time that it was created). All snapshots except the first one will be deleted.

  5. Add/remove snapshot manually.

  6. Modify VDB parameters.

  7. See info about the VDB.

  8. Remove VDB object – only stopped VDBs that do not have child VDBs can be removed.

For more information, see 21: Oracle VDB Management.

For more information, see 21: PostgreSQL VDB Management.

For more information, see 21: MS-SQL VDB Management.

Shared Snapshots

Non-admin users can share snapshots with other non-admin users that belong to a different User Role. It is permitted only for User Roles that have authorization to the same Test Data Environment. Shared Snapshots are used to share a point-in-time copy of a VDB between different uses groups (e.g. Dev and QA) for development and testing purposes.

The owner of a snapshot can do the following operations:

  1. Share - share any of its GI/VDB snapshot with another User Role

  2. Unshare - unshare its own shared snapshot

The receiver of a snapshot can do the following operations:

  1. Create VDB - create VDB on the shared snapshot that was received

  2. Unshare - unshare a shared snapshot that was received

For more information, see 21: Oracle Sharing Snapshots.

For more information, see 21: PostgreSQL Sharing Snapshots.

For more information, see 21: MS-SQL Sharing Snapshots.

Managing Duplicates

Duplicates are a full point-in-time copy of the golden image or of a VDB file container. It uses the full capacity of the source database. It is treated as a mutable golden image with one snapshot from which VDBs and duplicates are created. Duplicates reside on the Accelario server.

The following operations can be done on a duplicate object:

  1. Create a Duplicate – from any GI, VDB, or duplicate snapshot.

  2. Modify Duplicate – modify all the GI parameters.

  3. See info about the Duplicate.

  4. Remove Duplicate object – only a GI that does not have configured VDBs can be removed.

Users and Roles

The Data Virtualization Module uses a role-based user management system. Users and roles are divided into the following categories:

Admin user - can manage all the system resources including source hosts, test data environments, target hosts, target databases, and VDBs. An admin-user can also do user management, storage pool management, monitoring, and troubleshooting.

Non-admin user – can manage only the VDBs to which they have been assigned.

User authentication – an admin-user and a non-admin user can be authenticated locally or remotely with Active Directory.

User Authorization – a role-based mechanism is used to assign resources for a non-admin user. A non-admin user is authorized to use and manage VDBs defined on a group of assigned golden images and assigned target hosts. A non-admin user is only permitted to do the operations on the VDBs to which they have been assigned.

The following operations can be done on users and roles:

  • Create and modify users

  • Create and modify roles

For more information, see 21: Users Management.

Storage Pool Management

Storage pools are the storage containers of the Accelario server. These storage pools store the golden images and duplicates.

It is necessary for the administrator to create a primary storage pool to store all the golden images and duplicates. The administrator can also create a secondary storage pool to store all the duplicates. The secondary storage pool can be configured on a different storage device, thus it can optimize storage performance.

The following operations can be done on a target host:

  • Create a secondary storage pool.

  • Expand storage pool.

  • Reduce storage pool.

  • See info about the storage pool.

  • Remove (secondary) storage pool.

For more information, see 21: Storage Pool Management.

System Setup

The System Setup is used to define the system setup such as SMTP, Active Directory, etc. The following operations can be done in the System Setup window:

  • Setup the Active Directory Authorization.

  • Configure the SMTP.

  • Configure Oracle advanced parameters.

  • Configure Accelario Server Host/IP.

For more information, see 21: System Setup.

Event Viewer

Used to see and save all user events. The following operations can be done on Events:

  • See, filter, and search all user events.

  • Save all user events to a file.

For more information, see https://accelario.atlassian.net/wiki/pages/createpage.action?spaceKey=DVPD&title=21%3AEvent%20Viewer&linkCreation=true&fromPageId=2651652125

Â