MM_30482 [v2] DataOperator for all the isolated tables (#5182)

* MM_30475 : ADDED default schema

* MM_30475 : ADDED todo for field 'value' of default/Global entity

* MM_30476 : Created schema for SERVER DB

* MM_30476 : Server model [ IN PROGRESS ]

* MM_30476 : Including types for group, groups_in_channel and role

* MM_30476 : ADDED models for Group

- @typings absolute path has been added to the tsconfig.json

* MM_30476 : ADDED typings to current models

* MM_30476 : ADDED typings to current models

* MM_30476 : ADDED models related to TEAM section of the ERD

* MM_30476 : ADDED models for User section of the ERD

* MM_30476 : ADDED models for POST section of the ERD

* MM_30476 : ADDED models for Channel section of the ERD

* MM_30475 : Updated typings and references to MM_TABLES

* MM_30476 : Verified all field names

* MM_30476 : Verified every table associations

* MM_30476 : Verified all relation fields

* MM_30476 : Updated primary id of the main models

We will override the wdb id at component level when we create a new records.  This involves the models : channel, group, post, team and user.

* MM_30476 : Including 1:1 relationship amongs some entities

* MM_30476 : ADDED Schema Managers

* The migration array will hold all the migration steps.

*  The initial app release (e.g. v2 )will have an empty array and subsequent releases  (e.g.  v2.1 ) will have the steps listed in that array.

* On initialization, the database will perform the migration to accomodate for new columns/tables creation and while it will conserve the mobile phone's data, it will also make it conform to this new schema.

* If a migration fails, the migration process will rollback any changes.  This migration will be thoroughly tested in development before pushing it live.

* Revert "MM_30476 : ADDED Schema Managers"

This reverts commit a505bd5e11.

* MM_30478 : Converted schema_manager into a function

* MM_30478 : Updated schema manager and included patch for wdb

* MM_30478:  Updated watermelondb patch package

* MM_30478 : Update function create_schema_manager to createSqliteAdaptorOptions

* MM_30476 : Update constant name to reflect directory name

* MM_30476 : Updated msgCount from my_channel model to message_count in server schema

* MM_30482 : Added tests for schema_manager

* MM_30482 : Database Manager [ IN PROGRESS ]

* MM_30478 : Returning an sqliteAdapter instead of an object

* MM_30476 : Apply suggestions from code review

Co-authored-by: Elias Nahum <nahumhbl@gmail.com>

* MM_30476 : Updated all imports as per instruction.

* MM_30476 : Shortening object chains by destructuring

* MM_30476 : Updated schema file structure

* MM_30476 : Prettifying @typings folder

* MM_30476 : Removing useless ids

* MM_30476 : Prettify imports for decorators

* MM_30476 : ADDED documentations and lazy queries to Channel and Channel_Info

* MM_30476 : ADDED documentations for default schema

* MM_30476 : Documentation [ IN PROGRESS ]

- Following JSDoc syntax for single line comment
- Removed redundant fields in the 'membership' tables and left only the @relation records.

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS]

Updated
1)  my_team and team,
2) my_channel and  channel,

to each have 1:1 relationship  with one another

* MM_30476 : Updated all Typescript definitions

* MM_30476 :Updated @relation to @immutableRelation

* MM_30476 : Updated description for previous_post_id

* MM_30478 : Updated patch package for wdb module

* MM_30478: DB Manager [IN PROGRESS ]

* MM_30478: DB Manager [IN PROGRESS]

* MM_30478: DB Manager [IN PROGRESS]

* MM_30478 : DB Manager [IN PROGRESS]

* MM_30478 : Deleting .db file on iOS

* MM_30478: Successfully deleting .db files and directory on iOS side

* MM_30478 : Update definition for default/global

* MM_30478 : Updated all models

* MM_30478 : Doing a bit of house cleaning

* MM_30478: Record of new server connection  added to default/servers db

* TS Definitely Typed Assignment issue is now FIXED

* MM_30478 : TS Definitely Typed Assignment \n  Removed all the constructors but error still in editor tabs.  But this time the app is not crashing

* MM_30478 : Attempt 1 [SUCCESSFUL]

* MM_30478 : Removing useDefineForClassFields

* MM_30478 : Retrieving the servers in a list + Improved the DB Manager and Babel config

* MM_30478 : Updated babel.config.js

* MM_30478 : Minor UI correction

* MM_30478 : Jest and Typescript configuration

* MM_30478 : A bit of housekeeping

* MM_30478 : Installed WDB on Android

* MM_30478 : Deletes new server record from default DB

* MM_30478 : Returns subset of server db instances

* MM_30478 : Code clean up

* MM_30478 :  Code clean up on db manager

* MM_30478 : House keeping + Patch for WDB

* MM_30478 : Android - Saving & Deleting in FilesDir [COMPLETED]

* MM_30478 : Code clean up

* MM_30478 : Code clean up

* MM_30478 : Code clean up

* MM_30478 : Test successful on Android device

* MM_30478 : Rolling back change to jest.config.js

* MM_30478 : Updated test to test_integration

* MM_30478 : Fix imports

* MM_30478 : Refactored the manual testscript

* MM_30478 : Renamed database manager test file

* MM_30478 : Code clean up

* MM_30478 : Updated manual test file with a note.

* MM_30482 : DataOperator [ IN PROGRESS ]

* MM_30482 : DataOperator - setting up the factory [ IN PROGRESS ]

* MM_30482: Code refactoring

* MM_30482 : DataOperator - setting up the factory [ IN PROGRESS ]

* MM_30482 : DataOperator - code clean up [ IN PROGRESS ]

* MM_30482 : Minor code clean up

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Implementing JEST test cases

* MM_30478 : Implementing JEST last  test cases

* MM_30478 : Jest fixing ts errors

* MM_30478 : Database Manager Jest testing [ IN PROGRESS ]

* MM_30482 - Fixing DataOperator [ IN PROGRESS ]

* MM_30482 : Code clean up

* MM_30482 - Creates multiple records [ IN PROGRESS ]

* MM_30482 - Creates multiple records [ IN PROGRESS ]

* MM_30482 : Update operation [ COMPLETED ]

* MM_30482 : Code clean up

* MM_30482 : Updated TS for Data Operator

* Update mobile v2 detox deps

* MM_30482 : Added factories for all isolated tables

* MM_30482 : Refactored TS

* MM_30482 : Refactored base factory

* MM_30482 : Updated JSDoc for operateBaseRecord - Delete CASE

* MM_30482 : Implementing test for Data Operator

* MM_30482 : Completed tests for all isolated tables

* MM_30482 : Renamed entity_factory into operators

* MM_30482 : Fix all imports

* MM_30482 : Update multiple records

* MM_30482 : Edge case for existing records ( update instead of create )

* MM_30482 : Edge case  - create instead of update

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* MM_30482 : Imposing usage of correct table name for isolated entities

* MM_30482 : Code improvement as per Joseph reviews

* MM_30482 : Updated tests to validate choice of operator service wrt tableName

* MM_30482 : Updated PR as per suggestions

* MM_30482 : Updated comments to follow jsdoc conventions

Co-authored-by: Elias Nahum <nahumhbl@gmail.com>
Co-authored-by: Avinash Lingaloo <>
Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

app/database/admin/data_operator/index.ts

@@ -0,0 +1,183 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+import {MM_TABLES} from '@constants/database';
+import {
+    BatchOperations,
+    DBInstance,
+    HandleBaseData,
+    HandleIsolatedEntityData,
+    RecordValue,
+} from '@typings/database/database';
+
+import DatabaseManager from '../database_manager';
+
+import {
+    operateAppRecord,
+    operateCustomEmojiRecord,
+    operateGlobalRecord,
+    operateRoleRecord,
+    operateServersRecord,
+    operateSystemRecord,
+    operateTermsOfServiceRecord,
+} from './operators';
+
+export enum OperationType {
+    CREATE = 'CREATE',
+    UPDATE = 'UPDATE',
+    DELETE = 'DELETE'
+}
+
+export enum IsolatedEntities {
+    APP= 'app',
+    GLOBAL = 'global',
+    SERVERS = 'servers',
+    CUSTOM_EMOJI = 'CustomEmoji',
+    ROLE = 'Role',
+    SYSTEM = 'System',
+    TERMS_OF_SERVICE = 'TermsOfService'
+}
+
+class DataOperator {
+    private defaultDatabase: DBInstance;
+    private serverDatabase: DBInstance;
+
+    /**
+     * handleIsolatedEntityData: Operator that handles Create/Update operations on the isolated entities as
+     * described by the IsolatedTables type
+     * @param {HandleIsolatedEntityData} entityData
+     * @param {OperationType} entityData.optType
+     * @param {IsolatedEntities} entityData.tableName
+     * @param {Records} entityData.values
+     * @returns {Promise<void>}
+     */
+    handleIsolatedEntityData = async ({optType, tableName, values}: HandleIsolatedEntityData): Promise<void> => {
+        let recordOperator;
+
+        switch (tableName) {
+        case IsolatedEntities.APP : {
+            recordOperator = operateAppRecord;
+            break;
+        }
+        case IsolatedEntities.GLOBAL : {
+            recordOperator = operateGlobalRecord;
+            break;
+        }
+        case IsolatedEntities.SERVERS : {
+            recordOperator = operateServersRecord;
+            break;
+        }
+        case IsolatedEntities.CUSTOM_EMOJI : {
+            recordOperator = operateCustomEmojiRecord;
+            break;
+        }
+        case IsolatedEntities.ROLE : {
+            recordOperator = operateRoleRecord;
+            break;
+        }
+        case IsolatedEntities.SYSTEM : {
+            recordOperator = operateSystemRecord;
+            break;
+        }
+        case IsolatedEntities.TERMS_OF_SERVICE : {
+            recordOperator = operateTermsOfServiceRecord;
+            break;
+        }
+        default: {
+            recordOperator = null;
+            break;
+        }
+        }
+        if (recordOperator) {
+            await this.handleBaseData({optType, values, tableName, recordOperator});
+        }
+    };
+
+    /**
+     * batchOperations: Accepts an instance of Database (either Default or Server) and an array of
+     * prepareCreate/prepareUpdate values and executes the actions on the database.
+     * @param {BatchOperations} operation
+     * @param {Database} operation.db
+     * @param {Array} operation.models
+     * @returns {Promise<void>}
+     */
+    private batchOperations = async ({db, models}: BatchOperations) => {
+        if (models.length > 0) {
+            await db.action(async () => {
+                await db.batch(...models);
+            });
+        }
+    };
+
+    /**
+     * handleBaseData: Handles the Create/Update operations on an entity.
+     * @param {HandleBaseData} opsBase
+     * @param {OperationType} opsBase.optType
+     * @param {string} opsBase.tableName
+     * @param {Records} opsBase.values
+     * @param {(recordOperator: DataFactory) => void} opsBase.recordOperator
+     * @returns {Promise<void>}
+     */
+    private handleBaseData = async ({optType, tableName, values, recordOperator}: HandleBaseData) => {
+        const db = await this.getDatabase(tableName);
+        if (!db) {
+            return;
+        }
+
+        let results;
+        const config = {db, optType, tableName};
+
+        if (Array.isArray(values) && values.length) {
+            const recordPromises = await values.map(async (value) => {
+                const record = await recordOperator({...config, value});
+                return record;
+            });
+
+            results = await Promise.all(recordPromises);
+        } else {
+            results = await recordOperator({...config, value: values as RecordValue});
+        }
+
+        if (results) {
+            await this.batchOperations({db, models: Array.isArray(results) ? results : Array(results)});
+        }
+    };
+
+    /**
+     * getDatabase: Based on the table's name, it will return a database instance either from the 'DEFAULT' database or
+     * the 'SERVER' database.
+     * @param {string} tableName
+     * @returns {Promise<DBInstance>}
+     */
+    private getDatabase = async (tableName: string): Promise<DBInstance> => {
+        const isInDefaultDB = Object.values(MM_TABLES.DEFAULT).some((tbName) => {
+            return tableName === tbName;
+        });
+
+        if (isInDefaultDB) {
+            return this.defaultDatabase || this.getDefaultDatabase();
+        }
+
+        return this.serverDatabase || this.getServerDatabase();
+    };
+
+    /**
+     * getDefaultDatabase: Returns the default database
+     * @returns {Promise<DBInstance>}
+     */
+    private getDefaultDatabase = async () => {
+        this.defaultDatabase = await DatabaseManager.getDefaultDatabase();
+        return this.defaultDatabase;
+    };
+
+    /**
+     * getServerDatabase: Returns the current active server database (multi-server support)
+     * @returns {Promise<DBInstance>}
+     */
+    private getServerDatabase = async () => {
+        this.serverDatabase = await DatabaseManager.getActiveServerDatabase();
+        return this.serverDatabase;
+    };
+}
+
+export default new DataOperator();

app/database/admin/data_operator/operators.ts

@@ -0,0 +1,210 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+import {MM_TABLES} from '@constants/database';
+import {Q} from '@nozbe/watermelondb';
+import Model from '@nozbe/watermelondb/Model';
+
+import App from '@typings/database/app';
+import CustomEmoji from '@typings/database/custom_emoji';
+import {
+    DataFactory,
+    RawApp,
+    RawCustomEmoji,
+    RawGlobal,
+    RawRole,
+    RawServers,
+    RawSystem,
+    RawTermsOfService,
+} from '@typings/database/database';
+import Global from '@typings/database/global';
+import Role from '@typings/database/role';
+import Servers from '@typings/database/servers';
+import System from '@typings/database/system';
+import TermsOfService from '@typings/database/terms_of_service';
+
+import {OperationType} from './index';
+
+const {APP, GLOBAL, SERVERS} = MM_TABLES.DEFAULT;
+const {CUSTOM_EMOJI, ROLE, SYSTEM, TERMS_OF_SERVICE} = MM_TABLES.SERVER;
+
+/**
+ * operateAppRecord: Prepares record of entity 'App' from the DEFAULT database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateAppRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawApp;
+
+    const generator = (app: App) => {
+        app._raw.id = record?.id ?? app.id;
+        app.buildNumber = record?.buildNumber ?? '';
+        app.createdAt = record?.createdAt ?? 0;
+        app.versionNumber = record?.versionNumber ?? '';
+    };
+
+    return operateBaseRecord({db, optType, tableName: APP, value, generator});
+};
+
+/**
+ * operateGlobalRecord: Prepares record of entity 'Global' from the DEFAULT database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateGlobalRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawGlobal;
+
+    const generator = (global: Global) => {
+        global._raw.id = record?.id ?? global.id;
+        global.name = record?.name ?? '';
+        global.value = record?.value ?? 0;
+    };
+
+    return operateBaseRecord({db, optType, tableName: GLOBAL, value, generator});
+};
+
+/**
+ * operateServersRecord: Prepares record of entity 'Servers' from the DEFAULT database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateServersRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawServers;
+
+    const generator = (servers: Servers) => {
+        servers._raw.id = record?.id ?? servers.id;
+        servers.dbPath = record?.dbPath ?? '';
+        servers.displayName = record?.displayName ?? 0;
+        servers.mentionCount = record?.mentionCount ?? 0;
+        servers.unreadCount = record?.unreadCount ?? 0;
+        servers.url = record?.url ?? 0;
+    };
+
+    return operateBaseRecord({db, optType, tableName: SERVERS, value, generator});
+};
+
+/**
+ * operateCustomEmojiRecord: Prepares record of entity 'CustomEmoji' from the SERVER database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateCustomEmojiRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawCustomEmoji;
+
+    const generator = (emoji: CustomEmoji) => {
+        emoji._raw.id = record?.id ?? emoji.id;
+        emoji.name = record?.name ?? '';
+    };
+
+    return operateBaseRecord({db, optType, tableName: CUSTOM_EMOJI, value, generator});
+};
+
+/**
+ * operateRoleRecord: Prepares record of entity 'Role' from the SERVER database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateRoleRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawRole;
+
+    const generator = (role: Role) => {
+        role._raw.id = record?.id ?? role.id;
+        role.name = record?.name ?? '';
+        role.permissions = record?.permissions ?? [];
+    };
+
+    return operateBaseRecord({db, optType, tableName: ROLE, value, generator});
+};
+
+/**
+ * operateSystemRecord: Prepares record of entity 'System' from the SERVER database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateSystemRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawSystem;
+
+    const generator = (system: System) => {
+        system._raw.id = record?.id ?? system.id;
+        system.name = record?.name ?? '';
+        system.value = record?.value ?? '';
+    };
+
+    return operateBaseRecord({db, optType, tableName: SYSTEM, value, generator});
+};
+
+/**
+ * operateTermsOfServiceRecord: Prepares record of entity 'TermsOfService' from the SERVER database for update or create actions.
+ * @param {DataFactory} operator
+ * @param {Database} operator.db
+ * @param {OperationType} operator.optType
+ * @param {RecordValue} operator.value
+ * @returns {Promise<void>}
+ */
+export const operateTermsOfServiceRecord = async ({db, optType, value}: DataFactory) => {
+    const record = value as RawTermsOfService;
+
+    const generator = (tos: TermsOfService) => {
+        tos._raw.id = record?.id ?? tos.id;
+        tos.acceptedAt = record?.acceptedAt ?? 0;
+    };
+
+    return operateBaseRecord({db, optType, tableName: TERMS_OF_SERVICE, value, generator});
+};
+
+/**
+ * operateBaseRecord:  The 'id' of a record is key to this function. Please note that - at the moment - if WatermelonDB
+ * encounters an existing record during a CREATE operation, it silently fails the operation.
+ *
+ * In our case, we check to see if we have an existing 'id' and if so, we'll update the record with the data.
+ * For an UPDATE operation, we fetch the existing record using the 'id' value and then we do the update operation;
+ * if no record is found for that 'id', we'll create it a new record.
+ *
+ * @param {DataFactory} operatorBase
+ * @param {Database} operatorBase.db
+ * @param {OperationType} operatorBase.optType
+ * @param {string} operatorBase.tableName
+ * @param {any} operatorBase.value
+ * @param {((model: Model) => void)} operatorBase.generator
+ * @returns {Promise<any>}
+ */
+const operateBaseRecord = async ({db, optType, tableName, value, generator}: DataFactory) => {
+    // We query first to see if we have a record on that entity with the current value.id
+    const appRecord = await db.collections.get(tableName!).query(Q.where('id', value.id)).fetch() as Model[];
+    const isPresent = appRecord.length > 0;
+
+    if ((isPresent && optType === OperationType.CREATE) || (isPresent && optType === OperationType.UPDATE)) {
+        // Two possible scenarios:
+        // 1. We are dealing with either duplicates here and if so, we'll update instead of create
+        // 2. This is just a normal update operation
+        const record = appRecord[0];
+        return record.prepareUpdate(() => generator!(record));
+    }
+
+    if ((!isPresent && optType === OperationType.UPDATE) || (optType === OperationType.CREATE)) {
+        // Two possible scenarios
+        // 1. We don't have a record yet to update; so we create it
+        // 2. This is just a normal create operation
+        return db.collections.get(tableName!).prepareCreate(generator);
+    }
+
+    return null;
+};

app/database/admin/data_operator/test.ts

@@ -0,0 +1,481 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+import {MM_TABLES} from '@constants/database';
+import {Q} from '@nozbe/watermelondb';
+import App from '@typings/database/app';
+
+import DatabaseManager, {DatabaseType} from '../database_manager';
+import DataOperator, {IsolatedEntities, OperationType} from './index';
+import {
+    operateAppRecord,
+    operateCustomEmojiRecord,
+    operateGlobalRecord,
+    operateRoleRecord,
+    operateServersRecord,
+    operateSystemRecord,
+    operateTermsOfServiceRecord,
+} from './operators';
+
+jest.mock('../database_manager');
+
+const {APP} = MM_TABLES.DEFAULT;
+
+describe('*** Data Operator tests ***', () => {
+    it('=> should return an array of type App for operateAppRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.getDefaultDatabase();
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateAppRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {buildNumber: 'build-7', createdAt: 1, id: 'id-18', versionNumber: 'v-1'},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('App');
+    });
+
+    it('=> should return an array of type Global for operateGlobalRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.getDefaultDatabase();
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateGlobalRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {id: 'g-1', name: 'g-n1', value: 'g-v1'},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('Global');
+    });
+
+    it('=> should return an array of type Servers for operateServersRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.getDefaultDatabase();
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateServersRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {
+                dbPath: 'mm-server',
+                displayName: 's-displayName',
+                id: 's-1',
+                mentionCount: 1,
+                unreadCount: 0,
+                url: 'https://community.mattermost.com',
+            },
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('Servers');
+    });
+
+    it('=> should return an array of type CustomEmoji for operateCustomEmojiRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.createDatabaseConnection({
+            shouldAddToDefaultDatabase: true,
+            databaseConnection: {
+                actionsEnabled: true,
+                dbName: 'community mattermost',
+                dbType: DatabaseType.SERVER,
+                serverUrl: 'https://appv2.mattermost.com',
+            },
+        });
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateCustomEmojiRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {id: 'emo-1', name: 'emoji'},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('CustomEmoji');
+    });
+
+    it('=> should return an array of type Role for operateRoleRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.createDatabaseConnection({
+            shouldAddToDefaultDatabase: true,
+            databaseConnection: {
+                actionsEnabled: true,
+                dbName: 'community mattermost',
+                dbType: DatabaseType.SERVER,
+                serverUrl: 'https://appv2.mattermost.com',
+            },
+        });
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateRoleRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {id: 'role-1', name: 'role-name-1', permissions: []},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('Role');
+    });
+
+    it('=> should return an array of type System for operateSystemRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.createDatabaseConnection({
+            shouldAddToDefaultDatabase: true,
+            databaseConnection: {
+                actionsEnabled: true,
+                dbName: 'community mattermost',
+                dbType: DatabaseType.SERVER,
+                serverUrl: 'https://appv2.mattermost.com',
+            },
+        });
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateSystemRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {id: 'system-1', name: 'system-name-1', value: 'system'},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('System');
+    });
+
+    it('=> should return an array of type TermsOfService for operateTermsOfServiceRecord', async () => {
+        expect.assertions(3);
+
+        const db = await DatabaseManager.createDatabaseConnection({
+            shouldAddToDefaultDatabase: true,
+            databaseConnection: {
+                actionsEnabled: true,
+                dbName: 'community mattermost',
+                dbType: DatabaseType.SERVER,
+                serverUrl: 'https://appv2.mattermost.com',
+            },
+        });
+        expect(db).toBeTruthy();
+
+        const preparedRecords = await operateTermsOfServiceRecord({
+            db: db!,
+            optType: OperationType.CREATE,
+            value: {id: 'system-1', acceptedAt: 1},
+        });
+
+        expect(preparedRecords).toBeTruthy();
+        expect(preparedRecords!.collection.modelClass.name).toMatch('TermsOfService');
+    });
+
+    it('=> should create a record in the App table in the default database', async () => {
+        expect.assertions(2);
+
+        // Creates a record in the App table
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.APP,
+            values: {buildNumber: 'build-1', createdAt: 1, id: 'id-1', versionNumber: 'version-1'},
+        });
+
+        // Do a query and find out if the value has been registered in the App table of the default database
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', 'id-1')).fetch() as App[];
+
+        // We should expect to have a record returned as dictated by our query
+        expect(records.length).toBe(1);
+    });
+
+    it('=> should create several records in the App table in the default database', async () => {
+        expect.assertions(2);
+
+        // Creates a record in the App table
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.APP,
+            values: [
+                {buildNumber: 'build-10', createdAt: 1, id: 'id-10', versionNumber: 'version-10'},
+                {buildNumber: 'build-11', createdAt: 1, id: 'id-11', versionNumber: 'version-11'},
+                {buildNumber: 'build-12', createdAt: 1, id: 'id-12', versionNumber: 'version-12'},
+                {buildNumber: 'build-13', createdAt: 1, id: 'id-13', versionNumber: 'version-13'},
+            ],
+        });
+
+        // Do a query and find out if the value has been registered in the App table of the default database
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', Q.oneOf(['id-10', 'id-11', 'id-12', 'id-13']))).fetch() as App[];
+
+        // We should expect to have 4 records created
+        expect(records.length).toBe(4);
+    });
+
+    it('=> should update a record in the App table in the default database', async () => {
+        expect.assertions(3);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        // Update record having id 'id-1'
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.UPDATE,
+            tableName: IsolatedEntities.APP,
+            values: {buildNumber: 'build-13-13', createdAt: 1, id: 'id-1', versionNumber: 'version-1'},
+        });
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', 'id-1')).fetch() as App[];
+        expect(records.length).toBeGreaterThan(0);
+
+        // Verify if the buildNumber for this record has been updated
+        expect(records[0].buildNumber).toMatch('build-13-13');
+    });
+
+    it('=> should update several records in the App table in the default database', async () => {
+        expect.assertions(4);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        // Update records having id 'id-10' and 'id-11'
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.UPDATE,
+            tableName: IsolatedEntities.APP,
+            values: [
+                {buildNumber: 'build-10x', createdAt: 1, id: 'id-10', versionNumber: 'version-10'},
+                {buildNumber: 'build-11y', createdAt: 1, id: 'id-11', versionNumber: 'version-11'},
+            ],
+        });
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', Q.oneOf(['id-10', 'id-11']))).fetch() as App[];
+        expect(records.length).toBe(2);
+
+        // Verify if the buildNumber for those two record has been updated
+        expect(records[0].buildNumber).toMatch('build-10x');
+        expect(records[1].buildNumber).toMatch('build-11y');
+    });
+
+    it('=> [EDGE CASE] should UPDATE instead of CREATE record for existing id', async () => {
+        expect.assertions(3);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        // id-10 and id-11 exist but yet the optType is CREATE.  The operator should then prepareUpdate the records instead of prepareCreate
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.APP,
+            values: [
+                {buildNumber: 'build-10x', createdAt: 1, id: 'id-10', versionNumber: 'version-10'},
+                {buildNumber: 'build-11x', createdAt: 1, id: 'id-11', versionNumber: 'version-11'},
+            ],
+        });
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', Q.oneOf(['id-10', 'id-11']))).fetch() as App[];
+
+        // Verify if the buildNumber for those two record has been updated
+        expect(records[0].buildNumber).toMatch('build-10x');
+        expect(records[1].buildNumber).toMatch('build-11x');
+    });
+
+    it('=> [EDGE CASE] should CREATE instead of UPDATE record for non-existing id', async () => {
+        expect.assertions(3);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        // id-15 and id-16 do not exist but yet the optType is UPDATE.  The operator should then prepareCreate the records instead of prepareUpdate
+        await DataOperator.handleIsolatedEntityData({
+            optType: OperationType.UPDATE,
+            tableName: IsolatedEntities.APP,
+            values: [
+                {buildNumber: 'build-10x', createdAt: 1, id: 'id-15', versionNumber: 'version-10'},
+                {buildNumber: 'build-11x', createdAt: 1, id: 'id-16', versionNumber: 'version-11'},
+            ],
+        });
+
+        const records = await defaultDB!.collections.get(APP).query(Q.where('id', Q.oneOf(['id-15', 'id-16']))).fetch() as App[];
+
+        // Verify if the buildNumber for those two record has been created
+        expect(records[0].buildNumber).toMatch('build-10x');
+        expect(records[1].buildNumber).toMatch('build-11x');
+    });
+
+    it('=> should use operateAppRecord for APP entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.APP,
+            values: [
+                {buildNumber: 'build-10x', createdAt: 1, id: 'id-21', versionNumber: 'version-10'},
+                {buildNumber: 'build-11y', createdAt: 1, id: 'id-22', versionNumber: 'version-11'},
+            ],
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateAppRecord});
+    });
+
+    it('=> should use operateGlobalRecord for GLOBAL entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.GLOBAL,
+            values: {id: 'global-1-id', name: 'global-1-name', value: 'global-1-value'},
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateGlobalRecord});
+    });
+
+    it('=> should use operateServersRecord for SERVERS entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.SERVERS,
+            values: {
+                dbPath: 'server.db',
+                displayName: 'community',
+                id: 'server-id-1',
+                mentionCount: 0,
+                unreadCount: 0,
+                url: 'https://community.mattermost.com',
+            },
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateServersRecord});
+    });
+
+    it('=> should use operateCustomEmojiRecord for CUSTOM_EMOJI entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.CUSTOM_EMOJI,
+            values: {
+                id: 'custom-emoji-id-1',
+                name: 'custom-emoji-1',
+            },
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateCustomEmojiRecord});
+    });
+
+    it('=> should use operateRoleRecord for ROLE entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.ROLE,
+            values: {
+                id: 'custom-emoji-id-1',
+                name: 'custom-emoji-1',
+                permissions: ['custom-emoji-1'],
+            },
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateRoleRecord});
+    });
+
+    it('=> should use operateSystemRecord for SYSTEM entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.SYSTEM,
+            values: {id: 'system-id-1', name: 'system-1', value: 'system-1'},
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateSystemRecord});
+    });
+
+    it('=> should use operateTermsOfServiceRecord for TERMS_OF_SERVICE entity in handleBaseData', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: IsolatedEntities.TERMS_OF_SERVICE,
+            values: {id: 'tos-1', acceptedAt: 1},
+        };
+
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledWith({...data, recordOperator: operateTermsOfServiceRecord});
+    });
+
+    it('=> should not call handleBaseData if tableName is invalid', async () => {
+        expect.assertions(2);
+
+        const defaultDB = await DatabaseManager.getDefaultDatabase();
+        expect(defaultDB).toBeTruthy();
+
+        const spyOnHandleBase = jest.spyOn(DataOperator as any, 'handleBaseData');
+
+        const data = {
+            optType: OperationType.CREATE,
+            tableName: 'INVALID_TABLE_NAME',
+            values: {id: 'tos-1', acceptedAt: 1},
+        };
+
+        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+        // @ts-ignore
+        await DataOperator.handleIsolatedEntityData(data);
+
+        expect(spyOnHandleBase).toHaveBeenCalledTimes(0);
+    });
+});

app/database/admin/database_manager/__mocks__/index.ts

@@ -2,17 +2,15 @@
 // See LICENSE.txt for license information.
 
 import {Database, Model, Q} from '@nozbe/watermelondb';
-import LokiJSAdapter from '@nozbe/watermelondb/adapters/lokijs';
 import {Class} from '@nozbe/watermelondb/utils/common';
+import LokiJSAdapter from '@nozbe/watermelondb/adapters/lokijs';
 
 import {MM_TABLES} from '@constants/database';
-import type {DBInstance, DefaultNewServer, MMDatabaseConnection} from '@typings/database/database';
 import IServers from '@typings/database/servers';
+import type {DBInstance, DefaultNewServer, MMDatabaseConnection} from '@typings/database/database';
 
 import DefaultMigration from '../../../default/migration';
 import {App, Global, Servers} from '../../../default/models';
-import {defaultSchema} from '../../../default/schema';
-import ServerMigration from '../../../server/migration';
 import {
     Channel,
     ChannelInfo,
@@ -43,6 +41,8 @@ import {
     TermsOfService,
     User,
 } from '../../../server/models';
+import {defaultSchema} from '../../../default/schema';
+import ServerMigration from '../../../server/migration';
 import {serverSchema} from '../../../server/schema';
 
 const {SERVERS} = MM_TABLES.DEFAULT;

app/database/admin/database_manager/index.ts

@@ -1,21 +1,26 @@
 // Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See LICENSE.txt for license information.
 
-import {Database, Model, Q} from '@nozbe/watermelondb';
 import SQLiteAdapter from '@nozbe/watermelondb/adapters/sqlite';
-import {Class} from '@nozbe/watermelondb/utils/common';
+import {Database, Q} from '@nozbe/watermelondb';
 import {DeviceEventEmitter, Platform} from 'react-native';
 import {FileSystem} from 'react-native-unimodules';
 
 import {MIGRATION_EVENTS, MM_TABLES} from '@constants/database';
-import type {DBInstance, DefaultNewServer, MigrationEvents, MMDatabaseConnection} from '@typings/database/database';
 import IServers from '@typings/database/servers';
+import type {
+    ActiveServerDatabase,
+    DBInstance,
+    DatabaseConnection,
+    DefaultNewServer,
+    MigrationEvents,
+    Models,
+} from '@typings/database/database';
 import {deleteIOSDatabase, getIOSAppGroupDetails} from '@utils/mattermost_managed';
 
 import DefaultMigration from '../../default/migration';
 import {App, Global, Servers} from '../../default/models';
 import {defaultSchema} from '../../default/schema';
-import ServerMigration from '../../server/migration';
 import {
     Channel,
     ChannelInfo,
@@ -46,21 +51,11 @@ import {
     TermsOfService,
     User,
 } from '../../server/models';
+import ServerMigration from '../../server/migration';
 import {serverSchema} from '../../server/schema';
 
 const {SERVERS} = MM_TABLES.DEFAULT;
 
-type Models = Class<Model>[]
-
-// The elements needed to create a new connection
-type DatabaseConnection = {
-    databaseConnection: MMDatabaseConnection, shouldAddToDefaultDatabase
-        : boolean
-}
-
-// The elements required to switch to another active server database
-type ActiveServerDatabase = { displayName: string, serverUrl: string }
-
 // The only two types of databases in the app
 export enum DatabaseType {
     DEFAULT,

app/database/admin/database_manager/test_manual.ts

@@ -3,10 +3,10 @@
 
 import {Platform} from 'react-native';
 
-import DBManager, {DatabaseType} from './index';
-
 import {getIOSAppGroupDetails} from '@utils/mattermost_managed';
 
+import DBManager, {DatabaseType} from './index';
+
 export default async () => {
     // Test: It should return the iOS App-Group shared directory
     const testAppGroupDirectory = () => {

app/screens/server/index.tsx

@@ -1,10 +1,10 @@
 // Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See LICENSE.txt for license information.
 
-import React from 'react';
-import {SafeAreaView, ScrollView, StatusBar, StyleSheet, Text, View} from 'react-native';
 import {Colors, DebugInstructions, LearnMoreLinks, ReloadInstructions} from 'react-native/Libraries/NewAppScreen';
+import {SafeAreaView, ScrollView, StatusBar, StyleSheet, Text, View} from 'react-native';
 
+import React from 'react';
 import {Screens} from '@constants';
 import {goToScreen} from '@screens/navigation';
 

types/database/database.d.ts

@@ -1,7 +1,13 @@
 // Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See LICENSE.txt for license information.
-import {Migration} from '@nozbe/watermelondb/Schema/migrations';
+
 import {AppSchema, Database} from '@nozbe/watermelondb';
+import Model from '@nozbe/watermelondb/Model';
+import {Migration} from '@nozbe/watermelondb/Schema/migrations';
+import {Class} from '@nozbe/watermelondb/utils/common';
+
+import {IsolatedEntities, OperationType} from '../../app/database/admin/data_operator';
+import {DatabaseType} from '../../app/database/admin/database_manager';
 
 export type MigrationEvents = {
     onSuccess: () => void,
@@ -10,7 +16,7 @@ export type MigrationEvents = {
 }
 
 export type MMAdaptorOptions = {
-    dbPath : string,
+    dbPath: string,
     schema: AppSchema,
     migrationSteps?: Migration [],
     migrationEvents?: MigrationEvents
@@ -30,4 +36,83 @@ export type DefaultNewServer = {
 }
 
 // A database connection is of type 'Database'; unless it fails to be initialize and in which case it becomes 'undefined'
-type DBInstance = Database | undefined
+export type DBInstance = Database | undefined
+
+export type RawApp = {
+    buildNumber: string,
+    createdAt: number,
+    id: string,
+    versionNumber: string,
+}
+
+export type RawGlobal = {
+    id: string,
+    name: string,
+    value: string,
+}
+
+export type RawServers = {
+    dbPath: string,
+    displayName: string,
+    id: string,
+    mentionCount: number,
+    unreadCount: number,
+    url: string
+}
+
+export type RawCustomEmoji = {
+    id: string,
+    name: string
+}
+
+export type RawRole = {
+    id: string,
+    name: string,
+    permissions: []
+}
+
+export type RawSystem = {
+    id: string,
+    name: string,
+    value: string
+}
+
+export type RawTermsOfService = {
+    id: string,
+    acceptedAt: number
+}
+
+export type RecordValue = RawApp | RawGlobal | RawServers | RawCustomEmoji | RawRole | RawSystem | RawTermsOfService
+
+export type DataFactory = {
+    db: Database,
+    generator?: (model: Model) => void,
+    optType?: OperationType,
+    tableName?: string,
+    value: RecordValue,
+}
+
+export type Records = RecordValue | RecordValue[]
+
+export type HandleBaseData = {
+    optType: OperationType,
+    tableName: string,
+    values: Records,
+    recordOperator: (recordOperator: DataFactory) => void
+}
+
+export type BatchOperations = { db: Database, models: Model[] }
+
+export type HandleIsolatedEntityData = { optType: OperationType, tableName: IsolatedEntities, values: Records }
+
+export type Models = Class<Model>[]
+
+// The elements needed to create a new connection
+export type DatabaseConnection = {
+    databaseConnection: MMDatabaseConnection,
+    shouldAddToDefaultDatabase: boolean
+}
+
+// The elements required to switch to another active server database
+export type ActiveServerDatabase = { displayName: string, serverUrl: string }
+

types/database/index.d.ts

@@ -95,3 +95,4 @@ type PostType = 'system_add_remove' |
     'system_leave_channel' |
     'system_purpose_change' |
     'system_remove_from_channel';
+