Files
mattermost-mobile/app/database/admin/database_manager/index.ts
Avinash Lingaloo 040bf22264 MM_33223 [v2] Database Operator - Post section (#5227)
* MM_30475 : ADDED default schema

* MM_30475 : ADDED todo for field 'value' of default/Global entity

* MM_30476 : Created schema for SERVER DB

* MM_30476 : Server model [ IN PROGRESS ]

* MM_30476 : Including types for group, groups_in_channel and role

* MM_30476 : ADDED models for Group

- @typings absolute path has been added to the tsconfig.json

* MM_30476 : ADDED typings to current models

* MM_30476 : ADDED typings to current models

* MM_30476 : ADDED models related to TEAM section of the ERD

* MM_30476 : ADDED models for User section of the ERD

* MM_30476 : ADDED models for POST section of the ERD

* MM_30476 : ADDED models for Channel section of the ERD

* MM_30475 : Updated typings and references to MM_TABLES

* MM_30476 : Verified all field names

* MM_30476 : Verified every table associations

* MM_30476 : Verified all relation fields

* MM_30476 : Updated primary id of the main models

We will override the wdb id at component level when we create a new records.  This involves the models : channel, group, post, team and user.

* MM_30476 : Including 1:1 relationship amongs some entities

* MM_30476 : ADDED Schema Managers

* The migration array will hold all the migration steps.

*  The initial app release (e.g. v2 )will have an empty array and subsequent releases  (e.g.  v2.1 ) will have the steps listed in that array.

* On initialization, the database will perform the migration to accomodate for new columns/tables creation and while it will conserve the mobile phone's data, it will also make it conform to this new schema.

* If a migration fails, the migration process will rollback any changes.  This migration will be thoroughly tested in development before pushing it live.

* Revert "MM_30476 : ADDED Schema Managers"

This reverts commit a505bd5e11.

* MM_30478 : Converted schema_manager into a function

* MM_30478 : Updated schema manager and included patch for wdb

* MM_30478:  Updated watermelondb patch package

* MM_30478 : Update function create_schema_manager to createSqliteAdaptorOptions

* MM_30476 : Update constant name to reflect directory name

* MM_30476 : Updated msgCount from my_channel model to message_count in server schema

* MM_30482 : Added tests for schema_manager

* MM_30482 : Database Manager [ IN PROGRESS ]

* MM_30478 : Returning an sqliteAdapter instead of an object

* MM_30476 : Apply suggestions from code review

Co-authored-by: Elias Nahum <nahumhbl@gmail.com>

* MM_30476 : Updated all imports as per instruction.

* MM_30476 : Shortening object chains by destructuring

* MM_30476 : Updated schema file structure

* MM_30476 : Prettifying @typings folder

* MM_30476 : Removing useless ids

* MM_30476 : Prettify imports for decorators

* MM_30476 : ADDED documentations and lazy queries to Channel and Channel_Info

* MM_30476 : ADDED documentations for default schema

* MM_30476 : Documentation [ IN PROGRESS ]

- Following JSDoc syntax for single line comment
- Removed redundant fields in the 'membership' tables and left only the @relation records.

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS ]

* MM_30476 : Documentations [ IN PROGRESS]

Updated
1)  my_team and team,
2) my_channel and  channel,

to each have 1:1 relationship  with one another

* MM_30476 : Updated all Typescript definitions

* MM_30476 :Updated @relation to @immutableRelation

* MM_30476 : Updated description for previous_post_id

* MM_30478 : Updated patch package for wdb module

* MM_30478: DB Manager [IN PROGRESS ]

* MM_30478: DB Manager [IN PROGRESS]

* MM_30478: DB Manager [IN PROGRESS]

* MM_30478 : DB Manager [IN PROGRESS]

* MM_30478 : Deleting .db file on iOS

* MM_30478: Successfully deleting .db files and directory on iOS side

* MM_30478 : Update definition for default/global

* MM_30478 : Updated all models

* MM_30478 : Doing a bit of house cleaning

* MM_30478: Record of new server connection  added to default/servers db

* TS Definitely Typed Assignment issue is now FIXED

* MM_30478 : TS Definitely Typed Assignment \n  Removed all the constructors but error still in editor tabs.  But this time the app is not crashing

* MM_30478 : Attempt 1 [SUCCESSFUL]

* MM_30478 : Removing useDefineForClassFields

* MM_30478 : Retrieving the servers in a list + Improved the DB Manager and Babel config

* MM_30478 : Updated babel.config.js

* MM_30478 : Minor UI correction

* MM_30478 : Jest and Typescript configuration

* MM_30478 : A bit of housekeeping

* MM_30478 : Installed WDB on Android

* MM_30478 : Deletes new server record from default DB

* MM_30478 : Returns subset of server db instances

* MM_30478 : Code clean up

* MM_30478 :  Code clean up on db manager

* MM_30478 : House keeping + Patch for WDB

* MM_30478 : Android - Saving & Deleting in FilesDir [COMPLETED]

* MM_30478 : Code clean up

* MM_30478 : Code clean up

* MM_30478 : Code clean up

* MM_30478 : Test successful on Android device

* MM_30478 : Rolling back change to jest.config.js

* MM_30478 : Updated test to test_integration

* MM_30478 : Fix imports

* MM_30478 : Refactored the manual testscript

* MM_30478 : Renamed database manager test file

* MM_30478 : Code clean up

* MM_30478 : Updated manual test file with a note.

* MM_30482 : DataOperator [ IN PROGRESS ]

* MM_30482 : DataOperator - setting up the factory [ IN PROGRESS ]

* MM_30482: Code refactoring

* MM_30482 : DataOperator - setting up the factory [ IN PROGRESS ]

* MM_30482 : DataOperator - code clean up [ IN PROGRESS ]

* MM_30482 : Minor code clean up

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Fixed JEST issue with TS

* MM_30478 : Implementing JEST test cases

* MM_30478 : Implementing JEST last  test cases

* MM_30478 : Jest fixing ts errors

* MM_30478 : Database Manager Jest testing [ IN PROGRESS ]

* MM_30482 - Fixing DataOperator [ IN PROGRESS ]

* MM_30482 : Code clean up

* MM_30482 - Creates multiple records [ IN PROGRESS ]

* MM_30482 - Creates multiple records [ IN PROGRESS ]

* MM_30482 : Update operation [ COMPLETED ]

* MM_30482 : Code clean up

* MM_30482 : Updated TS for Data Operator

* Update mobile v2 detox deps

* MM_30482 : Added factories for all isolated tables

* MM_30482 : Refactored TS

* MM_30482 : Refactored base factory

* MM_30482 : Updated JSDoc for operateBaseRecord - Delete CASE

* MM_30482 : Implementing test for Data Operator

* MM_30482 : Completed tests for all isolated tables

* MM_30482 : Renamed entity_factory into operators

* MM_30482 : Fix all imports

* MM_30482 : Update multiple records

* MM_30482 : Edge case for existing records ( update instead of create )

* MM_30482 : Edge case  - create instead of update

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* MM_30482 : Code clean up

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* Update app/database/admin/data_operator/operators.ts

Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>

* MM_30482 : Imposing usage of correct table name for isolated entities

* MM_30482 : Code improvement as per Joseph reviews

* MM_30482 : Updated tests to validate choice of operator service wrt tableName

* MM_30482 : Updated PR as per suggestions

* MM_30482 : Updated comments to follow jsdoc conventions

* MM_33223 : Renamed DBInstance to DatabaseInstance

* MM_33223 : ADDED Prettier

* MM_33223 - Prettier formatting

* MM_33223 : Prettier formatting

* MM_33223 - Post section [ in progress ]

* MM_33223 : PostsInThread [99% completed ]

* MM_33223: Reaction entity completed

* MM_33223: Added Reaction to the Post

* MM_33223 : Refactored reactions utils

* MM_33223 : Added previous post id to all posts

* MM_33223 : Added File Metadata

* MM_33223 : Code clean up

* MM_33223 : Added PostMetadata

* MM_33223 : Added Draft

* MM_33223 - Removed Prettier

* MM_33223 - Undo files changes due to Prettier

* MM_33223 : Making use of MM eslint plugins

* MM_33223 : PostsInChannel [ IN PROGRESS ]

* MM_33223 : Including update_at in Post schema

* MM_33223: Code clean up

* MM_33223: Code clean up

* MM_33223 : Code clean up

* MM_33223: Testing Reaction [IN PROGRESS]

* MM_33223 : Updated typings for RawCustomEmoji in Reactions

* MM_33223 : Refactored DataOperator test

* MM_33223 : Jest - handleReactions - Completed

* MM_33223 : Jest - HandleDraft - Completed

* MM_33223 : Jest - HandleFiles - Completed

* MM_33223 : Refactored DataOperator-PostMetadata

* MM_33223 : Jest - HandlePostMetadata - Completed

* MM_33223 : Refactored posts into ordered and unordered

* MM_33223 : Refactoring + Jest Utils [ IN PROGRESS ]

* MM_33223 - Jest Utils - Completed

* MM_33223 : Jest - Remaining operators - Completed

* MM_33223 : Jest - Handler PostsInThread - Completed

* MM_33223 : Jest - HandlePostsInChannel - Completed

* MM_33223 : Refactored DataOperator class

* MM_33223 : DataOperator test clean up

* MM_33223 : DataOperator code clean up

* MM_33223 : Jest - HandlePosts - Completed

* MM_33223: JSDoc - Operators - Completed

* MM_33223 : Refactoring file types.ts

* MM_33223 : Refactored import statements

* MM_33223 : Added @database alias

* MM_33223 : Added missing JSDoc

* MM_33223 : Minor code clean up

* MM_33223 : Lint fixed

* MM_33223 : Disable eslint rules for Notification

* MM_33223 : Disable eslint rule for screens

* Update app/database/admin/data_operator/index.ts

Co-authored-by: Miguel Alatzar <migbot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Miguel Alatzar <migbot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Miguel Alatzar <migbot@users.noreply.github.com>

* MM_33223 : Update data_operatator as per suggestion

* Update app/database/admin/data_operator/index.ts

* MM_33223 : Removed OptType as the operator can do without it.

* MM_33223 : Code correction after review

* MM_33223 : Refactored Data Operator following reviews

* MM_33223 : Including a wrapper to DataOperator

* MM_33223 : Completing tests for wrapper

Co-authored-by: Elias Nahum <nahumhbl@gmail.com>
Co-authored-by: Avinash Lingaloo <>
Co-authored-by: Joseph Baylon <joseph.baylon@mattermost.com>
Co-authored-by: Miguel Alatzar <migbot@users.noreply.github.com>
2021-03-26 19:23:32 +04:00

433 lines
16 KiB
TypeScript

// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
import {Database, Q} from '@nozbe/watermelondb';
import SQLiteAdapter from '@nozbe/watermelondb/adapters/sqlite';
import logger from '@nozbe/watermelondb/utils/common/logger';
import {DeviceEventEmitter, Platform} from 'react-native';
import {FileSystem} from 'react-native-unimodules';
import {MIGRATION_EVENTS, MM_TABLES} from '@constants/database';
import DefaultMigration from '@database/default/migration';
import {App, Global, Servers} from '@database/default/models';
import {defaultSchema} from '@database/default/schema';
import ServerMigration from '@database/server/migration';
import {
Channel,
ChannelInfo,
ChannelMembership,
CustomEmoji,
Draft,
File,
Group,
GroupMembership,
GroupsInChannel,
GroupsInTeam,
MyChannel,
MyChannelSettings,
MyTeam,
Post,
PostMetadata,
PostsInChannel,
PostsInThread,
Preference,
Reaction,
Role,
SlashCommand,
System,
Team,
TeamChannelHistory,
TeamMembership,
TeamSearchHistory,
TermsOfService,
User,
} from '@database/server/models';
import {serverSchema} from '@database/server/schema';
import type {
ActiveServerDatabase,
DatabaseConnection,
DatabaseInstance,
DefaultNewServer,
MigrationEvents,
Models,
} from '@typings/database/database';
import {DatabaseType} from '@typings/database/enums';
import IServers from '@typings/database/servers';
import {deleteIOSDatabase, getIOSAppGroupDetails} from '@utils/mattermost_managed';
const {SERVERS} = MM_TABLES.DEFAULT;
if (!__DEV__) {
// To prevent logs leaking in production environment
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
logger.silence();
}
class DatabaseManager {
private activeDatabase: DatabaseInstance;
private defaultDatabase: DatabaseInstance;
private readonly defaultModels: Models;
private readonly iOSAppGroupDatabase: string | null;
private readonly androidFilesDirectory: string | null;
private readonly serverModels: Models;
constructor() {
this.defaultModels = [App, Global, Servers];
this.serverModels = [
Channel,
ChannelInfo,
ChannelMembership,
CustomEmoji,
Draft,
File,
Group,
GroupMembership,
GroupsInChannel,
GroupsInTeam,
MyChannel,
MyChannelSettings,
MyTeam,
Post,
PostMetadata,
PostsInChannel,
PostsInThread,
Preference,
Reaction,
Role,
SlashCommand,
System,
Team,
TeamChannelHistory,
TeamMembership,
TeamSearchHistory,
TermsOfService,
User,
];
this.iOSAppGroupDatabase = Platform.OS === 'ios' ? getIOSAppGroupDetails().appGroupDatabase : null;
this.androidFilesDirectory = Platform.OS === 'android' ? FileSystem.documentDirectory : null;
}
/**
* createDatabaseConnection: Creates database connection and registers the new connection into the default database. However,
* if a database connection could not be created, it will return undefined.
* @param {MMDatabaseConnection} databaseConnection
* @param {boolean} shouldAddToDefaultDatabase
*
* @returns {Promise<DatabaseInstance>}
*/
createDatabaseConnection = async ({
configs,
shouldAddToDefaultDatabase = true,
}: DatabaseConnection): Promise<DatabaseInstance> => {
const {
actionsEnabled = true,
dbName = 'default',
dbType = DatabaseType.DEFAULT,
serverUrl = undefined,
} = configs;
try {
const databaseName = dbType === DatabaseType.DEFAULT ? 'default' : dbName;
const databaseFilePath = this.getDatabaseDirectory(databaseName);
const migrations = dbType === DatabaseType.DEFAULT ? DefaultMigration : ServerMigration;
const modelClasses = dbType === DatabaseType.DEFAULT ? this.defaultModels : this.serverModels;
const schema = dbType === DatabaseType.DEFAULT ? defaultSchema : serverSchema;
const adapter = new SQLiteAdapter({
dbName: databaseFilePath,
migrationEvents: this.buildMigrationCallbacks(databaseName),
migrations,
schema,
});
// Registers the new server connection into the DEFAULT database
if (serverUrl && shouldAddToDefaultDatabase) {
await this.addServerToDefaultDatabase({databaseFilePath, displayName: dbName, serverUrl});
}
return new Database({adapter, actionsEnabled, modelClasses});
} catch (e) {
// console.log(e);
}
return undefined;
};
/**
* setActiveServerDatabase: Set the new active server database. The serverUrl is used to ensure that we do not duplicate entries in the default database.
* This method should be called when switching to another server.
* @param {string} displayName
* @param {string} serverUrl
* @returns {Promise<void>}
*/
setActiveServerDatabase = async ({displayName, serverUrl}: ActiveServerDatabase) => {
const isServerPresent = await this.isServerPresent(serverUrl);
this.activeDatabase = await this.createDatabaseConnection({
configs: {
actionsEnabled: true,
dbName: displayName,
dbType: DatabaseType.SERVER,
serverUrl,
},
shouldAddToDefaultDatabase: Boolean(!isServerPresent),
});
};
/**
* isServerPresent : Confirms if the current serverUrl does not already exist in the database
* @param {String} serverUrl
* @returns {Promise<boolean>}
*/
isServerPresent = async (serverUrl: String) => {
const allServers = await this.getAllServers();
const existingServer = allServers?.filter((server) => {
return server.url === serverUrl;
});
return existingServer && existingServer.length > 0;
};
/**
* getActiveServerDatabase: The DatabaseManager should be the only one setting the active database. Hence, we have made the activeDatabase property private.
* Use this getter method to retrieve the active database if it has been set in your code.
* @returns {DatabaseInstance}
*/
getActiveServerDatabase = (): DatabaseInstance => {
return this.activeDatabase;
};
/**
* getDefaultDatabase : Returns the default database.
* @returns {Database} default database
*/
getDefaultDatabase = async (): Promise<DatabaseInstance> => {
if (!this.defaultDatabase) {
await this.setDefaultDatabase();
}
return this.defaultDatabase;
};
/**
* retrieveDatabaseInstances: Using an array of server URLs, this method creates a database connection for each URL
* and return them to the caller.
*
* @param {string[]} serverUrls
* @returns {Promise<{url: string, dbInstance: DatabaseInstance}[] | null>}
*/
retrieveDatabaseInstances = async (
serverUrls?: string[],
): Promise<{ url: string; dbInstance: DatabaseInstance }[] | null> => {
if (serverUrls?.length) {
// Retrieve all server records from the default db
const allServers = await this.getAllServers();
// Filter only those servers that are present in the serverUrls array
const servers = allServers!.filter((server: IServers) => {
return serverUrls.includes(server.url);
});
// Creates server database instances
if (servers.length) {
const databasePromises = servers.map(async (server: IServers) => {
const {displayName, url} = server;
// Since we are retrieving existing URL ( and so database connections ) from the 'DEFAULT' database, shouldAddToDefaultDatabase is set to false
const dbInstance = await this.createDatabaseConnection({
configs: {
actionsEnabled: true,
dbName: displayName,
dbType: DatabaseType.SERVER,
serverUrl: url,
},
shouldAddToDefaultDatabase: false,
});
return {url, dbInstance};
});
const databaseInstances = await Promise.all(databasePromises);
return databaseInstances;
}
return null;
}
return null;
};
/**
* deleteDatabase: Removes the *.db file from the App-Group directory for iOS or the files directory on Android.
* Also, it removes its entry in the 'servers' table from the DEFAULT database
* @param {string} serverUrl
* @returns {Promise<boolean>}
*/
deleteDatabase = async (serverUrl: string): Promise<boolean> => {
try {
const defaultDB = await this.getDefaultDatabase();
let server: IServers;
if (defaultDB) {
const serversRecords = (await defaultDB.collections.
get(SERVERS).
query(Q.where('url', serverUrl)).
fetch()) as IServers[];
server = serversRecords?.[0] ?? undefined;
if (server) {
// Perform a delete operation for this server record on the 'servers' table in default database
await defaultDB.action(async () => {
await server.destroyPermanently();
});
const databaseName = server.displayName;
if (Platform.OS === 'ios') {
// On iOS, we'll delete the *.db file under the shared app-group/databases folder
deleteIOSDatabase({databaseName});
return true;
}
// On Android, we'll delete both the *.db file and the *.db-journal file
const androidFilesDir = `${this.androidFilesDirectory}databases/`;
const databaseFile = `${androidFilesDir}${databaseName}.db`;
const databaseJournal = `${androidFilesDir}${databaseName}.db-journal`;
await FileSystem.deleteAsync(databaseFile);
await FileSystem.deleteAsync(databaseJournal);
return true;
}
return false;
}
return false;
} catch (e) {
// console.log('An error occurred while trying to delete database with name ', databaseName);
return false;
}
};
/**
* factoryReset: Removes the databases directory and all its contents on the respective platform
* @param {boolean} shouldRemoveDirectory
* @returns {Promise<boolean>}
*/
factoryReset = async (shouldRemoveDirectory: boolean): Promise<boolean> => {
try {
//On iOS, we'll delete the databases folder under the shared AppGroup folder
if (Platform.OS === 'ios') {
deleteIOSDatabase({shouldRemoveDirectory});
return true;
}
// On Android, we'll remove the databases folder under the Document Directory
const androidFilesDir = `${FileSystem.documentDirectory}databases/`;
await FileSystem.deleteAsync(androidFilesDir);
return true;
} catch (e) {
// console.log('An error occurred while trying to delete the databases directory);
return false;
}
};
/**
* getAllServers : Retrieves all the servers registered in the default database
* @returns {Promise<undefined | Servers[]>}
*/
private getAllServers = async () => {
// Retrieve all server records from the default db
const defaultDatabase = await this.getDefaultDatabase();
const allServers =
defaultDatabase &&
((await defaultDatabase.collections.get(MM_TABLES.DEFAULT.SERVERS).query().fetch()) as IServers[]);
return allServers;
};
/**
* setDefaultDatabase : Sets the default database.
* @returns {Promise<DatabaseInstance>}
*/
private setDefaultDatabase = async (): Promise<DatabaseInstance> => {
this.defaultDatabase = await this.createDatabaseConnection({
configs: {dbName: 'default'},
shouldAddToDefaultDatabase: false,
});
return this.defaultDatabase;
};
/**
* addServerToDefaultDatabase: Adds a record into the 'default' database - into the 'servers' table - for this new server connection
* @param {string} databaseFilePath
* @param {string} displayName
* @param {string} serverUrl
* @returns {Promise<void>}
*/
private addServerToDefaultDatabase = async ({databaseFilePath, displayName, serverUrl}: DefaultNewServer) => {
try {
const defaultDatabase = await this.getDefaultDatabase();
const isServerPresent = await this.isServerPresent(serverUrl);
if (defaultDatabase && !isServerPresent) {
await defaultDatabase.action(async () => {
const serversCollection = defaultDatabase.collections.get('servers');
await serversCollection.create((server: IServers) => {
server.dbPath = databaseFilePath;
server.displayName = displayName;
server.mentionCount = 0;
server.unreadCount = 0;
server.url = serverUrl;
});
});
}
} catch (e) {
// console.log({catchError: e});
}
};
/**
* buildMigrationCallbacks: Creates a set of callbacks that can be used to monitor the migration process.
* For example, we can display a processing spinner while we have a migration going on. Moreover, we can also
* hook into those callbacks to assess how many of our servers successfully completed their migration.
* @param {string} dbName
* @returns {MigrationEvents}
*/
private buildMigrationCallbacks = (dbName: string) => {
const migrationEvents: MigrationEvents = {
onSuccess: () => {
return DeviceEventEmitter.emit(MIGRATION_EVENTS.MIGRATION_SUCCESS, {dbName});
},
onStarted: () => {
return DeviceEventEmitter.emit(MIGRATION_EVENTS.MIGRATION_STARTED, {dbName});
},
onFailure: (error) => {
return DeviceEventEmitter.emit(MIGRATION_EVENTS.MIGRATION_ERROR, {dbName, error});
},
};
return migrationEvents;
};
/**
* getDatabaseDirectory: Using the database name, this method will return the database directory for each platform.
* On iOS, it will point towards the AppGroup shared directory while on Android, it will point towards the Files Directory.
* Please note that in each case, the *.db files will be created/grouped under a 'databases' sub-folder.
* iOS Simulator : appGroup => /Users/{username}/Library/Developer/CoreSimulator/Devices/DA6F1C73/data/Containers/Shared/AppGroup/ACA65327/databases"}
* Android Device: file:///data/user/0/com.mattermost.rnbeta/files/databases
*
* @param {string} dbName
* @returns {string}
*/
private getDatabaseDirectory = (dbName: string): string => {
return Platform.OS === 'ios' ? `${this.iOSAppGroupDatabase}/${dbName}.db` : `${FileSystem.documentDirectory}${dbName}.db`;
};
}
if (!__DEV__) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
logger.silence();
}
export default new DatabaseManager();