Crypt Vault Storage

Copyright 2015, Bryan Nielsen <bnielsen1965@gmail.com>

Version 1.0

Quick Start

Unzip & Upload

Unzip the cryptvaultstorage.zip package and upload the entire cvs folder to the hosted directory on your web server.

The cvs folder contains the entire library, remote API, administration page, and installation page.

Create Database

Create the database that will be used to store the tables for the application.

If you are using MySQL or PostgreSQL then make a note of the hostname (often just localhost), the database name, and the user credentials used to access the database.

If you are using SQLite then make a note of the full path and filename for the database file. Also be sure that the web server use has read and write permissions on the directory and database file.

Adjust Settings

Edit the cvs/config.php file and enter your database settings, timezone, and set the audit settings for record retention and expiry.

Initialize

Open the cvs/install/install.php page in a web browser and create the database tables and create an administrative user.

REMOVE THE INSTALL FOLDER FROM THE SERVER TO PREVENT UNAUTHORIZED ACCESS!

First Login

Open the cvs/admin.php page in a browser and login with the administrative user that was created in the initialization step.

Create any additional users. Create any desired keys. Add keys to a keyring and save to other users if necessary.

Test

Test the key functionality using the Test tab on the admin.php.

Integration

Add the necessary code to integrate the vault into your web site or application.

Maintain

Routinely remove expired records in accordance with your data retention policy. Routinely export log list file to an archive and delete old log list.


Table of Contents

Quick Start

Unzip & Upload

Create Database

Adjust Settings

Initialize

First Login

Test

Integration

Maintain

Table of Contents

Requirements

Introduction

Key Pairs

Encrypt Process

Decrypt Process

Web Based Interface

Web Interface Security

Installation

Create Database

Upload Files

Configure Application

Database

Timezone

Data Retention Window

Initialize Application

Create Tables

Create Administrative User

Test Key

Remove Install Tool

Administration

User

Account Permissions

Associate Account

Clerk Account

Manager Account

Administrator Account

Keyring

Keys

Advanced Settings

Records

Log

Implementation

PHP

Saving Records

Remote API

Use Cases

Making Keys

Data Retention Policies

CVS_RECORD_RETAIN_DAYS

CVS_RECORD_EXPIRE_DAYS

Employee Levels

Associate

Clerk

Manager

Administrator

High Level Security


Requirements

Apache or NGINX

OpenSSL

PHP

MySQL or PostgreSQL or SQLite

Introduction

The Crypt Vault provides secure storage for user submitted data through encryption, a user interface that you and your employees use to retrieve decrypted data, and visual cues to assist with enforcement of data retention policies and audits.

Implementation of the Crypt Vault in your site will require a combination of installation and configuration of the Crypt Vault software as well as integration into your existing site source code. The integration process is relatively simple but will require some development work by your site developer.

A web server with PHP and a database server will be required to run the Crypt Vault. The web server is used to host the API and the web based user interface. The database will store the encryption keys, the encrypted records, and the user records for the web interface.

Key Pairs

The foundation of the Crypt Vault is based on encryption keys generated with OpenSSL. The keys are used to secure the user data stored on the server as well as user keyrings and sessions in memory when accessing the administration web page.

The encryption keys are generated in pairs as a public key and a private key. The public key is used to encrypt the data and the private key is used to decrypt the data.

The public key can encrypt data without any special conditions. While the private key will require a secret passphrase to enable the key before it can be used to decrypt the data.

The system can hold an unlimited number of key pairs and each key pair that is created is required to have a distinct name. When creating a new key pair a new name is provided and a secret passphrase that will enable the new private key. The resulting key pair is stored in the database under the name of the key pair.

Encrypt Process

When a record arrives for secure storage the request must include the name of the key pair to use in the encryption process. The public key in the key pair is pulled from the database and used to encrypt the record before it is stored in the database. The original record content is then cleared from memory when the request ends and the encrypted copy is saved in the database  for future use.

The encrypted record that is saved in the database includes the name of the key that was used in the encryption process. This key name reference is needed for the decryption process.

Decrypt Process

When the record is eventually retrieved from the database the key name in the record lets the system know which key pair is required for the decryption process. The decrypt process will require the secret passphrase for the private key in the key pair. Once the private key is enabled the record can be decrypted and the original content can be displayed..

Web Based Interface

The Crypt Vault includes a web based user interface where records can be decrypted, keys and users are managed, and users are maintained. The web interface includes visual cues to assist in identifying the state of records and it includes a log of user activity.

Web Interface Security

The web interface can be configured to require an HTTPS connection, which is recommended to insure all communications over the network are secure. The database stores user passwords in a one way hash code based on the best algorithm PHP can detect is available on the server. Each user account includes a keyring saved in their account and the keyring is encrypted with a Crypt Vault key which requires the user account password to read.

The server will never provide any keys or encrypted records through the web interface. All encryption processes take place on the server in memory and the interface only returns details from the results of the encryption process.

Account security in the web interface utilizes user account names, account passwords, a session token in a cookie, and a temporary GUID used to encrypt the user keyring in server memory and session files during the session.

A temporary session cookie is used to identify each user connection to the server. The user's web browser will use this cookie to identify the user's connection while they are active in the web interface.

In addition to the session cookie the server will provide the connected user with a temporary GUID value that is only transmitted from the server once when the user authenticates. This GUID is used on the server as a passphrase to decrypt the user's temporary keyring held in memory on the server. When the user attempts to use any command that requires the keyring in the session on the server they will need to send this GUID value back to the server to decrypt the keyring in memory.

Installation

The installation will require a web server to host the application and a database server to host the database when using MySQL or PostgreSQL. In many cases the web server and database server are the same host. When using SQLite the web server will host the database as a database file.

Create Database

Using your prefered database tools you must create a database to hold the records. If you are using MySQL or PostgreSQL then note the server hostname, database name, and the user credentials for the configuration step. If you are using SQLite then note the database filename and the full path to the file.

NOTE: When using SQLite make sure the folder containing the database file is readable and writable by the user running the web server service.

Upload Files

The Crypt Vault package includes a folder named cvs with all the source code required to run the application. Upload a copy of this folder to the hosted directory on your web server, i.e. public_html/.

After uploading the application to the server verify the upload by opening the installation page in a web browser and running a compatibility. The installation page will be located in the upload folder under cvs/install/install.php. I.E. Something like https://mydomain.net/tools/cvs/install/install.php

When the install page loads then click on the Compatibility Test button to verify the server will be capable of running the application.

Configure Application

The application configuration is located in the cvs folder in the file named config.php. Edit the config.php file and adjust the settings as needed.

Database

Adjust the CVS database settings with the values you noted from the database creation step. You must select a driver for the CVS_DATABASE_TYPE and then set the values for the host, user credentials, and database name if using mysql or pgsql, or set the file parameter to the full path for your sqlite database if using sqlite.

NOTE: When using SQLite you should specify the full path to the database file to avoid issues that may occur due to a relative path. I.E. /var/www/html/cryptvaultstorage/data/test.db

Timezone

Adjust the timezone setting for your location so the times and dates displayed in the administrative pages will show the correct time stamp.

Data Retention Window

 Edit the record retain and expire values to define the minimum  number of days that each record must be retained and the maximum number of days before a record expires and should be deleted.

Initialize Application

After installation you need to create the tables in the database and the first administrative user before the application can be utilized. The installation tool included in the package will assist with the initialization step. The installation page will be located in the upload folder under cvs/install/install.php and accessible via a web browser I.E. something like https://mydomain.net/tools/cvs/install/install.php.

Create Tables

Click on the Create Tables button and the tables will be created in the database. If this step fails then diagnose the database problem and try again. If the tables already exist or only some of the tables exist then it may be necessary to drop the tables and retry the create step with an empty database.

Create Administrative User

At least one administrative user is required and the first administrative user is created using the install tool. Enter the credentials for the first administrative account and click on the Create Admin button.

After creating the administrative user go to the cvs/admin.php page and verify the login credentials work.

Test Key

As the administrative user, create a test key in the application under the Manage Keys tab on the administrative page. This key will be used as a temporary key to test the application installation.

Add this new test key to the administrative user's keyring by clicking on the key link icon on the Manage Keys page and entering the pass phrase for the test key.

Next go to the Test tab on the administration page, select the test key you created, enter a description and content for a test record, and save the test record.

At this point you should have a test record in the list of records on the Record tab and it should be highlighted as readable with the key that was added to the user keyring.

Click on the read icon for the test record and verify that the administrative user can decrypt and read the test record.

Remove Install Tool

After all testing is complete and successful the installation tool should be removed from the server to prevent any unauthorized access. Leaving the install script on the server presents a security risk.

Delete the install subdirectory from the cvs folder on the server. If for any reason the install tool is required for diagnosis or repair of the application the install directory can be uploaded to the same location and reused.

Administration

The administration of the application involves the creation and maintenance of user accounts, creating and maintaining encryption keys and keyrings, and maintenance of the encrypted records stored in the system.

User

The web based user interface is designed for multiple user accounts with predefined account types. Administrator users will have the ability to create and edit user accounts on the User tab of the administration page.

The account types include an Associate, Clerk, Manager, and Administrator. Each account type has predefined permissions for various functions in the administration interface.

Account Permissions

Associate

Clerk

Manager

Administrator

Read Records

Yes

Yes

Yes

Yes

Delete Records

No

Yes

Yes

Yes*

Add Key To Keyring

No

Yes

Yes

Yes

Remove Key From Keyring

No

Yes

Yes

Yes

Save Current Keyring

No

Yes

Yes

Yes

Save Current Keyring To Other User**

No

No

Yes

Yes

Make Named Key Pair

No

No

Yes

Yes

Delete Named Key Pair

No

No

Yes

Yes

Administrate User Accounts

No

No

No

Yes

* Administrator users can delete any record. Other users can only delete a record if they have the named key pair for the record in their keyring.

** Saving a current keyring to another user's account will overwrite any keyring they currently hold in their account.

Associate Account

This is the lowest level account in the system with only basic read permissions. An Associate can only read records that have been encrypted with the keys given to them in their keyring. The Associate cannot manage their own keyring, a Manager or Administrator must create a keyring to save in an Associate account to enable these users to read records.

Clerk Account

A Clerk can read and delete records associated with the keys in their keyring. The Clerk can also manage their own keyring by adding and removing keys, and they can save their current keyring for future use. This user cannot save their keyring to another user account and they cannot add or delete named key pairs in the system.

Manager Account

The Manager account can read and delete records associated with the keys in their keyring. They can manage their own keyring by adding and removing keys, and they can save their current keyring to Associate and Clerk accounts. A Manager can also create and delete named key pairs in the system.

Administrator Account

An Administrator has all permissions required for record, key, keyring, and user administration. The Administrator also has the ability to delete records without holding the named key pair in their keyring.

Keyring

When a user signs into their account on the administration page they are given a keyring on the server to hold keys which they have activated. The keys on their keyring can be used to read records that have been encrypted with that key.

All user accounts, except the Associate accounts, can add keys to their keyring by clicking on the link icon on the Manage Keys tab and entering the activation credentials for that key. The keyring in the user's session can also be saved to an encrypted key safe under their account. A saved keyring will be reloaded whenever the user signs into their account.

Keyrings are not automatically saved to the user's key safe. When activated keys are added to a session keyring a user must click the Save button on the Keyring tab if they want to use the same keyring the next time the sign in.

Administrators and Managers can also save their current keyring to another user's key safe. This will overwrite any keyring the user had saved in their key safe. This capability is required for Associate accounts because an Associate cannot add keys to their own keyring and they cannot save their keyring. The Associate accounts can only use the keyring that has been given to them by an Administrator or Manager.

Keys

The keys are the central component of the Crypt Vault Storage system. Keys are created and maintained from the Manage Keys tab of the administration page. The Manage Keys tab provides a list of existing keys with usage statistics and icons that can toggle settings and add a key from the list to the current session keyring.

Once a key is created the name and encryption credentials for the key cannot be modified. An existing key can be deleted, the key can be disabled or enabled, and the flag for remote API usage can be enabled or disabled.

Advanced Settings

When creating a new key there are some advanced options that affect the behaviour of the new key. You can select if the private half of the key pair will be saved in the database, whether records saved with the key ever expire, and if the key can be used to save records using the remote API.

Records

When a record is saved in Crypt Vault Storage the record is encrypted with the key specified in the request and saved in the database with additional details such as the date and time when saved, the date and time when a user last decrypted and read the record, etc.

The web interface provides a list of the records currently in the database as well as color coding for the condition of each record in the list.

Records with a green background are still waiting to be read and the current user has the proper key in their keyring to read the record. The records with an orange colored background are also waiting to be read but the current user does not have the correct key in their keyring to read these records.

After a record has been read the background for the record will become grey. If a record has been read, the current user has a key for the record in their keyring, and it is past the retention setting in the application configuration then the user can delete the record.

Records that show up in bold red text have expired and are outside the window for data retention as specified by the application configuration settings. A well defined data retention policy is good for customer privacy and the highlight will help to maintain adherence to the policy.

Log

Many of the events from user activity in the application are recorded in an event log. The event log can be used to audit user and record access within the system.

The log should be exported and deleted on a regular basis. Log in as an Administrator and export the log list from the beginning up to the a recent day. A download link will appear to download the export. Save this file in a secure archival location for future reference. After the export and download then delete the log list up to the same recent day as the export.

Implementation

To use the Crypt Vault Storage on a web site or other platform requires a minor level of development to implement the Vault into a web site or application. The primary method of implementation is via PHP and the PHP library functions provided by the Crypt Vault Storage class files. In applications where PHP is not an option it is also possible to use the remote web API to request storage of records.

Refer to the included developer notes document and the documentation for the PHP library and the remote API for extensive details on the capabilities and use.

PHP

The Crypt Vault Storage code provides an initialization script and a class library of functions that simplify the process of saving and reading encrypted information in the database. The library uses exceptions to make error handling in your code simple as well.

Saving Records

As an example assume we have a customer form that is posted to a PHP script that processes the form and we need to save the form details in the Crypt Vault. A key has been created in the Crypt Vault for this form and the name of the key is New Subscriptions. When the form is submitted the contents can be stored in the Crypt Vault in a few simple steps:

  1. Process the form data into a description variable and a content variable. The content will be encrypted while the description is plain text in the database.
  2. Include the init.php script from the Crypt Vault to initialize the library.
  3. Create an instance of the StorageProcessor class.
  4. Save the record using the name of the key, the description and the content generated from the form.

The code for this example may look something like the following…

<?php

// process the form submission when post parameters are set

if ( isset($_POST['client_name']) && isset($_POST['client_email']) ) {

  try {

    // prepare the form data

    $description = 'New Subscription From: ' . $_POST['client_name'];

    $content = 'Client: ' . $_POST['client_name'] . "\n" . 'Email: ' . $_POST['client_email'] . "\n";

    // initialize the library

    include 'cvs/init.php';

    // create instance of the storage processor

    $storage = new \CryptVault\Storage\StorageProcessor();

    // save as a new record

    $recordId = $storage->saveRecord('New Subscriptions', $description, $content);

    // error handling

    if ( false === $recordId ) {

      // no record id, save failed

      $saveSuccess = false;

      $message = 'Failed to save the request.';

    }

    else {

      // record successfully saved

      $saveSuccess = true;

      $message = 'Thank you for your subscription request.(' . $recordId . ')';

    }

  }

  catch ( \Exception $e ) {

    // exception based failure, details are in the exception message

    $saveSuccess = false;

    $message = 'Failed to save the request. ' . $e->getMessage();

  }

 

  // display message based on success status

  if ( $saveSuccess ) {

      echo '<div class="message">' . $message . '</div>';

  }

  else {

      echo '<div class="error">' . $message . '</div>';

  }

}

?>

Remote API

In some cases a project may not be developed using PHP and the standard Crypt Vault Storage library cannot be used. It is still possible to integrate the vault into the project by using the web based remote API.

NOTE: Saving records using the remote API is normally disabled on each key. On the Manage Keys tab you can toggle the Remote Record Save option on each key to allow saving of records using a key through the remote API.

The remote API uses JSON messages in the body of HTTP POST requests. Each request must contain at a minimum the API method to execute and each method may require additional parameters. The following example is a raw HTTP POST to a Crypt Vault Storage installation with a request to save a new record...

POST /cryptvaultstorage/cvs/api.php HTTP/1.1

Host: getwebscripts.com

Content-Type: application/json

Cache-Control: no-cache

{

    "method": "saveRecord",

    "keyName": "New Subscriptions",

    "description": "New Subscription From: Bob Toad",

    "data": "Client: Bob Toad\nEmail: bobt@bobtoad.org"

}

Assuming the saveRecord request is successful the response may look something like the following…

HTTP/1.1 200 OK

Cache-Control: no-cache, must-revalidate
Connection: close
Content-Length: 72
Content-Type: application/json
Date: Wed, 07 Oct 2015 02:45:46 GMT
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
Server: Apache/2.2.15 (CentOS)
X-Powered-By: PHP/5.3.3

{

  "id": "fea109f5-ab87-4bf8-8d31-72cd711d994e",

  "success": true,

  "errors": []

}

Use Cases

Making Keys

The only restriction on keys is that each key in the system must have a unique name. New records are categorised by the name of the key that was used to encrypt each record. So it is recommended that keys be created for as many different categories as may be needed.

 I.E. Keys could be categorised and named by the web domain or application that uses each key, or the keys could be named based on the purpose of the data encrypted with the key such as subscriptions, prospectus requests, etc.

When creating a new key it is important to use a strong passphrase for the private key. The administration page has a Make Phrase button that will generate a random phrase or you can enter your own phrase.

Always make a record of the passphrase for each key you create and keep the passphrase in a secure location.

If you create a key where the private key is not saved then you will also need to keep a record of the private key for future use as well.

Record the passphrase and keep it in a safe location. You will be asked for the passphrase when you add a key to a keyring. Once the key is added to the keyring the user that holds the keyring will not need to use the passphrase again. However, if the key needs to be added to another keyring or if the key is removed from the keyring then the passphrase will be needed again to add the key back.

Provide the name of the key to the developer that will implement the Crypt Vault Storage in your web site or application. The developer will only need the name of the key when saving records in the Crypt Vault.

Data Retention Policies

A strong method to guarantee that private data does not leak into the possession of unauthorized entities is to remove the private data when it is no longer required. The Crypt Vault includes configuration settings that establish a data retention window that prevents data deletion too soon while providing visual indicators for records that have expired and should be removed from the system.

The configuration settings for the data retention window are defined in the cvs/config.php file.

CVS_RECORD_RETAIN_DAYS

The CVS_RECORD_RETAIN_DAYS parameter defines the number of days that each record must be retain in the system to provide an opportunity to process the data. New records must be held this number of days and must be read by a user before they can be removed from the system.

NOTE: The Administrator users can delete any record from the system at any time.

CVS_RECORD_EXPIRE_DAYS

The CVS_RECORD_EXPIRE_DAYS parameter determines when a record is expired and should be removed from the system. When a record remains in the system beyond the number of days defined in this parameter the record will be displayed with a warning level in the record list to signify that the record should be removed.

Employee Levels

Different employees within an organization require access to different data and functions. The administration page for the Crypt Vault Storage has four access levels that grant varying levels of data and function access.

Associate

If an employee should be highly restricted and only have the ability to read a specific set of records encrypted with a specific set of keys then they should be assigned an Associate account and given a keyring with only the keys they need to read specific records.

This is useful for employees who will read certain records when they arrive in the database and take some business action based on the content of the record. But they are not allowed to delete the record or modify anything in the Crypt Vault.

Clerk

Some employees are given a greater level of trust and are allowed to not only read records in the Crypt Vault but are allowed to delete old records and manage their own keyring by adding or removing keys to which they have the credentials.

These employees would be assigned a Clerk account and they would be given a keyring if they are not allowed to access the credentials for the keys. If they are given credentials for any keys they can add these keys to their keyring themselves.

A Clerk will see records in the database which they cannot read but they will not have access if they do not have the proper key or the credentials for the key. This will give them more insight into what records are waiting in the system while still restricting their access where necessary.

Manager

Employees at a management level will have greater trust and responsibility over other employees. Thus the Manager account has all the access capabilities of the lower employee accounts with the added capability of saving a keyring to other Associate or Clerk accounts.

This ability to save keyrings to other accounts gives a Manager the power to provide access to specific records to another employee without sharing the credentials to the keys themselves.

Administrator

And finally there are the employees who are responsible for maintaining the system and the user accounts. These employees would require an Administrator account where they would have control over all users, keys, and records.

High Level Security

Encrypted data is secured by the need for three components to decrypt the data, the encrypted message itself, the private key used for decryption, and the passphrase that enables the private key.

While the encryption of sensitive data reduces the susceptibility of inappropriate access there is still the risk of leaking the encryption keys along with the data. In such an event the credentials for the private key could be brute forced to crack the encryption. Once the passphrase is brute forced the data would be accessible through decryption.

The first step in lessening the risk of a brute force attack is to utilize a strong passphrase for keys. When these keys are stored in a user keyring the password on the user account must also be strong. Therefore a policy should be in place for passphrases and passwords.

Since the decryption process requires three components for completion the Crypt Vault provides an option to not save private keys in the database when creating a new key pair. When this option is selected the passphrase and the private key text must be saved externally.

By saving the private key externally from the Crypt Vault you have removed two of the three components required for decryption from the database. This adds a higher level of security at the cost of convenience. To read records encrypted with these keys it will be necessary to enter both the passphrase and the private key text.