These instructions should be followed for running the FileSystem/S3 demo in conjunction with the YOUnite stack and the Biz demo on your local system.

Overview

This demo demonstrates how the YOUnite File System/S3 Adaptor is configured and how it operates by watching a local file system directory and copying all file system operations that occur in the local files system directory to an AWS S3 bucket.

For more on the YOUnite File System and S3 Adaptor see YOUnite Off-the-Shelf File System and S3 Adaptor.

Prerequisites

  • The YOUnite Stack must be running

  • The Biz Demo must be configured

  • Write access to a local test directory (local-test-directory)

  • Write access to an AWS S3 bucket. Most importantly, the required aws access key id and aws secret access key.

  • Postman (and its command line utility newman). See Postman Downloads.

  • Python 3 or later. See Python Downloads.

Note
 Python 3 (version 3.7+) is required to run these scripts. They are not compatible with Python 2, which has been deprecated.

  • Copy fs-local-demo.tar to the same directory where biz-local-demo.tar has been extracted and then extract fs-local-demo.tar.

tar -xvf fs-local-demo.tar
python3 scripts/start.py -l

  • After this step, this directory should contain the directories: biz-demo and fs-demo

Setup and Run the Start Script

  • cd fs-demo

  • Login to the AWS console (or ask someone with the appropriate AWS permissions) to create the AWS bucket - For this demo you will need to be able to login to AWS and view the S3 bucket (alternatively, you can access it using the AWS CLI).

  • We need to avoid conflicting AWS access keys. We are already using AWS access keys to pull docker images from YOUnite’s AWS Elastic Container Repository (ECR) and for this demo we need a separate set of AWS keys to access our own S3 bucket (as covered above). A quick workaround for this demo is to assign the following vars :

    • export S3_AWS_ACCESS_KEY_ID=<aws access key id that has access to S3 bucket>

    • export S3_AWS_SECRET_ACCESS_KEY=<aws secret access key to the same>

  • Start the adaptor using the local-test-directory and S3 Bucket. For example:

python3 widgets/start.py -l -s file:///home/my-dir/purchase-orders -x s3://younite.local.fs.demo
Note
 On Windows use forward slashes, not backslashes for folder names.

Test the Demo

The purchase order and sales in the pdf files in the purchase_orders directory map to the test data files in the biz demo:

Purchase Order PDFs Path to Matching Test Data

fs-demo/purchase_orders/1a-QU65XIHZ.pdf

biz-demo/test-data/purchase_order_sale_1a.sql

fs-demo/purchase_orders/1a-QU11AQGR.pdf

biz-demo/test-data/purchase_order_sale_1b.sql

fs-demo/purchase_orders/1a-QU54BB0B.pdf

biz-demo/test-data/purchase_order_sale_1c.sql

  • Bring up a File System explorer (e.g. Finder on Mac, Windows Explorer in Windows) and copy a file from fs-demo/purchase-orders to the local-test-directory.

Note
 Copy (don’t move) the pdf or you won’t be able to re-use the pdf if you delete the destination directory.

  • Login to the AWS Console and Navigate to the S3 bucket. The file should have been replicated to the S3 bucket.

This demo can work between filesystems w/o an S3 bucket or many other types of filesystems. For a complete description of the File System Adaptor’s capabilities, see YOUnite Off-the-Shelf File System and S3 Adaptor.

Stop the Adaptors

cd fs-demo
python3 widgets/stop.py -l

`