Home

AlayaCare Flat File Guide

Documentation around AlayaCare’s Flat File integration processes.

Contents

• Overview
  • Scope
  • Architecture
  • Definitions
• Flat File Integration Process
  • System Architecture Model
  • Flat File Workflow Overview
  • Overall Architectural Considerations
  • System Architecture Component Definitions
• System Construction
  • Environments
• Configuration and Setup
  • Supported File Formats
  • Accessing S3 via AWS CLI

Overview

Scope

This document describes the process around flat file integrations. The goal of such integrations is to satisfy the need to allow data to flow from and into AlayaCare from external systems that clients or partners have in place to satisfy their business requirements.

The goal of this flat file integration process is to define the technologies, products, and techniques necessary to develop and support the system, and to ensure that the system components are compatible and comply with the enterprise-wide standards and direction defined by AlayaCare.

Architecture

Definitions

AWS: Amazon Web Services GPG: The GNU Privacy Guard SNS: Amazon Simple Notification Services S3: Amazon Simple Storage Service

Flat File Integration Process

System Architecture Model

The flat file integration process is based on the following components:

• S3 buckets to transfer files between systems
• AWS SNS notification to trigger a data migration process
• Public/Private encryption keys to encrypt/decrypt the files if needed (we use GPG)
• Predefined/Custom format flat files import/exports

Flat File Workflow Overview

The flat file process is triggered based on an event, for example, when a file is dropped on the tenant’s S3 bucket. The files must be dropped into the S3 bucket Inbound folder. Any naming convention can be used for flat files (for example: inbound/Employees/ExampleOfEmployeeFlatFile.csv).

Anything inside the Inbound folder within the S3 gets triggered and, after execution, the file is moved to the Outbound folder, until the Inbound folder is empty.

Once the process is done, an ErrorFile is generated and uploaded to the Outbound folder. The naming convention for the error files can also be chosen by the client, but the Alayacare technical support team needs to be notified.

Initial testing runs are executed in staging for validation. Once in Production, the files are executed everytime a file is dropped.

Overall Architectural Considerations

Security Strategy: The S3 buckets used are encrypted at rest.

Performance requirements and transaction volumes: Only one file can be migrated at a time. Therefore, subsequent files will be queued until the first file is completed.

Resilience and recovery: The flat file migration jobs are dependent on the data in the files. If a job fails, AlayaCare will be notified and will in turn work with the Client Success team to notify clients.

System Architecture Component Definitions

S3: A flat file integration bucket will be made available to the client to upload and download from. User credentials will be sent in an encrypted email.

• Upload: --flatfileintegration-s3bucket-inbound
• Download: --flatfileintegration-s3bucket-outbound

SNS: A SNS notification is triggered as soon as a file is uploaded to the inbound bucket. This notification will trigger the data migration process.

Data Migration Process: The data migration process is responsible for the following tasks

1. Mapping the file values and ids to what the software accepts
2. Getting authenticated with the AlayaCare platform
3. Load the data using AlayaCare’s internal APIs
4. Report (and encrypt if it’s part of the process) an error file listing skipped rows to the outbound S3 bucket

System Construction

Environments

Through the development cycle of this integration AlayaCare will be providing clients with the following environments:

Staging

Role: Development and Validation As development of required features is completed they would be deployed to the staging environment where data flow and interactions can be validated. Credentials for this environment will be provided as part of the setup phase.

UAT/Training

Role: Training and acceptance testing If required for training or user acceptance testing, a UAT/Training environment can be provisioned. Credentials for this environment will be provided on demand as per client needs.

Preview

Role: Impact assessment and change management When new features or change requirements are ready to be released to a prod environment they would be placed in a preview environment exposed to the client during a pre-agreed period of time. Credentials for this environment will be provided as part of the setup phase.

Production

Role: Live production environment Stable monitored environment with latest stable integration deployed complying to AlayaCare’s SLA.

Configuration and Setup

Supported File Formats

1. Flat Files must be single files. There cannot be any other preloaded files as part of the process.
2. As of right now, the S3DataSource only supports CSV files.
3. Encodings can be specified, but must be consistent between uploads. By default, the encoding is set to utf-8.
4. Files uploaded must be uploaded with AES256 server side encryption

Accessing S3 via AWS CLI

This is only one of many ways for someone to access an S3 bucket.

• Installing aws cli
• Setting credentials
• aws s3api Command Line Reference

Some helpful commands to know:

aws s3api help

aws s3api list-objects --profile={tenant-profile-in-credentials} --bucket={bucketname}

aws s3api get-object --profile={tenant-profile-in-credentials} --bucket={bucketname} --key={key within bucket} [filename to save locally]

aws s3api put-object --profile={tenant-profile-in-credentials} --bucket={bucketname} --key={key within bucket} --body={filename to upload} --server-side-encryption=AES256