An Introduction to YOUnite

Introduction

YOUnite is a robust, configurable, federated data discovery, catalog, governance and synchronization platform that unifies an organization’s data tier. YOUnite allows an organization to gently transition to these goals without forcing it into an "all-or-nothing" approach. It provides tools that allow the organization to:

Describe data models and criteria to maintain uniqueness of records across the entire YOUnite data fabric
Group applications/services into organizational boundaries
Manage access control to the data (governance)
Decide which applications/services contain data to be unified
Integrate and transform data between source systems using off-the-shelf and custom adaptors
Log data events including federate data events, access requests and exceptions
Make federated data available via the Data Virtualization Service through the YOUnite REST API
Register for data notification changes so applications can receive webhooks when date events (e.g. new customer record) are generated

These features are presented through the YOUnite UI and other mechanisms presenting a single pane view for the adminstrators and data stewards.

YOUnite includes a user-facing tool, YOUnite UI, that provides:

An administrative interface for managing the YOUnite Data Fabric
A Data Steward interface for managing data governance and access to data
Access federated data access (Glossary-dataVirtualization)including the ability to compare differences in a federated record between source systems

With YOUnite you can create zones to group the resources for a given entity. A single zone refers to a collection of systems/applications owned by groups inside of an organizations. Zones are defined within YOUnite by the Administrator/Stakeholders for each zone. They act as a boundary for which permissions and governance may be defined so that an one group in an organization can control data flowing in and out between other groups. For example, an educational institution may have a college district office responsible for many colleges within the district. You can create a "parent zone" for the college district office for accessing data across all of its district colleges. Each college within that district could then be grouped into a "child zone," with YOUnite securing and controlling data flow into and out of each. Read more on zones below.

YOUnite addresses the issue of "What is the record of truth?" for a resource in any given context. That source of truth is called Master Data. The resource might be a customer, a product order, a college, a course, etc. YOUnite provides multiple points for updating these resources. It also ensures that the data record is always updated while allowing each zone to determine where those updates are applied. For example, if a student updates their address in the college application system, YOUnite receives this change and consults the college zone’s Adaptor Capabilities Lists then determines if the updated address is applied to the Student Information System (SIS), Learning Management Systems (LMS), or any other application that stores the student’s address. The Adaptor Capabilities Lists consist of the capabilities that upon initialization broadcasts to the YOUnite Server:

The Data Domains it supports
The methods it supports for the given Data Domains (GET, PUT, POST and/or DELETE)

Data Domains

Data Domains are the heart of Data discovery and cataloging. In traditional database parlance, data domains are a collection of fields (values) that are encompassed by an attribute (database column). For example, using a Customer table example below, the timeZone attribute has a data domain of A, P, M, C, E, or null, which represent Alaska, Pacific, Mountain, Central and Eastern time zones. In other words, the data for timeZone is limited to this data set, or data domain.

The timeZone data for customer records might appear as in the following example:

With YOUnite, data domains refer to versions of a specific data type, such as employee, student or course and is defined by the parties responsible for data governance. The goal is to:

Create data domains that will normalize data across an organization
Manage access to their organization’s disjointed data sets (referred to as data governance or governance).

YOUnite allows data architects to:

Create data domains that reflect their organization’s requirements or its unique organizational structure
Version their data domains to accommodate new applications and application versions

Once a domain is created, and the data in the source systems is linked, it can be referenced by other data domains, data stewards, and by API consumers as a source of truth.

YOUnite domains are defined:

In JSON format.
Have universally-adopted domain versions agreed upon stakholders in the data fabric.
Create matching algorithms for identifying duplicate records in different systems where some data may or may not completely match.

YOUnite employs Federated data domains. Federated data records are NOT stored inside YOUnite but are created in real-time by referencing the properties (data elements) as they are stored in the company’s multiple source systems. An example of a good candidate for a federated domain is if one division vigorously guards its sales data but the entire company would benefit from the sharing of the updated data. With a federated domain, the data would remain in the sales division’s system for them to manage and maintain and the company as a whole would benefit from the shared updated sales data.

For each data record associated with a given federated domain, The YOUnite catalog stores:

Registry links of where the data is stored
When it was last updated
Which zone it is shared with
The unique global ID(s) known as the Data Record or DR Key.

The permission-related information (which zone the data is shared with) is handled with data governance and is implemented with ACLs.

Domain Property Types

Standard primitives types (integer, floating point numbers, strings, and enumerations) are supported as well as arrays and cross-references between domains.

For more a more on domains see theData Domains guide.

The Operational Side of YOUnite: Zones

One of YOUnite’s key design solutions is the ability to group an organization’s data by the organization’s structure (e.g. divisions, departments, districts, schools, etc.) and create relationships between these groups. YOUnite calls these groupings zones. An example might be found in a college school system. For instance, YOUnite could mirror a college system’s structure where the top-level zone is the Chancellor’s Office, with college district zones in the middle and individual college zones underneath.

Zones contain users, groups, roles, permissions, adaptors, logs, ACLs, and other resources. Zones are associated with each other in a hierarchical structure with parent, sibling, and child zones. In the example of the college school district, the schools within that district might be considered child zones of the district (parent zone).

Zone Users

When a zone is created, two distinct users are defined:

Zone Admin
Zone Data Steward

The Zone Admin controls the operational aspects of the zone while the Zone Data Steward controls the data. See Managed Roles for a more comprehensive overview of these two roles.

Zone Admin

The Zone Admin is responsible for operational tasks for that zone, such as:

Setting policies
Creating subordinate zones
Creating users
Managing operational notifications that are sent and received from other zones
Managing and viewing operational logs and notifications
Managing operational permissions between zones (i.e. allowing users of other zones to perform the responsibilities listed above)

Zone Data Steward

The Zone Data Steward (ZDS) is responsible for the accuracy of the data being handled by YOUnite for the zone(s) and how it is shared. They

Has access rights to all of the data relevant to the zone
Configures and manages Cataloging and Synchronization. Adaptors connect source systems to the YOUnite Data Fabric Platform and more specifically to the zone.
Manage inbound and outbound ACLs for permissions to the data
Can limit or expand user access to the data
Grant API access to the data
Manage data related notifications that are sent and received from other zones
Manage and view domain and data logs

YOUnite works with Single Sign-On (SSO) services. When a zone is created, the SSO IDs for the parent zone’s zone admin and zone data steward are used.

Note	As a security measure, new zone admins and zone data stewards can be assigned to a new zone and the zone administrator and data steward from the parent zone can be removed.

You can add additional users (individually or as part of a group) to a zone and grant them permissions. YOUnite’s permissions model involves roles and groups so that roles can be assigned to groups. This creates an easy-to-manage permission paradigm:

Identity	Description
User	A user or, SSO User, in the YOUnite Data Fabric that is tied to an SSO ID. An SSO User can access the various services that make up the YOUnite Data Fabric Platform such as the YOUnite Server and YOUnite UI. Everyone must be authenticated through an authorization application. A single SSO user may have multiple zone users associated with them.
Groups	A group contains multiple users.
Permissions	Specifies access or denial to operations and resources.
Roles	A grouping of permission settings.

Identity

Description

User

A user or, SSO User, in the YOUnite Data Fabric that is tied to an SSO ID.

An SSO User can access the various services that make up the YOUnite Data Fabric Platform such as the YOUnite Server and YOUnite UI. Everyone must be authenticated through an authorization application. A single SSO user may have multiple zone users associated with them.

Groups

A group contains multiple users.

Permissions

Specifies access or denial to operations and resources.

Roles

A grouping of permission settings.

For more a more on zones see the Zones, Users, Groups, Roles and Permissions guide.

Adaptors: The Key to Federated Data Discovery, Cataloging and Synchronization

With Federated, adaptors are the interface between an organization disjointed systems. Adaptors perform CRUD (create, read, update, and delete) operations on the data while the YOUnite Server performs the task of distributing (routing) the data between the systems.

Adaptors represent a significant development effort in the overall process of data discovery, cataloging and synchronization.

Following is a brief description of how adaptors function in the overall YOUnite Data Fabric. Further reading on adaptors can be found in the following pages:

A more detailed description about adaptors can be found on the Adaptors page.
Creating and managing adaptors can be found on the Creating and Managing Adaptors page.
The Adaptor SDK Summary page covers the Java SDK and how to develop an adaptor how to use applications can leverage YOUnite.
The Developing YOUnite Aware Applications page covers how applications can leverage YOUnite.
Several off-the-shelf adaptors are available and are covered in the in the Adaptor Guide section of the Knowledge Base.

In the example below, the organization’s CRM, ERP, and MIS systems need to be integrated with YOUnite through the adaptors.

Extending the example above, each adaptor belongs to a zone as in the image below. The ellipse represents a data record in the form of a data event originating at an adaptor in Zone-A and being routed to adaptors in other zones or in response to an API consumer’s request to the YOUnite Server. The data event and data record payloas are in the form of a JSON object.

Note	Adaptors need to be developed if pre-existing "out-of-the-box" adaptors do not exist.

Data Governance: ACLs

ACLs are what allow data visibility between zones and adaptors and are typically managed by the zone’s data steward.

ACLs are a key component of Data Governance.

ACLs are:

Permissions on out-bound data flowing from a zone and its adaptors
Controls on what in-bound data a zone or adaptor should receive
Permissions for creating, deleting and updating data records in YOUnite.

ACLs can be thought of as a series of filters that get applied to a data operation.

For example, if an update (PUT) operation is performed on a data record in a source system, the adaptor detects and routes it as a data event to the YOUnite Server. From there ACL filters would control:

What zones and adaptors would have visibility to the change
What zones and adaptors have subscribed to receive the change

Notifications

YOUnite can notify applications within an organization’s IT Ecosystem when:

Operational changes occur within YOUnite
When federated data records are added, updated, or removed in the source systems

This allows business processes to take action when certain events occur and provides a mechanism to unify source systems in the enterprise data tier.

Putting it All Together - Unifying an Organization’s Data Tier

The YOUnite Data Fabric is complete once zones, users and data domains are defined and adaptors are developed and deployed. The YOUnite Layers is an abstraction that makes it easy to see how all of the pieces work together:

Layer 1 - Data Synchronization

Adaptors makes the connections between YOUnite and the source system in the organization’s IT Ecosystem targeted to be part of the YOUnite Data Fabric. Adaptors route data events between the YOUnite Server and the source systems.

Note	Business logic is generally not part of an adaptor.

Layer 2 - Data Governance

Data governance defines what data a zone chooses to share and receive (inbound and outbound ACLs) with other zones and adaptors. For example, the HR Zone may choose to restrict changes it receives (inbound) from a system that is part of the Manufacturing zone. Or, the HR Zone may restrict changes it receives from an entire zone representing a company spin-off subsidiary and all its systems. Or, the HR zone may apply governance ACLs on its own system, preventing personal information from being shared outside of the system (outbound).

Layer 3 - Data Virtualization, Global Delete, Notifications and Federated Data Event Tracing

Federated GET (Data Virtualization)

Users and applications can make requests and gain data access through the YOUnite Server API. YOUnite becomes an operation data store when accessing data records through the API; this is known as a Federated GET.

Users can make requests for YOUnite to retrieve data records from various source systems and assemble it into one federated data record relative to the user’s unique requirements based on which systems they consider the best source of truth while considering data governance rules that need to be applied to the request.

Global DELETE

YOUnite provides data stewards through a single interface the ability to forget data across source systems in the YOUnite Data Fabric when needed for compliance; for example the user’s "right to be forgotten."

Single Platform for Data Change Notification

Notifications allow events to be generated and delivered to legacy applications to trigger business logic. If, for example, an employee is promoted, and the HR system updates the employee’s record, the adaptor attached to the HR system can detect the change and pass it on to YOUnite where it can then notify other systems that have registered interest in employee status changes.

Trace Data Lineage

All data events that flow through YOUnite Data Fabric Platform are logged and can be traced providing data compliance officers the ability to see when a record was created, updated, routed to other systems, retrieved or deleted.

Note	By default, data events are logged but the data itself is not included in the log entry. However, the data event data can be logged if using the optional YOUnite Temporal Adaptor.