Practical AWS for Large Organisations

Table of Contents

  1. Overview
    1.1. Service Catalogs
    1.2. Automated Push Security
    1.3. Standardised Support Wrapper Patterns
    1.4. Alignment to Industry Standards
    1.5. Scalable Account Management
  2. Accounts Structure
    2.1. Landing Zone
  3. Pipelined Data Flows and Reactive Architecture
  4. Central Services
    4.1. Base Infrastructure
    4.2. Persistence Infrastructure
    4.3. Service Infrastructure
  5. Pluggable Compute Solutions
    5.1. Standard Quick Start Products
    5.2. Commercial Marketplace Products
    5.3. Custom In-House Products
  6. CI/CD Pipelines
  7. Solutions

Overview

There are a number of components that enable business AWS management at scale. Key is to bring them together in a consistent and coherent way through combining and enhancing AWS templates and professional services solutions. Those that are most useful for scaled management such as multi-account landing zones and cross-account management are available but not necessarily fully automated. However, using these as the basis of our scaled strategy allows building on good foundations for future improvement. Looking at the AWS account landscape as a whole we can break it down into a handful of important areas.

Centralisation of Key Functionality

Through centralising identity and access management we can achieve an ease of management in user access and security. Additions of a security and audit account allows for guarantees on log integrity and provides a high level view across the threat spectrum.

Service Catalogs

By using a central store of defined Cloudformation products we enable users to one-click deploy from a set of standard and assured approaches to account creation, application development, live production service usage and so on. As a feedback loop, the most useful patterns from around the business can be taken in and shared with all other parts of the business. This means that patterns improvements and fixes can be centrally pushed out to all the accounts using them. This combines the best balance of innovation by all users of AWS, and simple off-the-shelf deployments of large and complex usage-hardened and IA passed infrastructure. Business criteria on the treatment of logs and the quality of live services can also be specified and made coherent.

Automated Push Security

By deploying centrally managed service catalog patterns across the organisation we can utilise security alerting and policy enforcement automatically with relative simplicity. This gives us push notifications on security policy mis-configurations and known attacks or insecure designs. We are also able to apply defined security policies through AWS Config across the whole estate. Automated responses to infrastructure problems or attacks can be put in-place and at scale with ease.

Standardised Support Wrapper Patterns

Areas such as monitoring and logging and network set-up have a limited set of patterns that are industry standards. By providing catalogued solutions for these, all services present a equivalent look, quality and security level. However, there are many reasons why different compute and persistence solutions might be required. For instance where we have containerised services compared to more traditional virtual machine server layouts. As the links between this and elements such as Cloudwatch are abstracted and standardised, these parts should be able to be varied to meet the needed solution very easily while still being encompassed by the logging and security outer components.

Alignment to Industry Standards

We utilise the templates already in existence and published by AWS, adapting and enhancing when required. There are unlikely to be many non-standard use cases for government in the cloud and harnessing already established patterns and templates will accelerate our adoption and share the development burden.

Scalable Account Management

Using scripted account creation and harnessing StackSets and AWS Organisations we can group functionality and Cloudformation products together to develop rapidly deployable account flavours, accounts that come with a set of pre-baked functionalities and a definable flexability to apply a range of solutions.

Accounts Structure

Landing Zone

References: [references_accounts], [references_services] Working to the standard AWS Enterprise Landing Zone guidance and multi-account strategy the focus is on providing standardised accounts, IAM groups, networks, and so on in a simple manner. Importantly this balances the standardisation of the central functions such as user management, the provision of out-of-the-box templated solutions, with keeping the flexibility within accounts to create needed solutions to problems. This gives the main benefits of,

  • Minimisation of management burden arising from cloud scale.

  • Account and operational structures are designed and assured from configuration

  • Rapid development of solutions from centrally stored templates.

  • Solve once across large and disparate organisations as successful solutions are pulled in and shared from all areas.

To implement this we lay out a handful of useful accounts defining this functionality .

Overview of AWS account structure. Central accounts are created for defined once-only functionality and management functions. Service accounts are created in defined flavours from catalog products. Gateway accounts hold common external network links.

Master Organisation Account

Root of the organisational tree and consolidation of billing. We apply our top-level organisational control policies here, the base the policy frameworks that apply organisation-wide. By grouping accounts into organisational units such as non-resource accounts, development/production accounts, or any other logical grouping, we can restrict access and behaviours in accounts in an easily managed way.

Cross-Account Management Account

Implementing the AWS Cross Account Manager Solution to allow for centralisation of IAM groups and access with delegation to other-accounts.

Cross account management utilises a master account backed by Active Directory to manager users, groups, and policies centrally, with the roles mirrored to sub accounts. On login to the central portal, users get temporary access delegated to the subaccounts.

Shared Services Account

Central store for Service Catalog patterns. Used for initial deployment of templates to set up accounts. Contains deployable products to set up full service VPC types and compute deployment layouts.

AWS Reference Solutions

AWS solutions already cover setting up continuous deployment pipelines for Service Catalog products. AWS: Building a Continuous Delivery Pipeline for AWS Service Catalog

The shared services account serves as the collection point for templates that can be deployed to many accounts either as part of central management or by users of those accounts. Security fixes, new patterns and enhancements can be pushed centrally or pulled by each account.

Security and Audit Account

[@Blend] Centralised logging and auditing. All services Cloudwatch, cloudtrail, and VPC logs are shipped here through Kinesis pipelines. Data is backed up to S3 and then to glacier with MFA delete. This collection of data can then be used for broad spectrum security alerting and analytics.

Metrics, logs, VPC flow logs are centralised in per-account Cloudwatch. This data is used within account for automated security responses. Additionally the data is filtered and streamed using Kinesis to a central account for further analytics and audit.

Billing Account

Centralisation, visualisation and alerting on all accounts billing data.

Pipelined Data Flows and Reactive Architecture

Alongside the standardised deployment code-* tools and logging to Cloudwatch, the pipelines are extended to include Cloudwatch events to Lambda reactions, and Kinesis pipelines to a centralised security account (Figure [fig:moj_aws_cloud_pluggable_compute]).

While the actual compute solutions differs, the surrounding infrastructure remains the same. Code-* tools provide the pipeline, Cloudwatch, the logging and alerting. Lambda reactions trigger from Cloudwatch Events to provide reactive actions on services and infrastructure. AWS Config overlays the infrastructure to provide governance. Through Kinesis, selections of the Cloudwatch data are centralised into a secure account.

Central Services

Service infrastructure components

Base Infrastructure

VPC and networking

The underlying VPC, NAT, VPN layout and other networking.

Persistence Infrastructure

The database, S3 and other storage attached to the base environment.

Service Infrastructure

These are components that are service specific and group together conceptually.

Deploy pipeline

The CICD pipeline from git repo to deploy job.

DNS Routing

The DNS entries defined in Route 53 that redirect to the correct endpoint.

Load Balancer Routing

The listener, SSL certificate, healthcheck and hostname routing that point to the correct compute resource.

Compute

The actual running compute resource such as EC2 instances, ECS services etc.

A service is broadly made up of 4 components. CICD, Compute backends, Route 53 entries, and Load balancer routing

Pluggable Compute Solutions

Different Service Catalog product stacks can be deployed into accounts to provide compute in whatever standardised variation required by a service. We can describe a number of possible approached here but in reality this is where the most flexibility lies for innovation and differentiation.

Standard Quick Start Products

A range of Service Catalog products deployed from a central account that provide standardised AWS platforms such as CI/CD Windows layouts and commercial stacks such as Vault. References: [references_services], [references_general]

Commercial Marketplace Products

Products pre-approved or purchased from the AWS Marketplace that are then placed in the Service Catalog for business wide use.

Custom In-House Products

Custom in-house solutions provided as one-click deployments. Pre-designed stacks that cover specialised in-house uses or improvements and innovation.

Example: ECS Backed Beanstalk

Simple ECS Support with .ebextensions to provide full service support (Figure [fig:moj_aws_pluggable_compute_beanstalk]). This pattern allows for using the AWS Beanstalk interface, which is a nice frontend for users. In addition, standard AWS Code Pipeline tools are used for CI/CD, also giving easy to use frontends for users.

Compute

Elastic Beanstalk environments deployed into pre-existing account VPC’s. (Service Catalog Product)

CI/CD

Standard Github to Code Pipeline pattern using Code Build and Code Deploy. (Service Catalog Product)

Routes

Using ALB host header matching and Route 53 to provide standardised internal or external routing to service pods on ECS. (ebextensions)

Certificates

The ACM SES pattern for automated certificate manager use. (ebextensions)

Cron Jobs

Cron jobs are eb-deployed onto beanstalk. There is the more AWS-specific utilisation of workers as an option. (ebextensions)

Container Env Variables

Dummy variables can be supplied through ebexetensions and then filled with real credentials inside of beanstalk. This avoids the risks of storing secrets outside of the beanstalk environment and the cost of HA credential storage and orchestration. (ebextensions)

option_settings:
  - option_name: MY_SECRET
    value: DUMMY

Using standard beanstalk gives a simple method of working with compute infrastructure. Standard ebextensions can augment the basic beanstalk setup to provide elements such as ACM, routes into microservices on ECS through host header matching, etc.

Example: Multi-tier VM platforms

More customised traditional platforms utilising Patch Manager and more config management.

CI/CD Pipelines

AWS Reference Solutions

Separation of non-production and production accounts can be achieved while allowing for transferring artefacts through the different environments through the cross-account deployment pattern provided by AWS. AWS: Building a Secure Cross-Account Continuous Delivery Pipeline

Solutions

Base

Monitoring and Logging

Compute

Beanstalk

EC2 Container Service

Enabler Solutions

SES Mail Forwarder

Parameter Store Sync

Security Policy

Organisational Solutions

Centralised Logging and Security

Service Catalog Continuous Deployment

References

Accounts

ARC314 Enabling Enterprise Migrations: Creating an AWS Landing Zone ENT203 Enterprise Fundamentals: Design Your Account and VPC Architecture SAC323 Manage Multiple AWS Accounts with AWS Organizations SAC320 Deep Dive: Security and Governance Across a Multi-Account Strategy AWS Multi-Account Security Strategy AWS Cross Account Manager AWS: Building a Continuous Delivery Pipeline for AWS Service Catalog
AWS: Building a Secure Cross-Account Continuous Delivery Pipeline Blend: Centralizing Logs in an Isolated AWS Account

Security

SAC304 Predictive Security: Using Big Data to Fortify Your Defenses

Services

Copying AWS Marketplace products to the Service Catalog

Automation

General

AWS Quick Start Repository AWS Tagging Strategies