UWA Recommendations
UWA - Ingest VNET.
The approach, BIG-IP VE HA pair load balancing for the transit VNET, deployed with HA Failover via LB template can easily fit into existing infrastructure without asking the customer to re-architect the entire infrastructure. Of course, the main requirement of this use case is that SNAT is allowed when traffic reaches the F5 BIG-IP VE’s.
Benefits
- This solution works for both Reverse and Forward proxies use cases
- Minimal affect to the customer’s existing setup
- All the same benefits as HA failover-via LB solution
Requirements:
- SNAT is required
- Azure Native Internal load balancer
Following these deployment patterns it enables native Azure functionality, this includes the integration into Azure Security Center and consolidation logging framework that aligns the previously mentioned Cloud Security posture.
Application Deployment Lifecycle.
As with all Cloud Journeys this document will outline the each stage and how F5’s vision of Code-To-Customer, it will be broken down into the following stages:
With each stage outlined and tools details recommendations will be made of the Code-To-Customer approach.
Provisioning
ARM Templates
Using Azure ARM templates it is possible to create a high availability (active-active/active-standby) pair of BIG-IP VE instances in Microsoft Azure. F5 Networks have grouped the available templates into the following categories:
Standalone
These templates are used to deploy a single BIG-IP VE, these are primarily used for development testing or replacing/upgrading instances that form part of traditional fail-over clusters.
Failover
These templates are designed for the deployment of more that one BIG-IP VE in a ScaleN cluster, this would be similar to UWA existing, traditional, deployment methodologies. These clusters are primary deployed in replication of traditional Active/Standby BIG-IP deployments, in the case of UWA the nominated deployment pattern is from active/active.
Autoscale
These template types deploy a group of BIG-IP VE’s that scale in and out based on thresholds nominated, BIG-IP VE’s are all deployed as Active and are primarily used to scale out an individual L7 service on a single wildcard virtual. Addition services can be provisioned using port remapping for these, these types of deployments rely upstream service to distribute traffic like DNS/GSLB or a platform’s built-in load balancer.
With the recommended configuration of Active/Active, SourceNAT is needed to ensure the egress traffic traverse the same ingress BIG-IP. Another, more advanced, alternative is to use Direct Server Return (DSR) in Azure LB that means that the Azure LB will not perform Destination NAT on the traffic which will arrive at the backend pool with the correct destination IP address, this is commonly used in scenarios that require 1 VIP per application within BIG-IP Cluster.
Declarative Onboarding (DO)
F5 Declarative Onboarding (DO) is an F5 offering that provides a framework to automate BIG-IP onboarding via Declarative REST APIs. Similar to AS3, DO provides a foundation to enable F5’s Infrastructure as Code (IaC) deployment methodologies.
DO automates L1-L3 on-boarding for BIG-IP, making BIG-IP available on the network and ready to accept L4-L7 Application Services configurations. The following example declaration on-boards a clustered BIG-IP system, further explanation of this can be found locate at Composing a Declarative Onboarding declaration for a cluster of BIG-IPs.
{
"schemaVersion": "1.0.0",
"class": "Device",
"async": true,
"label": "Onboard BIG-IP into an HA Pair",
"Common": {
"class": "Tenant",
"hostname": "bigip1.example.com",
"myLicense": {
"class": "License",
"licenseType": "regKey",
"regKey": "AAAAA-BBBBB-CCCCC-DDDDD-EEEEEEE"
},
"myDns": {
"class": "DNS",
"nameServers": [
"8.8.8.8",
"2001:4860:4860::8844"
],
"search": [
"f5.com"
]
},
"myNtp": {
"class": "NTP",
"servers": [
"0.pool.ntp.org",
"1.pool.ntp.org",
"2.pool.ntp.org"
],
"timezone": "UTC"
},
"root": {
"class": "User",
"userType": "root",
"oldPassword": "foo",
"newPassword": "bar"
},
"admin": {
"class": "User",
"userType": "regular",
"password": "asdfjkl",
"shell": "bash"
},
"anotherUser": {
"class": "User",
"userType": "regular",
"password": "foobar",
"partitionAccess": {
"Common": {
"role": "guest"
}
}
},
"myProvisioning": {
"class": "Provision",
"ltm": "nominal"
},
"internal": {
"class": "VLAN",
"tag": 4093,
"mtu": 1500,
"interfaces": [
{
"name": "1.2",
"tagged": false
}
]
},
"internal-self": {
"class": "SelfIp",
"address": "10.10.0.100/24",
"vlan": "internal",
"allowService": "default",
"trafficGroup": "traffic-group-local-only"
},
"external": {
"class": "VLAN",
"tag": 4094,
"mtu": 1500,
"interfaces": [
{
"name": "1.1",
"tagged": false
}
]
},
"external-localself": {
"class": "SelfIp",
"address": "10.20.0.100/24",
"vlan": "external",
"allowService": "none",
"trafficGroup": "traffic-group-local-only"
},
"external-self": {
"class": "SelfIp",
"address": "10.20.0.200/24",
"vlan": "external",
"allowService": "none",
"trafficGroup": "traffic-group-1"
},
"default": {
"class": "Route",
"gw": "10.10.0.1",
"network": "default",
"mtu": 1500
},
"configsync": {
"class": "ConfigSync",
"configsyncIp": "/Common/internal-self/address"
},
"failoverAddress": {
"class": "FailoverUnicast",
"address": "/Common/internal-self/address"
},
"failoverGroup": {
"class": "DeviceGroup",
"type": "sync-failover",
"members": ["bigip1.example.com", "bigip2.example.com"],
"owner": "/Common/failoverGroup/members/0",
"autoSync": true,
"saveOnAutoSync": false,
"networkFailover": true,
"fullLoadOnSync": false,
"asmSync": false
},
"trust": {
"class": "DeviceTrust",
"localUsername": "admin",
"localPassword": "pass1word",
"remoteHost": "/Common/failoverGroup/members/0",
"remoteUsername": "admin",
"remotePassword": "pass2word"
}
}
}
Configuration
F5 BIG-IP VE’s, once deployed, may be configured to either suit UWA DevOps methodology levering Azure DevOps or in-house deployment pipelines. As with the variations of provisioning of BIG-IP VE’s, the same variety exists to suit the provisioning of application services such as;
F5 Application Services Templates (FAST)
F5 FAST, provides a way to streamline deployment of AS3 applications onto BIG-IP using AS3 deployment patterns, or templates. FAST is the next phase of evolution for F5, unlocking new capabilities, aligning to multi-cloud, injecting automation, and empowering customers with our best-in-class application services.
This allows, through the use of a tabbed gui within the BIG-IP console, the construction of Virtual Server configuration that produces a AS3 declaration that can be copied, committed to source or templates to be set as AS3 payloads on REST API operations.
Further information, how-to’s and additional examples are located at the FAST documentation site.
Application Services 3 Extension (AS3)
Application Services 3 Extension is a flexible, low-overhead mechanism for managing application-specific configurations on a BIG-IP system. AS3 uses a declarative model, meaning you provide a JSON declaration rather than a set of imperative commands.
The declaration represents the configuration which AS3 creates on a BIG-IP system. AS3 is well-defined according to the rules of JSON Schema, and declarations validate according to JSON Schema. AS3 accepts declaration updates via REST (push), reference (pull), or CLI (flat file editing).
An example AS3 is as follows, this contains;
- Partition (Tenant) named Sample_http_01
- HTTP VIP called servceMain
- A pool named web_pool
- Persistence provide based on JSESSIONID cookie
{
"class": "AS3",
"action": "deploy",
"persist": true,
"declaration": {
"class": "ADC",
"schemaVersion": "3.0.0",
"id": "fghijkl7890",
"label": "Sample 1",
"remark": "HTTP with custom persistence",
"Sample_http_01": {
"class": "Tenant",
"A1": {
"class": "Application",
"template": "http",
"serviceMain": {
"class": "Service_HTTP",
"virtualAddresses": [
"10.0.6.10"
],
"pool": "web_pool",
"persistenceMethods": [{
"use": "jsessionid"
}]
},
"web_pool": {
"class": "Pool",
"monitors": [
"http"
],
"members": [{
"servicePort": 80,
"serverAddresses": [
"192.0.6.10",
"192.0.6.11"
]
}]
},
"jsessionid": {
"class": "Persist",
"persistenceMethod": "cookie",
"cookieMethod": "hash",
"cookieName": "JSESSIONID"
}
}
}
}
}
Further information, along with VSCode Schema validation, is currently located at Application Services 3 Extension Documentation
Ansible
BIP-IP’s, both physical and virtual appliances, can also be provisioned, configured and managed with the application-deployment tool Anisble. From Ansible 2.9+ BIG-IP and BIG-IQ supports Ansible Galaxy, a website - Galaxy - where users can obtain collection of roles that also support F5 Ansible Galaxy Modules
An example Ansible playbook declaration for configuration HA Pair of BIG-IP’s, Ansible templates - as below - using Jinja2
---
- name: Common HA configuration on all devices
hosts: f5-test
connection: local
vars:
provider:
server: "{{ ansible_host }}"
server_port: "{{ f5_server_port }}"
user: "{{ f5_username }}"
password: "{{ f5_password }}"
validate_certs: "{{ validate_certs }}"
tasks:
- include_tasks: validate.yaml
- name: Add VLANs
bigip_vlan:
name: "{{ item.vlan.name }}"
tag: "{{ item.vlan.tag }}"
untagged_interfaces: "{{ item.vlan.interfaces }}"
provider: "{{ provider }}"
loop: "{{ nets }}"
- name: Add Self-IPs
bigip_selfip:
name: "{{ item.name }}"
address: "{{ item.address }}"
netmask: "{{ item.netmask }}"
vlan: "{{ item.vlan.name }}"
provider: "{{ provider }}"
allow_service: default
loop: "{{ nets }}"
- name: Configure hostname
bigip_hostname:
hostname: "{{ inventory_hostname }}"
provider: "{{ provider }}"
- name: Set CMI device parameters
bigip_device_connectivity:
config_sync_ip: "{{ config_sync_ip }}"
mirror_primary_address: "{{ mirror_primary_address }}"
unicast_failover: "{{ unicast_failover }}"
provider: "{{ provider }}"
- name: Primary device specific configuration
hosts: f5-test[0]
connection: local
vars:
provider:
server: "{{ ansible_host }}"
server_port: "{{ f5_server_port }}"
user: "{{ f5_username }}"
password: "{{ f5_password }}"
validate_certs: "{{ validate_certs }}"
tasks:
- name: Trust peer device
bigip_device_trust:
peer_server: "{{ hostvars[item].ansible_host }}"
peer_hostname: "{{ hostvars[item].inventory_hostname }}"
peer_user: "{{ f5_username }}"
peer_password: "{{ f5_password }}"
provider: "{{ provider }}"
loop: "{{ groups['f5-test'][1:] }}"
- name: Add Device Group
bigip_device_group:
name: deviceGrp
auto_sync: yes
type: sync-failover
provider: "{{ provider }}"
- name: Add members to the device group
bigip_device_group_member:
device_group: deviceGrp
name: "{{ item }}"
provider: "{{ provider }}"
loop: "{{ groups['f5-test'] }}"
- name: Perform a config sync
bigip_configsync_action:
device_group: deviceGrp
sync_device_to_group: yes
provider: "{{ provider }}"
Further information, along with further references are located;
Or how to run F5 Ansible Playbooks for Tower and AWX.
HashiCorp
HashiCorp, the creators of Terraform OpenSource IaC, also have Terraform Cloud, the enterprise offering that supports divisions along with MFA and SSO, that incorporates Sentinel that enables the shift to multi-cloud infrastructure. Like DO, AS3 and Ansible Terraform also support DevOps pipelines and GitFlow.
An example Terraform Plan, main.tf, is defined as follows;
#
# Set minimum Terraform version and Terraform Cloud backend
#
terraform {
required_version = ">= 0.12"
}
/*
# Create a random id
*/
resource "random_id" "id" {
byte_length = 2
}
/*
# Create VPC as per requirements
*/
module "vpc" {
source = "../modules/services/network"
providers = {
aws = aws.secops
}
prefix = "${var.project}-${var.environment}"
cidr = var.cidr
azs = var.azs
env = var.environment
random = random_id.id
}
/*
# Create BIG-IP host as per requirements
*/
module "bigip" {
source = "../modules/functions/bigip"
providers = {
aws = aws.secops
}
prefix = "${var.project}-${var.environment}"
cidr = var.cidr
azs = var.azs
env = var.environment
vpcid = module.vpc.vpc_id
public_subnets = module.vpc.public_subnets
private_subnets = module.vpc.private_subnets
database_subnets = module.vpc.database_subnets
random = random_id.id
keyname = var.ec2_key_name
keyfile = var.ec2_key_file
}
# Create Jump host as per requirements
module "jumphost" {
source = "../modules/functions/jumphost"
providers = {
aws = aws.secops
}
prefix = "${var.project}-${var.environment}"
region = var.region
cidr = var.cidr
azs = var.azs
env = var.environment
vpcid = module.vpc.vpc_id
public_subnets = module.vpc.public_subnets
public_nic_ids = module.bigip.public_nic_ids
docker_private_ip = module.docker.docker_private_ip
random = random_id.id
keyname = var.ec2_key_name
keyfile = var.ec2_key_file
bigip_mgmt_addr = module.bigip.mgmt_addresses
bigip_mgmt_dns = module.bigip.mgmt_public_dns
bigip_password = module.bigip.bigip_password
bigip_private_add = module.bigip.private_addresses
}
Further information on the use of Terraform;
along with F5 Terraform Resources that can also be found on F5 CloudDocs.
Monitoring & Analytics
F5 Automation Tool Chain - Telemetry Streaming (TS)
Consistent with IaC, Telemetry Streaming (TS) is F5 Networks JSON declarative model of streaming events and statistics to the customers preferred data visualization, it also supports native integration with both Microsoft Azure Log Analytics and Azure Application Insights along with other known logging solutions.
For complete visibility BIG-IP and TS also integrate with Azure Sentinel.
F5 Telemetry Streaming also provides metrics and analytics to F5-aaS Cloud Offers along with rich application insights and understanding when deployed alongside BIG-IQ Centralised Management (CM).
An example stanza for the configuration and declaration of TS is as follows:
{
"class": "Telemetry",
"TS_System": {
"class": "Telemetry_System",
"systemPoller": {
"interval": 60,
"enable": true,
"trace": false,
"actions": [
{
"setTag": {
"tenant": "`T`",
"application": "`A`"
},
"enable": true
}
]
},
"enable": true,
"trace": false,
"host": "localhost",
"port": 8100,
"protocol": "http"
},
"TS_Listener": {
"class": "Telemetry_Listener",
"port": 6514,
"enable": true,
"trace": false,
"match": "",
"actions": [
{
"setTag": {
"tenant": "`T`",
"application": "`A`"
},
"enable": true
}
]
},
"Poller":{
"class":"Telemetry_System_Poller",
"interval":60,
"enable":true,
"trace":false,
"allowSelfSignedCert":false,
"host":"localhost",
"port":8100,
"protocol":"http"
},
"Beacon_Consumer":{
"class":"Telemetry_Consumer",
"type":"Generic_HTTP",
"host":"ingestion.ovr.prd.f5aas.com",
"protocol":"https",
"port":50443,
"path":"/beacon/v1/ingest-telemetry-streaming",
"method":"POST",
"enable":true,
"trace":false,
"headers":[
{
"name":"grpc-metadata-x-f5-ingestion-token",
"value":"`>@/passphrase`"
}
],
"passphrase":{
"cipherText":"<it's one of those secret things>"
}
},
"Statsd_Consumer": {
"type": "Statsd",
"host": "elk.local",
"protocol": "udp",
"class": "Telemetry_Consumer",
"port": 8125,
"enable": true,
"trace": false
},
"ElasticSearch_Consumer": {
"index": "f5telemetryindex",
"protocol": "http",
"dataType": "_doc",
"class": "Telemetry_Consumer",
"host": "elk.local",
"type": "ElasticSearch",
"port": 9200,
"enable": true,
"trace": false
},
"SumoLogic_Consumer": {
"class": "Telemetry_Consumer",
"type": "Sumo_Logic",
"host": "collectors.sumologic.com",
"protocol": "https",
"port": 443,
"enable": true,
"trace": false,
"path": "/receiver/v1/http/",
"passphrase": {
"cipherText": "<another one of those secrets>"
}
},
"schemaVersion": "1.6.0"
}
The previous TS example creates a Virtual Server (VS) local listener on the BIG-IP appliance, port 6517, then a system poller with an interval of 60 seconds on port 8100 then finally configures three consumers of TS:
- F5a-aaS Beacon
- StatsD local listener
- ElasticSearch Indexes.
BIG-IP High Speed Logging (HSL) shares a similar framework to that of Telemetry Streaming, this enables an ease of upgrade from HSL deployment. Telemetry Streaming also support the redaction/masking of information on instance.
Backup and Restoration
As with all Cloud Journeys the requirements for the backup and restoration of configuration is nullified to align with Cloud Native Immutable Infrastructure Principals.
_”The gold standard for cloud infrastructure is for it be able to be provisioned without any assistance. The tools that provision that infrastructure should accept declarative configuration as inputs.”_
F5 Automated Tool Chain combined with Azure ARM templates enables the ease of secure deployments in an agile manner, if the both the workload and data classification require it F5 Networks Secure Cloud Architecture (SCA) Solutions.
As with migrations, workload components need configuration up-lifted or refreshed and understanding this F5 offers assistance through both professional services and community channels within Slack or GitHub.
Recommendations
With cloud to align with the elasticity and immutability I believe that operating in the cloud requires automation, therefore one should not shy away from automated updates to User Defined Routing (UDR’s) given that tools such as F5’s Cloud Failover Extension (CFE) to be native with mature cloud operations.
The use of BIG-IP v15.1.x also brings support for Accelerated Networking on Azure that has support for SR-IOV, also improvements to cloud-init both an upgrade to version 18.5 and additional support two custom modules, Set password and TMOS Declared. Using these modules you can change the built-in TMOS admin and root passwords and leverage F5 Automation Toolchain (including, Declarative Onboarding and F5 Application Services Extension) respectively.
BIG-IP v15.1.x also brings support for the previously mention F5 Application Services Templates (FAST) that allows the creation application templates for virtual server configuration and for these to be deployed as AS3 applications.
With outbound traffic traversing the BIG-IP as per the 3nic deployment pattern, it allows for the inspection, analysis, securing, etc - not only because it allows apps to see the true source IP.
Finally, best practices to follow when on the early stages of a cloud journey;
- use of GitFlow or Git Branching DevOps
- use CI/CD tools, Azure DevOps, for speed of deployments
- template based deployments both non-prod and prod (IaC)
- use of DO and AS3 to keep configuration off box
- App & Dev teams configuring partitioned BIG-IP using declarative deployments.
Conclusion
As it has been touched on there is multiple ways to deploy and dictate the architectural requirements of individual workloads, in a immutable declarative way that removes the need for break-glass and ClickOps configuration steps.
By framing the application flows/workloads as a series of objects rather than a single blob/entity it allows this flow, Code to Customer, to be simplified in all stages of the applications life-cycle.