AWS Notes

🚩 Issue #83 | 6 August 2023

▪️ Amplify – CloudWatch Logger Feature for Swift and Android
▪️ Application Composer – Updates: Undo and Redo, Export Canvas, Local Sync Mode
▪️ Aurora
     ▫️ Local Write Forwarding for MySQL
     ▫️ Supports Automatic Backup of Deleted Clusters
▪️ Backup – Improved Performance for S3 Buckets
▪️ Batch on Fargate – Support for Linux ARM64 and Windows x86 Containers in Console
▪️ Batch – Price Capacity Optimized Allocation Strategy for Spot Instances
▪️ Bedrock – Claude 2 Foundation Model from Anthropic
▪️ Billing Console – Invoice Balance Due Information and Invoice Amount
▪️ Clean Rooms – Custom Query and Template Capabilities
▪️ CloudFormation – Enhanced Deletion Policies for Dev-Test Cycle
▪️ CloudWatch Internet Monitor – Customizable Thresholds for Health Events
▪️ Config – Supports 19 New Resource Types
▪️ Connect
     ▫️ Archiving and Deleting Flows from the UI
     ▫️ Automatic Activity Scheduling Based on Shift Duration
     ▫️ Custom Flow Block Titles
     ▫️ Flows UI Toolbar and Ability to Add Notes
     ▫️ Scheduling Agent Time-off Balance and Group Allowance Support
     ▫️ Automated Flexible Days Scheduling
     ▫️ Support for Flow-only Attributes
▪️ Database Migration Service – Enhanced Homogeneous Migrations for MariaDB
▪️ EBS – Up to 128 Volume Attachments per EC2 Instance
▪️ EC2 – Introduction of M7i-flex and M7i Instances
▪️ EC2 VT1 – Improved Control of Stream Quality, Latency, and Bandwidth
▪️ EFS – Supports up to 55,000 IOPS per File System
▪️ EKS – Easier Configuration and Use of EFS for Persistent Shared File Storage
▪️ EKS Fargate – Additional Ephemeral Storage
▪️ ElastiCache – Support for M7g and R7g Graviton3-based Nodes
▪️ EMR Serverless – API Support for Application UIs
▪️ EventBridge Scheduler – Schedule Deletion after Completion
▪️ Inspector – Enhanced Vulnerability Intelligence in Findings
▪️ Instance Refresh – Rollbacks Triggered by CloudWatch Alarms
▪️ IoT Core – New Certificate Signing & Key Generation Algorithms
▪️ Lake Formation – Availability in Two Additional Regions
▪️ NAT Gateway – Availability in US West Phoenix Local Zone
▪️ Polly – New Belgian French Female Voice Isabelle
▪️ RDS Custom for SQL Server – Supports Local Time Zones
▪️ RDS for Oracle – Support for Replicas for Single-tenant Instances
▪️ RDS for SQL Server – Supports Secondary Host Metrics in Enhanced Monitoring
▪️ RDS – Simplifies Connectivity Between an RDS or Aurora Database and AWS Lambda
▪️ Redshift – Open Sourced Integration for Apache Spark
▪️ Resilience Hub – Application Resilience Drift Detection
▪️ SageMaker – New Direct Integration with Salesforce Data Cloud
▪️ SageMaker Studio – Pre-built Docker ‘Distribution’ for Machine Learning
▪️ SageMaker – Support for ml.p5 Instances for Model Training
▪️ SQS – Increased Throughput Quota for FIFO High Throughput Mode
▪️ Thinkbox Deadline – Release of 10.3
▪️ Transit Gateway Flow Logs – Available in Five Additional Regions
▪️ Translate – Language Detection for Real Time Document Translation
▪️ VPC IP Address Manager – Publishes IP Address Utilization Metrics for AWS Resources
▪️ VPC – Supports Primary IPv6 Address on an Elastic Network Interface
▪️ WorkSpaces – Microsoft 365 Apps for Enterprise Availability

Новый AWS Region — Тель-Авив, Израиль: 🎉

https://aws.amazon.com/blogs/aws/now-open-aws-israel-tel-aviv-region/

Идентификатор il-central-1, как и в подавляющем большинстве других регионов, имеет 3 AZ.

✅ Итого на теперь всего — 32 региона.

#AWS_Regions

В Route 53 завезли .tech, .store, .press, .games домены.

https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-route-53-support-14-top-level-domains/

Issue #82 | 30 July 2023 2/2

▪️ RDS Custom for Oracle CloudFormation Templates support
▪️ RDS for Oracle Oracle Application Express version 23.1
▪️ RDS storage optimization process indicator
▪️ Redshift
     ▫️ automatic mounting of AWS Glue Data Catalog
     ▫️ querying Apache Iceberg tables
▪️ Route 53 +14 Top-Level Domains
▪️ SageMaker Canvas
     ▫️ custom Amazon S3 output location for ML artifacts
     ▫️ Document Queries powered by Amazon Textract
     ▫️ expands data preparation with five new capabilities
     ▫️ sharing ML models with QuickSight
     ▫️ supports training ML models with different objective metrics
▪️ SageMaker Feature Store cross account sharing, discovery and access
▪️ SageMaker JumpStart SDXL 1.0 foundation model from Stability AI
▪️ SimSpace Weaver
     ▫️ custom container images and new clock
     ▫️ multiple spatial domains per simulation
▪️ SQS JSON protocol | Preview
▪️ Supply Chain Demand Planning
     ▫️ better UI
     ▫️ CloudTrail support for event history
     ▫️ multiple override management
▪️ Transcribe Toxicity Detection for spoken conversations
▪️ Transfer Family SFTP connectors
▪️ VPC IP Address Manager Public IP Insights
▪️ Wickr federated data retention for internal and external conversations

😶‍🌫️ Issue #82 | 30 July 2023 1/2

▪️ ACE Pipeline Manager bulk import in-line editing
▪️ AppStream 2.0 Graphics G5 instances
▪️ Billing Conductor auto-account association
▪️ Chime SDK more compositing options for live connector and media capture
▪️ CloudFormation looping function
▪️ CloudFormation StackSets direct access to resources and stack drift info
▪️ CloudWatch Application Insights multi-app instance deployments
▪️ CloudWatch Internet Monitor Network Load Balancers as a resource type
▪️ CloudWatch Synthetics new Synthetics NodeJS runtime version 5.0
▪️ CodeCatalyst Elastic Kubernetes Service depoyments
▪️ Connect Customer Profiles rule-based identity resolution
▪️ Control Tower additional proactive controls
▪️ Cost Management visual Savings Plans recommendations
▪️ DataSync copying data to and from Azure Blob Storage
▪️ DocumentDB document compression support
▪️ DynamoDB Database Encryption SDK | GA
▪️ EC2
     ▫️ C7gd, M7gd, and R7gd Instances
     ▫️ P5 instances, optimized for generative AI and HPC | GA
▪️ EC2 Spot Instances Blueprints ECS templates
▪️ ECS console now has enhanced support for task definition workflows
▪️ ECS Service Discovery  +5 regions
▪️ Elastic Disaster Recovery +5 regions
▪️ EMR enhanced cluster scaling experience
▪️ EMR on EC2 native LDAP authentication
▪️ EMR Serverless
     ▫️ retrieving secrets from Secrets Manager
     ▫️ storing logs in CloudWatch
▪️ GameLift expands SDK support for Unity based game developers
▪️ Glue DataBrew Recipes now include Glue jobs
▪️ Glue for Apache Spark  Snowflake connectivity | GA
▪️ Glue Studio Redshift Serverless support
▪️ Health delegated administrator
▪️ HealthImaging process medical images at petabyte scale | GA
▪️ IAM Roles Anywhere credential helper adds support for OS certificate stores
▪️ IoT Core MQTT support for Device Location feature
▪️ Lake Formation Read-Only Administrator role
▪️ Lambda Python 3.11 runtime
▪️ Managed Blockchain Access and Query | GA
▪️ Marketplace Vendor Insights PCI DSS, HIPAA, and GDPR certifications
▪️ MQ
     ▫️ customer managed configuration for RabbitMQ brokers
     ▫️ RabbitMQ version 3.11 and new operator policies
▪️ Omics workflow run queuing
▪️ Partnership
     ▫️ Connect Ready Program
     ▫️ ECS Delivery Specialization
▪️ Personalize peroonalize your search results with OpenSearch
▪️ QuickSight
     ▫️ introduces Analysis file menu
     ▫️ Snapshot Export APIs

Первый тост за localhost!

Lambda + Python 3.11

https://aws.amazon.com/blogs/compute/python-3-11-runtime-now-available-in-aws-lambda/

#Lambda

​​Loops in CloudFormation: !ForEach 💪

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-foreach.html

AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::LanguageExtensions'
Resources:
'Fn::ForEach::Tables':
- TableName
- [Points, Score, Name, Leaderboard]
- 'DynamoDB${TableName}':
Type: 'AWS::DynamoDB::Table'
Properties:
TableName: !Ref TableName
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: '5'
WriteCapacityUnits: '5'

Fn::ForEach can be used in the sections:

▫️ Resource
▪️ Resource properties
▫️ Conditions
▫️ Outputs

Fn::ForEach parameters

Identifier (String) → Identifier is used to refer to the current element we’re iterating over within the Collection (Array of Strings).

Collection (Array of Strings) → Array of values that the Identifier can take.

OutputKey (String) → The key of the resulting key-value pair for the given element in the collection that will be merged to the parent object.

OutputValue (Any) → The value of the resulting key-value pair for the given element in the collection that will be merged to the parent object.

Note: the syntax of Fn::ForEach declaration has a suffix where the UniqueLoopName is used to identify the loop. This allows multiple Fn::ForEach function references to be declared on a given level.

#CloudFormation

Самые производительные AI/ML EC2 инстансы - P5 c Nvidia H100 вышли в GA

https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/

Good Demo for GameTech on Unity Engine + TeamCity CI/CD + Rider IDE
https://www.twitch.tv/videos/1882050229

🟢 Issue #81 | 23 July 2023

▪️ Amplify JS lib better performance
▪️ AppConfig Agent simplifies feature flag and config use for EC2
▪️ CloudWatch ML backed Logs Insights pattern query command
▪️ CodeCatalyst workflows triggered by GitHub pull requests
▪️ Config advanced queries +65 resource types
▪️ Connect
     ▫️ CloudFormation support for routing profiles and queues
     ▫️ pre-defined Contact Lens conversational analytics metrics
▪️ Connect Cases case assignment
▪️ Connect Wisdom chat agents
▪️ ECS domainless gMSA authentication
▪️ Elemental MediaTailor cue ad tags in Channel Assembly
▪️ EMR on EKS Apache Spark with Java 17
▪️ Fargate faster container startup using Seekable OCI
▪️ Glue Crawlers Apache Hudi Tables
▪️ IoT Device Defender monitoring of device disconnect durations
▪️ IVS rendition filtering and higher frequency thumbnails
▪️ Lake Formation delegation of LF-Tag management
▪️ Lambda & EventBridge Pipes enhanced filtering
▪️ Lex Introducing Analytics
▪️ PrivateLink CloudWatch Contributor Insights integration
▪️ Redshift QUALIFY clause in SELECT SQL statement
▪️ Redshift ML integration with Amazon Forecast
▪️ Route 53 Resolver is now available on AWS Outposts rack
▪️ SageMaker JumpStart Meta Llama 2 foundation models
▪️ SNS mobile push notifications in 12 new regions
▪️ Tools Lambda Annotations Framework for .NET.
▪️ Translate real time translation of Docx files
▪️ WAF URI path aggregation key for rate-based rules

✅ Issue #80 | 16 July 2023

▪️  Aurora PostgreSQL pgvector for vector storage and similarity search & version updates
▪️  Batch on Fargate Linux ARM64 and Windows x86 containers in CLI/SDK
▪️  CloudFront 3072-bit RSA certificates
▪️  Connect programmatically delete Routing Profiles and Queues
▪️  DMS Redshift Serverless support
▪️  DocumentDB index improvements
▪️  Elemental MediaLive
      ▫️  1-second metrics
      ▫️  alert categories in Channel Assembly
▪️  EMR on EKS programmatic execution for managed endpoints
▪️  FSx for NetApp ONTAP
      ▫️  IPSec encryption of data in transit
      ▫️  two additional monitoring and troubleshooting capabilities
      ▫️  write once, read many (WORM) protection with SnapLock
▪️  Karpenter Windows containers support
▪️  Lambda now detects and stops recursive loops in Lambda functions
▪️  Location Service
      ▫️  API Keys for Maps, Places, and Routes
      ▫️  publishing device position updates on EventBridge
▪️  Mainframe Modernization expands control and visibility of runtime
▪️  Omics FedRAMP Moderate authorization
▪️  OpenSearch Service version 2.7
▪️  Personalize add columns to existing datasets
▪️  Proton deployment history
▪️  QuickSight
      ▫️  axis customization options for small multiples and radar chart
      ▫️  unified color experience for analysis and dashboards
▪️  RDS for SQL Server self-managed Active Directory
▪️  S3 Inventory ACLs as object metadata in inventory reports

AWS is offering 7-day free trial on Skill Builder - a learning platform built by the experts at AWS.

Practice building with AWS Builder Labs, develop your role-based skills using gamified learning with AWS Cloud Quest, verify your skills by taking a Jam Journey challenge, and prepare for an AWS Certification with enhanced exam prep materials.

Details: https://pages.awscloud.com/GLOBAL-Other-GC-Skill-Builder-Subscription-Free-Trial.html

NOTE: This promotion is redeemable through July 30th, 2023. Terms and conditions apply. If you do not cancel your free trial after 7 days, you will be automatically subscribed at $29 USD per month.

10 полезных советов по ускорению OpenSearch:

https://www.tecracer.com/blog/2023/07/performance-boost-10-expert-tips-for-optimizing-your-amazon-opensearch-service-cluster.html

▫️ Choose the right instance type
▫️ Start big
▫️ Use bulk ingest requests and employ multi-threading
▫️ Minimize frequent updates to the same document
▫️ Monitoring
▫️ Profile queries
▫️ Find an optimal shard number and size
▫️ Optimize shard locating
▫️ Use filters
▫️ Use search templates

#OpenSearch

☸️ Confidential Kubernetes

https://kubernetes.io/blog/2023/07/06/confidential-kubernetes/

Реально хорошая статья по состоянию дел с Confidential Computing в отношении Kubernetes. Жаль, без авторов со стороны AWS, потому для человека в теме, по части AWS будут сразу видны некоторые, скажем так, моменты.

1️⃣ «A managed CloudHSM from AWS costs around $1.50 / hour or ~$13,500 / year.»

А-ха-ха. В год, страшное дело, для бизнеса с такими требованиями по безопасности. И особенно смешно с учётом стоимости HSM в Azure: 😃

Hourly usage fee per HSM
Azure Dedicated HSM $4.85

2️⃣ Технология Confidential Computing на AWS или AWS Nitro Enclaves, лишь кратко упомянута из-за «have a different threat model compared to the CPU-based solutions by Intel and AMD».

Тут всё верно, целиком согласен. Nitro Enclaves — крутая фича, однако годность её AWS придётся всю жизнь доказывать, т.к. простых путей проверить этого нет и нужно целиком полагаться на авторитет AWS и аудиторов, а не техническую невозможность доступа в изолированное окружение.

3️⃣ AMD SEV — упомянуты лишь Azure и Google. Хотя на AWS теперь тоже доступны SEV-SNP (в то время как на Google лишь SEV-ES).

4️⃣ Скорость работы — реализация Confidential Computing от AMD очень эффективна: «SEV-SNP VM overhead is <10%». Про реализацию от Intel сказано расплывчато, что "hard to benchmark". Перевожу на простой — полный тормоз. 😁

5️⃣ Смешное название CoCo (Confidential Containers) и возможность запускать Confidential Kubernetes worker nodes вновь распространяется лишь Azure и Google. А правильно было упомянуть, что AWS Nitro Enclaves работает на EKS.

6️⃣ Хорошее и важное замечание «they don't offer a dedicated confidential control plane» — пока никакое облако не предлагает Confidential Kubernetes мастер-ноды, речь только о workers.

7️⃣ Constellation — интересная штука, буду признателен, если кто-то поделится опытом использования.

Итого, хорошая статья, очень рекомендую ознакомиться.

#security #ConfidentialComputing #ConfidentialKubernetes

⛅ Issue #79 | 9 July 2023

▪️  Application Migration Service multi-account migrations
▪️  Backup expands cross-account backup AWS Region coverage
▪️  CloudWatch Cross-Account Service Quotas
▪️  CodeBuild GitHub Actions support
▪️  Config +16 resource types
▪️  Connect autorun based on agent activity
▪️  DynamoDB
      ▫️ Distributed Cache Provider .NET
      ▫️ local version 2.0
▪️  EKS increases pod density limits for Windows containers
▪️  Elemental MediaLive input thumbnail images
▪️  Glue Crawlers Apache Iceberg Tables
▪️  GuardDuty EKS Runtime Monitoring expands OS and processor support
▪️  Mainframe Modernization Blu Age runtime deployment
▪️  OpenSearch Service higher IOPS and throughput for gp3 volumes
▪️  Personalize latest streamed data
▪️  RDS
      ▫️ PostgreSQL 16 Beta 2 Preview Environment
      ▫️ PostgreSQL Multi-AZ logical replication with 2 readable standbys
▪️  SageMaker Model Cards now integrated with model versions in Registry
▪️  Systems Manager Parameter Store increases API throughput limit
▪️  Textract AnalyzeDocument - Forms

​​🆕 CodeBuild + GitHub Actions:

https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html

You can use an action runner to run GitHub Actions within CodeBuild. This can be done by adding steps to any phase in your buildspec file.

#CodeBuild #GitHub

​​☁️ Отличия облачных провайдеров по публичным проектам:

https://iot-analytics.com/global-cloud-projects

Качественная аналитика, которая даёт представление о специфике клиентской базы провайдеров на основании порядка 7 тысяч публично доступных проектов, которые указывают используемое облако.

Company / Market share / Share of public projects
AWS 39% 37%
Azure 27% 30%
Google 9% 22%
Oracle 2% 7%
Alibaba 5% 4%
Others 18% n/a

Интересное:

🔹 Oracle для публичных проектов идёт четвёртым

🔸 У Google 2/3 клиентов мелкие, у AWS половина, а у Azure лишь треть (т.е. 2/3 большие и очень большие)

🔹 AWS доминирует в Индии, Azure в Японии, Google во Франции

🔸 Alibaba на 87% это Китай и Азия

#info

У настоящего безопасника нет цели, у него есть лишь сертификации:

https://pauljerimy.com/security-certification-roadmap/

#security