AWS Notes

Уже третий эпизод - и только о AWS организациях!
В первой части поговорили о Organization Units, во второй - о service control policies (SCPs). В новом выпуске продолжаем обсуждать безопасность: начали с IAM Access Analyzer, затронули то, как работает SOO и работу c SSO через CLI. А на сладкое поделились с вами проверенными практиками написания SCP политик, способами защиты root account и MFA, а также описали важные шаги в случае, если MFA была утеряна.

Тема настолько обширная, что, кажется, будет продолжение в 4 выпуске. Будем рады ответить на вопросы!

#podcast
Послушать можно тут:
- Apple Podcasts
- Google Podcasts
- Spotify
- PodBean
- YandexMusic

​​cfn-teleport

A command-line tool which can move CloudFormation resources between stacks.

https://github.com/udondan/cfn-teleport

cfn-teleport --source Stack1 --target Stack2 --resource Bucket21D68F7E8 --resource Bucket182C536A1 --yes

#CloudFormation

How to create and manage Redshift clusters, as well as store, process, and analyze large data scopes?
How does AWS QuickSight facilitate data-driven decisions in companies?

📍Join the Data Analytics Meetup by N-iX!

Rostyslav Fedyshyn, Solution Architect | Head of Data & Analytics Practice at N-iX
and Oleksiy Ivanchenko, Solution Architect at AWS, will give deep answers to these questions and share best practices on:
• setting up and optimizing Redshift;
• using Redshift in real projects and optimal integration with other services;
• how QuickSight allows you to share analytical results with colleagues, which contributes to better decision-making and team collaboration.

Don't miss it!
April 26 at 18:00 (GMT+3)
Online

Participation is free of charge: https://bit.ly/3H8SyOG

See you there!

☁️ Issue #68 | 23 April 2023

▪️  Amazon Corretto Quarterly Updates | aarch64 Alpine Linux  is GA
▪️  Amplify Flutter web and desktop support | GA
▪️  Amplify Push Notifications for mobile and cross platform apps
▪️  Backup SAP HANA databases on EC2
▪️  Chime SDK Hindi and Thai languages for live transcription
▪️  CloudFormation Template Sync Controller for Flux
▪️  CodeCatalyst
      ▫️ Dev Environment dashboard | Preview
      ▫️ General Availability
▪️  Competency Manufacturing and Industrial
▪️  Comprehend improves accuracy of document classification using layout data
▪️  Connect Customer Profiles now shows cases information in the agent workspace
▪️  Control Tower +7 additional Regions
▪️  DocumentDB ODBC driver to connect from BI tools
▪️  DynamoDB up to 50 concurrent table restores
▪️  EC2 Ubuntu Pro operating system in a subscription-included model
▪️  EFS up to 10 GiB/s of throughput
▪️  Elastic Disaster Recovery simplifies launch settings management
▪️  EMR enhanced error details
▪️  EMR Serverless  job-level billed resources for efficient cost management
▪️  Glue monitor usage of Glue resources
▪️  GuardDuty support for AWS Lambda
▪️  Inspector deep inspection of EC2 instances
▪️  IoT Core for LoRaWAN public network and roaming with Everynet | Preview
▪️  Kafka Distro for OpenTelemetry support
▪️  Lake Formation Apache Hive Metastore resources
▪️  Lambda Python 3.10 runtime
▪️  Migration Hub Refactor Spaces +7 regions
▪️  Partner Central  redesigned opportunity management
▪️  Personalize
      ▫️ enables popularity tuning for similar item recommendations
      ▫️ integration with SageMaker Data Wrangler
      ▫️ Kafka Sink connector to ingest real-time data with ease
▪️  RDS events now include tags for filtering and routing
▪️  Redshift
      ▫️ MERGE SQL command | GA
      ▫️ centralized access control for data sharing with Lake Formation
      ▫️ Dynamic Data Masking | GA
▪️  SageMaker organize models in the Model Registry with Collections
▪️  SageMaker Studio Lab combats bots with CAPTCHA
▪️  SAM CLI local testing support for API Gateway Lambda authorizers
▪️  Snowball Edge Compute Optimized S3 compatible storage
▪️  Systems Manager Incident Manager collaboration with MS Teams
▪️  VPC Prefix Lists +3 regions
▪️  WAF
      ▫️ Captcha JavaScript API support
▪️  WAF Ready Partner Offerings

Команда девопсов демонстрирует клиенту как просто выкатывается новая версия на прод.

#пятничное

🆕 Lambda + Python 3.10

https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html#w354aac23c27:~:text=Mar%2031%2C%202023-,Python%203.10,-python3.10

#Lambda

​​📆 Ровно через неделю, 24 апреля начнётся AWS All Stars Meetup:

https://wearecommunity.io/events/aws-user-group-3city-allstars

Приглашённые звёзды и их доклады:

⭐ Viktor Vedmich: Senior Developer Advocate at AWS — "Chaos engineering Why breaking things should be practiced"

⭐ Denis Astahov: AWS Hero, Solutions Architect at OpsGuru — "Why you should use AWS Organization"

⭐ Pawel Piwosz: AWS Community Builder, DevOps Institute Ambassador, Lead Systems Engineer at Epam Systems — "Security for IaC with Terraform and Cloudformation(Live demo)"

Мероприятие бесплатное, пройдёт онлайн 24 апреля в 17:00 GMT+2 (18:00 по Минску/Москве).

Присоединяйтесь!

IAM Roles, прикреплённые к виртуалкам, работают через EC2 Instance Profiles. Это даёт некоторую специфику, например, когда роль убирается у виртуалки, а она ещё некоторое время продолжает отрабатывать права, что были у виртуалки ранее с уже отсутствующей ролью.

Как это происходит и почему в деталях описано здесь:

https://www.uptycs.com/blog/aws-iam-roles-instance-profiles

AWS знает про такое поведение, так задумано. Короче —всё нормально, расходимся.

#IAM #EC2

🌤 Issue #67 | 16 April 2023

▪️   AppFlow +6 connectors
▪️   AppSync EventBridge support
▪️   Backup +3 new regions
▪️   Chime SDK
       ▫️  updated video background blur and replacement
       ▫️  updates Service Level Agreement
▪️   CodeWhisperer is now generally available
▪️   Competency Media & Entertainment Competency
▪️   Config advanced queries supports 27 new resource types
▪️   Connect
       ▫️  agents to handle voice calls, chats, and tasks concurrently
       ▫️  Voice ID multiple fraudster watchlists per Voice ID domain
▪️   EC2
       ▫️  I4i  available in additional regions
       ▫️  Inf2 optimized for generative AI| GA
       ▫️  Trn1n optimized for network-intensive generative AI models | GA
▪️   EC2 Image Builder vulnerability detection with Amazon Inspector for custom images
▪️   EC2 Serial Console EC2 bare metal instances
▪️   ECS/Batch Split Cost Allocation Data
▪️   ECS Linux containers AL2023 and gMSA
▪️   ECS on Fargate supports extensible ephemeral storage for Windows Tasks
▪️   EKS Kubernetes version 1.26
▪️   Elemental MediaConnect Gateway
▪️   Elemental MediaConvert media metrics
▪️   Firewall Manager +6 WAF features
▪️   FSx for Lustre is now available in three additional regions
▪️   FSx for NetApp ONTAP is now available in three additional regions
▪️   FSx for Windows File Server is now available in the 3 additional regions
▪️   GameLift adds support for Unreal Engine 5
▪️   Glue visual ETL new native Redshift capabilities
▪️   Ground Station Wideband Digital Intermediate Frequency
▪️   GuardDuty +3 threat detections to Alert Customers on Suspicious DNS Traffic
▪️   IoT Core MQTT5 Shared Subscriptions and new CloudWatch metrics
▪️   Lambda SnapStart for Java functions in 6 additional regions
▪️   MSK is now available in Hyderabad, Spain, and Zurich Regions
▪️   MWAA Apache Airflow version 2.5
▪️   Pinpoint PrivateLink support
▪️   QuickSight Row Level Security tags with OR condition
▪️   RDS Optimized Reads up to 2X faster queries on RDS for PostgreSQL
▪️   Redshift enhances string query performance by up to 63x
▪️   Rekognition Face Liveness to deter fraud in facial verification
▪️   SageMaker Inference Recommender improves usability and launches new features
▪️   Service Management Connector AWS Support and Automation integrations in Jira Cloud
▪️   WAF
       ▫️  increases web ACL capacity units limits
       ▫️  larger request body inspections for CloudFront distributions
▪️   Well-Architected Framework strengthens prescriptive guidance

#friday

​​RDS + db.m7g & db.c7g:

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.DBInstanceClass.html#Concepts.DBInstanceClass.Support

db.m6g.large $0.159
db.m7g.large $0.168

db.r6g.large $0.225
db.r7g.large $0.239

#RDS

Можно ли ускорить базу данных в 6-7 раз?

Продолжаем говорить про базы данных. В этот раз обсудим из чего базы данных состоят, где узкие горлышки, и какие продукты приходят на смену устоявшимся Postgres, MongoDB, Redis, Neo4J. И в этом мне помог разобраться - основатель https://www.unum.cloud/ Ashot Vardanian

Например, знали ли вы, что большая часть современных БД хранит данные в Log Structured Merged Tree структуре, а если точнее - в одной ее реализации - RocksDB от Facebook? А что Postgres не умеет работать с асинхронными интерфейсами Linux, и уже на этом уровне в 10 раз медленнее чем новые аналоги построенные на io_uring и SPDK? Или что можно отправить данные с диска на видеокарту в обход процессора?
Заходите на подкаст чтобы узнать больше.

#podcast
Послушать можно тут:
- Apple Podcasts
- Google Podcasts
- Spotify
- PodBean
- YandexMusic

​​Kubernetes 1.26 для EKS и EKS Distro 🎉

https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-26/

Спустя официального релиза 1.26 прошло чуть более 4 месяцев, то есть задержка поддержки очередной версии сократилась настолько резко, что выйди эта версия на день раньше, то на AWS даже была бы актуальная версия, ведь версия 1.27 вышла буквально вчера. 😃Сделанный в прошлый раз прогноз на эту версию снова был очень неточным — ошибся почти на полтора месяца.

https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-1.26

Теперь отставание по версиям сократилось до одной версии, в качестве прогноза версии 1.27 на AWS поставлю на 19 июня 2023 года.

Отдельно стоит отметить, что окончание поддержки версии EKS 1.22 будет в самом начале лета — 4 июня.

#EKS

☁️  Issue #66 | 9 April 2023

▪️  Amazon Aurora PostgreSQL 15
▪️  Amplify Library for Swift macOS Support is GA
▪️  App Runner +7 new compute configurations
▪️  Athena external data sources
▪️  Aurora PostgreSQL 14.7, 13.10, 12.14, and 11.19 versions
▪️  Billing Conductor pricing change
▪️  CDK policies validations during synthesis
▪️  CloudFront S3 Object Lambda Access Point origin
▪️  CodeCatalyst Dev Environments GitHub repositories support
▪️  Competency Partners Cloud Operations
▪️  Config 23 new resource types
▪️  Controllers for Kubernetes (ACK) for Amazon MemoryDB
▪️  EC2
      ▫️  EPYC3 instances now support EBS-optimized
      ▫️  Graviton2-based instances are available in additional regions
      ▫️  On-Demand Capacity Reservations – Utilization Notifications
▪️  ElastiCache for Redis new cluster creation UX
▪️  EMR on EKS Spark with Java 11
▪️  Glue  available in AWS Europe (Spain) and AWS Europe (Zurich)
▪️  Kendra Microsoft OneDrive Connector
▪️  Lambda introduces response payload streaming
▪️  Migration Hub High Availability SAP HANA systems
▪️  Monitron extends data stream with closure codes and status from sensors
▪️  MWAA Shell Launch Scripts
▪️  Network Firewall IPv6-only subnets
▪️  NICE DCV DCV and DCV Extension SDK | GA
▪️  Proton Git management of service configurations
▪️  RDS
     ▫️  add ElastiCache cache from Console
      ▫️  Custom for SQL Server Multi-AZ deployments
      ▫️  MySQL up to 15 read replicas for RDS Multi-AZ with 2 readable standbys
▪️  Resource Explorer export Search Results in csv
▪️  S3
      ▫️   beginning to apply two security best practices to all new buckets by default
      ▫️  object replication status
▪️  SageMaker
      ▫️  is now available in Asia Pacific (Hyderabad) Region
      ▫️  sharing predictions with QuickSight
      ▫️  Canvas 45+ data sources for no-code ML
      ▫️  Feature Store hard deletion in online store
▪️  Security Hub +4 new security best practice controls
▪️  Service Catalog Terraform open source
▪️  Supply Chain general availability
▪️  SWF PrivateLink support
▪️  Systems Manager Distributor New Relic Infrastructure Monitoring agent
▪️  Textract
      ▫️  AnalyzeDocument - Tables feature
      ▫️  Bulk Document Uploader
▪️  Trusted Advisor
      ▫️  fault tolerance checks for ECS
      ▫️  introduces Engage for Enterprise On-Ramp Support customers | Preview
▪️  VPC bring your own IP in 2 additional AWS Regions
▪️  WorkSpaces Core introduces MS Office 2019 Professional Plus bundle

​​EKS + Grafana Operator to manage Amazon Managed Grafana:

https://aws.amazon.com/blogs/mt/using-open-source-grafana-operator-on-your-kubernetes-cluster-to-manage-amazon-managed-grafana/

The grafana-operator is a Kubernetes operator built to help you manage your Grafana instances inside Kubernetes. Grafana Operator enables you to create and manage Grafana resources such as dashboards and data sources, declaratively between multiple instances in an easy and scalable way.

#AMG #EKS

Terraform + RDS & Secrets Manager:

В terraform-provider-aws v4.61 добавили поддержку секретов для пароля RDS:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance

Свои секреты использовать нельзя, их создаст RDS при manage_master_user_password = true, параметр password при этом должен отсутствовать.

Для секретов RDS пароля можно указать свой KMS ключ.

resource "aws_db_instance" "postgres15" {
...
storage_encrypted = true
kms_key_id        = var.kms_key_id

manage_master_user_password = true
master_user_secret_kms_key_id = var.kms_key_id
}

Получить созданный RDS секрет с паролем:

data "aws_secretsmanager_secrets" "postgres15" {
  filter {
    name   = "owning-service"
    values = ["rds"]
  }
  filter {
    name   = "tag-value"
    values = [aws_db_instance.postgres15.arn]
  }
}

data "aws_secretsmanager_secret" "postgres15" {
  arn = tolist(data.aws_secretsmanager_secrets.postgres15.arns)[0]
}

output "rds_master_password_secret" {
  description = "RDS master user secret details"
  value       = aws_db_instance.postgres15.master_user_secret
}

output "rds_master_password_secret_data" {
  description = "RDS master user secret data resource in Secrets Manager"
  value       = data.aws_secretsmanager_secret.postgres15
}

#RDS #Terraform

On April 6 we invite you to a webinar “Protecting Your Organization with AWS WorkSpaces: From Security Perimeter to Business Continuity"

Whether you're looking to protect your organization's security perimeter or prepare for unexpected disruptions, AWS WorkSpaces can help.


Join us for this informative session to learn more and discuss two use cases:
1. Protecting the security perimeter of your organization
benefits of using AWS WorkSpaces to enforce security policies and manage devices;
best practices for securing your organization's data with AWS WorkSpaces.

2. Supporting business failover scenarios
benefits of using AWS WorkSpaces for disaster recovery and business continuity;
best practices for ensuring that your critical business functions remain operational, even in the face of unexpected disruptions.

Speaker: Vadym Kovalenko, Cloud Architect at Triangu
Date: April 6, 6:00 (GMT+3)
Format: online
100% free of charge

Register 👉 https://bit.ly/3lORzvs

See you!