AWS Notes

Docker Systems Status Page

November 11, 2023 16:23 UTC [Investigating] We're investigating a partial disruption of registry services.

November 11, 2023 18:15 UTC [Monitoring] Docker Hub was temporarily disrupted (with unusual error rates and high latency on registry operations such as push/pull and auth) due to unusual traffic patterns.

The incident has been handled, and operation error rates/latency have returned to normal.

We will continue monitoring before resolving.

https://www.dockerstatus.com/

Хотите сэкономить на использовании AWS Lambda и повысить производительность функций? В этом выпуске подкаста мы подробно разберем способы оптимизации serverless приложений и раскроем неочевидные на первый взгляд тонкости настройки Lambda.

Мы начнем с того, как правильно выбрать объем оперативной памяти для функций. Оказывается, это напрямую влияет на производительность и стоимость.

Далее разберем технологию Snap Start, которая позволяет минимизировать холодные старты Java функций.

Поговорим о предварительном создании пула функций с Provisioned Concurrency. Это хороший способ ускорить отклик критичных Lambda.

Также узнаем, как использование компилируемых языков вроде Rust и Go вместо интерпретируемых может значительно прибавить скорость выполнения кода.

И еще много других полезных советов - от фильтрации ненужных событий до оптимальной инициализации функций.

В общем, если хотите выжать максимум из Lambda - обязательно послушайте этот выпуск!
Послушать можно тут:
- Podbean
- Yandex
- Google Podcasts
- Apple Podcasts
- Spotify
#podcast #AWSpodcast

​​Awesome Amazon CodeWhisperer:

🔗 https://github.com/cremich/awesome-codewhisperer

#CodeWhisperer

Issue #95 | 29 October 2023

▪️  Application Discovery Service new actions to remove servers, data collectors and imports
▪️  AppStream 2.0 multi-session fleets for Windows Server OS
▪️  Aurora MySQL
      ▫️  3.04 as long-term support (LTS) release
      ▫️  3.05 (compatible with MySQL 8.0.32) is generally available
      ▫️  optimizations that reduce the database restart time by up to 65%
▪️  Aurora PostgreSQL
      ▫️  15.4, 14.9, 13.12, 12.16, and 11.21 versions
      ▫️  federated query to MySQL and MariaDB databases
      ▫️  pgvector v0.5.0 with HNSW indexing
      ▫️  query plans on replica instances with query plan management
▪️  Cloud WAN Tunnel-less to enable high-performant global SD-WANs
▪️  CodePipeline
      ▫️  parameterized pipelines
      ▫️  triggering on git tags
▪️  Config +19 new resource types
▪️  Console Mobile better sign-in UX for iOS App
▪️  EC2
      ▫️  +2 new sizes I4i.12xlarge and I4i.24xlarge
      ▫️  Hibernate +20 additional instance families on EC2 Spot
      ▫️  M2 Mac instances for macOS | GA
      ▫️  Multi-VPC ENI Attachments
▪️  ECS increases applications resiliency to unpredictable load spikes
▪️  EKS
      ▫️  customer managed IAM policies
      ▫️  modification of cluster subnets and security groups
▪️  Entity Resolution data protection capabilities
▪️  FinSpace
      ▫️  Managed kdb Insights +5 regions
      ▫️  Managed kdb Insights adds more historical database cache options
▪️  Glue example jobs, drag-and-drop, and a preview-first layout
▪️  IoT Core Credential Provider supports VPC endpoints
▪️  IoT ExpressLink Technical Specification v1.2 with BLE connectivity
▪️  IVS audio-only pricing for Real-Time Streaming
▪️  Kendra feature to collapse and expand query results
▪️  Marketplace enhanced private offer UX for sellers
▪️  MemoryDB Graviton3-based R7g nodes
▪️  Migration Hub ability to change the AWS Region used to store migration data
▪️  Network Firewall egress TLS inspection in 2 regions
▪️  Neuron Llama-2 70b model and PyTorch 2.0
▪️  OpenSearch Serverless automated time-based data deletion
▪️  OpenSearch Service
      ▫️  efficient vector query filters for FAISS
      ▫️  now supports Internet Protocol Version 6 (IPv6)
▪️  RDS Blue/Green Deployments for Aurora and RDS PostgreSQL
▪️  RDS Custom for SQL Server M6i and R6i instances
▪️  RDS for PostgreSQL, MySQL, MariaDB M7g, R7g DB instances in additional regions
▪️  RDS for SQL Server minor version 2019 CU22
▪️  re:Post introduces Selections
▪️  SageMaker Feature Store Build ML feature pipelines from custom data sources
▪️  Security Lake +4 Regions
▪️  SNS
      ▫️  in-place message archiving and replay for FIFO topics
      ▫️  message data protection with custom data identifiers
▪️  SQS increased throughput quota for FIFO High Throughput mode
▪️  Systems Manager OpsCenter deletion of operational issues
▪️  Systems Manager Patch Manager additional MacOS and Linux versions
▪️  Transfer Family Basic authentication for outbound AS2 messages
▪️  User Notifications is now available in Israel Region
▪️  X-Ray W3C format trace IDs for distributed tracing

​​AWS-EU — новая будущая отдельная часть AWS European Sovereign Cloud и новый будущий AWS-EU регион в Германии:

https://aws.amazon.com/blogs/aws/in-the-works-aws-european-sovereign-cloud/

AWS-EU будет физически располагаться только на территории Евросоюза, работать там будут только граждане Евросоюза, все метаданные пользователей будут находиться также лишь в Евросоюзе, то есть совершенно независимо от N.Virginia. Cоответственно, и падать ЕвроAWS будет совершенно независимо от AWS. Но это не точно.

https://press.aboutamazon.com/2023/10/amazon-web-services-to-launch-aws-european-sovereign-cloud

ЕвроAWS стартует с одним еврорегионом в Германии. Однако в длинной компании заинтересованных сторон много стран, потому уже вскоре можно ожидать Францию, а затем, видимо, Чехию и Финляндию.

Стоит отметить, что ЕвроAWS не первый среди "евро-облаков", т.к. уже есть ЕвроAzure и ЕвроGCP.

#AWS_Regions #AWS_EU_Regions

Where can you find AWS learn study guides?
AWS Ramp-Up Guides provide you a step-by-step guides for a specific goal. They can be browsed by cloud role, solution or even industry-specific.

Explore them here: https://bit.ly/awsguides

Each guide is downloadable and features suggested digital trainings, whitepapers, relevant certifications, labs, workshops and more.

#training #guide

✈️ Issue #94 | 22 October 2023

▪️  AMI Block Public Access now enabled for all new and existing accounts with no public AMIs
▪️  AWS member account level credit sharing preferences
▪️  Bedrock now available in Europe (Frankfurt) AWS Region
▪️  CloudWatch out-of-the-box alarm recommendations for AWS services
▪️  CodeCatalyst scheduled workflow triggers
▪️  CodePipeline retrying a pipeline execution from the first action in a failed stage
▪️  CodeWhisperer customization capability | Preview
▪️  Connect additional levels of granular access control
▪️  Data Provider for SAP Oracle Linux OS, more JDK versions, and Linux logrotate
▪️  EC2
     ▫️  M6gd instances are now available in additional regions
     ▫️  C6a, M6a, R6a instances now available in additional regions
     ▫️  M6id instances are now available in additional regions
     ▫️  new R7i instances
▪️  EKS get add-ons from ISV in the EKS console
▪️   Elastic Disaster Recovery Introducing Recover into Existing Instance
▪️  Entity Resolution launches records matching with data service providers
▪️  Glue for Apache Spark native connectivity for Google BigQuery
▪️  IoT TwinMaker is now available in Tokyo, Seoul, and Mumbai
▪️  Kendra 11 JDBC data sources to enable structured data search
▪️  Managed Blockchain Query
     ▫️  Bitcoin Testnet and new Asset APIs
     ▫️  Sepolia Testnet support
▪️  MSK EventBridge Pipes console integration
▪️  MSK Replicator cross-region and same-region streaming data replication
▪️  OpenSearch Service
     ▫️  Im4gn instances
     ▫️  new administrative options
     ▫️  new language analyzers
      ▫️  OCSF and custom logs
▪️  Partner quick setup Metric Streams
▪️  Personalize OpenSearch service integration
▪️  QuickSight
     ▫️  direct connectivity to Starburst
     ▫️  direct connectivity to Trino
▪️  RDS
      ▫️  Dedicated Log Volume
     ▫️  Optimized Writes using RDS Blue/Green Deployments
▪️  Redshift
      ▫️  integration with AWS Secrets Manager
     ▫️  integration with Visual Studio Code
     ▫️  Query Editor V2 Autocomplete suggestions
▪️  Resource Explorer +12 new resource types
▪️  Route 53
      ▫️  Share Application Recovery Controller clusters across multiple AWS accounts
     ▫️  traffic flow now supports geoproximity routing for AWS Local Zones
▪️  SageMaker Inference
      ▫️  ml.p5.48xlarge Instances
     ▫️  Regional Expansion for ml.p4d, ml.trn1, and ml.g5 instances
▪️  Security Hub +19 new security controls
▪️  Service Catalog additional Infrastructure as Code (IaC) provisioning tools
▪️  Wickr FedRAMP Moderate authorized
▪️  WorkSpaces
      ▫️  introduces Graphics G4dn bundles for Ubuntu desktops
     ▫️  Windows Server 2022 bundles

Всем привет вчера зарелизили 0.4.1 версию SRE Learning Platform!

Features:

Spot ec2 is moved from spot request to spot fleet, which gives spot ec2 more stability
Terraform code is updated to use terraform version v1.5.6
Terraform aws provider is updated to v5.17.0
Ubuntu_version is added to k8s_self_management module, which is the way to get ami_id by default
k8s version 1.28 is used in cka-mock and cks-mock exams
The cks mock exam 01 uses version Graviton instances t4g.medium
The cks labs use version 1.28 and Graviton instances t4g.medium
Cks labs are actualized, list is added to them
Cks mock solutions are added
Video for run mock exam is added
Video for cks mock solutions is added

🟠 Issue #93 | 15 October 2023

▪️ Billing Link Your Bank Account for SEPA direct debit payments in Germany
▪️ CDK cdk8s include synthesize, import Helm charts, resolve cloud tokens
▪️ CloudWatch
     ▫️ launch Internet Monitor from VPC console
     ▫️ metric monitors EC2 instance reachability to EBS volumes
▪️ Direct Connect new location in Seoul, South Korea
▪️ EC2
     ▫️ C7gd, M7gd, and R7gd instances now available in additional regions
     ▫️ M6in and M6idn instances are now available in new regions
     ▫️ setting AMIs to a disabled state
▪️ Elemental MediaTailor ad break discovery in Channel Assembly
▪️ Global Accelerator second Edge location in Vietnam
▪️ Glue GitLab, BitBucket in its Git integration feature
▪️ Lambda IPv6 support for outbound connections in VPC
▪️ Linux Ansible and Corretto 21 with AL2023.2
▪️ Network Load Balancer (NLB)
     ▫️ Availability Zone DNS affinity
     ▫️ Disable connection termination for unhealthy targets
     ▫️ UDP connection termination by default
▪️ OpenSearch Search Pipelines
▪️ Organizations service control policies (SCPs) in AWS China Regions
▪️ QuickSight
     ▫️ running queries as an IAM role for Redshift data sources
     ▫️ total positioning options for pivot table
▪️ RDS for PostgreSQL, MySQL, and MariaDB M6in, M6idn, R6in, R6idn db instances
▪️ RDS pgactive: Active-active Replication Extension for PostgreSQL
▪️ Rekognition Enhance accuracy of pre-trained model with Custom Moderation
▪️ SageMaker Canvas
▪️ built in Deploy ML models; real-time endpoints
▪️ expands content summarization and information extraction capabilities
▪️ Step Functions launches Optimized Integration for Amazon EMR Serverless
▪️ Systems Manager
     ▫️ Application Manager SAP HANA
     ▫️ enable-by-default for all EC2 instances in an organization
▪️ Textract Custom Queries to improve information extraction for business-specific docs

#анонсы #машины_aws

Лучшая книга по AWS CloudFormation получила второе издание!

В новой версии:
1. Переработаны почти все главы в соответствии с изменениями у самого AWS'а, обновлен весь код, и мне за него даже почти не стыдно
2. В главе Advanced Template Development добавлена секция по работе с Language Extensions
3. Глава про Macros была переписана полностью и теперь включает в себя Nested Stacks и Modules
4. Новая глава, посвященная работе с CloudFormation Registry
5. В главе по CDK используется CDK v2

Отдельная благодарность @VasilyPantyukhin и @patrick239 за то, что заставили меня вылизывать все почти до мельчайшего знака препинания!

⚠️ EKS 1.23 — End of Standard support

https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-release-calendar

Сегодня, 11 октября 2023-го года, заканчивается стандартная поддержка EKS 1.23.

Extended support

Кто не хочет или не может обновиться до 1.24+, то можно запросить расширенную поддержку 1.23, которая начнётся завтра, 12 октября, и продлится ровно 1 год.

Расширенная поддержка прекратится 11 октября 2024-го года, продлить поддержку 1.23 дальше будет нельзя, кластеры автоматом будут проапгрейжены на 1.24.

https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#extended-support-faqs

За расширенную поддержку нужно будет платить дополнительно, правда вот стоимость этого пока не ясна.

#EKS

Active-active Replication for RDS PostgreSQL

https://aws.amazon.com/blogs/database/using-pgactive-active-active-replication-extension-for-postgresql-on-amazon-rds-for-postgresql/

▫️ Active-active Replication can be used to maintain availability throughout different database operations and reduce write latency for applications distributed in multiple AWS Regions.

▫️ PostgreSQL pgactive extension makes it possible to deploy RDS PostgreSQL instances using an active-active topology, and provides the foundational features for managing active-active replication such as conflict detection and resolution. 

#RDS #PostgreSQL

🤗 Issue #92 | 8 October 2023

▪️  ALB/NLB registering instances addressed by IPv6 as targets
▪️  Amplify GraphQL API capabilities as CDK construct
▪️  AppConfig feature flags new encryption options
▪️  App Runner custom domains improvements
▪️  Catalog API JSON object alongside String for “Details” attribute
▪️  CodeDeploy multiple load balancers for EC2 applications
▪️  Connect Contact Lens manager alerts on real-time metrics
▪️  Connect
      ▫️ enhances API for accessing historical agent and contact metrics
      ▫️ multiple out-of-the-box chat widgets per instance
      ▫️ new contact and agent metrics in the API
▪️  Control Tower 22 proactive controls, +12 Security Hub Detective controls
▪️  DataZone is now generally available
▪️  EC2
      ▫️ C7a compute optimized instances
      ▫️ R7a bare metal memory optimized instances
      ▫️ Hibernate more OS support
▪️  ECR Public new navigation and search features to the ECR Public Gallery
▪️  EKS Extended support for Kubernetes Versions | Preview
▪️  Elemental MediaConvert video source replacement
▪️  EMR on EKS Interactive Endpoints is now GA
▪️  EventBridge support for wildcard filters in rules
▪️  Firewall Manager referencing of Security Groups
▪️  Health Health events in your organization to EventBridge
▪️  HealthOmics sequence store with auto-calculated ETags for read sets
▪️  IoT Device Management Software Package Catalog | GA
▪️  IVS introduces in-console broadcasting for low-latency streaming
▪️  Kendra releases Drupal Connector to enable Drupal document search
▪️  Location Service bounding box search for device positions
▪️  OpenSearch Service
      ▫️ alerting and anomalies on dashboards
      ▫️ announced OpenSearch Service Integrations
      ▫️ OpenSearch version 2.9
      ▫️ expands geospatial aggregations support
▪️  QuickSight predictive analytics using Amazon SageMaker Canvas
▪️  RDS for PostgreSQL pgvector v0.5.0 with HNSW indexing
▪️  SageMaker Canvas ready-to-use models to include foundation models
▪️  SageMaker Feature Store low latency feature retrieval
▪️  SageMaker Model Registry private model repositories
▪️  SAM CLI Lambda test events
▪️  SQS binding extension support in CoreWCF
▪️  Supply Chain Demand Planning now supports Product Lineage
▪️  Verified Access customer managed KMS keys
▪️  Well-Architected Framework
      ▫️ New updates
      ▫️ Templates Reduce duplication
▪️  WorkSpaces
      ▫️ bundle selection guidance in AWS console
      ▫️ expand Microsoft productivity apps offerings

🚀 Join us on September 30th online to build applications faster and more securely with your AI coding companion - Amazon CodeWhisperer.

In this session we will use Amazon CodeWhisperer to learn the basics of prompt engineering. We will start with several examples highlighting how the service is leveraging your comments and existing code for the results. Then you will dive into challenges designed to improve your productivity.

Details: https://www.meetup.com/aws-user-group-3city/events/295753448/

🕛 Date/Time: September 30, 2023 at 9:00 GMT+2
📍️️️️️️ Location: Online (Russian / English)
📕️️️️ Topic: Amazon CodeWhisperer Immersion Day

We care about your privacy, and we want to assure you that your personal information will be kept confidential, used only for event-related purposes, and deleted right after the event.

‼️ IMPORTANT: Please make sure to fill out the Google Form [https://forms.gle/xGqTSQEFatY2nHax6] if you want to attend the workshop. This is needed for us to send you workshop credentials.

⚡️Amazon Bedrock GA has arrived! 🚀 А у нас вышел новый эпизод подкаста! Где мы как раз со специалистом по Gen AI - Nikita Fedkin обсуждаем, что за зверь такой Generative AI.
В этом выпуске подкаста мы погружаемся в увлекательный мир Generative AI.
Разбираемся с ключевыми аспектами, такими как модели диффузии, Stable Diffusion и особенностями обучения больших языковых моделей (LLMs). Обсуждаем процесс FineTuning, детали RAG и его сравнение с FineTuning, а также принципы работы VectorSearch.
И как раз (28.09.2023) вышел новый сервис от AWS - BedRock, который мы обсуждали в выпуске.

Послушать можно тут:
- Podbean
- Yandex
- Google Podcasts
- Apple Podcasts
- Spotify
#podcast #AWSpodcast

P.S. Да-да картинка была сгенерирована с использованием GenAI - но голоса настоящие :)

​​EKS + Kubernetes 1.28 🎉

https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-28/

Спустя официального релиза 1.28 прошло лишь полтора месяца и теперь на AWS снова актуальная версия. 💪 Сделанный в прошлый раз прогноз на эту версию опять был весьма точным — ошибся лишь на 4 дня.

https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-1.28

Следующая версия должна официально выйти где-то в начале декабря, так что поставлю, что 1.29 на AWS появится 17 января 2024 года.

На текущий момент на AWS доступны для установки версии Kubernetes: 1.23 - 1.28.

⚠️Версия EKS 1.23 перестанет поддерживаться уже через две недели - 11 октября.

#EKS

✈️ Issue #90 | 24 September 2023

▪️  App Runner Auto-Scaling configuration management improvements
▪️  Bottlerocket new ECS-optimized AMI
▪️  Cloud Map introduces new API for retrieving service revision
▪️  CloudWatch new Metric Math for RDS Performance Insights
▪️  CloudWatch Synthetics new Synthetics Python runtime version 2.0
▪️  CodeArtifact Swift Package Manager
▪️  Competency Partners Telecom Services
▪️  Connect CloudFormation support for security profiles
▪️  DocumentDB in-place major version upgrade
▪️  EBS Multi-Attach io2 volumes now supports NVMe reservations
▪️  EC2 Console simulate interruptions in your Spot Fleet directly
▪️  EC2 M2 Pro Mac instances for macOS | GA
▪️  Glue Data Quality identify records that failed the CustomSQL rule type
▪️  Glue Interactive Sessions new kernel and support IAM Conditionals
▪️  HealthImaging supports CloudFormation
▪️  IAM Roles Anywhere credential helper now supports PKCS #11 modules
▪️  Location Services price reduction of up to 75% for tracking and geofencing
▪️  Lookout for Equipment model retraining
▪️  Management Console UX and navigation bar enhancements
▪️  Open-Source Corretto 21 | GA
▪️  QuickSight new layout and sparkline to KPI visual
▪️  RDS X2iedn instances for SQL Server
▪️  RDS for Oracle M6i, R6i, and R5b instances in new regions
▪️  RDS Performance Insights SQL-level metrics for RDS for SQL Server
▪️  SNS de-identify outbound messages via redaction or masking

OpenTF теперь в Linux Foundation с именем OpenTofu:

https://www.linuxfoundation.org/press/announcing-opentofu

Переименование произошло во избежание юридических проблем с названием.

https://github.com/opentofu

Команды выглядят как:

tofu init
tofu plan
tofu apply

#OpenTofu

Join us on September 23rd online to gain hands on experience building and deploying a serverless app with Amazon SNS, Amazon SQS, and AWS Lambda.

In this builder session, we will create a demo serverless architecture with data protection specifically as the goal. Dive deep with AWS Key Management Service (AWS KMS) to encrypt data at rest with a customer managed key (CMKs) and use key policies and resource policies to enforce least privilege.

Details: https://www.meetup.com/ru-RU/aws-user-group-3city/events/295753260/

🕛Date/Time: September 23, 2023 at 09:00 GMT+2
📍Location: Online (Russian / English)
📕Topic: Data Protection in Event-Driven Architectures

‼️ IMPORTANT: Please make sure to fill out the form (link) if you want to attend the workshop. This is needed for us to send you workshop credentials.