​​🆕 Step Functions Distributed Map:

https://aws.amazon.com/blogs/aws/step-functions-distributed-map-a-serverless-solution-for-large-scale-parallel-data-processing/

Step Function’s map state executes the same processing steps for multiple entries in a dataset. The existing map state is limited to 40 parallel iterations at a time.

Glue vs EMR vs S3 Batch Operations

◆ Data scientists and data engineers use Glue and EMR to process large amounts of data.
◆ On the other hand, application developers will use Step Functions to add serverless data processing into their applications. Step Functions is able to scale from zero quickly, which makes it a good fit for interactive workloads where customers may be waiting for the results.
◆ System administrators and IT operation teams are likely to use S3 Batch Operations for single-step IT automation operations such as copying, tagging, or changing permissions on billions of S3 objects.

#Step_Functions #serverless