💥Top 5 Data Engineering Trends in 2022: Astronomer Research
Astronomer, which commercializes and promotes the popular batch automation tool for working with data, Apache AirFlow, conducted a series of interviews with experts in the field of data engineering to identify the most pressing trends in the IT field.
• Data lineage, Data provenance and Data Quality
• Decentralization of data across different contexts and teams, but within a single consistent infrastructure with centralization of resources
• Consolidation of data tools, including orchestration of processing pipelines
• Data Mesh, eliminating silos between processing teams through the connection of used platforms
• mutual integration of DataOps, MLOps, AIOps for more efficient and faster use of consistent data and tools for seamless work with them.
https://www.astronomer.io/blog/top-data-management-trends-2022