DataSpoof

PowerBI complete curriculum from basic to advance

Introduction to charts
Filters
Dropdown
Drill down and Drill through in PowerBI
Add images, buttons, bookmarks
Top N
Dax expression


Introduction to Power query for data cleaning
- basic exploration summary statistics
- values count function
- Checking for wrong data entry errors
- Missing values
- Duplicate values
- Outliers
- Checking data imbalance
- data skewness
- Handling json types columns
- Handling time stamp columns
- Feature engineering
- Query folding in PowerBI


Data modelling in PowerBI
- Creating relationships between several tables
- Active vs inactive relationships
- Cardinality and Cross-Filtering


Security and Data governance
- Row level security (static vs dynamic)
- Object level security
- Combining row level and object level security


Other topics

- Real time streaming dashboard in PowerbI
- Data lineage in powerbi
- PowerBI Gateway
- Incremental refresh vs Scheduled refresh in powerbi
- import mode vs direct mode in PowerBI
- native query
- row context and filter context in powerbi
- Powerbi dataflow

10 great Python packages for Data Science not known to many:

1️⃣ CleanLab

Cleanlab helps you clean data and labels by automatically detecting issues in a ML dataset.

2️⃣ LazyPredict

A Python library that enables you to train, test, and evaluate multiple ML models at once using just a few lines of code.

3️⃣ Lux

A Python library for quickly visualizing and analyzing data, providing an easy and efficient way to explore data.

4️⃣ PyForest

A time-saving tool that helps in importing all the necessary data science libraries and functions with a single line of code.

5️⃣ PivotTableJS

PivotTableJS lets you interactively analyse your data in Jupyter Notebooks without any code 🔥

6️⃣ Drawdata

Drawdata is a python library that allows you to draw a 2-D dataset of any shape in a Jupyter Notebook.

7️⃣ black

The Uncompromising Code Formatter

8️⃣ PyCaret

An open-source, low-code machine learning library in Python that automates the machine learning workflow.

9️⃣ PyTorch-Lightning by @LightningAI

Streamlines your model training, automates boilerplate code, and lets you focus on what matters: research & innovation.

🔟 Streamlit

A framework for creating web applications for data science and machine learning projects, allowing for easy and interactive data viz & model deployment.

MLOPS Tools available in Market

1. Version Control and Experiment Tracking:

- DVC (Data Version Control): Manages datasets and models using version control, similar to how Git handles code.

- MLflow: An open-source platform to manage the ML lifecycle, including experiment tracking, model versioning, and deployment.

- Weights & Biases: Offers experiment tracking, model management, and visualization tools.

2. Model Deployment:

- Kubeflow: An open-source toolkit that runs on Kubernetes, designed to make deployments scalable and portable.

- AWS SageMaker: Amazon’s fully managed service that provides tools for building, training, and deploying machine learning models at scale

- TensorFlow Serving: A flexible, high-performance serving system for machine learning models, designed for production environments.

3. CI/CD for Machine Learning:

- GitHub Actions: Automates CI/CD pipelines for machine learning projects, integrating with other MLOps tools.

- Jenkins: An automation server that can be customized to manage CI/CD pipelines for machine learning.

4. Model Monitoring and Management:

- Prometheus & Grafana: Combined, they provide powerful monitoring and alerting solutions, often used for ML model monitoring.

- Seldon Core: An open-source platform for deploying, scaling, and managing thousands of machine learning models on Kubernetes.

5. Data Pipeline Management:

- Apache Airflow: An open-source platform to programmatically author, schedule, and monitor workflows.

- Prefect: A modern workflow orchestration tool that handles complex data pipelines, including those involving ML models.

Lifecycle policies in S3 bucket

Purpose: Automate the management of objects in S3 buckets over time.

Components: Define rules based on object age or storage class.

Actions: Transition objects to different storage classes or delete them.

Benefits: Reduce storage costs, optimize performance, and ensure compliance.

https://youtu.be/XkDu-haZgvc?si=99Ci3blI9lb0hTpA

A python library to call all LLM API

AWS #video3

* Like it and subscribe to our channel for daily aws content

https://youtu.be/BaVQoUnWAYE?si=rMjSq_ckC-oBi3hQ

Build delightful web apps quickly in Python using Google Mesop library

https://github.com/google/mesop

AWS #video2

* Like It and subscribe to our channel for daily aws content

https://youtu.be/LSMLtBBfNzE?si=fbRaXrx18wUlESw3

Giveaway

Complete AWS for data science will be totally free on our YouTube channel.

* On next 100 days 100 videos will be published

* Like it and subscribe to our channel

* Daily video will be released at 7pm ist

https://youtu.be/Bzrf13Xd-KU?si=1m1M4lsA6ePDKqKs

Pdoc is a python library which help us to visualize the code step by step.

pip install pdoc3

How to run

pdoc filename.py