Workflow Element Store

  1. Handling Missing Data
  2. Annotation
  3. Dealing with Outliers
  4. Time-Based Features
  5. Interaction Features
  6. Feature Extraction from Images
  7. Polynomial Features
  8. Auto-Preprocessing libraries
  9. Data Partitioning - Train, Validation, & Test
  10. Binning / Discretization
  11. Handling Noisy Data
  12. AutoEDA libraries
  13. Augmentation
  14. Data Scaling and Normalization
  15. Data Transformations
  16. Dimensionality Reduction
  17. Handling Imbalanced Classes
  18. Domain-Specific Feature Engineering
  19. Textual Feature Extraction
  20. Handling Time-Series Data
  21. Feature Selection
  22. Handling Categorical Data
  1. Data Augmentation
  2. Binary Classification Techniques
  3. Regularization
  4. Association Rules
  5. AutoML
  6. Blackbox - Neural Network Models
  7. Early Stopping
  8. Performance Visualization
  9. Multiclass Classification Techniques
  10. Transfer Learning
  11. Model Interpretability
  12. Model Comparison
  13. External Validation
  14. Recommendation Engine
  15. Hyperparameter Tuning
  16. Weight Initialization
  17. Batch Normalization
  18. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  19. Cross-Validation
  20. Regression Analysis
  21. Reinforcement Learning
  22. Ensemble Techniques
  23. Clustering
  24. Cross-Validation
  25. Regular Monitoring and Logging
  26. Regularization Techniques
  27. Transfer Learning
  28. Batch Size Selection
  29. Learning Rate Scheduling
  30. Evaluation Metrics
  31. Natural Language Processing
  32. Forecasting Techniques
  33. Word Embeddings
  34. Network Analytics/ GeoSpatial Analytics
  1. Github Actions
  2. model registry
  3. Databases
  4. Data Preprocessing pipeline models
  5. Kafka Brokers
  6. Evidently.ai
  7. Datawarehouse
  8. Github
  9. code repository
  10. Apache Airflow
ML Workflow Advanced - Architecture
  • Element belongs to model
  • Element not belongs to model
Data Sources

Data Sources

Streaming Data

Streaming Data

Batch Data

Batch Data

Cloud Storage

Cloud Storage

Labeled Data

Labeled Data

Feature Engineering Pipeline

Feature Engineering Pipeline

Experimentation

Experimentation

ML Model

ML Model

Repository

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Offline DB Online DB

Orchestration Component

Artifact Store

CI/CD Component

Model Registry

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Automation ML Workflow Pipeline

Monitoring Component

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)