Workflow Element Store

  1. Feedback Data
  2. Data Bases - SQL
  3. Mobile Applications or IoT Applications
  4. Public Datasets
  5. Experiments (DoE)
  6. APIs and Data Feeds
  7. Data bases - NoSQL
  8. Surveys and Questionnaires
  9. Flat files
  10. WebScraping
  11. Data Collaboration and Partnerships
  1. AWS Redshift
  2. AWS RDS
  3. Azure ADF
  4. GCP Dataflow
  5. Apache Kafka
  6. AWS Glue
  7. GCP BigQuery
  8. PostgreSQL
  9. ETL/ELT pipeline
  10. Azure Synapse
  11. MySQL
  12. GCP Data Fusion
  13. Oracle DB
  14. GCS
  15. RDBMS
  16. s3
  17. Azure blob storage
  18. Azure Streaming Analytics
  19. MS SQL server
  20. AWS Kinesis
  21. MongoDB
  1. Feature Extraction from Images
  2. Dimensionality Reduction
  3. Interaction Features
  4. Dealing with Outliers
  5. Handling Missing Data
  6. Polynomial Features
  7. Data Scaling and Normalization
  8. Handling Noisy Data
  9. Handling Time-Series Data
  10. Handling Categorical Data
  11. Time-Based Features
  12. Augmentation
  13. Textual Feature Extraction
  14. Data Transformations
  15. Data Partitioning - Train, Validation, & Test
  16. Auto-Preprocessing libraries
  17. AutoEDA libraries
  18. Domain-Specific Feature Engineering
  19. Annotation
  20. Feature Selection
  21. Handling Imbalanced Classes
  22. Binning / Discretization
  1. Batch Size Selection
  2. Regularization Techniques
  3. Hyperparameter Tuning
  4. Regular Monitoring and Logging
  5. Association Rules
  6. Learning Rate Scheduling
  7. Transfer Learning
  8. Performance Visualization
  9. Clustering
  10. Batch Normalization
  11. Regression Analysis
  12. Ensemble Techniques
  13. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  14. Word Embeddings
  15. Transfer Learning
  16. Evaluation Metrics
  17. External Validation
  18. Forecasting Techniques
  19. Early Stopping
  20. Binary Classification Techniques
  21. Reinforcement Learning
  22. Multiclass Classification Techniques
  23. Natural Language Processing
  24. AutoML
  25. Weight Initialization
  26. Cross-Validation
  27. Cross-Validation
  28. Regularization
  29. Model Comparison
  30. Data Augmentation
  31. Recommendation Engine
  32. Blackbox - Neural Network Models
  33. Network Analytics/ GeoSpatial Analytics
  34. Model Interpretability
  1. Databases
  2. Data Preprocessing pipeline models
  3. Datawarehouse
  4. model registry
  5. code repository
  1. Data Drift Monitoring
  2. Performance Metrics
  3. Serverless Computing
  4. Edge Deployment
  5. Flask
  6. Prediction Logging
  7. Model Drift
  8. Alerting and Notification
  9. Containerization
  10. Concept Drift Detection
  11. Streamlit
  12. Bias and Fairness Assessment
  13. Feedback Collection
  14. Cloud Deployment
  15. Model Versioning
  16. Model Serialization
  17. Model Health Monitoring
  18. FastAPI
ML Workflow Intermediate - Architecture
  • Element belongs to model
  • Element not belongs to model
Training Pipeline
Data Collection

Data Collection

API Stream

Web crawler

API Stream

Web crawler

Selenium

Data Ingestion

Data Ingestion

Data Landing Zone

Store Data from all the Sources

Data Cleaning / Preprocessing

Data Cleaning / Preprocessing

Derived & Base features

Data Training & Modelling

Data Training & Modelling

Inference Pipeline
Input Data for Forecasting

Input Data for Forecasting

Input Data

Cleaned & Processed Data

Inference

Inference

streamlit