ML Notebooks Execution Guide
This directory contains machine learning notebooks for the Cyber Forge AI platform. Follow this guide to run the notebooks in the correct order for optimal results.
📋 Prerequisites
Before running any notebooks, ensure you have:
Python Environment : Python 3.9+ installedDependencies : Install all required packages:
1 cd .. /
2 pip install -r requirements.txt
Jupyter : Install Jupyter Notebook or JupyterLab:
pip install jupyter jupyterlab
🎯 Execution Order
Run the notebooks in this specific order to ensure proper model training and dependencies:
1. Basic AI Agent Training 📚
File : ai_agent_training.py
Purpose : Initial AI agent setup and basic training
Runtime : ~10-15 minutes
Description :
Sets up the foundational AI agent
Installs core dependencies programmatically
Provides basic communication and cybersecurity skills
RUN THIS FIRST - Required for other notebooks
1 cd ml-services/notebooks
2 python ai_agent_training.py
2. Advanced Cybersecurity ML Training 🛡️
File : advanced_cybersecurity_ml_training.ipynb
Purpose : Comprehensive ML model training for threat detection
Runtime : ~30-45 minutes
Description :
Data preparation and feature engineering
Multiple ML model training (Random Forest, XGBoost, Neural Networks)
Model evaluation and comparison
Production model deployment preparation
jupyter notebook advanced_cybersecurity_ml_training.ipynb3. Network Security Analysis 🌐
File : network_security_analysis.ipynb
Purpose : Network-specific security analysis and monitoring
Runtime : ~20-30 minutes
Description :
Network traffic analysis
Intrusion detection model training
Port scanning detection
Network anomaly detection
jupyter notebook network_security_analysis.ipynb4. Comprehensive AI Agent Training 🤖
File : ai_agent_comprehensive_training.ipynb
Purpose : Advanced AI agent with full capabilities
Runtime : ~45-60 minutes
Description :
Enhanced communication skills
Web scraping and threat intelligence
Real-time monitoring capabilities
Natural language processing for security analysis
RUN LAST - Integrates all previous models
jupyter notebook ai_agent_comprehensive_training.ipynb📊 Expected Outputs
After running all notebooks, you should have:
Trained Models : Saved in ../models/ directoryPerformance Metrics : Evaluation reports and visualizationsAI Agent : Fully trained agent ready for deploymentConfiguration Files : Model configs for production use
🔧 Troubleshooting
Common Issues:
Memory Errors :
Reduce batch size in deep learning models
Close other applications to free RAM
Consider using smaller datasets for testing
Package Installation Failures :
Update pip: pip install --upgrade pip
Use conda if pip fails: conda install <package>
Check Python version compatibility
CUDA/GPU Issues :
For TensorFlow GPU: Install CUDA 11.8+ and cuDNN
For CPU-only: Models will run slower but still work
Check GPU availability: tensorflow.test.is_gpu_available()
Data Download Issues :
Ensure internet connection for Kaggle datasets
Set up Kaggle API credentials if needed
Some notebooks include fallback synthetic data generation
📝 Notes
First Run : Initial execution takes longer due to package installation and data downloadsSubsequent Runs : Much faster as dependencies are cachedCustomization : Modify hyperparameters in notebooks for different resultsProduction : Use the saved models in the main application
🎯 Next Steps
After completing all notebooks:
Deploy Models : Copy trained models to production environmentIntegration : Connect models with the desktop applicationMonitoring : Set up model performance monitoringUpdates : Retrain models with new data periodically
🆘 Support
If you encounter issues:
Check the troubleshooting section above
Verify all prerequisites are met
Review notebook outputs for specific error messages
Create an issue in the repository with error details
Happy Training! 🚀
