GAIA Benchmark Agent
This Hugging Face Space hosts a GAIA (General AI Assistant) benchmark agent designed to solve certification challenges across various domains of AI and machine learning.
Features
- Processes questions from the GAIA benchmark
- Uses LangChain and OpenAI's language models
- Analyzes questions and identifies their types
- Retrieves relevant context when needed
- Generates accurate, well-reasoned answers
- Integrates with external information sources:
- SerpAPI for real-time web search capabilities
- YouTube for video content search and transcript analysis
- Tavily for AI-optimized search results
- Audio processing for speech-to-text conversion and analysis
Usage
- Log in to your Hugging Face account using the button
- Click 'Run Evaluation & Submit All Answers' to:
- Fetch questions from the GAIA benchmark
- Run the agent on all questions
- Submit answers and see your score
Implementation Details
The agent uses a modular architecture with specialized handlers for different question types:
- Factual knowledge questions
- Technical implementation questions
- Mathematical questions
- Context-based analysis questions
- Ethical/societal impact questions
- Media content questions (videos, podcasts, audio recordings)
- Current events questions
- Categorization questions with enhanced botanical classification
Botanical Classification
The agent has been enhanced with comprehensive botanical classification capabilities, allowing it to:
- Accurately distinguish between botanical fruits and vegetables
- Provide detailed explanations of botanical classifications
- Correctly identify commonly misclassified items (tomatoes, bell peppers, cucumbers, etc.)
- Explain the difference between botanical and culinary classifications
External Information Sources
The agent can access external information to provide more accurate and up-to-date answers:
- SerpAPI Integration: Enables real-time web search capabilities for current events and factual information
- YouTube Integration:
- Search for relevant videos on specific topics
- Extract and analyze video transcripts for information
- Tavily Search: AI-optimized search engine that provides relevant results for complex queries
Audio Processing Capabilities
The agent has been enhanced with audio processing capabilities, allowing it to:
- Transcribe audio files using OpenAI's Whisper API with Google Speech Recognition fallback
- Extract ingredients from recipe audio recordings
- Process and analyze spoken content from various audio formats
- Format responses according to user requests for audio content
API Keys Configuration
To use the external information sources, you need to set the following API keys in your environment:
SERPAPI_API_KEY: For web search capabilitiesYOUTUBE_API_KEY: For YouTube video search and transcript analysisTAVILY_API_KEY: For AI-optimized search resultsWHISPER_API_KEY: For audio transcription (defaults to OPENAI_API_KEY if not set)
Repository
The code for this agent is available at:
https://huggingface.co/derkaal/GAIA-agent
