RAVANA AGI

Shared State Management

Shared State Management

Update Summary

Changes Made

  • Added documentation for state persistence and restoration during shutdown/startup
  • Updated State Data Model section with Snake Agent state integration
  • Enhanced State Initialization and Access Patterns with state restoration logic
  • Added new section on State Persistence and Serialization covering Snake Agent integration
  • Updated Common Issues and Best Practices with state persistence considerations
  • Added configuration details for state persistence features

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. State Data Model
  7. State Initialization and Access Patterns
  8. State Transitions and Propagation
  9. Thread-Safe Access in Asynchronous Environment
  10. State Persistence and Serialization
  11. Debugging and Monitoring State Evolution
  12. Common Issues and Best Practices
  13. Conclusion

Introduction

The SharedState class serves as the central repository for dynamic runtime data in the Ravana AGI system. It encapsulates critical information such as mood, goals, current situation, and system status, enabling coordinated behavior across multiple modules. This document provides a comprehensive analysis of the state management system, detailing its data model, initialization process, access patterns, and integration with other components. The design emphasizes thread-safe operations in an asynchronous environment while supporting complex state transitions driven by actions, emotional responses, and external inputs. Recent updates have enhanced the system with state persistence capabilities, allowing for state restoration during shutdown and startup.

Project Structure

The project follows a modular architecture with clear separation of concerns. The core functionality resides in the core directory, while specialized capabilities are organized into distinct modules. The SharedState class is located in the core package, reflecting its fundamental role in the system's operation.

Diagram sources

Section sources

Core Components

The core components of the state management system include the SharedState class, AGISystem class, and supporting modules that interact with and modify the shared state. The SharedState class acts as a data container, while the AGISystem orchestrates state updates through various processes and modules. The recent addition of the ShutdownCoordinator enables state persistence across system restarts, while the SnakeAgent integrates its internal state with the broader system state management.

Section sources

Architecture Overview

The architecture centers around the AGISystem class, which maintains a single instance of SharedState accessible to all modules. State changes propagate through a well-defined sequence of operations: situation generation, memory retrieval, decision making, action execution, and mood updating. This loop ensures consistent state evolution while allowing for autonomous operation. The ShutdownCoordinator manages the graceful shutdown process, including state persistence, while the SnakeAgent operates as a background process with its own state that is integrated into the system's persistence mechanism.

Diagram sources

Detailed Component Analysis

SharedState Class Analysis

The SharedState class is a simple yet powerful data structure designed to hold the dynamic state of the AGI system. It uses Python's built-in types for maximum compatibility and ease of serialization.

Diagram sources

Section sources

State Data Model

The SharedState class defines a comprehensive data model that captures various aspects of the AGI system's runtime state:

State Attributes

  • mood: Dictionary of mood components with float values representing intensity
  • current_situation: Dictionary containing the current context or prompt being processed
  • current_situation_id: Integer identifier for the current situation in the database
  • recent_memories: List of recent memory objects retrieved from the memory service
  • long_term_goals: List of strings representing high-level objectives
  • mood_history: Historical record of mood vectors for tracking emotional trends
  • curiosity_topics: List of topics generated by the curiosity trigger module
  • search_results: List of recent web search results for immediate reference
  • current_task: String describing the current task in single-task mode

The data model is designed to be extensible, allowing modules to add new attributes as needed without modifying the core class. This flexibility supports the system's adaptive nature while maintaining a consistent interface. The SnakeAgent maintains its own state model (SnakeAgentState) that is integrated into the system's persistence mechanism during shutdown.

Section sources

State Initialization and Access Patterns

The SharedState instance is created during AGISystem initialization with an initial mood vector derived from the EmotionalIntelligence module:

self.shared_state = SharedState(
    initial_mood=self.emotional_intelligence.get_mood_vector()
)

Access to the shared state follows a consistent pattern throughout the codebase. Modules access the state through the AGISystem's shared_state attribute, ensuring a single source of truth. The state is updated directly by assigning new values to its attributes, leveraging Python's dynamic nature for simplicity.

Configuration parameters from Config.py influence the initial state and behavior:

  • CURIOSITY_CHANCE: Probability of triggering curiosity-based state updates
  • REFLECTION_CHANCE: Probability of initiating reflection when mood doesn't improve
  • POSITIVE_MOODS/Negative_MOODS: Lists defining which moods contribute to overall emotional score

The system now includes state restoration capabilities during startup. The AGISystem attempts to load previous state data from the shutdown_state.json file, restoring mood, current plans, shared state, and Snake Agent state when available:

async def _load_previous_state(self):
    """Load previous system state if available."""
    try:
        previous_state = await load_previous_state()
        if not previous_state:
            logger.info("No previous state found, starting fresh")
            return
        
        # Restore mood if available
        if "mood" in agi_state and hasattr(self, 'emotional_intelligence'):
            self.emotional_intelligence.set_mood_vector(agi_state["mood"])
        
        # Restore shared state
        if "shared_state" in agi_state and hasattr(self, 'shared_state'):
            shared_data = agi_state["shared_state"]
            if "current_task" in shared_data:
                self.shared_state.current_task = shared_data["current_task"]
            if "current_situation_id" in shared_data:
                self.shared_state.current_situation_id = shared_data["current_situation_id"]
        
        # Restore Snake Agent state
        if "snake_agent" in agi_state and self.snake_agent and SnakeAgentState:
            snake_data = agi_state["snake_agent"]
            if "state" in snake_data:
                restored_state = SnakeAgentState.from_dict(snake_data["state"])
                self.snake_agent.state = restored_state
    except Exception as e:
        logger.error(f"Error loading previous state: {e}")

Section sources

State Transitions and Propagation

State transitions occur through a well-defined sequence in the AGISystem's run_iteration method. The process begins with external data updates and progresses through several stages:

  1. External Updates: Search results are added to shared_state.search_results
  2. Behavior Handling: Mood-based behavior modifiers are applied
  3. Curiosity Processing: New curiosity topics are generated and stored
  4. Situation Generation: A new situation is created and assigned to current_situation
  5. Memory Retrieval: Relevant memories are fetched and stored in recent_memories
  6. Decision Making: The decision engine accesses the current state to make choices
  7. Action Execution: Actions may modify various state attributes
  8. Mood Update: Emotional intelligence processes the action output to update mood

The propagation of state changes is primarily push-based, with the AGISystem actively pushing updates to the shared state. However, some modules like CuriosityTrigger can also push updates directly when they generate new information.

Diagram sources

Section sources

Thread-Safe Access in Asynchronous Environment

While the current implementation does not explicitly use locks or thread-safe data structures, it operates within an asyncio event loop that provides inherent thread safety for single-threaded execution. The AGISystem's run_iteration method is designed to complete one full cycle of state updates atomically, minimizing the risk of race conditions.

Background tasks interact with the shared state through the main event loop, ensuring serialized access:

  • Data collection task updates the system through the data service
  • Event detection task processes articles and may trigger state changes
  • Knowledge compression task updates knowledge without direct state modification
  • Memory consolidation task optimizes memory storage

The system relies on Python's asyncio primitives to coordinate access, with shared state modifications occurring within coroutine functions that yield control appropriately. This design assumes that the event loop will serialize access to the shared state, preventing concurrent modifications.

Section sources

State Persistence and Serialization

The system now includes comprehensive state persistence capabilities, allowing for state restoration during shutdown and startup. The ShutdownCoordinator manages this process through a dedicated state_persistence phase:

async def _phase_state_persistence(self):
    """Phase 5: Persist system state for recovery."""
    if not Config.STATE_PERSISTENCE_ENABLED:
        logger.info("State persistence disabled")
        return
    
    logger.info("Persisting system state...")
    
    try:
        state_data = await self._collect_system_state()
        
        state_file = Path(Config.SHUTDOWN_STATE_FILE)
        with open(state_file, 'w', encoding='utf-8') as f:
            json.dump(state_data, f, indent=2, default=str)
        
        logger.info(f"System state saved to {state_file}")
    except Exception as e:
        logger.error(f"Error persisting system state: {e}")

The state persistence mechanism captures various components of the system state:

  • AGI System State: Mood, current plans, shared state, research progress, invention history
  • Snake Agent State: Analysis state, pending experiments, communication queue, learning history
  • Configuration: System version and timestamp

The SnakeAgent integrates with this system through its own state management:

@dataclass
class SnakeAgentState:
    """State management for Snake Agent"""
    last_analysis_time: datetime = None
    analyzed_files: Set[str] = None
    pending_experiments: List[Dict[str, Any]] = None
    communication_queue: List[Dict[str, Any]] = None
    learning_history: List[Dict[str, Any]] = None
    current_task: Optional[str] = None
    mood: str = "curious"
    experiment_success_rate: float = 0.0
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert state to dictionary for persistence"""
        return {
            "last_analysis_time": self.last_analysis_time.isoformat() if self.last_analysis_time else None,
            "analyzed_files": list(self.analyzed_files),
            "pending_experiments": self.pending_experiments,
            "communication_queue": self.communication_queue,
            "learning_history": self.learning_history,
            "current_task": self.current_task,
            "mood": self.mood,
            "experiment_success_rate": self.experiment_success_rate
        }
    
    @classmethod
    def from_dict(cls, data: Dict[str, Any]) -> 'SnakeAgentState':
        """Create state from dictionary"""
        state = cls()
        if data.get("last_analysis_time"):
            state.last_analysis_time = datetime.fromisoformat(data["last_analysis_time"])
        state.analyzed_files = set(data.get("analyzed_files", []))
        state.pending_experiments = data.get("pending_experiments", [])
        state.communication_queue = data.get("communication_queue", [])
        state.learning_history = data.get("learning_history", [])
        state.current_task = data.get("current_task")
        state.mood = data.get("mood", "curious")
        state.experiment_success_rate = data.get("experiment_success_rate", 0.0)
        return state

Configuration options control the state persistence behavior:

  • STATE_PERSISTENCE_ENABLED: Enables or disables state persistence (default: True)
  • SHUTDOWN_STATE_FILE: Specifies the file path for state persistence (default: shutdown_state.json)
  • SNAKE_STATE_PERSISTENCE: Controls Snake Agent state persistence (default: True)

After successful state restoration, the system automatically cleans up the state file:

def cleanup_state_file():
    """Clean up the state file after successful recovery."""
    try:
        state_file = Path(Config.SHUTDOWN_STATE_FILE)
        if state_file.exists():
            state_file.unlink()
            logger.info("Previous state file cleaned up")
    except Exception as e:
        logger.warning(f"Could not clean up state file: {e}")

Section sources

Debugging and Monitoring State Evolution

The system provides extensive logging and monitoring capabilities for tracking state evolution:

Logging Mechanisms

  • JSON or text-formatted logs based on LOG_FORMAT configuration
  • Detailed logging of state changes at each iteration
  • Mood score calculations logged with old and new values
  • Decision-making context logged with raw LLM responses

Monitoring Tools

  • get_state_summary method provides a quick overview of key state elements
  • Mood history tracking enables analysis of emotional trends
  • Invention history records creative outputs over time
  • Database logs provide persistent records of situations, decisions, and moods

The EmotionalIntelligence module includes a demonstration script that shows mood evolution in response to different action outputs, serving as a valuable debugging tool. Additionally, the system logs the rationale behind behavior modifiers and reflection triggers, providing insight into the decision-making process.

The ShutdownCoordinator provides comprehensive shutdown logging:

🛑 Initiating graceful shutdown - Reason: manual
📋 Shutdown timeout: 30s, Force timeout: 60s
🔄 Executing shutdown phase: tasks_stopping
✅ Phase 'tasks_stopping' completed in 2.34s
🔄 Executing shutdown phase: memory_service_cleanup
✅ Phase 'memory_service_cleanup' completed in 1.87s
...
✅ Graceful shutdown completed successfully

Section sources

Common Issues and Best Practices

Common Issues

Stale State References: Since modules hold references to the shared state, they may occasionally work with outdated data. This is mitigated by the sequential nature of the main loop.

Concurrent Modification Conflicts: While the asyncio event loop prevents true concurrency, background tasks could potentially modify state simultaneously. The current design minimizes this risk by limiting direct state modifications in background tasks.

Incorrect State Initialization: If the EmotionalIntelligence module fails to initialize properly, the initial mood vector may be incomplete. The system handles this by using default mood values from configuration.

State Persistence Failures: During shutdown, state persistence might fail due to file system issues or permission errors. The system logs these errors but continues with the shutdown process.

Best Practices

  • Always access shared state through the AGISystem instance
  • Use the provided configuration constants rather than hardcoding values
  • Limit direct state modifications to the main iteration loop when possible
  • Utilize the logging system to trace state changes for debugging
  • Regularly monitor mood history to ensure emotional stability
  • Keep state attributes focused on runtime data, delegating persistent storage to specialized services
  • Enable state persistence in production environments to maintain continuity across restarts
  • Monitor the shutdown_state.json file size to prevent excessive growth
  • Implement regular backup strategies for critical state files

Section sources

Conclusion

The SharedState class provides a robust foundation for managing dynamic runtime data in the Ravana AGI system. Its simple yet comprehensive data model captures essential aspects of the system's state, while its integration with the AGISystem enables coordinated behavior across multiple modules. The design prioritizes accessibility and flexibility, allowing for easy extension and modification. By leveraging Python's built-in data types and the asyncio event loop, the implementation achieves efficient state management without complex synchronization mechanisms. The recent addition of state persistence capabilities through the ShutdownCoordinator enhances system reliability by enabling state restoration during shutdown and startup, with specific integration for the SnakeAgent's internal state. The extensive logging and monitoring capabilities ensure that state evolution can be effectively tracked and analyzed, supporting both debugging and long-term system improvement.

Referenced Files in This Document