🗑️ Remove PostgreSQL database dependencies and related files

- Delete docker-compose.yaml (PostgreSQL and pgAdmin services)
- Remove src/preprocess/db.py (database connection file)
- Clean up setup.py: remove psycopg2-binary and SQLAlchemy dependencies
- Remove docker/ directory and initialization files
- Update README.md to reflect simplified structure
- Add Excel processing dependencies (openpyxl, xlrd) to setup.py
- Project now uses CSV files directly without database layer
- Simplified deployment without Docker dependencies

README.md

@@ -32,7 +32,6 @@ SD_roster_real/
 ├── main.py                          # Main entry point
 ├── README.md
 ├── requirements.txt
-├── docker-compose.yaml
 ├── setup.py
 │
 ├── src/                             # Core business logic

STREAMLIT_README.md

@@ -1,161 +0,0 @@
-# SD Roster Optimization Tool - Multi-Page Streamlit Interface
-
-A comprehensive multi-page Streamlit web application for supply chain roster optimization using OR-Tools.
-
-## 🏗️ Application Structure
-
-### 🏠 Home Page
-- **Welcome Dashboard**: Overview and navigation hub
-- **Global Settings**: Shared data path and date configuration  
-- **System Status**: Component availability and health checks
-- **Quick Navigation**: Direct links to main functionalities
-
-### 📊 Dataset Metadata Page
-Comprehensive data analysis across five detailed tabs:
-
-#### 📋 Data Overview Tab
-- **Key Metrics**: Orders, quantities, products, employees, production lines
-- **Data Quality Analysis**: Completeness scores and missing data indicators
-- **Data Freshness**: Latest data timestamps and age indicators
-
-#### 📦 Demand Analysis Tab  
-- **Demand Metrics**: Total, average, max, min order sizes
-- **Top Products**: Ranking by demand volume with visualizations
-- **Daily Patterns**: Trend analysis and demand variability
-- **Distribution Analysis**: Order quantity and frequency distributions
-
-#### 👥 Workforce Analysis Tab
-- **Employee Metrics**: Total staff, types, distribution
-- **Cost Structure**: Hourly rates by employee type and shift
-- **Productivity Analysis**: Performance metrics by employee type
-
-#### 🏭 Production Capacity Tab
-- **Line Metrics**: Total lines, types, maximum capacities
-- **Capacity Distribution**: Line allocation and utilization potential
-- **Theoretical Analysis**: Maximum capacity calculations by shift
-
-#### 💰 Cost Analysis Tab
-- **Cost Structure**: Min/max/average hourly rates and ranges
-- **Budget Planning**: Minimum and maximum cost scenarios
-- **Projections**: Weekly and monthly cost estimates
-
-### 🎯 Optimization Page
-Advanced optimization interface with comprehensive results:
-
-#### 📊 Summary Tab
-- Total optimization cost and key metrics
-- Cost efficiency analysis (cost per day, cost per unit)
-- Optimization parameters used
-
-#### 📈 Production Tab
-- Production vs. demand comparison by product
-- Fulfillment rate analysis with interactive charts
-- Production schedule visualization
-
-#### 👷 Labor Tab
-- Labor allocation by employee type and shift
-- Required headcount analysis
-- Daily and average staffing requirements
-
-#### 💰 Costs Tab
-- Detailed cost breakdown by employee type and shift
-- Cost distribution visualizations
-- Priority mode analysis (when applicable)
-
-## Quick Start
-
-### 1. Install Dependencies
-```bash
-pip install -r requirements.txt
-```
-
-### 2. Run the Application
-```bash
-# Option 1: Using the runner script
-python run_streamlit.py
-
-# Option 2: Direct streamlit command
-streamlit run Home.py
-```
-
-### 3. Access the Application
-Open your browser to: `http://localhost:8501`
-
-## Usage Guide
-
-### Navigation Flow
-1. **Start at Home**: Configure global settings and navigate to specific functions
-2. **Explore Metadata**: Analyze your data across the comprehensive metadata tabs
-3. **Run Optimization**: Configure parameters and execute optimization on the dedicated page
-4. **Analyze Results**: Review detailed results across multiple result tabs
-
-### Page-by-Page Guide
-1. **Home Page**: Set data paths, select date ranges, check system status
-2. **Dataset Metadata**: Deep dive into demand, workforce, capacity, and cost analysis
-3. **Optimization**: Configure optimization parameters, run optimization, analyze results
-
-## Technical Details
-
-### Optimization Engine
-- Built on Google OR-Tools for mixed-integer programming
-- Supports multiple constraint modes for realistic business scenarios
-- Handles complex multi-product, multi-shift, multi-line scheduling
-
-### Data Sources
-The application automatically loads data from:
-- `COOIS_Released_Prod_Orders.csv` - Production orders and demand
-- Employee data files - Staff availability and costs
-- Production line configuration - Line capacities and capabilities
-
-### Configuration
-Key optimization parameters can be adjusted in `src/config/optimization_config.py`:
-- Employee types and costs
-- Shift definitions and durations
-- Production line capacities
-- Constraint modes and business rules
-
-## Business Scenarios
-
-### Priority Mode (Recommended)
-- Uses UNICEF Fixed term staff first
-- Engages Humanizer staff only when fixed staff at capacity
-- Reflects realistic business operations
-
-### Mandatory Mode
-- Forces all fixed staff to work full hours
-- More expensive but ensures full utilization
-- Useful for guaranteed staffing scenarios
-
-### Demand-Driven Mode
-- Purely cost-optimized scheduling
-- No mandatory fixed hours
-- Most cost-efficient but may underutilize staff
-
-## Troubleshooting
-
-### Common Issues
-1. **No Date Ranges Available**: Ensure your data files are in the correct location
-2. **Optimization Fails**: Check that demand data exists for the selected date range
-3. **Import Errors**: Verify all dependencies are installed
-
-### Performance Tips
-- Smaller date ranges optimize faster
-- Reducing product count can improve solve time
-- Priority mode typically solves faster than mandatory mode
-
-## File Structure
-```
-Home.py                   # Main home page (entry point)
-pages/
-  ├── 1_📊_Dataset_Metadata.py  # Comprehensive data analysis page
-  └── 2_🎯_Optimization.py      # Optimization interface and results
-run_streamlit.py          # Convenient runner script  
-src/
-  ├── models/optimizer_real.py  # Core optimization engine
-  ├── config/optimization_config.py  # Configuration parameters
-  └── etl/                # Data extraction and transformation
-streamlit_app_old.py      # Backup of original single-page app
-```
-
-## Support
-For technical issues or feature requests, refer to the main project documentation or contact the development team.

data/real_data_excel/converted_csv/Kit_Composition_and_relation_cleaned_with_line_type_TEST.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b371c129393480a757a7839ba092dfe076037954aeb3f3c9bc962ae83ef8b9f
+size 23800

docker-compose.yaml

@@ -1,36 +0,0 @@
-version: '3.8'
-services:
-  db:
-    image: postgres:16-alpine
-    container_name: sd_postgres
-    restart: unless-stopped
-    ports:
-      - "${DB_PORT:-5432}:5432"
-    environment:
-      POSTGRES_USER: hjun
-      POSTGRES_PASSWORD: alsdfjwpoejfkd
-      POSTGRES_DB: sd_roster_real
-    volumes:
-      - db_data:/var/lib/postgresql/data
-      - ./docker/init:/docker-entrypoint-initdb.d:ro   # 초기 스키마/권한 스크립트
-    healthcheck:
-      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
-      interval: 5s
-      timeout: 3s
-      retries: 20
-
-  pgadmin:
-    image: dpage/pgadmin4
-    container_name: sd_pgadmin
-    restart: unless-stopped
-    environment:
-      PGADMIN_DEFAULT_EMAIL: [email protected]
-      PGADMIN_DEFAULT_PASSWORD: alsdfjwpoejfkd
-    ports:
-      - "${PGADMIN_PORT:-5050}:80"
-    depends_on:
-      db:
-        condition: service_healthy
-
-volumes:
-  db_data:

requirements-viz.txt

@@ -1,7 +0,0 @@
-# Optional visualization dependencies for enhanced hierarchy dashboard
-# Install with: pip install -r requirements-viz.txt
-
-networkx>=2.8.0       # For dependency network graphs
-plotly>=5.0.0         # For interactive charts (should already be installed)
-pandas>=1.3.0         # For data processing (should already be installed)
-numpy>=1.20.0         # For numerical operations (should already be installed)

requirements.txt

@@ -6,3 +6,7 @@ scipy>=1.9.0
 ortools>=9.0.0
 openpyxl>=3.0.0
 xlrd>=2.0.0
+# Optional visualization dependencies for enhanced hierarchy dashboard
+# Install with: pip install -r requirements-viz.txt
+
+networkx>=2.8.0       # For dependency network graphs

setup.py

@@ -9,22 +9,21 @@ setup(
     packages=find_packages(),
     install_requires=[
         "absl-py>=2.3.1",
-        "dotenv>=0.9.9", 
         "immutabledict>=4.2.1",
         "numpy>=2.2.0",
         "ortools>=9.14.0",
         "pandas>=2.3.0",
         "plotly>=5.24.0",
         "protobuf>=3.20,<6",
-        "psycopg2-binary>=2.9.9",
         "python-dateutil>=2.9.0",
         "python-dotenv>=1.0.0",
         "pytz>=2025.2",
         "six>=1.17.0",
-        "SQLAlchemy>=2.0.36",
         "streamlit>=1.39.0",
         "typing_extensions>=4.14.0",
         "tzdata>=2025.2",
+        "openpyxl>=3.0.0",
+        "xlrd>=2.0.0",
     ],
     python_requires=">=3.10,<3.11",
 )

src/preprocess/db.py

@@ -1,17 +0,0 @@
-from sqlalchemy import create_engine
-from dotenv import load_dotenv
-import os
-
-load_dotenv()
-USER = os.getenv("POSTGRES_USER", "myuser")
-PWD = os.getenv("POSTGRES_PASSWORD", "mypass")
-DB = os.getenv("POSTGRES_DB", "mydb")
-PORT = os.getenv("DB_PORT", "5432")
-HOST = "localhost"
-
-engine = create_engine(
-    f"postgresql+psycopg2://{USER}:{PWD}@{HOST}:{PORT}/{DB}", future=True
-)
-if __name__ == "__main__":
-    with engine.begin() as conn:
-        print(conn.execute("select version();").scalar())

test_hierarchy_viz.py

@@ -1,167 +0,0 @@
-#!/usr/bin/env python3
-"""
-Test script for hierarchy visualization
-Run this to see a demo of the hierarchy dashboard
-"""
-
-import sys
-import os
-sys.path.append('src')
-
-# Sample data for testing
-def create_sample_results():
-    """Create sample optimization results for testing"""
-    
-    # Sample schedule data showing hierarchy flow
-    sample_schedule = [
-        # Day 1: Prepacks first
-        {'day': 1, 'line_type_id': 6, 'line_idx': 1, 'shift': 1, 'product': 'PREPACK_A', 'run_hours': 4.0, 'units': 200},
-        {'day': 1, 'line_type_id': 7, 'line_idx': 1, 'shift': 1, 'product': 'PREPACK_B', 'run_hours': 3.5, 'units': 150},
-        
-        # Day 2: Subkits using prepacks
-        {'day': 2, 'line_type_id': 6, 'line_idx': 1, 'shift': 1, 'product': 'SUBKIT_X', 'run_hours': 5.0, 'units': 100},
-        {'day': 2, 'line_type_id': 7, 'line_idx': 2, 'shift': 2, 'product': 'SUBKIT_Y', 'run_hours': 4.5, 'units': 80},
-        
-        # Day 3: Master kits using subkits
-        {'day': 3, 'line_type_id': 6, 'line_idx': 2, 'shift': 1, 'product': 'MASTER_FINAL', 'run_hours': 6.0, 'units': 50},
-        {'day': 3, 'line_type_id': 7, 'line_idx': 1, 'shift': 3, 'product': 'MASTER_DELUXE', 'run_hours': 5.5, 'units': 40},
-    ]
-    
-    # Sample hierarchy data
-    sample_kit_levels = {
-        'PREPACK_A': 0,  # Prepack
-        'PREPACK_B': 0,  # Prepack
-        'SUBKIT_X': 1,   # Subkit
-        'SUBKIT_Y': 1,   # Subkit
-        'MASTER_FINAL': 2,   # Master
-        'MASTER_DELUXE': 2,  # Master
-    }
-    
-    # Sample dependencies
-    sample_dependencies = {
-        'SUBKIT_X': ['PREPACK_A'],
-        'SUBKIT_Y': ['PREPACK_B'],
-        'MASTER_FINAL': ['SUBKIT_X', 'PREPACK_A'],
-        'MASTER_DELUXE': ['SUBKIT_Y', 'PREPACK_B'],
-    }
-    
-    # Sample production totals
-    weekly_production = {
-        'PREPACK_A': 200,
-        'PREPACK_B': 150,
-        'SUBKIT_X': 100,
-        'SUBKIT_Y': 80,
-        'MASTER_FINAL': 50,
-        'MASTER_DELUXE': 40,
-    }
-    
-    # Sample workforce data
-    person_hours_by_day = [
-        {'day': 1, 'emp_type': 'UNICEF Fixed term', 'used_person_hours': 16, 'cap_person_hours': 64},
-        {'day': 1, 'emp_type': 'Humanizer', 'used_person_hours': 40, 'cap_person_hours': 80},
-        {'day': 2, 'emp_type': 'UNICEF Fixed term', 'used_person_hours': 20, 'cap_person_hours': 64},
-        {'day': 2, 'emp_type': 'Humanizer', 'used_person_hours': 45, 'cap_person_hours': 80},
-        {'day': 3, 'emp_type': 'UNICEF Fixed term', 'used_person_hours': 18, 'cap_person_hours': 64},
-        {'day': 3, 'emp_type': 'Humanizer', 'used_person_hours': 42, 'cap_person_hours': 80},
-    ]
-    
-    return {
-        'objective': 12500.75,  # Total cost
-        'run_schedule': sample_schedule,
-        'weekly_production': weekly_production,
-        'person_hours_by_day': person_hours_by_day,
-        'kit_levels': sample_kit_levels,
-        'kit_dependencies': sample_dependencies
-    }
-
-def test_hierarchy_flow():
-    """Test the hierarchy flow visualization components"""
-    print("🧪 Testing Hierarchy Visualization Components")
-    print("=" * 50)
-    
-    # Create sample data
-    results = create_sample_results()
-    print(f"✅ Created sample results with {len(results['run_schedule'])} production runs")
-    
-    try:
-        # Test imports
-        from src.visualization.hierarchy_dashboard import (
-            prepare_hierarchy_flow_data,
-            prepare_hierarchy_analytics_data,
-            calculate_hierarchy_line_utilization,
-            get_hierarchy_level_summary
-        )
-        print("✅ Successfully imported hierarchy dashboard functions")
-        
-        # Test flow data preparation
-        flow_data = prepare_hierarchy_flow_data(results)
-        print(f"✅ Prepared flow data: {len(flow_data)} flow records")
-        
-        # Test analytics data
-        analytics = prepare_hierarchy_analytics_data(results)
-        print(f"✅ Prepared analytics data: {analytics['dependency_violations']} violations detected")
-        
-        # Test line utilization calculation
-        line_util = calculate_hierarchy_line_utilization(results)
-        print(f"✅ Calculated line utilization for {len(line_util)} lines")
-        
-        # Test hierarchy summary
-        summary = get_hierarchy_level_summary(flow_data)
-        print("✅ Generated hierarchy level summary:")
-        for level, data in summary.items():
-            print(f"   - {level.title()}: {data['count']} products, {data['total_units']} units")
-        
-        print("\n🎉 All hierarchy visualization components working correctly!")
-        print("\nTo see the full visualization:")
-        print("1. Run your Streamlit app: streamlit run app.py")
-        print("2. Go to Settings page and run optimization")
-        print("3. Check the '🔄 Hierarchy Flow' tab in results")
-        
-        return True
-        
-    except Exception as e:
-        print(f"❌ Error testing hierarchy visualization: {e}")
-        import traceback
-        traceback.print_exc()
-        return False
-
-def display_sample_hierarchy_info():
-    """Display information about the sample hierarchy"""
-    print("\n📊 Sample Hierarchy Structure:")
-    print("=" * 30)
-    
-    print("🟢 PREPACKS (Level 0):")
-    print("   - PREPACK_A: Basic components")
-    print("   - PREPACK_B: Basic components")
-    
-    print("\n🟡 SUBKITS (Level 1):")
-    print("   - SUBKIT_X: Uses PREPACK_A")
-    print("   - SUBKIT_Y: Uses PREPACK_B")
-    
-    print("\n🔴 MASTERS (Level 2):")
-    print("   - MASTER_FINAL: Uses SUBKIT_X + PREPACK_A")
-    print("   - MASTER_DELUXE: Uses SUBKIT_Y + PREPACK_B")
-    
-    print("\n📅 Production Flow:")
-    print("   Day 1: Produce prepacks first (dependencies)")
-    print("   Day 2: Produce subkits (using prepacks)")  
-    print("   Day 3: Produce masters (using subkits)")
-    
-    print("\nThis demonstrates the optimal hierarchy flow!")
-
-if __name__ == "__main__":
-    print("🔄 Hierarchy Visualization Test")
-    print("=" * 40)
-    
-    # Display sample info
-    display_sample_hierarchy_info()
-    
-    # Test the components
-    success = test_hierarchy_flow()
-    
-    if success:
-        print(f"\n✅ Test completed successfully!")
-    else:
-        print(f"\n❌ Test failed - check error messages above")
-    
-    print("\n" + "=" * 40)

test_kit_relationships.py

@@ -1,162 +0,0 @@
-#!/usr/bin/env python3
-"""
-Test script for kit relationships visualization
-Tests the actual kit dependency relationships from kit_hierarchy.json
-"""
-
-import sys
-import os
-sys.path.append('src')
-
-def test_kit_relationships():
-    """Test kit relationships visualization"""
-    print("🔗 Testing Kit Relationships Visualization")
-    print("=" * 50)
-    
-    try:
-        # Test importing the kit relationships module
-        from src.visualization.kit_relationships import (
-            load_kit_hierarchy,
-            build_relationship_data,
-            get_production_timing,
-            find_dependency_violations
-        )
-        print("✅ Successfully imported kit relationships module")
-        
-        # Test loading hierarchy data
-        hierarchy_data = load_kit_hierarchy()
-        if hierarchy_data:
-            print(f"✅ Loaded kit hierarchy: {len(hierarchy_data)} master kits")
-            
-            # Show some example relationships
-            print("\n📋 Sample Kit Relationships:")
-            count = 0
-            for kit_id, kit_info in hierarchy_data.items():
-                if kit_info.get('dependencies') and count < 5:
-                    deps = kit_info['dependencies']
-                    kit_name = kit_info.get('name', kit_id)[:50] + "..." if len(kit_info.get('name', '')) > 50 else kit_info.get('name', kit_id)
-                    print(f"  • {kit_id} ({kit_name})")
-                    print(f"    Depends on: {deps}")
-                    count += 1
-            
-            # Test with sample production data
-            sample_produced_kits = set(list(hierarchy_data.keys())[:10])  # First 10 kits
-            print(f"\n🧪 Testing with {len(sample_produced_kits)} sample produced kits")
-            
-            relationships = build_relationship_data(hierarchy_data, sample_produced_kits)
-            print(f"✅ Found {len(relationships)} dependency relationships")
-            
-            if relationships:
-                print("\n🔗 Sample Relationships:")
-                for i, rel in enumerate(relationships[:5]):
-                    print(f"  {i+1}. {rel['source']} → {rel['target']} ({rel['source_type']} → {rel['target_type']})")
-            
-            # Test production timing analysis
-            sample_timing = {kit: i % 5 + 1 for i, kit in enumerate(sample_produced_kits)}  # Random days 1-5
-            violations = find_dependency_violations(sample_timing, relationships)
-            print(f"✅ Dependency analysis: {len(violations)} violations found")
-            
-            print("\n🎉 Kit relationships visualization components working!")
-            return True
-            
-        else:
-            print("⚠️ No kit hierarchy data found - please check kit_hierarchy.json")
-            return False
-            
-    except FileNotFoundError:
-        print("❌ Kit hierarchy file not found at data/hierarchy_exports/kit_hierarchy.json")
-        return False
-    except Exception as e:
-        print(f"❌ Error testing kit relationships: {e}")
-        import traceback
-        traceback.print_exc()
-        return False
-
-def display_hierarchy_structure():
-    """Display the structure of the hierarchy data"""
-    print("\n📊 Kit Hierarchy Structure Analysis")
-    print("=" * 40)
-    
-    try:
-        from src.visualization.kit_relationships import load_kit_hierarchy
-        hierarchy_data = load_kit_hierarchy()
-        
-        if not hierarchy_data:
-            print("No hierarchy data available")
-            return
-        
-        # Analyze hierarchy structure
-        masters = []
-        subkits = []
-        prepacks = []
-        
-        total_dependencies = 0
-        
-        for kit_id, kit_info in hierarchy_data.items():
-            kit_type = kit_info.get('type', 'unknown')
-            dependencies = kit_info.get('dependencies', [])
-            total_dependencies += len(dependencies)
-            
-            if kit_type == 'master':
-                masters.append(kit_id)
-            elif kit_type == 'subkit':
-                subkits.append(kit_id)
-            elif kit_type == 'prepack':
-                prepacks.append(kit_id)
-        
-        print(f"📦 Total Kits: {len(hierarchy_data)}")
-        print(f"  • Masters: {len(masters)}")
-        print(f"  • Subkits: {len(subkits)}")
-        print(f"  • Prepacks: {len(prepacks)}")
-        print(f"🔗 Total Dependencies: {total_dependencies}")
-        
-        # Find most complex kit (most dependencies)
-        max_deps = 0
-        most_complex = None
-        
-        for kit_id, kit_info in hierarchy_data.items():
-            deps = len(kit_info.get('dependencies', []))
-            if deps > max_deps:
-                max_deps = deps
-                most_complex = kit_id
-        
-        if most_complex:
-            print(f"🏆 Most Complex Kit: {most_complex} ({max_deps} dependencies)")
-        
-        # Show dependency chains
-        print(f"\n🔄 Sample Dependency Chains:")
-        chains_shown = 0
-        for kit_id, kit_info in hierarchy_data.items():
-            if kit_info.get('dependencies') and chains_shown < 3:
-                deps = kit_info['dependencies']
-                kit_name = kit_info.get('name', kit_id)[:40] + "..." if len(kit_info.get('name', '')) > 40 else kit_info.get('name', kit_id)
-                print(f"  Chain {chains_shown + 1}: {kit_name}")
-                for dep in deps[:3]:  # Show first 3 dependencies
-                    dep_info = hierarchy_data.get(dep, {})
-                    dep_name = dep_info.get('name', dep)[:30] + "..." if len(dep_info.get('name', '')) > 30 else dep_info.get('name', dep)
-                    print(f"    ↳ Needs: {dep_name}")
-                chains_shown += 1
-        
-        print(f"\nThis data will be visualized in the dashboard! 🎨")
-        
-    except Exception as e:
-        print(f"Error analyzing hierarchy: {e}")
-
-if __name__ == "__main__":
-    # Display hierarchy structure
-    display_hierarchy_structure()
-    
-    # Test kit relationships
-    success = test_kit_relationships()
-    
-    if success:
-        print(f"\n✅ Kit relationships test completed successfully!")
-        print(f"\nTo see the visualization:")
-        print(f"1. Run: streamlit run app.py")
-        print(f"2. Go to Settings → Run Optimization")  
-        print(f"3. Check 'Hierarchy Flow' → 'Kit Relationships' tab")
-        print(f"4. See the interactive network graph! 🕸️")
-    else:
-        print(f"\n❌ Test failed - check error messages above")
-    
-    print(f"\n" + "=" * 50)