Add example for EntitySynchronizer.sync_generator method

[serra]

Description

This issue proposes adding an example that demonstrates the use of the generator pattern for entity synchronization using the EntitySynchronizer.sync_generator method.

Proposed Example

# Example demonstrating the use of the generator pattern for entity synchronization.
#
# This example shows how to use the EntitySynchronizer.sync_generator method
# to process entities in batches as they arrive, rather than waiting for the
# entire sync to complete.

import logging
from floriday_supplier_client import TradeItemsApi
from floriday_supplier_client.api_factory import ApiFactory
from floriday_supplier_client.sync.entity_sync import EntitySynchronizer

# Configure logging
logging.basicConfig(
    level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)


def example_generator_pattern():
    # Example demonstrating the generator pattern for entity synchronization.
    #
    # This approach is useful when:
    # 1. Processing large datasets that might not fit in memory
    # 2. Implementing custom processing logic for each entity
    # 3. Providing real-time feedback during synchronization
    # 4. Implementing pause/resume functionality
    
    print("\n=== Example: Using Generator Pattern for Trade Items Sync ===\n")

    # Initialize API client
    factory = ApiFactory()
    client = factory.get_api_client()
    api_instance = TradeItemsApi(client)

    # Create a synchronizer without persistence callback
    # We'll handle the entities ourselves as they arrive
    synchronizer = EntitySynchronizer(
        entity_type="trade_items",
        fetch_entities_callback=api_instance.get_trade_items_by_sequence_number,
        start_seq_number=0,
        batch_size=10,  # Using a smaller batch size for demonstration
    )

    # Track statistics
    total_processed = 0
    batches_processed = 0
    
    # Custom processing logic
    def process_trade_item(item):
        # Custom processing logic for each trade item.
        print(f"Processing trade item: {item.trade_item_id} - {getattr(item, 'trade_item_name', 'N/A')}")
        # In a real application, you might:
        # - Transform the data
        # - Apply business rules
        # - Save to a different database
        # - Generate reports
        # - etc.
        return item.trade_item_id

    print("Starting synchronization using generator pattern...")
    
    # Use the generator to process entities in batches
    for batch_number, batch in enumerate(synchronizer.sync_generator(), 1):
        batches_processed += 1
        batch_size = len(batch)
        total_processed += batch_size
        
        print(f"\nProcessing batch #{batch_number} with {batch_size} items")
        
        # Process each entity in the batch
        for item in batch:
            process_trade_item(item)
            
        # Provide real-time feedback
        print(f"Progress: {total_processed} items processed in {batches_processed} batches")
        
        # Example of conditional logic - we could pause, change processing, etc.
        if batches_processed >= 5:  # Limit to 5 batches for this example
            print("\nReached batch limit, stopping early")
            break
    
    print(f"\nFinished processing {total_processed} items in {batches_processed} batches")


if __name__ == "__main__":
    example_generator_pattern()

Benefits

The generator pattern provides several benefits:

1. Memory efficiency: Only one batch of entities is in memory at a time
2. Real-time processing: Process entities as they arrive rather than waiting for all data
3. Custom control flow: Ability to pause, resume, or stop processing based on custom conditions
4. Separation of concerns: Decouples entity retrieval from processing logic

Implementation Plan

1. Create a new example file (e.g., example_generator.py)
2. Add the example code demonstrating the generator pattern
3. Update documentation to reference this example
4. Consider adding a test that verifies the generator functionality

Related

This is part of the refactoring work for issue #3 to move the generic entity synchronization code from vine-floriday-adapter to floriday-supplier-client.