2. File Handling Implementation for Submission System

Status

Provisional 2025-02-14

Implemented by openedx/edx-submissions#286

Context

As part of the XQueue migration effort detailed in ADR 0001, we need to implement a file handling system within edx-submissions. Currently, XQueue manages file submissions through a tightly coupled approach.

### Current Limitations

1. Inadequate File Management: XQueue’s approach relies on JSON strings in character fields, with size constraints and manual URL manipulation for file handling.

2. Process Inefficiencies: The current system uses synchronous HTTP for file retrieval, lacks proper validation, and tightly couples submission processing with file handling.

3. Integration Challenges: External graders depend on specific URL formats with HTTP-based retrieval, embedding file information directly in submission payloads.

Decision

We will implement a dedicated file management system for the assessment submission process, focusing on workflow and educational needs:

1. Centralized Storage: Create a unified repository for student-submitted files, ensuring they are properly associated with their assessments and accessible throughout the grading process.

2. Streamlined Workflow: Design a clear process where files are automatically processed during submission creation, securely stored, and efficiently delivered to grading systems.

3. Consistent Experience: Maintain compatibility with existing systems to ensure a smooth transition, allowing instructors and external graders to access files without changes to their established workflows.

Consequences

Positive:

1. Architecture: Clean separation of concerns, improved maintainability, better error handling, optimized database access

2. Integration: Seamless xqueue-watcher compatibility, support for existing workflows, minimal client code changes

3. Operations: Robust file validation, improved tracking, better error visibility, simplified lifecycle management

Negative:

1. Technical: Additional database structures

2. Migration: Temporary system complexity, additional testing needs

3. Performance: File processing overhead

References

Current System Documentation:

• XQueue Repository: openedx/xqueue

• XQueue Watcher Repository: openedx/xqueue-watcher

Related Repositories:

• edx-submissions: openedx/edx-submissions

• edx-platform: openedx/edx-platform

• XQueue Repository: openedx/xqueue

Related Documentation:

• ADR 0001: Creation of ExternalGraderDetail Model for XQueue Migration

Future Event Integration:

• Open edX Events Framework: openedx/openedx-events

• Event Bus Documentation: https://openedx.atlassian.net/wiki/spaces/AC/pages/124125264/Event+Bus

Related Architecture Documents:

• Open edX Architecture Guidelines: https://openedx.atlassian.net/wiki/spaces/AC/pages/124125264/Architecture+Guidelines