cardosofelipe/syndarix
1
0
Fork 0
forked from cardosofelipe/fast-next-template
Code Issues 18 Pull Requests Wiki Activity

[SPIKE-009] Issue Sync with External Trackers #9

New Issue
Closed
opened 2025-12-29 03:51:01 +00:00 by cardosofelipe · 1 comment
cardosofelipe commented 2025-12-29 03:51:01 +00:00
Owner
Copy Link

Objective

Design bidirectional sync architecture for external issue trackers (Gitea, GitHub, GitLab).

Key Questions

  1. How do we handle sync conflicts?
  2. What's the sync frequency/trigger strategy? (webhook, polling, on-demand)
  3. How do we map fields between systems?
  4. How do we handle external issue creation (create locally, create remote, or both)?
  5. How do we store remote_url and compute it when needed?

Sync Strategy

  • External tracker is source of truth
  • Local mirror for unified navigation and agent access
  • Changes in Syndarix sync back via MCP

Research Areas

  • Webhook handling for real-time sync
  • Conflict resolution strategies
  • Field mapping between trackers
  • Partial sync (only certain labels/milestones)

Expected Deliverables

  • Sync architecture design
  • Field mapping specification
  • Conflict resolution rules
  • remote_url computation logic
  • ADR documenting the approach

Acceptance Criteria

  • Can sync issues from Gitea
  • Local changes sync back to remote
  • Conflicts are detected and handled
  • remote_url navigates to external tracker

Labels

spike, architecture, integrations

## Objective Design bidirectional sync architecture for external issue trackers (Gitea, GitHub, GitLab). ## Key Questions 1. How do we handle sync conflicts? 2. What's the sync frequency/trigger strategy? (webhook, polling, on-demand) 3. How do we map fields between systems? 4. How do we handle external issue creation (create locally, create remote, or both)? 5. How do we store remote_url and compute it when needed? ## Sync Strategy - External tracker is source of truth - Local mirror for unified navigation and agent access - Changes in Syndarix sync back via MCP ## Research Areas - [ ] Webhook handling for real-time sync - [ ] Conflict resolution strategies - [ ] Field mapping between trackers - [ ] Partial sync (only certain labels/milestones) ## Expected Deliverables - Sync architecture design - Field mapping specification - Conflict resolution rules - remote_url computation logic - ADR documenting the approach ## Acceptance Criteria - [ ] Can sync issues from Gitea - [ ] Local changes sync back to remote - [ ] Conflicts are detected and handled - [ ] remote_url navigates to external tracker ## Labels `spike`, `architecture`, `integrations`
cardosofelipe commented 2025-12-29 12:22:38 +00:00
Author
Owner
Copy Link

SPIKE-009 Research Complete

The comprehensive spike document has been created at docs/spikes/SPIKE-009-issue-synchronization.md.

Executive Summary

After researching bi-directional sync patterns, conflict resolution strategies, and analyzing Gitea/GitHub/GitLab API capabilities, the recommended architecture is:

Webhook-first, polling-fallback with Last-Writer-Wins (LWW) conflict resolution using version vectors. External trackers serve as the source of truth with Syndarix maintaining local mirrors for unified agent access.

Key Architectural Decisions

  1. Sync Strategy: Webhooks for real-time sync (< 1s latency), polling every 15-30 min for reconciliation and fallback
  2. Conflict Resolution: Version vectors for causality tracking with tiered resolution:
    • Same field, different times → LWW
    • Concurrent edits → Flag for manual review
    • Different fields → Merge both changes
  3. Provider Abstraction: Unified IssueProvider interface with adapters for Gitea, GitHub, GitLab
  4. Reliability: Outbox pattern for outbound sync with exponential backoff retry
  5. Infrastructure: Redis Streams for webhook event queuing/deduplication, Celery workers for processing

Database Schema

New tables designed:

  • external_connections - Provider configuration and credentials
  • issue_sync_log - Audit trail of all sync operations
  • sync_outbox - Pending outbound syncs with retry tracking

Field Mapping

Canonical model abstracts provider differences:

  • Gitea: number → GitHub: number → GitLab: iid → Syndarix: external_number
  • Gitea: body → GitHub: body → GitLab: description → Syndarix: description
  • State normalization: opened/open → IssueStatus.OPEN

Implementation Roadmap

  • Phase 1-2 (Weeks 1-3): Foundation + Gitea inbound sync
  • Phase 3 (Week 3-4): Outbound sync with outbox pattern
  • Phase 4 (Week 4-5): GitHub & GitLab providers
  • Phase 5-6 (Weeks 5-7): Conflict resolution + production readiness

Research Sources

Next Steps

  1. Review spike document for any gaps
  2. Create ADR-009 to formalize the architecture decision
  3. Begin Phase 1 implementation with database schema

Spike research completed by Claude Code Agent

## SPIKE-009 Research Complete The comprehensive spike document has been created at `docs/spikes/SPIKE-009-issue-synchronization.md`. ### Executive Summary After researching bi-directional sync patterns, conflict resolution strategies, and analyzing Gitea/GitHub/GitLab API capabilities, the recommended architecture is: **Webhook-first, polling-fallback** with **Last-Writer-Wins (LWW)** conflict resolution using version vectors. External trackers serve as the **source of truth** with Syndarix maintaining local mirrors for unified agent access. ### Key Architectural Decisions 1. **Sync Strategy**: Webhooks for real-time sync (< 1s latency), polling every 15-30 min for reconciliation and fallback 2. **Conflict Resolution**: Version vectors for causality tracking with tiered resolution: - Same field, different times → LWW - Concurrent edits → Flag for manual review - Different fields → Merge both changes 3. **Provider Abstraction**: Unified `IssueProvider` interface with adapters for Gitea, GitHub, GitLab 4. **Reliability**: Outbox pattern for outbound sync with exponential backoff retry 5. **Infrastructure**: Redis Streams for webhook event queuing/deduplication, Celery workers for processing ### Database Schema New tables designed: - `external_connections` - Provider configuration and credentials - `issue_sync_log` - Audit trail of all sync operations - `sync_outbox` - Pending outbound syncs with retry tracking ### Field Mapping Canonical model abstracts provider differences: - Gitea: `number` → GitHub: `number` → GitLab: `iid` → Syndarix: `external_number` - Gitea: `body` → GitHub: `body` → GitLab: `description` → Syndarix: `description` - State normalization: `opened`/`open` → `IssueStatus.OPEN` ### Implementation Roadmap - **Phase 1-2** (Weeks 1-3): Foundation + Gitea inbound sync - **Phase 3** (Week 3-4): Outbound sync with outbox pattern - **Phase 4** (Week 4-5): GitHub & GitLab providers - **Phase 5-6** (Weeks 5-7): Conflict resolution + production readiness ### Research Sources - [StackSync - Two-Way Sync Best Practices](https://www.stacksync.com/blog/2025-best-two-way-sync-tools-a-comprehensive-guide-for-data-integration) - [MuleSoft - Bi-Directional Sync Patterns](https://blogs.mulesoft.com/api-integration/patterns/data-integration-patterns-bi-directional-sync/) - [Medium - Conflict Resolution in Distributed Systems](https://medium.com/@priyasrivastava18official/system-design-pattern-from-chaos-to-consistency-the-art-of-conflict-resolution-in-distributed-9d631028bdb4) ### Next Steps 1. Review spike document for any gaps 2. Create ADR-009 to formalize the architecture decision 3. Begin Phase 1 implementation with database schema --- *Spike research completed by Claude Code Agent*
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
Agent orchestration
 architecture
Architectural decisions
 backend
Backend work
 blocked
Blocked by dependency
 database
Database/migrations
 design
UI/UX design work
 devops
DevOps/Infrastructure work
 epic
Epic/parent issue
 frontend
Frontend work
 infrastructure
Infrastructure / DevOps
 mcp
MCP related
 phase-0
Phase 0: Foundation
 phase-1
Phase 1: Core Platform
 phase-2
Phase 2: MCP Integration
 priority:high
High priority
 spike
Architecture spike / research task
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
cardosofelipe
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: cardosofelipe/syndarix#9
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?