# Cross-App Data Sync & Deduplication

Design reliable one-way or two-way synchronization between business apps with field ownership, conflict rules, deduplication, replay safety, and reconciliation.

## Prompt

You are a senior workflow automation designer. Your task is to design a cross-application data synchronization workflow that avoids loops, conflicting updates, duplicate records, and silent data loss.

Design the workflow so it can be implemented in Zapier, Make, n8n, a native integration, or custom code. Do not pretend that you have connected to, configured, tested, or deployed any system. Do not invent field names, API capabilities, permissions, prices, limits, credentials, business rules, or approval authority.

USER INPUT
Source and destination systems: {{systems}}
Objects to synchronize: {{objects}}
System of record for each object and field: {{systems_of_record}}
Field schemas and identifiers: {{schemas_identifiers}}
Sync direction and frequency: {{sync_direction_frequency}}
Create, update, archive, and deletion rules: {{lifecycle_rules}}
Conflict-resolution rules: {{conflict_rules}}
Privacy and access rules: {{privacy_rules}}
Volume, API, and timing constraints: {{constraints}}
Reconciliation and recovery requirements: {{reconciliation}}

DESIGN METHOD
1. Define whether the synchronization is one-way, bidirectional, scheduled, event-driven, or hybrid.
2. Identify the system of record at object and field level.
3. Select stable cross-system identifiers and create a mapping table.
4. Define create, update, merge, archive, unlink, and deletion behavior.
5. Add loop prevention using origin markers, versions, timestamps, or event IDs.
6. Define conflict detection and route unresolved conflicts to review.
7. Handle missing records, late events, out-of-order updates, schema changes, pagination, rate limits, and partial batches.
8. Add idempotent retries and a dead-letter queue.
9. Design periodic reconciliation between the systems.
10. Define monitoring for drift, lag, failure rate, duplicates, and unmatched records.

REQUIRED OUTPUT
A. Synchronization scope and architecture
B. System-of-record matrix
C. Object and field mapping
D. Identifier and deduplication strategy
E. Create, update, conflict, and lifecycle rules
F. Loop and replay prevention
G. Error handling and dead-letter queue
H. Reconciliation process
I. Monitoring and alert thresholds
J. Migration and test plan
K. Implementation checklist

CORE RELIABILITY RULES
1. Separate confirmed requirements from assumptions and unresolved questions.
2. Define one precise trigger and the conditions that should block false or duplicate runs.
3. Identify the system of record for every important field.
4. Map data explicitly between source and destination fields.
5. Validate required fields before taking downstream actions.
6. Use a stable unique identifier or idempotency key to prevent duplicate records and repeated messages.
7. Add human approval before external messages, publishing, payments, deletion, permission changes, or other difficult-to-reverse actions.
8. Define retries, timeouts, error logging, alert ownership, and a manual fallback.
9. Minimize movement and storage of personal or sensitive data.
10. Keep secrets in the automation platform's secure credential manager. Never request real API keys, passwords, access tokens, or webhook secrets.
11. Include test cases for the happy path, missing data, duplicates, invalid data, unavailable services, permission failures, and partial completion.
12. End with a clear list of items that require human confirmation before implementation.
13. Do not use last-write-wins unless the user explicitly accepts its data-loss risk.
14. Do not propagate deletion across systems without an approved reversible process and human review.

## Best for

Operations and engineering teams synchronizing contacts, companies, orders, tickets, projects, or other records between business applications.

## Compatible tools

- Claude
- ChatGPT

## How to use

- List the systems, objects, fields, identifiers, sync direction, and frequency.
- Define field ownership, conflict, archive, and deletion rules.
- Review API constraints, permissions, and sensitive-data movement.
- Test replays, loops, conflicts, late events, partial failures, and reconciliation.

## Customization tips

- Prefer stable vendor IDs over names or email addresses when available.
- Use soft deletion or inactive states before considering destructive propagation.
- Document every field transformation and accepted null behavior.
- Add a dashboard for lag, drift, failures, duplicates, and unmatched records.

## Example input

Synchronize HubSpot companies to Airtable one way. HubSpot is the system of record. Use HubSpot company ID as the external key. Create missing Airtable rows and update approved fields. Never delete rows automatically; mark them inactive when HubSpot is archived. Run every 15 minutes and perform nightly reconciliation.

## Example output

Architecture:
Scheduled one-way sync from HubSpot to Airtable every 15 minutes.
Nightly full reconciliation verifies completeness.

Identity:
HubSpot company ID is the external key stored in Airtable.
Never match by company name alone.

Lifecycle:
New HubSpot company → create Airtable row.
Updated approved HubSpot field → update mapped Airtable field.
Archived HubSpot company → set Airtable status to Inactive.
No automatic deletion.

Replay safety:
Use HubSpot company ID plus source update version as the idempotency key.
Record the last successful source update processed.

Failure handling:
Retry temporary failures.
Move permanently failed records to a review queue.
Alert Operations when sync lag or unmatched-record count exceeds the approved threshold.