# OARSI – move_aa_to_ym Development Requirements

## 1. Purpose & Scope
- Capture the tasks required to migrate OARSI from the legacy `aa` AMS integration to `ym`.
- Track shared module work (`public_html/modules/custom/move_aa_to_ym`) and OARSI-specific site automation/scripts.
- Document assumptions, blockers, and next steps so this file can evolve with future requirements.

## 2. High-Level Requirements (shared across OARSI & ASHT)
1. **Module foundation**: Build `move_aa_to_ym` (custom module) providing batch processes + Drush commands to orchestrate the move.
2. **Service reuse**: Leverage `httpServices` and `memberServices` already declared in `modules/custom/ams/ams.services.yml` for outbound `aa`/`ym` calls.
3. **Field mapping**:
   - Add `field_aa_id` anywhere entities currently sync with the remote AMS.
   - Populate `field_ams_id` with the new `ym` profile ID; leave blank when no `ym` match exists.
   - Persist the legacy `aa` ID into the `ym` profile’s `ConstituentID` field via `PeopleProfileFindID` API and follow-up update calls.
4. **YM lookup flow**:
   - Use `https://ws.yourmembership.com/json/metadata?op=PeopleProfileFindID` with ImportID / ConstituentID / ProfileID / Username / Email as needed.
   - For records not migrated, keep `field_ams_id` empty, but still populate `field_aa_id` locally for traceability.
5. **Non-migrated user handling**:
   - Build a view/report of users lacking `field_ams_id`.
   - Provide tooling to delete or block those users; any destructive path must emit a report (CSV/log) of affected accounts.
6. **Automation scripts**: Per-site bash scripts should call Drush/batch flows to execute migrations in production, once validated in dev/stage.
7. **Environments**: Use staging to validate the full process before production rollout.

## 3. OARSI-Specific Notes
- Start with OARSI because its requirements match the shared workflow closely; use it to validate the reusable module.
- Confirm which entity bundles within `sites/oarsi` consume AMS data (users, memberships, custom nodes) and ensure `field_aa_id` is present there.
- When updating `field_ams_id`, update the user entity and the directory profile (`profile` entity, type `main`) in the same transaction so both references stay aligned to the same YM profile.
- OARSI profiles currently lack `field_ams_id`; once YM IDs are known, add the field and backfill it by copying the user’s `field_ams_id`, mimicking the logic in `sites/sfb/modules/core_settings/src/Form/copyUserAmsIdToProfile.php`.
- Define the local script location (e.g., `sites/oarsi/scripts/move_aa_to_ym.sh`) and required environment variables/API keys.
- Capture any OARSI-only business rules (membership tiers, role assignments) once discovered.

## 3a. Bundles Currently Using AMS IDs (OARSI)
| Entity | Bundle / Type | Notes |
| --- | --- | --- |
| `user` | (core user) | Holds existing `field_ams_id`; will gain `field_aa_id`. |
| `profile` | `main` (from `sites/oarsi/modules/directory`) | Needs `field_ams_id` added + backfilled after YM IDs exist; tied 1:1 to users for directory display. |
- Use `/admin/config/ams/field-mapping` to confirm/adjust these relationships or extend the list if new bundles surface.

## 4. Reporting & Audit
- The user view listing empty `field_ams_id` should live in this site split so content admins can review without code deploys.
- When deleting/blocking unmigrated users, write a summary file (timestamped) that includes UID, email, and the `field_aa_id` that failed migration.

## 5. API Reference Snapshot & Migration Flow Clarification

### API Endpoint
```
GET /Ams/{ClientID}/PeopleProfileFindID
Query params: ImportID, ConstituentID, ProfileID, Username, Email, PageNumber, PageSize
Response: { "ID": ["String"], "ProfileIDs": [0], "ImportID": "String", "ConstituentID": "String", ... }
```

### How the Migration Works
**IMPORTANT**: Users have already been migrated from AA to YM in the external AMS system. This migration updates Drupal to reflect that external migration.

**Current State in Drupal**: 
- `field_ams_id` contains the old AA ID (e.g., "12345")

**Current State in YM**: 
- The user profile was migrated from AA to YM
- YM assigned a new ProfileID (e.g., "67890")
- YM stored the old AA ID in the `ConstituentID` field for reference

**Migration Process**:
1. Read the AA ID from Drupal's `field_ams_id` (e.g., "12345")
2. Call YM API: `PeopleProfileFindID` with parameter `ConstituentID=12345`
3. YM returns the new `ProfileID` from the `ProfileIDs` array (e.g., "67890")
4. Store the old AA ID → `field_aa_id` (for historical tracking)
5. Update `field_ams_id` with the new YM ProfileID (e.g., "67890")

**No data is written to YM** - we are only reading from YM to get the new ProfileID that corresponds to each old AA ID.

- Use pagination inputs when bulk-checking users.

## 6. Progress Tracker
| Area | Task | Owner | Status | Notes |
| --- | --- | --- | --- | --- |
| Discovery | Confirm entity bundles needing `field_aa_id` + confirm data sources |  | ☐ Not started | Known bundles already listed; if uncertainty remains, run `drush @oarsi sql:query "SHOW TABLES LIKE '%field_ams_id%';"` to discover additional references |
| Module | Scaffold `move_aa_to_ym` module structure + info.yml |  | ☐ Not started | Shared module under `modules/custom` |
| Module | Implement YM lookup batch + Drush commands |  | ☐ Not started | Reuse `httpServices`/`memberServices` |
| Data | Backfill `field_aa_id` + `field_ams_id` for OARSI users |  | ☐ Not started | Requires API credentials |
| Reporting | Build view of users missing `field_ams_id` |  | ☐ Not started | Access for site admins |
| Cleanup | Implement delete/block workflow + CSV log |  | ☐ Not started | Include dry-run mode |
| Automation | Author OARSI deployment script hooking module commands |  | ☐ Not started | Script to run in prod/stage |
| Validation | Smoke-test full flow in staging |  | ☐ Not started | Required before prod cutover |

## 7. Open Questions
- What percent of OARSI users are expected to stay in `aa`, and do we retain any of them locally?
- Are there downstream systems reading `field_ams_id` that need coordination during the gap between unset → populated?
- Should delete vs. block be configurable per run (CLI flags) or per-user in the view UI?

## 8. Component Migration Checklist
| Component | Path | Work Needed | Owner | Status | Notes |
| --- | --- | --- | --- | --- | --- |
| Shared batch/Drush tooling | `public_html/modules/custom/move_aa_to_ym` | Scaffold module, wire services, expose commands for ID backfill + cleanup |  | ☐ Not started | Foundation for both sites |
| AMS services reuse | `public_html/modules/custom/ams` | Confirm `httpServices`/`memberServices` configs cover YM endpoints; extend if missing |  | ☐ Not started | May only need config updates |
| AMS iframe controllers | `public_html/modules/custom/ams/ams_iframes/src/Controller` | Replace `aa` references with `ym` equivalents; verify render data matches new IDs |  | ☐ Not started | Impacts embedded member views |
| Directory sync | `public_html/sites/oarsi/modules/directory` | Ensure profile/node data loads new `field_ams_id`, store legacy ID in `field_aa_id` |  | ☐ Not started | Regression-test directory pages |
| Site automation script | `public_html/sites/oarsi/scripts/move_aa_to_ym.sh` (TBD) | Script Drush calls + logging for prod/stage execution |  | ☐ Not started | Define env var contract |

## 9. Implementation Roadmap (hand-off ready)
1. **Scaffold shared tooling**
   - Create `modules/custom/move_aa_to_ym/move_aa_to_ym.info.yml` + Drush services.
   - Expose a Drush command (e.g., `drush move-aa-ym:import-from-csv --csv-path=/path/to/export.csv`) that accepts filters (`--uid`, `--batch-size`, `--dry-run`).
   - Keep the shared module responsible for all reusable logic; inject site-specific handlers via config/service tags so each site can register the bundles it owns.
2. **Fetch YM IDs**
   - Command calls `memberServices` → `PeopleProfileFindID`.
   - Lookup priority: ImportID (legacy `field_ams_id`), then ConstituentID (legacy AA ID), then Username/Email.
   - Handle pagination via `PageNumber/PageSize`.
3. **Update entities atomically**
   - When a YM profile ID is found, wrap updates in a transaction:
     - Update user entity (`field_aa_id` = old AA ID, `field_ams_id` = new YM ID).
     - Load linked `profile` entity (type `main`) and update both fields with identical values.
     - Provide hooks so additional bundles can be added later.
   - If no YM match, write `field_aa_id`, leave `field_ams_id` empty, flag the account.
   - After YM IDs exist, run a follow-up batch (modeled after `copyUserAmsIdToProfile.php`) to populate the newly added `field_ams_id` on profiles that were missing it prior to migration.
4. **Persist YM ConstituentID**
   - After a successful match, push the legacy AA ID to YM via the appropriate API call so YM’s `ConstituentID` mirrors `field_aa_id`.
5. **Logging & reporting**
   - Emit structured logs/CSV storing UID, profile ID, AA ID, YM ID, timestamp, operator.
   - Produce a secondary report for “unmatched users” to seed the Drupal view.
6. **Error handling & retries**
   - Surface API errors (rate limiting, invalid credentials) with actionable messages.
   - Support `--resume` / checkpointing so long-running batches can recover.
7. **Testing expectations (to be executed outside this chat)**
   - Unit-test service wrappers where possible.
   - Run Drush command against staging subset, verify both entities receive the same YM ID.
   - Manually confirm the directory pages render the updated IDs.

## 10. Field Mapping Follow-up
- Manual field mapping updates live in `public_html/modules/custom/ams/ams_field_mappings`.
- After YM IDs are obtained/backfilled, export site-specific configs (e.g., `public_html/sites/oarsi/modules/directory/config/install/ams.profile-main-field_mappings.yml`) so new fields are synced properly.
- The admin UI for these mappings is `/admin/config/ams/field-mapping` (see `modules/custom/ams/ams_field_mappings/src/Form/amsFieldMappingForm.php`); use it or static configs to register new entity/bundle relationships for `field_aa_id`/`field_ams_id`.

## 11. Operational Notes
- Configuration changes (field mappings, views, etc.) are managed via the Features module; use `drush @oarsi features:export` / `features:import` as needed after code updates.
- Always target the multisite alias when running commands: `drush @oarsi <command>`, `drush @asht <command>`, etc., to avoid cross-site contamination.
- When configuration files are missing or unclear, inspect the database directly with Drush SQL commands (e.g., `drush @oarsi sql:query "SELECT entity_type, bundle FROM field_config WHERE field_name = 'field_ams_id';"`) to confirm every bundle storing `field_ams_id`.