# WP Connection Dead — Operations Runbook

**Covers:** WordPress site auth_failed or unreachable status set by `wp:check-connections`  
**Related code:** `app/Console/Commands/CheckWpConnectionsCommand.php`, `app/Notifications/WpConnectionAuthFailedNotification.php`, `app/Services/WpPublishService.php`

---

## Trigger

- User receives a `WpConnectionAuthFailedNotification` email/in-app notification.
- Site dashboard shows a publish error: "WordPress authentication failed" or "WordPress site is currently unreachable".
- `wp_connections.status` is `auth_failed` or `unreachable`.

---

## Symptoms

| Symptom | Likely cause |
|---------|-------------|
| `auth_failed` status, user gets 401 error | WordPress security keys (AUTH_KEY / SECURE_AUTH_KEY) rotated, or a security plugin changed them. Stored shared secret is encrypted with old keys and is now undecryptable. |
| `auth_failed` status, user gets 403 error | Site was disconnected / deleted on the Laravel side; HMAC is valid but site is unknown. |
| `unreachable` status | WordPress host is down, firewall blocking inbound HTTPS from the RankWizAI server, domain DNS failure, or expired SSL certificate. |
| `wp:check-connections` not running | Laravel queue workers are down, or scheduler is not configured. Check `php artisan schedule:list`. |

---

## Diagnosis

```bash
# Check status of all WP connections
php artisan tinker --execute="App\Models\WpConnection::select('id','site_id','status','health_check_consecutive_failures','last_health_check_at')->get()->toArray();"

# Run a dry-run health sweep to see current probe results without mutating DB
php artisan wp:check-connections --dry-run

# Look up the affected connection
php artisan tinker --execute="App\Models\WpConnection::with('site.user')->where('status','auth_failed')->orWhere('status','unreachable')->get(['id','site_id','status','wp_url','last_health_check_at','health_check_notified_at'])->toArray();"

# Check recent logs for probe failures
# grep storage/logs/laravel.log for 'wp.check_connections.connection_degraded'
```

**WordPress plugin side:**
1. Log into the affected WordPress admin → RankWizAI settings page.
2. Check the "Debug Information" panel (shown in debug builds) for decrypt failures or handshake log rejects.
3. Look for an admin notice about security key rotation.
4. Try "Test Connection" button; note the error code returned.

---

## Resolution

### auth_failed (HMAC 401 — shared secret mismatch)

The HMAC secret stored in WordPress can no longer be verified against the one stored in Laravel. This happens when WordPress security keys rotate (AUTO_UPDATE or security plugin), or after a wp-config.php edit.

1. **User action (preferred):** Instruct the user to disconnect and reconnect:
   - In WordPress admin → RankWizAI → "Disconnect"
   - In the RankWizAI dashboard → site settings → "Connect WordPress Plugin" → generate a new setup token
   - Paste the new token in WordPress admin → RankWizAI → "Finish setup"
   - Return to RankWizAI dashboard → "Verify connection"

2. **After reconnect:** The `wp_connections.status` will flip back to `connected` on the next `/status` probe, or immediately if the user triggers a publish.

3. **Reset health counters (optional, operator-only):**
   ```bash
   php artisan tinker --execute="
   App\Models\WpConnection::where('id', <id>)
     ->update(['health_check_consecutive_failures' => 0, 'last_health_check_at' => null, 'health_check_notified_at' => null]);
   "
   ```

### unreachable (network / 5xx failures)

1. Verify the WordPress site is up: `curl -I https://<wp_url>/rankwiz/v1/status`
2. If the site is up, the firewall may be blocking requests from the RankWizAI server. Ask the user to whitelist the RankWizAI server IPs in their WAF/firewall.
3. If the site is down, wait for the hosting provider to restore it. The health sweep will auto-recover once 3 consecutive probes succeed (the current code only escalates on 3 consecutive failures, but does not auto-recover on success — it resets the failure counter, but does NOT auto-restore `status='connected'` from `unreachable`).
4. **Manual status reset after the site recovers:**
   ```bash
   php artisan tinker --execute="
   App\Models\WpConnection::where('id', <id>)
     ->update(['status' => 'connected', 'health_check_consecutive_failures' => 0]);
   "
   ```

---

## Escalation

- If the user has rotated their WordPress security keys repeatedly, suggest disabling the security plugin's key-rotation feature and using a static `AUTH_KEY` set in `wp-config.php`.
- If the health sweep is not running at all (no `last_health_check_at` updates), check that at least one queue worker is running and the scheduler is configured (`* * * * * php artisan schedule:run`).
- For cluster deployments, verify `onOneServer()` is functioning (requires `cache.default` != `array`).