Admin Operations Runbook

This runbook gives tenant administrators a practical operating rhythm for managing users, roles, departments, approvals, overdue work, and monthly platform health.

1. Purpose

Tenant administrators keep the platform usable, accurate, and governed. Their work is not to own every risk or control, but to ensure the right people, permissions, workflows, and data quality practices are in place.

Use this runbook for routine administration:

• onboarding and offboarding users
• assigning roles and access
• maintaining departments and ownership
• reviewing pending approvals
• monitoring overdue work
• keeping lookup values clean
• performing monthly health checks

2. Daily Admin Checklist

Task	What to Check	Action
Pending approvals	Policies, assessments, exceptions, or workflow items waiting for approval.	Remind approvers or reassign if ownership changed.
Overdue work	Issues, findings, assessments, risk reviews, policy reviews, vendor reviews.	Escalate to owner and manager when needed.
Critical alerts	Critical risks, expiring exceptions, urgent findings.	Confirm owner action and management visibility.
Failed ownership	Records assigned to inactive or wrong users.	Reassign to the correct owner.
New requests	User access, role change, department change, report access.	Validate request and apply least-privilege access.

3. Weekly Admin Checklist

Task	What to Check	Action
User access review	New users, inactive users, role changes.	Remove unnecessary access and confirm business need.
Assessment progress	Assessments stuck, overdue, or missing evidence.	Follow up with assessment owners and control owners.
Risk review progress	High and critical risks approaching review date.	Ask risk owners for update or treatment decision.
Policy lifecycle	Drafts, pending approvals, acknowledgement campaigns.	Push policies through approval or update audience.
Vendor and asset hygiene	Missing owners, criticality, or review dates.	Correct ownership and required fields.
Scheduled reports	Recipient list and report usefulness.	Remove obsolete recipients and stale schedules.

4. Monthly Admin Checklist

1. Review dashboard and KPI trends.
2. Review data quality exceptions across modules.
3. Confirm high and critical risks have current treatment and review dates.
4. Review overdue findings and issues with accountable owners.
5. Review policy exceptions expiring within 30 days.
6. Review pending policy acknowledgements.
7. Review vendor assessments and high-risk vendors.
8. Review inactive users and excessive privileges.
9. Review scheduled report recipients.
10. Prepare a short management summary with actions, blockers, and escalations.

5. User Onboarding

When adding a user:

1. Confirm the business need for access.
2. Confirm department, manager, and job responsibility.
3. Assign the minimum role needed.
4. Add module-specific access only when required.
5. Confirm whether the user owns controls, risks, policies, findings, assets, or vendors.
6. Notify the user of their expected responsibilities.
7. Add the user to relevant assessment, policy acknowledgement, or workflow assignments.

Recommended checks:

• Do not assign administrator access for normal business ownership.
• Do not give approval permission to the same user who submits the item unless policy allows it.
• Make sure control owners can see and respond to assigned controls.
• Make sure executives have read-only access unless they also perform approvals.

6. User Offboarding

Before deactivating or removing access:

1. Search for records owned by the user.
2. Reassign open risks, controls, issues, findings, policies, vendors, assets, and assessments.
3. Reassign pending approvals.
4. Remove scheduled report delivery if no longer needed.
5. Remove elevated roles first.
6. Deactivate access after ownership is transferred.
7. Record any unresolved ownership gaps for follow-up.

Common mistake: removing a user before reassigning ownership. This creates orphaned work and weakens reporting.

7. Assigning Roles

Use least privilege: assign the smallest role that allows the user to complete their job.

User Type	Recommended Access
Tenant administrator	User, role, department, workflow, setup, and reporting administration.
Compliance manager	Assessments, controls, evidence, findings, compliance reports.
Risk manager	Risk register, appetite, treatments, risk reports, related issues.
Control owner	Assigned controls, evidence, assessment responses, remediation tasks.
Auditor	Audit plans, findings, evidence review, audit reports.
Executive viewer	Dashboards, KPIs, high-level reports, read-only views.
Vendor owner	Assigned vendor records, assessments, evidence, risk follow-up.
Policy owner	Drafting, review, acknowledgement tracking, exception review where appropriate.

8. Departments and Ownership

Departments support filtering, reporting, and accountability.

When maintaining departments:

• keep department names consistent with the organization structure
• avoid duplicate department names with small spelling differences
• assign users to the correct department
• review department ownership when teams reorganize
• update record ownership when departments merge or split

If departments are inaccurate, reports by business area become unreliable.

9. Lookup Values and Configuration Hygiene

Lookup values should be stable and meaningful.

Review:

• risk categories
• risk treatment options
• issue priorities
• finding severities
• asset types
• vendor criticality values
• policy categories
• incident categories
• workflow statuses where configurable

Avoid creating duplicate values such as “High”, “high”, and “High Risk” for the same meaning.

10. Pending Approvals Review

For each pending approval, check:

• who submitted it
• who must approve it
• how long it has been waiting
• whether the approver is active
• whether the item has enough information for approval
• whether the decision is urgent

Escalate approvals that block assessments, policy publication, risk acceptance, or exception expiry.

11. Overdue Work Review

Overdue work should be managed from the source module.

Overdue Item	Primary Follow-Up
Assessment	Assessment owner and control owners.
Control evidence	Control owner.
Risk review	Risk owner.
Treatment action	Treatment action owner.
Policy review	Policy owner.
Policy acknowledgement	Target audience owner or department manager.
Audit finding	Finding owner.
Issue	Assigned issue owner.
Vendor review	Vendor owner.

Escalate repeated overdue items to the accountable manager, not only the assigned user.

12. Monthly Health Summary

A useful monthly admin summary should include:

• compliance score and major movement
• active and overdue assessments
• high and critical risks
• overdue issues and findings
• pending approvals
• expiring exceptions
• policy acknowledgement gaps
• high-risk vendors
• KPI breaches
• data quality concerns
• decisions required from management

Keep the summary action-oriented. Management should be able to see what needs a decision, who owns it, and by when.

13. Common Admin Mistakes

• granting administrator access too broadly
• leaving inactive users with ownership
• treating reports as the source of truth instead of correcting source records
• allowing overdue work without escalation
• scheduling reports without a clear audience
• publishing policies before the audience and acknowledgement requirement are correct
• closing findings or issues without evidence
• accepting risks without documented justification
• creating duplicate departments, categories, or lookup values

14. Escalation Template

Use this structure when escalating:

Field	What to Include
Record	Name or reference of the item.
Owner	Current accountable owner.
Issue	What is overdue, incomplete, blocked, or high risk.
Impact	Compliance, risk, audit, operational, or reporting impact.
Required Decision	What management must decide or approve.
Due Date	When action is required.
Recommended Next Step	Reassign, approve, remediate, accept, extend, or close.

15. Production Deployment

Before deployment:

1. Confirm the release tag, commit SHA, and approved change ticket.
2. Confirm database backup completed successfully.
3. Confirm required secrets and environment variables are present for the target environment.
4. Confirm ASPNETCORE_ENVIRONMENT=Production.
5. Confirm allowed CORS origins match the production frontend domain.
6. Confirm SMTP settings are valid.
7. Confirm DataProtection keys are mounted to persistent storage or an approved key store.
8. Run the build and test commands from the production action plan.

Deployment sequence:

1. Put the deployment in the maintenance window or announce the no-downtime release.
2. Deploy backend and frontend artifacts.
3. Run pending database migrations.
4. Restart backend workers and API instances.
5. Verify /health or the configured health endpoint.
6. Sign in as a platform administrator and tenant administrator.
7. Run the smoke workflow: login, dashboard load, tenant list or tenant dashboard, one list export, one report export.
8. Monitor logs, error reports, and failed background jobs for at least 30 minutes.

16. Rollback

Rollback is required when deployment causes login failure, data corruption risk, unavailable critical workflows, or broken executive reporting.

Rollback sequence:

1. Stop new deployments and capture current logs.
2. Identify whether rollback is application-only or requires database restore.
3. If no destructive migration ran, redeploy the previous backend and frontend artifacts.
4. If a destructive or incompatible migration ran, restore the approved database backup to a replacement database and point the app to it.
5. Restore the previous environment variables if they changed.
6. Restart services and verify health checks.
7. Run smoke checks and confirm with the business owner.
8. Record the incident, root cause, customer impact, and follow-up fix.

17. Database Backup

Minimum backup controls:

• run automated daily full backups
• run point-in-time recovery if supported by the database platform
• encrypt backups at rest
• store backups outside the application host
• restrict restore permissions to operations owners
• test restore at least monthly

Before high-risk operations:

1. Take an on-demand backup.
2. Record backup name, timestamp, database name, and operator.
3. Confirm backup completion and size.
4. Keep the backup until the release is accepted.

18. Database Restore

Restore sequence:

1. Confirm restore approval and target recovery point.
2. Restore to a separate database first.
3. Run schema compatibility checks against the deployed application version.
4. Validate tenant count, admin users, critical records, and recent audit logs.
5. Switch the application connection string only after validation.
6. Restart application services.
7. Run smoke checks and document the restored timestamp.

Never overwrite the production database before validating the backup in an isolated target.

19. Migration Execution

Before migrations:

1. Review migration files and identify destructive operations.
2. Confirm backup is complete.
3. Confirm no long-running imports, exports, or scheduled reports are active.
4. Run migrations once per environment from the approved deployment job.

After migrations:

1. Confirm migration history table includes the expected migration.
2. Confirm backend starts without schema errors.
3. Run login, dashboard, list, report, and export smoke checks.
4. Keep migration output with the release evidence.

20. DataProtection Key Backup

DataProtection keys protect cookies and tokens. Losing them can invalidate sessions and break encrypted payloads.

Required controls:

• store keys in persistent storage or a managed key store
• back up the key location with production backups
• restrict write access to the application identity and operations owners
• do not rotate by deleting old keys
• verify all backend instances share the same key ring

Recovery:

1. Restore the key ring before restarting replacement application instances.
2. Confirm users can sign in without forced global logout unless key loss is accepted.
3. If keys are lost, announce session invalidation and require users to sign in again.

21. Log Retention

Minimum retention:

Log Type	Minimum Retention
Application logs	90 days
Security and audit logs	1 year
Deployment logs	1 year
Database backup and restore logs	1 year
Error reports	180 days

Operational checks:

• confirm logs include environment, tenant, user id where safe, correlation id, and timestamp
• avoid storing passwords, tokens, secrets, or unnecessary personal data
• review critical errors daily during launch and weekly after stabilization
• preserve logs for open incidents until closure

22. Incident Response

Use this process for outages, security events, data integrity issues, failed deployments, and critical reporting failures.

1. Declare severity and incident owner.
2. Preserve logs, deployment version, database state, and screenshots.
3. Stabilize service: rollback, disable feature, scale, or restore as needed.
4. Communicate impact, workaround, and next update time.
5. Track timeline, decisions, and actions.
6. Confirm recovery with smoke checks and business owner validation.
7. Complete root-cause analysis.
8. Add preventive action to the backlog and assign an owner.

23. Secret Rotation

Rotate secrets when scheduled, when an employee with access leaves, after suspected exposure, or before customer launch if test secrets were used.

Rotation sequence:

1. Identify dependent services and rollback plan.
2. Create the new secret in the approved secret store.
3. Update application configuration without exposing the value in logs.
4. Restart affected services.
5. Validate login, email, database, storage, webhooks, and SSO integrations as applicable.
6. Revoke the old secret after validation.
7. Record the rotation date, operator, systems affected, and next rotation due date.

24. Tenant Onboarding

Tenant onboarding sequence:

1. Confirm customer name, tenant code, domain, admin contact, support contact, and go-live date.
2. Create or onboard the tenant from the platform tenant administration screen.
3. Configure first administrator and require password change or invitation acceptance.
4. Enable only purchased or approved modules.
5. Sync tenant permissions and workflows.
6. Configure SMTP sender, branding, CORS/domain settings, and documentation URL if tenant-specific.
7. Seed approved frameworks, roles, and starter data only when accepted.
8. Run tenant smoke checks: admin login, dashboard, user invite, one module list, one report.
9. Record launch approval and accepted risks.

25. Tenant Offboarding

Tenant offboarding sequence:

1. Confirm commercial, legal, and data-retention approval.
2. Disable tenant access or put the tenant in suspended state.
3. Export customer-requested records where contractually required.
4. Preserve audit logs and backups for the agreed retention period.
5. Remove scheduled reports, webhooks, and external integrations.
6. Delete or anonymize data only after legal approval and backup-retention review.
7. Record final status, retained data, deletion date, and approver.

26. Support Impersonation

Support impersonation is high risk and must be exceptional.

Required controls:

• use only approved platform administrator accounts
• require a support ticket, customer approval, reason, tenant, user, and time window
• grant the minimum duration needed
• record every impersonation start and end in audit logs
• never change customer data unless the ticket explicitly authorizes it
• never use impersonation to bypass segregation of duties

Impersonation closure:

1. End impersonation immediately after the support action.
2. Record actions taken and records viewed or changed.
3. Attach evidence to the support ticket.
4. Ask the customer administrator to verify the issue is resolved.

27. Customer Launch Checklist

Complete this checklist before giving a customer production access.

Area	Required Check	Evidence
Domain and TLS	Production domain resolves correctly and TLS certificate is valid.	Domain, certificate expiry date, verification screenshot.
SMTP	Production SMTP sender works and SPF/DKIM/DMARC are aligned where applicable.	Test email, sender domain, delivery result.
CORS	Backend allows only approved production frontend origins.	Environment variable or configuration screenshot.
Backups	Automated backups are enabled and one restore test has passed.	Backup job name, latest backup timestamp, restore test date.
Monitoring	Health checks, application logs, error reports, and alert recipients are configured.	Monitoring dashboard URL and alert recipient list.
Admin accounts	Platform and tenant admin accounts are named users, active, and protected.	Account list and owner approval.
Seed data	Seeded frameworks, roles, users, and starter records are approved for this customer.	Seed data approval or "none" confirmation.
First tenant	Tenant code, display names, modules, workflows, and permissions are correct.	Tenant detail screenshot or checklist note.
Support contacts	Customer admin, technical contact, escalation contact, and support mailbox are recorded.	Contact list.
Docs URL	Customer-facing documentation URL is configured and reachable.	URL and access check.
Legal and privacy notice	Privacy notice, terms, data processing, and retention positions are accepted.	Legal approval reference.
Accepted risks	Any known production limitations are documented and accepted by product, security, and operations owners.	Accepted-risk register or signed launch note.

Launch approval:

1. Product owner confirms user-facing workflow readiness.
2. Security owner confirms access, tenant isolation, secrets, and logging readiness.
3. Operations owner confirms deployment, backup, restore, monitoring, and support readiness.
4. Customer owner confirms first tenant, admin users, contacts, and go-live timing.
5. Release owner records the final go/no-go decision.