HL7 Pipeline¶
Technology Stack: HAPI FHIR + HAPI HL7v2
- HL7v2 Parsing: HAPI HL7v2 library for ADT, ORU, ORM, SIU, RDE, VXU message types
- FHIR Conversion: HAPI FHIR for R4 bundle generation and ABDM profile validation
- License: Apache 2.0 (fully open-source)
UC-ING-001: Receive HL7 Message¶
Purpose: Accept raw HL7 v2 messages via MLLP (TCP) and acknowledge receipt.
| Property | Value |
|---|---|
| Actor | MLLP Listener Service |
| Trigger | Incoming TCP connection on port 2575 |
| Preconditions | Port 2575 open, Firewall rules allow EMR IP |
| Priority | P0 |
Main Success Scenario:
1. Listener accepts TCP connection from EMR IP (Port 2575)
2. Listener reads stream until Message Terminator
Bytes: \x0B (Start Block) ... data ... \x1C\x0D (End Block)
3. Listener generates HL7 ACK message
MSA-1: AA (Application Accept)
MSA-2: <Control ID from incoming MSH-10>
4. Listener writes ACK bytes to socket immediately (sync)
5. Listener closes connection (or keeps alive if persistent)
6. Listener passes raw message payload to Validation Queue
Observability:
- Metric: hl7_connections_total, hl7_bytes_received
- Log: {"event": "hl7_received", "remote_ip": "10.0.0.5", "size_bytes": 450}
Acceptance Criteria: 1. [ ] ACK sent within 200ms 2. [ ] Handles concurrent connections (up to 50) 3. [ ] Correctly identifies Start/End block markers
UC-ING-002: Validate HL7 Structure¶
Purpose: Ensure message has required segments before processing.
| Property | Value |
|---|---|
| Actor | Validation Worker |
| Trigger | New message in Validation Queue |
| Priority | P0 |
Main Success Scenario:
1. Worker parses raw string into HL7 Segment objects
2. Worker checks for MSH (Header) segment
- Verify MSH-9 (Message Type) is ORU^R01 (Results) or ADT^A01 (Admit)
- Verify MSH-12 (Version) is 2.3 or higher
3. Worker checks for PID (Patient ID) segment (Mandatory)
4. Worker checks for OBX (Observation) segments (if ORU)
5. If valid, enqueue to Normalization Queue
Alternative Flows:
Alt-1: Malformed Message
- Missing MSH or PID segment - Log error: `{"error": "missing_segment", "segment": "PID"}` - Move to Dead Letter Queue (DLQ) - Do NOT send NACK (connection already closed in UC-ING-001)Acceptance Criteria: 1. [ ] Rejects messages without PID 2. [ ] Logs validation failures with full message dump
UC-ING-003: Normalize Patient Identity¶
Purpose: Map Hospital MRN to System ABHA ID.
| Property | Value |
|---|---|
| Actor | Identity Worker |
| Trigger | Message in Normalization Queue |
| Priority | P0 |
Main Success Scenario:
1. Extract identifiers from PID segment:
- **MRN/UHID**: PID-3 (Patient Identifier List)
- **Mobile**: PID-13 (Phone Number - Home) -> Normalize to +91 format
- **ABHA ID**: PID-3 (if tagged with authority 'ABHA' or 'NDHM')
- **Aadhaar**: PID-3 (if present, strictly for matching, do not persist raw)
2. Query Patient Identity Service (MPI) with extracted IDs
3. **Match Logic**:
- Exact Match on ABHA ID -> High Confidence
- Exact Match on MRN + Hospital Code -> High Confidence
- Match on Mobile + Name (Fuzzy) -> Medium Confidence (Flag for review)
4. If found, attach internal `patient_id` to message context
5. If not found, create new provisional patient profile
6. Enqueue to Parsing Queue
Alternative Flows:
Alt-1: Patient Not Found
- Configured to "Reject Unknown": - Log warning: `{"event": "unknown_patient", "mrn": "12345"}` - Move to DLQ - Alert Data ManagerAcceptance Criteria: 1. [ ] Correctly parses PID-3 complex types 2. [ ] Caches mapping lookups for performance (<10ms)
UC-ING-004: Parse Lab Results¶
Purpose: Extract clinical data from ORU message into structured format.
| Property | Value |
|---|---|
| Actor | Parsing Worker |
| Trigger | Message in Parsing Queue |
| Priority | P0 |
Main Success Scenario:
1. Iterate through OBX segments
2. For each OBX:
- Extract Test Code (OBX-3): "718-7^Hemoglobin^LN" OR "HGB^Hemoglobin^L" (Local)
- Extract Value (OBX-5): "13.5"
- Extract Units (OBX-6): "g/dL"
- Extract Ref Range (OBX-7): "12-16"
3. **Code Normalization**:
- If LOINC provided, use directly.
- If Local Code, query `LabCodeMap` to find LOINC equivalent.
- If unknown, map to `UncodedObservation` type.
4. Construct `LabResult` JSON objects
5. Aggregate all results for this message
6. Enqueue to Persistence Queue
Acceptance Criteria: 1. [ ] Handles numeric and string values in OBX-5 2. [ ] Captures abnormal flags (OBX-8) 3. [ ] Preserves observation timestamp (OBX-14)
UC-ING-005: Update Patient Bundle (Labs)¶
Purpose: Persist data to the Canonical Patient Bundle.
| Property | Value |
|---|---|
| Actor | Persistence Worker |
| Trigger | Data in Persistence Queue |
| Priority | P0 |
Main Success Scenario:
1. Acquire lock on `bundle.json` for patient
2. Read existing bundle from S3/Disk
3. Append new `LabResult` objects to `labs` section of the Longitudinal Record
4. Deduplicate based on Test ID + Timestamp (Idempotency)
5. Write updated bundle atomically
6. Release lock
7. Emit event: `patient.updated` (Triggers Analytics/Alerts)
Observability:
- Metric: bundle_write_latency_ms
- Log: {"event": "bundle_updated", "patientId": "...", "added_labs": 5}
Acceptance Criteria: 1. [ ] No data corruption on concurrent writes 2. [ ] Atomic file replacement (write temp -> rename) 3. [ ] History preserved (append only, don't overwrite existing history)
UC-ING-009: Detect Duplicate HL7 Messages¶
Purpose: Prevent reprocessing of the same HL7 message by inspecting MSH control IDs.
| Property | Value |
|---|---|
| Actor | Deduplication Worker |
| Trigger | Message exits Validation Queue |
| Priority | P0 |
Main Success Scenario:
1. Worker reads `MSH-10` (Message Control ID)
2. Build cache key `hl7:dedup:{sendingApp}:{controlId}`
3. Check Redis for existing key with TTL 24h
4. If key not found, set it with NX flag and TTL
5. Publish `"hl7_dedup_pass"` metric
6. Forward message to Normalization Queue
7. If key exists, increment `hl7_duplicates_total` and drop payload
Alternative Flows:
Alt-1: Cache Unavailable
- Worker cannot reach Redis cluster - Fallback to local Bloom filter (1h retention) - Emit warning log: `{"event":"dedup_degraded"}`Acceptance Criteria: 1. [ ] Duplicate detection window configurable (1h-48h) 2. [ ] False positive rate < 0.1% 3. [ ] Metrics emitted for both kept and dropped messages
UC-ING-010: Persist Raw HL7 Payloads¶
Purpose: Store compressed HL7 payloads for audit and replay.
| Property | Value |
|---|---|
| Actor | Archival Worker |
| Trigger | Message accepted for processing |
| Priority | P1 |
Main Success Scenario:
1. Worker receives validated HL7 string and metadata
2. Compress payload using zstd (level 6)
3. Generate key `raw/hl7/{eventDate}/{controlId}.hl7.zst`
4. Upload to S3 with server-side encryption (SSE-S3)
5. Persist pointer in `hl7_archive` table (patientId, controlId, s3Key)
6. Emit event `hl7.raw_archived`
Acceptance Criteria: 1. [ ] Archive write latency < 500ms p95 2. [ ] Supports replay by control ID within 30 days 3. [ ] Storage class configurable (STANDARD vs GLACIER)
UC-ING-011: Extract Visit Context¶
Purpose: Derive encounter metadata from PV1/PV2 segments for downstream routing.
| Property | Value |
|---|---|
| Actor | Context Enrichment Worker |
| Trigger | Post-normalization HL7 message |
| Priority | P1 |
Main Success Scenario:
1. Parse PV1 segment into structured object
2. Capture attending provider (PV1-7) and location (PV1-3)
3. Determine encounter class (inpatient/outpatient) from PV1-2
4. Build `visitContext` JSON block with admission/discharge timestamps
5. Attach context to message envelope
6. Route to downstream queues (Labs, Imaging) based on visit type
Acceptance Criteria:
1. [ ] Handles messages without PV2 gracefully (fills defaults)
2. [ ] Normalizes provider IDs using provider_mapping table
3. [ ] Logs when encounter transitions (status change)
HL7 Additional Message Types¶
UC-ING-015: Receive HL7 Radiology Order (ORM)¶
Purpose: Ingest radiology orders from RIS/EMR.
| Property | Value |
|---|---|
| Actor | HL7 ORM Handler |
| Trigger | ORM^O01 message received |
| Priority | P0 |
Main Success Scenario:
1. Validate ORM structure (MSH, PID, ORC, OBR)
2. Extract order details: Exam Code (OBR-4), Priority (ORC-7)
3. Normalize patient identity (reuse UC-ING-003)
4. Create RadiologyOrder entry in system
5. Emit `radiology_order_received` event
Acceptance Criteria: 1. [ ] Supports STAT vs Routine ordering 2. [ ] Links to future imaging study via Accession Number 3. [ ] Order status tracking enabled
UC-ING-016: Receive HL7 Radiology Result (ORU)¶
Purpose: Ingest radiology reports.
| Property | Value |
|---|---|
| Actor | HL7 ORU Handler |
| Trigger | ORU^R01 message with OBX containing report |
| Priority | P0 |
Main Success Scenario:
1. Parse ORU segments (PID, OBR, OBX)
2. Extract report text from OBX-5 (TX datatype)
3. Extract Accession Number (OBR-18)
4. Trigger UC-IMG-014 (Report Linking)
5. Persist raw report if no imaging study found yet
Acceptance Criteria: 1. [ ] Handles multi-OBX reports (pagination) 2. [ ] Preserves formatting (newlines, tables) 3. [ ] Links to radiology order if exists
UC-ING-017: Receive HL7 Scheduling (SIU)¶
Purpose: Sync appointment/visit schedules.
| Property | Value |
|---|---|
| Actor | HL7 SIU Handler |
| Trigger | SIU^S12 (Appointment Notification) |
| Priority | P1 |
Main Success Scenario:
1. Parse SIU segments (SCH, PID, RGS)
2. Extract appointment time, location, provider
3. Normalize patient identity
4. Upsert appointment in scheduling index
5. Link to future encounters
Acceptance Criteria: 1. [ ] Handles appointment cancellations (SIU^S15) 2. [ ] Supports multi-provider appointments 3. [ ] Appointment reminders triggered
UC-ING-018: Receive HL7 Medication Orders (RDE)¶
Purpose: Ingest pharmacy orders.
| Property | Value |
|---|---|
| Actor | HL7 RDE Handler |
| Trigger | RDE^O11 message |
| Priority | P1 |
Main Success Scenario:
1. Parse RXO/RXE segments (medication, dose, route)
2. Normalize drug to RxNorm
3. Extract prescriber info (RXO-14)
4. Add to patient medication list
5. Emit `medication_ordered` event
Acceptance Criteria: 1. [ ] Detects duplicate orders 2. [ ] Flags drug-drug interactions 3. [ ] Supports PRN and scheduled medications
UC-ING-019: Receive HL7 Immunization (VXU)¶
Purpose: Ingest vaccination records.
| Property | Value |
|---|---|
| Actor | HL7 VXU Handler |
| Trigger | VXU^V04 message |
| Priority | P2 |
Main Success Scenario:
1. Parse RXA segments (vaccine, dose, site)
2. Extract CVX code for vaccine type
3. Add to patient immunization history
4. Check for contraindications based on history
Acceptance Criteria: 1. [ ] Supports batch import of historical vaccines 2. [ ] Validates CVX codes against registry 3. [ ] Tracks vaccination schedule compliance
UC-ING-020: Handle HL7 Error ACKs (AE/AR)¶
Purpose: Process error acknowledgments and retry logic.
| Property | Value |
|---|---|
| Actor | HL7 Error Handler |
| Trigger | Receiving AE (Application Error) or AR (Application Reject) ACK |
| Priority | P1 |
Main Success Scenario:
1. Parse MSA segment (MSA-1 = AE/AR, MSA-3 = Error Code)
2. Log error with original message control ID
3. If retriable error (network timeout), schedule retry
4. If permanent error (invalid patient), move to DLQ
5. Alert integration team for persistent failures
Acceptance Criteria: 1. [ ] Retries up to 3 times for transient errors 2. [ ] Tracks error patterns for monitoring 3. [ ] Escalates persistent errors to ops team