Data Engineer at Insight Global in Hybrid - Hyderabad

Job Description

Design and implement scalable data ingestion, transformation, and reconciliation pipelines for large-scale billing and document-processing workloads. Develop efficient data models, indexes, and query strategies to support high-volume retrieval and reporting requirements. Design retention, archival, and retrieval strategies that balance performance, compliance, and AWS storage costs. Process and transform large billing files, including sorting, filtering, grouping, consolidation, and complex business-rule application. Develop data mapping and transformation logic supporting PDF, AFP, API, and reporting requirements. Design data structures to support complex output generation, including grouping, sorting, and print-distribution requirements (e.g., zip code, print vendor, billing cycle). Build operational dashboards and executive reporting providing visibility into billing-cycle progress, completion status, throughput, exception volumes, and overall platform health. Support reconciliation, exception management, audit reporting, and historical bill retrieval requirements.