Databricks Solution Architect at IBM in Paramus, NJ

Databricks Solution Architect to define and lead the end-to-end architecture for a large data modernization program. You will bring deep expertise across cloud-based engineering, Databricks platform, data migration, data modeling, data architecture, data engineering, and reporting, and connect governed data products to ML, GenAI, and advanced AI solutions across hyperscalers.

Responsibilities

•    Define the target data and AI architecture across the AWS/Databricks data platform and GCP-based ML/GenAI services, and the integration patterns between them.

•    Lead the data migration architecture: re-engineering approach, migration patterns, Integration & Ingestion patterns, sequencing, cutover, reconciliation, and risk management.

•    Own data architecture and data modeling standards, covering conceptual, logical, and physical models, lakehouse/medallion layers, and data product design.

•    Define data engineering patterns and standards for ingestion (Fivetran), transformation, orchestration, CI/CD, observability, and cost control on cloud-based platforms.

•    Architect the reporting and analytics layer, including semantic models, governed metrics, and self-service access on top of data products.

•    Design the ML/GenAI lifecycle: feature engineering, model training and serving, RAG and agentic patterns, MLOps/LLMOps, monitoring, and evaluation.

•    Architect secure, scalable cross-cloud data movement and integration across hyperscalers (AWS, Microsoft Azure, GCP), selecting services based on fit, cost, and performance.

•    Ensure the architecture uses data products from the Databricks lakehouse, including SAP-sourced data via SAP BTP / BDC Connect.

•    Define security, data privacy, responsible AI, and governance guardrails for AI use cases.

•    Provide architectural oversight on related initiatives, such as the Tier-1 application migration from MySQL to Databricks Lakebase.

•    Evaluate and shape the roadmap for advanced AI capabilities (agentic AI, frontier models, and emerging superintelligence-class technologies) as they apply to the client.

•    Lead architecture reviews and communicate design decisions to executive and technical stakeholders.

•    Design end-to-end architectures that embed AI capabilities into enterprise applications and workflows, defining reference patterns for AI services, models, and orchestration layers that integrate with existing systems, APIs, and data platforms.

•    Collaborate with stakeholders to translate business objectives into scalable, resilient, and policy-compliant designs, balancing vendor-managed services with custom development and meeting performance, security, compliance, and cost objectives.

•    Drive governance for AI integration: prompts, guardrails, and lifecycle management, with observability, privacy, and responsible AI principles embedded from design through operations.

This job can be performed from anywhere in the US