Data Engineer at Beghou in New York, NY

Summary: Beghou Consulting is a company with over three decades of experience helping life sciences companies optimize their commercialization through strategic insight and advanced analytics. The Data Engineer role is responsible for building, optimizing, and supporting Beghou’s AI forward data platform, requiring hands-on development in Databricks, Python/Pyspark, and cloud environments. Responsibilities: - Works extensively in Databricks, including installing packages, configuring clusters, managing jobs, overseeing user/permission administration, managing Unity Catalog, using Databricks APIs and AI tools, and resolving configuration issues - Follows software development best practices including Agile workflows, version control (Git/DevOps), code reviews, testing, and documentation - Administers and configure identity/access systems such as AzureAD, SAML, Okta, OAuth, and apply AWS or Azure security best practices - Utilizes WYSIWYG ETL tools such as Azure Data Factory, Informatica, SnapLogic, or Boomi - Deploys and manages container orchestration environments using Docker, Kubernetes, or AWS ECS - Supports web application development using frameworks such as Flask, Django, and front-end technologies like JavaScript, Ajax, CSS/HTML - Uses Microsoft Office tools (Excel, PowerPoint, Word) for documentation, reporting, and presentations Required Qualifications: - Bachelor's or advanced degree in Engineering, Data Science, Statistics, Computer Science, or related quantitative field - 1-3 year of data engineering experience using Python, including pandas and/or PySpark - Strong experience with relational databases such as PostgreSQL, Oracle, MySQL, Redshift, or Snowflake - Experience with Git-based workflows and CI/CD practices - Experience with identity and access management tools (AzureAD/SAML/Okta/OAuth) and cloud security best practices (AWS or Azure) Preferred Qualifications: - Experience with ETL tools (Azure Data Factory, Informatica, SnapLogic, Boomi) - Experience with containerization tools (Docker, Kubernetes, AWS ECS) - At least 6 months of web application development experience with Flask, Django, JavaScript, Ajax, or CSS/HTML - Experience working with AI tools in development workflows - Recognized data engineering certifications preferred (e.g., Databricks Data Engineer, Google Cloud Professional Data Engineer, Azure/Fabric Data Engineer, AWS Certified Data Engineer) - Life Sciences industry experience Required Skills: Databricks, Python/PySpark, Cloud environments (AWS/Azure) Important Skills: Relational databases, CI/CD practices, ETL tools, Containerization tools Nice-to-Have Skills: Web application development, Identity/access management, AI tools experience, Data engineering certifications, Life Sciences experience Benefits: Comprehensive medical, dental, and vision insurance, Life & AD&D coverage, Disability insurance, Lifestyle Wellness Account, Generous PTO, Company holidays, Paid parental & bonding leave, Flexible hybrid work policy, 401(k) and employer match, Tax-free commuter, health, and childcare accounts, Performance bonuses, Monthly phone stipend, Employee discount program, Mentorship, Training, Tuition reimbursement, Charitable donation match