Large Language Model (LLM) Systems… at PassiveLogic · Holladay

Large Language Model (LLM) Systems Scientist

Research and develop multi-agent orchestration and decision-policy algorithms including inference time agent routing, task decomposition and coordination across both open weight and proprietary LLM and Transformer-based models using python, PyTorch and/or Tensorflow. Develop and scale vector-based retrieval and augmented generation (RAG) systems including vector index optimization, retrieval-latency tuning, hybrid (dense + sparse) search across SQL datastore, and embedding-pipeline throughput to support high-volume agentic reasoning across multiple AI applications. Research and develop novel training algorithms, including contrastive learning, using PyTorch and/or Tensorflow to specialize and align multi-agent LLM systems, and productionize them as platform-level training capabilities managed under Git-based version control and CI/CD. Design and scale the platform-level evaluation framework, including statistical benchmarking pipelines implemented using Python, pandas and numpy, automated regression suites, and quality and reliability dashboards, for evaluating LLM-driven generation and inference systems. Create graph-based knowledge representation and reasoning algorithms using graph neural networks (DGL) over semantic knowledge graphs supporting graph generation, node/edge inference, and graph-grounded retrieval for multi-agent LLM workflows. Develop the distributed multi-agent AI platform infrastructure on AWS that hosts production agent deployments serving multiple product lines, including the platform’s application pipelines developed in Swift. Research and implement safety and robustness methods for multi-agent LLM systems, including adversarial and prompt-injection robustness, data-exfiltration defenses, agent action sandboxing, and behavioral safety evaluation and red-teaming of agentic workflows operating against building protocol networks.

Requires: Master's Degree in Computer Science, 2 years of experience as an LLM Systems Scientist or AI Engineer, and proficiency with Python, PyTorch, TensorFlow, Numpy, Pandas, DGL, SQL, Git, AWS, Swift, LLM, Transformer-based systems, retrieval-augmented generation (RAG) and vector-based retrieval, and graph neural networks.