Applied AI Engineer at Allied… · Scarborou…

Job Description: Applied AI Engineer

Location: Toronto, ON (Scarborough)

Reports To: Staff AI Engineer

Company: ATS Software:

Position Overview

You will select, integrate and evaluate the models that power PDF data extraction, automated product cross-referencing and intelligent quote recommendations across the ATS software suite — combining frontier LLM APIs, specialized document AI services and self-hosted open-source models depending on the accuracy, latency and cost profile each feature demands.

You will work closely with Staff and Lead AI Engineers, Data Engineers and Core Product Teams to turn raw technical data into workflow features that engineers and estimators rely on. The core ATS application is Node.js. We are open to strong engineers from either a Node.js or a Python background, and expect you to work in both. This suits an engineer with solid backend fundamentals, practical MLOps exposure and real experience shipping LLM, RAG or document extraction applications to production.

Key Responsibilities1. AI Feature Development & Model Selection

• Build production features on frontier LLMs (Claude, Gemini, OpenAI) and document AI services (AWS Textract, LandingAI) to extract data from complex technical spec sheets, submittals and engineering drawings.
• Benchmark vendor and open-source options per task on accuracy, latency and cost per document; own the routing and fallback logic between them.
• Design structured extraction with schema-constrained outputs and validation, including confidence scoring that routes uncertain results to human review rather than into quotes.
• Develop and maintain retrieval-augmented generation (RAG) pipelines and vector search indexing for automated product cross-referencing and specification matching.
• Deploy and run self-hosted open-source models where document volume, latency or data sensitivity makes vendor APIs impractical.
• Write clean, maintainable and well-tested microservices in Node.js or Python to expose model functionality to the core web applications.

2. Evaluation & Quality

• Build and maintain golden datasets and automated evaluation suites for every AI feature; no prompt, model or retrieval change ships without a regression run.
• Define accuracy targets per extracted field and retrieval quality targets for cross-referencing, and report against them.
• Monitor production output quality to catch drift and identify common failure modes.
• Own the confidence thresholds that decide what passes straight through and what a human reviews.

3. Pipeline Integration & MLOps

• Integrate AI services into ATS cloud microservices (AWS), ensuring high efficiency, low latency and stability.
• Assist in establishing CI/CD pipelines for AI features, including automated testing and evaluation runs before deployment.
• Monitor production performance and cost: accuracy, inference speed, latency, API failure rates and token spend per document.

4. Data Engineering & Quality

• Clean, process and label complex structured and unstructured technical data from manufacturer catalogs, equipment spec sheets and price lists.
• Build automated data prep pipelines that streamline golden dataset creation and retrieval evaluation benchmarks.
• Maintain the labeled corpora that make model comparison and regression testing possible as the catalog grows.

5. Cross-Functional Collaboration

• Partner with Product Managers and Software Engineers to turn feature specs into working AI capabilities, including setting realistic accuracy expectations before a feature is committed.
• Participate in code reviews, technical spike evaluations and sprint planning to ensure timely, robust delivery.

Qualifications & ExperienceRequired

• Experience & Education: 3+ years of professional software development experience, with at least 1–2 years focused on building and deploying practical AI solutions.
• Core Technical Stack: Advanced proficiency in Node.js/TypeScript or Python. The core product is Node.js; AI services may be written in either. Strong candidates in one are expected to become productive in the other.
• Generative AI & Modern NLP: Hands-on experience building applications using LLMs (Anthropic, OpenAI, Google, or open-source models), prompt engineering, structured outputs and vector databases (e.g. Pinecone, Qdrant, Weaviate, pgvector). Familiarity with LangChain/LlamaIndex, or a clear point of view on when to skip them.
• Document Processing: Experience working with unstructured data extraction (PDF parsing, OCR frameworks like Tesseract/Textract, or multi-modal LLMs).
• Evaluation: A demonstrated approach to measuring AI output quality — golden sets, regression testing, accuracy measurement. Candidates should be able to describe how they knew a change made things better.
• Backend Fundamentals: Experience developing RESTful APIs (Express, NestJS, FastAPI or Flask) and working with relational databases (PostgreSQL, MySQL).
• Software Engineering Best Practices: Experience with Git, Docker, CI/CD, and basic unit/integration testing principles.

Nice to Have

• Bachelor’s degree in Computer Science, Data Science, Software Engineering, or a related technical field.
• Experience fine-tuning open-source models, or fine-tuning embedding models for domain-specific retrieval (PyTorch).
• Experience running self-hosted open-source models in production.
• Experience in AEC (Architecture, Engineering, Construction), PropTech, CPQ, e-commerce, or B2B SaaS applications.
• Cloud platform experience with AWS or Azure services (SageMaker, Lambda, S3, ECS).

Compensation

$100,000 – $140,000 CAD base per year, plus benefits. Scarborough, ON.

Your Life and Career at ATS:

ATS is dynamic, industrious, innovative and collaborative. We believe diverse and inclusive organizations create work environments that are inventive and open-minded, where people spark new ideas and explore alternatives. At ATS, we offer the following to make sure you have a rewarding and enjoyable experience:

· This is a new position

This is an in office position

· Career advancement opportunities

· Benefits package for all eligible full-time employees (including medical, vision and dental).

· A culture that promotes a healthy, fulfilling work-life balance

· Free parking

· Foosball, Ping Pong Table & Basketball net

· Gym facilities

· Epic year-round employee events!

TO APPLY: Please email hr@atsspec.com and include your resume and salary expectations. NO PHONE CALLS PLEASE.

To learn more about our company visit our web page: www.atssoftware.com/ and our promotional video: http://youtu.be/MPyk3BdN-8o

Allied Technical Sales Inc. values diversity and is proud to be an Equal Opportunity Employer. We are committed to the principles and practices of employment equity and encourage all qualified individuals, including women, persons with disabilities, visible minorities, and Aboriginal Peoples to apply. Should you be individually selected to participate in an assessment or selection process, accommodations are available upon request in relation to the materials or processes to be used.

Pay: $100,000.00-$140,000.00 per year

Benefits:

• Casual dress
• Company events
• Disability insurance
• Discounted or free food
• Extended health care
• Life insurance
• On-site gym
• On-site parking
• Paid time off
• RRSP match

Application question(s):

• Backend & Generative AI Experience: Can you describe your hands-on experience building and deploying production-grade AI solutions using LLMs (such as Claude, Gemini, or OpenAI) alongside your proficiency in Node.js/TypeScript or Python?
• Document Processing & Unstructured Data: What is your experience working with document processing pipelines (e.g., PDF parsing, OCR, or multi-modal LLMs) to extract structured data from complex technical spec sheets or engineering drawings?
• Model Evaluation & Quality Assurance: How have you designed golden datasets, automated evaluation suites, or confidence scoring thresholds to measure and monitor accuracy and catch drift in production AI features?

Work Location: In person