We are Generac, a leading energy technology company committed to powering a smarter world.
Over the 60 plus years of Generac’s history, we’ve been dedicated to energy innovation. From creating the home standby generator market category, to our current evolution into an energy technology solutions company, we continue to push new boundaries.
- Architect, build, and operate production AI/ML systems and agentic workflows, including the tooling, infrastructure, and processes needed to develop, test, and deploy them reliably.
- Define and implement quality and reliability practices for AI systems — testing frameworks, monitoring, and pass/fail criteria that gate releases and catch regressions before they reach customers.
- Instrument and operate production systems, building sufficient visibility into system behavior (e.g., logging, tracing, dashboards) to diagnose issues and support continuous improvement.
- Translate production feedback and failures into concrete engineering improvements, driving iteration on models, prompts, retrieval, or other system components based on data and stakeholder input.
- Define and track the technical metrics that matter for a given system or product area (e.g., accuracy, latency, cost, reliability) and produce artifacts that product, engineering, and risk stakeholders can rely on.
- Establish technical standards, tooling, and patterns that can be reused across teams building AI-driven products, promoting consistency in how AI system quality is measured and maintained.
- Partner closely with engineering, product, and design; mentor other engineers; and serve as a technical resource on AI system design, quality, and reliability trade-offs.
- Translate ambiguous, evolving product and technical goals into concrete, measurable engineering plans and deliverables.
Minimum Job Requirements
Education:
Bachelor's or Master's degree in Computer Science, Machine Learning, or a related technical field, or equivalent experience.
Certification/License:
None required.
Work Experience:
5+ years building and operating production AI/ML or software systems, including hands-on experience with large language models (LLMs) and/or AI agent systems.
Knowledge, Skills & Abilities:
- Strong software engineering fundamentals, including proficiency in Python and experience with distributed data pipelines and CI/CD.
- Demonstrated experience building and/or operating evaluation, testing, or observability capabilities for ML/AI systems in production.
- Experience with LLM-based system design, including techniques such as LLM-as-judge evaluation, golden-dataset curation, and regression/drift detection.
- Experience instrumenting or operating complex, multi-step systems (e.g., multi-step reasoning, tool use, retrieval, multi-agent handoffs).
- Experience assessing agentic or AI systems for security, reliability, or failure modes (e.g., red-teaming or similar practices).
- Ability to translate ambiguous product or technical goals into measurable, defensible engineering plans.
- Strong cross-functional collaboration and communication skills; able to represent technical trade-offs to non-technical stakeholders.
Preferred Job Requirements
- Experience with cloud AI/ML platforms (e.g., AWS Bedrock) and agent frameworks (e.g., LangGraph, Strands, or similar).
- Experience with cloud-native MLOps/LLMOps practices: packaging, CI/CD, A/B testing, and model or prompt versioning.
- Hands-on experience with evaluation or observability platforms (e.g., Langfuse, Braintrust, LangSmith, or similar).
- Track record of standing up a new technical capability (e.g., an evaluation or observability function) as a reusable, cross-team resource.
- Experience using synthetic data to increase test coverage for AI/ML systems.
Experience across multiple product domains or business contexts
