Carl Broker
Advisory AI Engineer, IBM
Duluth, Minnesota
I build AI systems that are meant to be used, not just demoed.
Experience
From neuroscience research to enterprise AI engineering.
Advisory AI Engineer
IBM — Customer Success Engineering
Building enterprise AI systems, multi-agent architectures, and retrieval pipelines — from prototype to production.
- Architected multi-agent orchestration systems with orchestrator and specialist sub-agents
- Built RAG systems with chunking, embeddings, vector retrieval, and grounded generation
- Developed Python tool-calling frameworks for enterprise APIs and workflows
- Built document processing solutions combining vision-language models with PDF extraction tools
- Built evaluation and testing pipelines for agent quality and workflow success
- Authored production deployment of enterprise AI systems and published agents to IBM Agent Catalog
- watsonx
- Orchestrate ADK
- Python
- RAG
- embeddings
- vector databases
- LoRA
- LLM evaluation
- tool calling
- multi-agent systems
AI Engineer
IBM — Client Engineering
Delivering client-facing machine learning, NLP, and generative AI systems for enterprise clients.
- Delivered client-facing generative AI solutions, translating business requirements into production-ready applications
- Built RAG systems with multilingual embedding models and Milvus vector databases
- Developed testing frameworks and evaluation harnesses to validate agent routing logic and model performance
- Led customer-facing architecture and design sessions
- Python
- NLP
- LLMs
- LangChain
- Milvus
- data science
- enterprise architecture
Research Data Scientist
General Dynamics IT — EPA (Federal Contractor)
OCR, NLP, and document intelligence across a large federal document repository.
- Deployed Tesseract OCR across a large document database, automating text extraction from legacy records
- Trained and deployed NLP document classifiers via ensemble learning
- Used transfer learning with RoBERTa and Hugging Face tooling
- Built similarity workflows using transformer embeddings and cosine similarity
- PyTorch
- Hugging Face
- OCR
- Tesseract
- RoBERTa
- NLP
- cosine similarity
Data (Policy) Analyst
State of Washington, Healthcare Authority
Data engineering and policy analysis supporting the audit of Washington's multi-year healthcare expansion.
- Refactored data processing scripts to produce state and federal dataframes used to audit the state's healthcare expansion
- Python
- data processing
- healthcare policy
Senior Data Analyst
Johns Hopkins Health System
Analytics, reporting systems, and healthcare data workflows.
- Produced scheduled and ad-hoc analytical reports for executive leadership
- Built Python and pandas workflows for difficult-to-access datasets
- Redesigned a case management system to accommodate increased patient volume
- Partnered with IT departments and third-party vendors to integrate data sources and streamline reporting pipelines
- Python
- pandas
- analytics
- factor analysis
- reporting systems
Research
Johns Hopkins Medical Institute · NIH (NIDA)
Neuroscience, psychology, and clinical research grounded in data, signals, and behavior.
- Investigated neural mechanisms of motivation and learning using PCA on electrophysiological data and bivariate time-series analysis
- Led startup of a multi-year NIH-funded clinical research study and managed two research interns
- Co-authored 3 peer-reviewed papers, published on PubMed
- statistics
- PCA
- time series analysis
- experimental design
- research methods
- electrophysiology
Selected Work
AI systems, software, experiments, and tools.
Ask Veno: A Fully Local YouTube RAG Pipeline with a Retro-Terminal Dashboard
Turning 257 YouTube livestreams into a searchable, source-anchored knowledge base — Whisper, ChromaDB, and Ollama running entirely on local hardware.
Enterprise-Scale Document Intelligence Pipeline
End-to-end local OCR, NLP, and machine learning system for classifying 250,000+ scanned unclassified records.
Bayesian Optimization at Scale: Optuna-Driven XGBoost for Engineering Alert Classification
How Optuna's Bayesian optimization replaced manual hyperparameter tuning across hundreds of XGBoost classifiers for engineering alert triage.
I build software, AI systems, and tools — then write about what I learn along the way.
Writing
Build logs, benchmarks, and lessons from real systems.
- TensorRT-LLM on an RTX A6000: Part 2 — Concurrency, TTFT, and the Benchmark Trap Pushing TensorRT-LLM under load on an RTX A6000: why synthetic prompt size misleads, why TTFT is a cleaner signal, and what it shows about Dynamo.Jun 2026
- TensorRT-LLM on an RTX A6000: Part 1 — GPU Passthrough, KV Cache, and Single-GPU Serving Serving Qwen2.5-7B-Instruct with NVIDIA TensorRT-LLM on a single RTX A6000 — Docker GPU passthrough, KV cache, prefill/decode, Dynamo, and parallelism strategies.May 2026
About
I build AI systems that are meant to be used, not just demoed.
My work sits at the intersection of agentic AI, retrieval systems, LLM platforms, and production engineering. I like designing systems where models, tools, and data work together to solve real problems — whether that means multi-agent orchestration, vector retrieval, OCR pipelines, or enterprise AI deployment.
Over the course of my career, I’ve moved through neuroscience research, healthcare analytics, government research, data science, and enterprise AI engineering. That path gave me a strong bias toward systems that are grounded, testable, and useful in the real world.
Today, I focus on building enterprise AI systems that include:
- multi-agent orchestration
- retrieval-augmented generation
- embeddings and vector databases
- tool-calling architectures
- evaluation and guardrails
- production-ready LLM workflows
I enjoy the engineering side of AI just as much as the strategy side: choosing the right model for the job, designing the surrounding system, and getting the full thing to work reliably.
Education
- Johns Hopkins University — M.S. in Government Analytics, 2022
- National Institutes of Health — Post-Bacc. IRTA Research Fellowship, 2014
- The College of St. Scholastica — B.S. in Psychology, 2011
Interests
Outside of work, I’m into mountain biking, flying my paramotor, scuba diving, and building side projects that usually start as “just for fun” and turn into something much bigger.
