Databricks08.05.2026
Senior Staff Applied AI Engineer - Context Retrieval
Mountain View
Обязанности
- 01Build the full retrieval stack from scratch
- 02Own the end-to-end system: query understanding, content understanding and indexing, hybrid retrieval, ranking, and evaluation
- 03Make the architectural calls that will define how Databricks agents access context for years to come
- 04Retrieve across heterogeneous data — structured and unstructured
- 05Index and rank across structured assets (tables, columns, SQL queries, dashboards, code, notebooks, jobs) and unstructured content (docs, wikis, tickets, chat, images, video, audio)
- 06Build connectors and retrieval adapters for the systems where enterprise knowledge lives
- 07Optimize for two consumers at once: LLMs and humans
- 08Crack query understanding for agents
- 09Crack content understanding at scale
- 10Build search subagents that reason about retrieval
- 11Build the evaluation flywheel for both retrieval and subagents
- 12Set technical direction and grow the team
Требования
- 0110+ years of software engineering experience, with significant time spent building production retrieval, search, or RAG systems at scale
- 02Deep Information Retrieval (IR) expertise: lexical retrieval (BM25, Lucene/Elasticsearch/OpenSearch), dense retrieval (embeddings, ANN indexes — FAISS, ScaNN, HNSW), hybrid retrieval, and learning-to-rank
- 03Hands-on experience with modern LLM-era retrieval: RAG architectures, query rewriting, re-ranking with cross-encoders, long-context strategies, and grounding techniques that reduce hallucination
- 04Experience designing agentic systems on top of retrieval — search planners, multi-hop / iterative retrieval, self-reflection and sufficiency checks, tool-using agents that decide what to fetch and verify what came back
- 05Strong grasp of relevance evaluation: nDCG, MRR, Precision@K, Recall@K; offline/online experimentation; LLM-as-judge frameworks; building human labeling pipelines
- 06Experience working across structured and unstructured data — you've indexed and ranked over tables, code, and documents in the same system, and have opinions about how to do it well
- 07Track record of building 0→1: you've stood up a retrieval system from an empty repo, made the foundational architectural decisions, and grown it into something that customers depend on
- 08Demonstrated ability to operate as a technical leader: setting direction across teams, mentoring senior engineers, and influencing roadmap with research, product, and platform partners