AI Engineer World’s Fair 2025 Recap

taurin

Junior Member
Joined
Oct 24, 2023
Messages
182
Reaction score
207

I watched 8 hours of key sessions from the AI Engineer World’s Fair 2025 and put together the following recap:
  • Swix — Opening Keynote & the State of AI Engineering
    AI engineering is evolving fast; the spotlight is shifting toward agent engineering.
    – We need standard frameworks; Swix suggests SPA (Sync, Plan, Analyze, Deliver, Evaluate).
    – Tracking the input ↔ output ratio could be a more practical metric than arguing over what “agent” means.
    – Core lesson: don’t over-complicate things.

  • Asha Sharma — The Open Agentic Web
    Reasoning models are exploding, unlocking new speed and possibilities.
    – Envisions an agentic web where agents interact regardless of cloud, company, or device.
    – Shifts: pair-programming → peer-programming, software factory → agent factory, cloud models → local/on-device.
    – Introduces the “signals loop” (a continuous cycle for agents).
    Foundry platform: model ensembles, agentic RAG (+40 % accuracy on hard queries), tools-as-infrastructure, MCP, evaluation & observability.

  • Sarah Goa — An Investor’s View on AI: What Works & What to Build
    – AI is the largest tech revolution yet; real uptake proven by metrics (ChatGPT, Copilot).
    – Reasoning enables transparent decisions and sequential tasks.
    Agents = software that plans, uses AI, executes tasks, and keeps a goal in mind.
    – Start-ups in the agent space are booming and getting traction.
    – Other modalities (voice, video, image generation) are maturing; model prices are falling.
    – Opportunity: build “Cursor-for-X” apps—code-editor principles for other domains.
    – Conservative industries are adopting AI fastest (“AI leapfrog”).
    Execution is the moat; copilots remain the least-friction path today.

  • Simon Willis — Six-Month LLM Retrospective
    – >30 significant models released in the last six months.
    – Personal benchmark: “Pelican on a Bicycle.”
    – Good local models exist (e.g., Mistral Small 3); high-quality model prices have crashed.
    – GPT-4o shows context interference from memory.
    – Notable bugs: overly deferential ChatGPT, “Snitchbench” (models reporting users).
    Tools + reasoning is currently the most powerful AI-engineering technique.
    – Risks: prompt injection & the “lethal trifecta” (private data access + malicious instructions + exfiltration path).

  • Steven Chin & Andreas Kleger — Agentic Graph RAG
    – Stresses quality data and grounding.
    – Social responsibility grows as we edge toward AGI.
    – Discuss agent memory (e.g., Zep) and the broad concept of Graph RAG.
    – Demo: Neo4j LLM Graph Builder—turn unstructured data into knowledge graphs; query them.
    – Announced a Neo4j start-up program.

  • Theo — The History & Future of MCP
    – MCP arose from the need for LLMs to act beyond copy-paste.
    – Built as an open, standardized protocol for scale.
    – Adoption in tools like Cursor spurred momentum; now backed by major AI labs.
    – Principle: optimize for server simplicity.
    – Roadmap: elicitation (servers ask users for more info), registry API, open-source samples, governance.
    Call to action: build proper MCP servers (not just API wrappers), improve dev-ex, grow safety & observability tooling.

  • John — Scaling MCP Clients at Anthropic
    – Tool explosion caused integration chaos that naturally converged on MCP.
    – MCP standardizes context delivery to models.
    – “Pit of success” model: use an MCP gateway for centralized OAuth, credential management, observability.
    – Standard message format centralizes context representation.

  • Harold — Hidden Powers of MCP
    – Gap between spec and implementations (“API-wrapper syndrome”).
    – Full spec support in VS Code Insiders unlocks multi-level, stateful interactions.
    – Spec enables dynamic tool discovery, semantic resource links, and sampling (server-initiated LLM calls via client).
    – Needs better debugging & logging; spec updates in OAuth & Streamable HTTP are vital.
    – Community registry emerging; action-, context-, and semantics-oriented servers are key.

  • David Kramer — Challenges & Lessons from Using MCP
    – MCP is a plug-in architecture for agents.
    – At Sentry, MCP moved error context into the editor.
    – OAuth 2.1 is hard but doable.
    – MCP ≠ OpenAPI wrapper; design for how models handle context.
    – Client support is inconsistent; remote servers + OAuth spec matter for B2B SaaS.
    – Output must be model- and human-readable (Markdown > raw JSON).
    – Devs don’t control the consumer/model—must account for cost pass-through.
    – Lack of streaming tool responses hinders agent-to-agent UX.

  • Samuel — MCP Is All You Need (for Autonomous Agents)
    – People over-complicate agent-to-agent interaction.
    – MCP works for autonomous code-agents too; prompts/resources matter less, tool invocation more.
    – MCP vs OpenAPI: dynamic tools, logging, sampling, tracing, stdio subprocesses.
    Sampling is powerful: tools/agents can piggy-back on the parent agent’s model.
    – Demo: Pydantic AI agent using BigQuery (MCP server) to generate SQL via sampling, with execution logging & tracing.

  • Alex Vulov & Ben Ekl — Observability in MCP
    – MCP creates observability blind spots that grow with tool count.
    – Enterprises need seamless tracing.
    – W&B Weave integrates MCP today, but custom integrations aren’t vendor-neutral.
    – Proposes OpenTelemetry (OT) for standardized MCP observability (traces, spans, sinks).
    – Demo: trace flow from TS client → Python server via OT & Weave.
    – MCP Run will export telemetry to OT-compatible sinks.
    – Community standards & conventions are needed.

  • Jan Churn — The Rise of the Agent Economy on MCP’s Shoulders
    – General intelligence may emerge from many goal-driven agents.
    – MCP is the communication backbone for this B2A/A2A agent network.
    Appify marketplace hosts 5 000+ actors, discoverable via MCP.
    – Issue: agents need API tokens for third-party services.
    – Fix: a centralized marketplace where one token unlocks many MCP services; actors pay for upstream APIs and monetize.
    – Demo: Claude Desktop + Appify MCP actors (Twitter, Browserbase) for rapid ecosystem scaling.
    – Open questions: trust, autonomous tool discovery, agents & AGI.

  • Antje Bar — Building Agents at Cloud Scale
    – All customer interactions will be re-imagined with AI; new agent opportunities abound.
    – Amazon has >1 000 AI apps in development; Alexa re-imagined with agentic capabilities (600 M+ devices).
    – Future: specialized agents working together.
    Strand Agents (Python SDK) launched: built-in Bedrock, multi-provider support, 20+ tools (RAG, multimodal, multi-agent).
    – Native MCP support; remote MCP servers can run on Lambda with Streamable HTTP.
    – Demo: deploying an MCP server on Lambda & consuming it via Strand Agents.
    – Next: agent-to-agent (A2A) interaction; joined MCP steering committee.
    – “The atomic unit of every digital interaction will be an agent call.”

  • Kevin Hoe — What’s Next for “Identisides” (Agent + IDE)
    – Windsurf Editor launched 6 months ago and spread rapidly.
    – Secret sauce: shared timeline between human & AI – feels like mind-reading.
    – Windsurf must “be everywhere” for context gathering and must act & write everywhere (terminals, browsers, docs).
    – Features: terminal commands, live preview, GitHub MCP PRs, async reviews, one-click deploy.
    – Goal: 99 % agent work, 1 % human approval.
    – Built Suite One (SU1): a model trained on workflows, judged on end-to-end & mid-flow tasks; near-SOTA with lower resources.
    – “Data flywheel”: ship → users hit frontier → insights → refine model/tools → repeat.

  • Greg Brockman — Fireside Chat: What It Means to Be an AI Engineer
    – Love of coding = turning ideas into reality to help people.
    – Stripe taught challenging conventions & first-principles speed.
    – Effective self-learning needs passion & grit.
    – AGI felt possible after reading Turing’s “child machine” and watching deep learning win across domains.
    Engineering is as important as research; harmony scales ideas to impact.
    – Engineers need technical humility to keep learning.
    – ChatGPT/4o launches proved viral demand & the need to scale fast.
    – Vibe coding expands capability; agentic approaches will intersect and take over more.
    – CodeX changes coding style toward modular, testable, model-friendly structures.
    – Scaling large AI requires checkpointing & state management.
    – Future AI infra will diversify for compute vs latency; predicting resource mix is hard.
    Bottleneck for GPT-6: fundamental research rises again—balancing compute, data, algorithms, power, money.
 
Back
Top