Engineering Implementations

ReAct and the Birth of Agent Memory: Why Reasoning Is Memory Creation

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.

ICLR 2023

A reinterpretation of ReAct through the lens of memory systems. Most engineers see ReAct as reasoning plus acting. The deeper contribution is that it introduced explicit working memory into LLM agents — every thought is a state update, every observation is a memory write, and the evolving context becomes the foundation for all modern agent memory architectures from MemGPT to Letta to SimpleMem.

First paper to introduce working memory into LLM agents via accumulated thought-action-observation context
Every reasoning trace is a state transition — thoughts are memory writes, not just text generation
Augmented action space A-hat = A union L treats language itself as a state-modifying operation
Direct intellectual ancestor of MemGPT, Letta, SimpleMem, Mem0, and all modern agent memory systems
71% success on ALFWorld vs 37% for agents trained on 100,000+ expert demonstrations — memory enables this

AI Agents Memory Systems Working Memory ReAct Tool Use Agent Architecture State Management Production Systems

Read Implementation → View Original Paper ↗

Chain-of-Thought Prompting: How Eight Examples Changed the Future of AI Reasoning

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., Zhou, D.

NeurIPS 2022

The foundational discovery that inserting step-by-step reasoning traces into few-shot prompts unlocks emergent multi-step reasoning in large language models — without any fine-tuning, specialized datasets, or model changes. This single prompting technique became the intellectual ancestor of every modern agent architecture: ReAct, Tree-of-Thoughts, LLMCompiler, and structured planning systems.

Eight manually written reasoning exemplars unlock emergent multi-step reasoning at 100B+ parameter scale
PaLM 540B jumps from 17.9% to 56.9% on GSM8K math benchmark — surpassing fine-tuned GPT-3 with a verifier
Reasoning before the answer is causally necessary — post-hoc reasoning provides zero improvement
Emergent phase transition around 100B parameters — smaller models generate fluent but logically incorrect chains
Zero training cost, zero infrastructure changes — purely a prompting technique applicable to any sufficiently large LLM

Chain-of-Thought Prompting Reasoning Emergent Abilities Few-Shot Learning LLM Engineering AI Agents Foundational

Read Implementation → View Original Paper ↗

LLMCompiler: Why DAG Planning Is the Future of Tool-Calling AI Agents

Kim, S., Moon, S., Tabrizi, R., Lee, N., Mahoney, M. W., Keutzer, K., Gholami, A.

ICML 2024

A compiler-inspired orchestration framework that automatically identifies parallelizable LLM function calls, executes them concurrently via a DAG-based planner, and delivers up to 3.7x lower latency and 6.7x cost reduction over sequential ReAct-style execution. This architectural shift from sequential reasoning to parallel compilation fundamentally changes how production AI agents should orchestrate tool use.

Reduces multi-tool agent execution from O(N) sequential LLM calls to 2 LLM calls regardless of task count
DAG-based planning with dependency resolution enables true parallel tool execution without sacrificing correctness
Task Fetching Unit provides LLM-free scheduling — pure orchestration logic with zero inference cost
3.74x latency speedup and 6.73x cost reduction on Movie Recommendation vs ReAct
Dynamic replanning loop handles workloads that cannot be statically compiled upfront

AI Agents Tool Use Parallel Execution DAG Planning LLM Orchestration Function Calling Production Systems Compiler Theory

Read Implementation → View Original Paper ↗

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., Polosukhin, I.

NeurIPS 2017

Proposes the Transformer — a sequence model built entirely on self-attention, eliminating recurrence and convolution. The architecture achieves state-of-the-art machine translation quality while being dramatically faster to train in parallel. It became the architectural foundation for every modern large language model.

Multi-head self-attention allows the model to jointly attend to information from different representation subspaces
Positional encoding injects sequence-order information without recurrence, enabling full parallelism
Encoder-decoder architecture with cross-attention generalises to any seq2seq task
Achieves 28.4 BLEU on WMT 2014 English-to-German, surpassing all prior ensembles

Transformer Self-Attention NLP Sequence Modeling Architecture Deep Learning

Read Implementation → View Original Paper ↗

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.

NAACL 2019

Introduces BERT — a deeply bidirectional pre-training approach for language representations. Unlike prior models that read text left-to-right or as a shallow concatenation of left-to-right and right-to-left passes, BERT pre-trains using masked language modelling and next-sentence prediction, producing contextual embeddings that can be fine-tuned across a wide range of NLP tasks with minimal task-specific architecture changes.

Masked Language Modelling (MLM) enables true bidirectional context — the model sees the full sentence when predicting masked tokens
Next Sentence Prediction (NSP) pre-trains sentence-level relationship understanding
Fine-tuning with a single additional output layer achieves state-of-the-art on 11 NLP tasks
BERT-Large sets a new GLUE score of 80.5, outperforming prior work by over 7 points

BERT Pre-training Bidirectional Masked Language Model NLP Transfer Learning Transformers Fine-tuning

Read Implementation → View Original Paper ↗